Tomcat - Logging

java-logging
Log4j
slf4j
http://www.moonlit-software.com/
http://logging.apache.org/chainsaw/index.html

http://stackoverflow.com/questions/869945/how-to-send-java-util-logging-to-log4j
http://tomcat.apache.org/tomcat-6.0-doc/logging.html
http://www.mulesoft.com/tomcat-logging - done reading
http://www.mulesoft.com/tcat/tomcat-logging - done reading

http://www.mulesoft.com/tomcat-catalina
http://www.badllama.com/content/tomcat-7-logging-syslog-log4j
http://www.unixpowered.com/unixpowered/2012/05/29/configuring-tomcat-to-log-via-syslog/
http://blog.trifork.nl/2010/01/14/logging-to-the-syslog-from-a-java-application/
http://www.epischel.de/wordpress/2005/09/tomcat-and-syslog-and-accesslog/
http://marcoscorner.walther-family.org/2012/06/apache-tomcat-access-log-to-syslogd/
http://minaret.biz/tips/tomcatLogging.html
http://www.gossamer-threads.com/lists/rsyslog/users/7140
http://www.kubuntuforums.net/showthread.php?61376-Using-rsyslog-to-log-messages-from-Tomcat-Log4j-cluster-M-Cs-in-Lan
http://kb.monitorware.com/rsyslog-imfile-issue-t11609.html
http://marc.info/?l=syslog-ng&m=126994137420470
http://www.engardelinux.org/modules/index/list_archives.cgi?list=syslog-ng-users&page=0099.html&month=2010-03
http://markmail.org/thread/ohxqacaprr23bnea
http://serverfault.com/questions/96720/open-source-tomcat-log-viewer

http://commons.apache.org/proper/commons-logging/
Logback is intended as a successor to the popular log4j project, picking up where log4j leaves off.
jLo
Monolog
Lumberjack
Houston
jTracedump

https://www.loggly.com/docs/tomcat-application-server/
http://www.loggly.com/docs/java-log4j/
http://www.loggly.com/ultimate-guide/centralizing-windows-logs/
https://tomcat.apache.org/tomcat-8.0-doc/logging.html
https://pzolee.blogs.balabit.com/2010/10/collecting-and-forwarding-logs-from-windows/
https://www.liferay.com/community/forums/-/message_boards/message/11954727
http://qnalist.com/questions/910802/redirecting-tomcat-logs-to-rsyslog-server

https://wiki.apache.org/tomcat/Logging_Tutorial

Get rid of the logging.properties file and see if we can use log4j.properties

What am I trying to do?

I need to somehow forward all the log messages from MSTR Web to the Windows Event Log or syslog. I should learn how to use JULI before going the route of forcing it to use log4j. Not only do I need to collect all log messages from MSTR, I also need to collect the access log and merge them.

How can we enable access log and where can I find access log?

Tomcat doesn't log requests by default, but will do if you uncomment this valve in conf/server.xml:

<Valve className="org.apache.catalina.valves.AccessLogValve"
    directory="logs" prefix=access_log." suffix=".txt"
    resolveHosts="false"
    pattern="common"
/>

<Valve className="org.apache.catalina.valves.AccessLogValve" 
    directory="logs" prefix="access_log." suffix=".txt"  
    resolveHosts="false"
    pattern="%{x-Forwarded-For}i %l %u %t %r %s   %{Cookie}i"
/>

<Valve className="org.apache.catalina.valves.AccessLogValve"
    directory="logs"  prefix="access." suffix=".log"
    resolveHosts="false"
    pattern="%h %l %u %t &quot;%r&quot; %s %b &quot;%{Referer}i&quot; &quot;%{User-Agent}i&quot; %D"
/>

Tomcat doesn’t delete the created log files. So it keeps on present in the log directory unless you run a script periodically to delete those files.

The access log entry format we are using is almost identical to the standard combined logging format with an addition: the %D which is used for documenting the Time taken to process the request, in millis will appear at the end of each log entry:

pattern="%h %l %u %t &quot;%r&quot; %s %b &quot;%{Referer}i&quot; &quot;%{User-Agent}i&quot; %D"

See http://tomcat.apache.org/tomcat-7.0-doc/config/valve.html or http://tomcat.apache.org/tomcat-6.0-doc/config/valve.html

How does Tomcat handle logging?

On Windows, if we use the GUI application to configure Tomcat as a service, I see:

-Djava.util.logging.config.file=C:\Program Files (x86)\Apache Software Foundation\Tomcat 6.0\conf\logging.properties

I am not sure what this mean yet.

What does JCL abbreviate for?

Java Common Logging.

As of Tomcat 5.5, JCL technology is used throughout Tomcat. JCL is a lightweight API for Java application that allows hierarchical logging to be supported across all log levels, independent of logging implementation. This means rather than being limited to specific hard-coded framework, you can choose the solution that works for you with only a small amount of extra configuration.

What does JUL abbreviate for?

java.util.logging

What does JULI abbreviate for?

Java Utility Logging Implementation ( Tomcat's modification made on top of java.util.logging)

Although all JDKs ship with logging functionality provided by java.util.logging, this default implementation was not designed for container-based environments. To get around this limitation, Tomcat replaces the default LogManager with JULI, a modified implementation with a number of additional features for handling containers

What is the most notable functionalities of JULI?

The most notable of JULI's extended functionalities is the ability to set property files on a per class loader basis. This features makes redeployment of web application significantly less labor intensive.

What are the ways that we can configure JULI?

JULI can be configured by:

  • The logging.properties file located in your JDK's $JAVA_HOME/jre/lib directory, or at a custom location defined by editing the java.util.logging.config.file property. Look for Java within the Control Panel
  • java.util.logging.class, which is used for programmatic configuration
  • The logging.properties file located within each classloader you wish to configure. This allows you to create separate configuration for each of your web application.

How does JULI configuration syntax compared to JUL?

JULI uses similar configuration syntax to that used by java.util.logging with some additional functionality to interface more flexibility into Tomcat's container-based environment. JULI's features:

  • To allow multiple handlers of a single class to be defined, handler prefixes consisting of a string starting with a digit and ending with a dot (i.e., "1example.") are supported.
  • A list of logger's handlers can be defined using the property loggerName.handlers
  • The loggerName.useParentHandlers boolean value dictates whether or not loggers delegate to a parent with associated handlers. By default, this feature is disabled.
  • The .handlers property can be used to define the root loggers handler set.
  • Logger buffering can be configured on a per-handler basis using the bufferSize property. Values greater than 0 will cause the handler to use a BufferedOutputStream of equal value, in addition to system default buffering, while setting the value of 0 will use only the system default value. Setting a value less than 0 will cause the buffer to flush immediately on every write to the log.
  • Setting a property value to ${systemPropertyName} will cause the property value to be replaced by the specified system property.

What are JULI features?

  • To allow multiple handlers of a single class to be defined, handler prefixes consisting of a string starting with a digit and ending with a dot (i.e., "1example.") are supported.
  • A list of logger's handlers can be defined using the property loggerName.handlers
  • The loggerName.useParentHandlers boolean value dictates whether or not loggers delegate to a parent with associated handlers. By default, this feature is disabled.
  • The .handlers property can be used to define the root loggers handler set.
  • Logger buffering can be configured on a per-handler basis using the bufferSize property. Values greater than 0 will cause the handler to use a BufferedOutputStream of equal value, in addition to system default buffering, while setting the value of 0 will use only the system default value. Setting a value less than 0 will cause the buffer to flush immediately on every write to the log.
  • Setting a property value to ${systemPropertyName} will cause the property value to be replaced by the specified system property.

What is SLF4J?

SLF4J stand for Simple Logging Facade for Java (SLF4J). It serves as a simple facade or abstraction for various logging frameworks (e.g. java.util.logging, logback, log4j) allowing the end user to plug in the desired logging framework at deployment time.

How can we use log4j with Tomcat?

Log4J is, by far, the most popular logging framework / implementation for Java web application because it's speed-optimized, hierarchical, fail-stop, thread-safe, highly configurable, extensible, and specifically designed to mitigate the performance hit caused by log statements. Tomcat is, by default, hard coded to use JULI out of the box. However, if you want to use Log4J, you have to download and replace Tomcat's included JCL support, which is hard-coded to use JULI.

You can either download the full JCL implementation from the "bin/extras" directory of the Apache Tomcat download page, or build it from the Tomcat source tree, using the command -f extras.xml which will generate tomcat-juli.jar and tomcat-juli-adapters.jar in Tomcat's output/extras directory. Copy these files to $CATALINA_HOME/bin (replacing the existing tomcat-juli.jar) and your Tomcat will have full JCL support. Now that Tomcat is ready to work with Log4J, download Log4J from the project website and place log4jx.y.z.jar in $CATALINA_HOME/lib. Log4J use log4j.properties. It does not use the existing logging.properties file. You have to create the log4j.properties file in $CATALINA_HOME/lib.

log4j.rootLogger=INFO, R 
log4j.appender.R=org.apache.log4j.RollingFileAppender
log4j.appender.R.File=${catalina.base}/logs/tomcat.log
log4j.appender.R.MaxFileSize=10MB
log4j.appender.R.MaxBackupIndex=10 
log4j.appender.R.layout=org.apache.log4j.PatternLayout
log4j.appender.R.layout.ConversionPattern=%p %t %c - %m%n

This will generate the tomcat.log file in your main Tomcat logs folder, with a 10MB maximum size, and a maximum of 10 backups. Log4J has a very useful feature that you should not neglect: package-specific logging. Log4J uses Engine and Host names enclosed in square brackets to allow very specific levels of logging for different components:

log4j.logger.org.apache.catalina.core.ContainerBase.[Catalina].[localhost]=INFO
log4j.logger.org.apache.catalina.core=WARN
log4j.logger.org.apache.catalina.session=ERROR

How can we use log4j to log JSP servlets running on a specific Tomcat server at the lowest threshold?

org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/].[jsp] = ALL

You can also use this syntax and configuration layout on a per-application basis by including a custom "log4j.properties" in "WEB-INF/classes" and copying log4jx.y.z.jar to "WEB-INF/lib". Settings such as DEBUG or ALL are extremely verbose, and will noticeably slow Tomcat's startup time.

Why did Tomcat implement JULI instead of JUL or log4j?

Although all JDKs ship with logging functionality provided by java.util.logging, this default implementation is not designed for container-based environments. To get around this limitation, Tomcat replaces the default LogManager with JULI, a modified implementation with a number of additional features for handling containers.

The most notable of JULI's extended functionalities is the ability to set property files on a per-classloader basis. This feature makes redeployment of web applications significantly less labor-intensive.

The logging frameworks used by different web applications run independently of each other. See class loading for more details. The exception to this rule is java.util.logging, if it used directly or indirectly by your logging library. That is because it is loaded by the system and is shared across web applications.

Apache Tomcat has its own implementation of several key elements of java.util.logging API. This implementation is called "JULI". The key component there is a custom LogManager implementation, that is aware of different web applications running on Tomcat (and their different class loaders). It supports private per-application logging configurations. It is also notified by Tomcat when a web application is unloaded from memory, so that the references to its classes can be cleared, preventing memory leaks.

This java.util.logging implementation is enabled by providing certain system properties when starting Java. The Apache Tomcat startup scripts do this for you, but if you are using different tools to run Tomcat (such as jsvc, or running Tomcat from within an IDE), you should take care of them by yourself.

The default implementation of java.util.logging provided in the JDK is too limited to be useful. A limitation of JDK Logging appears to be the inability to have per-web application logging, as the configuration is per-VM. As a result, Tomcat will, in the default configuration, replace the default LogManager implementation with a container friendly implementation called JULI, which addresses these shortcomings. It supports the same configuration mechanisms as the standard JDK java.util.logging, using either a programmatic approach, or properties files. The main difference is that per-classloader properties files can be set (which enables easy redeployment friendly webapp configuration), and the properties files support slightly extended constructs which allows more freedom for defining handlers and assigning them to loggers.

JULI is enabled by default, and supports per classloader configuration, in addition to the regular global java.util.logging configuration. This means that logging can be configured at the following layers:

  1. Globally. That is usually done in the ${catalina.base}/conf/logging.properties file. The file is specified by the java.util.logging.config.file System property which is set by the startup scripts. If it is not readable or is not configured, the default is to use the ${java.home}/lib/logging.properties file in the JRE.
  2. In the web application. The file will be WEB-INF/classes/logging.properties

How can we configure JULI?

Here's a look at all the layers at which JULI's logging can be configured, including both layers standard to java.util.logging and JULI's per-classloader layers:

  • logging.properties, located in your JDK's $JAVA_HOME/jre/lib directory, or at a custom location defined by editing the java.util.logging.config.file property.
  • java.util.logging.config.class, which is used for programmatic configuration
  • a logging.properties file located within each classloader you wish to configure. This allows you to create separate configurations for each of your web applications with lifecycles equivalent to the given application, in addition to your core Tomcat logging configuration.

JULI uses similar configuration syntax to that used by java.util.logging, with some additional functionality to integrate more flexibly into Tomcat's container-based environment. Very detailed documentation on the default java.util.logging configuration syntax is available on Sun's J2SE site, but here's an overview of JULI's additional features:

  • To allow multiple handlers of a single class to be defined, handler prefixes consisting of a strings starting with a digit and ending with a "." (e.g. "1example.") are supported.
  • A list of a logger's handlers can be defined using the property loggerName.handlers, a functionality you might recognize from Java 6.x.
  • The loggerName.useParentHandlers boolean value dictates whether or not loggers delegate to a parent with associated handlers. By default, this feature is disabled.
  • The .handlers property can be used to define the root loggers handler set.
  • Logger buffering can be configured on a per-handler basis using the bufferSize property. Values great than 0 will cause the handler to use a BufferedOutputStream of equal value, in addition to system default buffering, while setting the value to 0 will use only the system default value. Setting a value less than zero will cause the buffer to flush on every write to the log, effectively disabling buffering.
  • Setting a property value to ${systemPropertyName} will cause the property value to be replaced by the specified system property.

The default logging.properties in the JRE specifies a ConsoleHandler that routes logging to System.err. The default conf/logging.properties in Apache Tomcat also adds several FileHandlers that write to files.

A handler's log level threshold is INFO by default and can be set using SEVERE, WARNING, INFO, CONFIG, FINE, FINER, FINEST or ALL. You can also target specific packages to collect logging from and specify a level.

Here is how you would set debugging from Tomcat. You would need to ensure the ConsoleHandler's (or FileHandler's') level is also set to collect this threshold, so FINEST or ALL should be set. Please refer to java.util.logging documentation in the JDK for the complete details:

org.apache.catalina.level=FINEST

The configuration used by JULI is extremely similar to the one supported by plain java.util.logging, but uses a few extensions to allow better flexibility in assigning loggers. The main differences are:

  • A prefix may be added to handler names, so that multiple handlers of a single class may be instantiated. A prefix is a String which starts with a digit, and ends with '.'. For example, 22foobar. is a valid prefix.
  • System property replacement is performed for property values which contain ${systemPropertyName}.
  • As in Java 6.0, loggers can define a list of handlers using the loggerName.handlers property.
  • By default, loggers will not delegate to their parent if they have associated handlers. This may be changed per logger using the loggerName.useParentHandlers property, which accepts a boolean value.
  • The root logger can define its set of handlers using the .handlers property.

There are several additional implementation classes, that can be used together with the ones provided by Java. The notable one is org.apache.juli.FileHandler.

org.apache.juli.FileHandler supports buffering of the logs. The buffering is not enabled by default. To configure it, use the bufferSize property of a handler. A value of 0 uses system default buffering (typically an 8K buffer will be used). A value of <0 forces a writer flush upon each log write. A value >0 uses a BufferedOutputStream with the defined value but note that the system default buffering will also be applied.

Example logging.properties file to be placed in $CATALINA_BASE/conf:

handlers = 1catalina.org.apache.juli.FileHandler, \
           2localhost.org.apache.juli.FileHandler, \
           3manager.org.apache.juli.FileHandler, \
           java.util.logging.ConsoleHandler

.handlers = 1catalina.org.apache.juli.FileHandler, java.util.logging.ConsoleHandler

############################################################
# Handler specific properties.
# Describes specific configuration info for Handlers.
############################################################

1catalina.org.apache.juli.FileHandler.level = FINE
1catalina.org.apache.juli.FileHandler.directory = ${catalina.base}/logs
1catalina.org.apache.juli.FileHandler.prefix = catalina.

2localhost.org.apache.juli.FileHandler.level = FINE
2localhost.org.apache.juli.FileHandler.directory = ${catalina.base}/logs
2localhost.org.apache.juli.FileHandler.prefix = localhost.

3manager.org.apache.juli.FileHandler.level = FINE
3manager.org.apache.juli.FileHandler.directory = ${catalina.base}/logs
3manager.org.apache.juli.FileHandler.prefix = manager.
3manager.org.apache.juli.FileHandler.bufferSize = 16384

java.util.logging.ConsoleHandler.level = FINE
java.util.logging.ConsoleHandler.formatter = java.util.logging.SimpleFormatter

############################################################
# Facility specific properties.
# Provides extra control for each logger.
############################################################

org.apache.catalina.core.ContainerBase.[Catalina].[localhost].level = INFO
org.apache.catalina.core.ContainerBase.[Catalina].[localhost].handlers = \
   2localhost.org.apache.juli.FileHandler

org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/manager].level = INFO
org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/manager].handlers = \
   3manager.org.apache.juli.FileHandler

# For example, to log debug messages in ContextConfig and HostConfig
# classes and to log only warnings and errors in other
# org.apache.catalina.** classes, uncomment these lines:
#org.apache.catalina.startup.ContextConfig.level = FINE
#org.apache.catalina.startup.HostConfig.level = FINE
#org.apache.catalina.level = WARNING

Example logging.properties for the servlet-examples web application to be placed in WEB-INF/classes inside the web application:

handlers = org.apache.juli.FileHandler, java.util.logging.ConsoleHandler

############################################################
# Handler specific properties.
# Describes specific configuration info for Handlers.
############################################################

org.apache.juli.FileHandler.level = FINE
org.apache.juli.FileHandler.directory = ${catalina.base}/logs
org.apache.juli.FileHandler.prefix = servlet-examples.

java.util.logging.ConsoleHandler.level = FINE
java.util.logging.ConsoleHandler.formatter = java.util.logging.SimpleFormatter
  • Consider removing ConsoleHandler from configuration. By default (thanks to the .handlers setting) logging goes both to a FileHandler and to a ConsoleHandler. The output of the latter one is usually captured into a file, such as catalina.out. Thus you end up with two copies of the same messages.
  • Consider removing FileHandlers for the applications that you do not use. E.g., the one for host-manager.
  • The handlers by default use the system default encoding to write the log files. It can be configured with encoding property. See Javadoc for details.
Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License