Where Are Your Log4j Log Files?

Although you can configure Log4j programmatically, it’s far more common to put log4j.properties or log4j.xml in your classpath to trigger autoconfiguration. But one problem that I always have is specifying the location of my log files. From what I can tell, these are your options:

Hard-code an absolute path

Hard-coding the path is brittle, unless every deployment in every environment uses the same location, which is very unlikely. For example, “c:\dev\tomcat\logs” doesn’t make any sense in Linux (although Log4j will treat it as a relative path and helpfully create that folder hierarchy in the app startup directory, e.g. “/usr/local/tomcat/bin/c:/dev/tomcat/logs”).

Hard-code a relative path

Hard-coding a relative path tends to assume too much about the deployment, e.g. “../logs/my_app.log” would work for a Tomcat web app. This assumes that the app is launched from $TOMCAT_HOME/bin and that there is a $TOMCAT_HOME/logs directory. But if the application can be deployed outside of Tomcat, then you either need to mimic Tomcat’s directory structure or wonder where your log files went.

Use a relative path with system properties

The system property approach can work well. Log4j will expand system properties and replace missing properties with a blank string. In this case you’d specify the log location as something like “${app.log.dir}/myapp.log”. The down side to this is the requirement that all users of the configuration file must specify the system property from the command line, which adds a configuration step and may not be possible in strictly controlled environments.

Subclass FileAppender

You could subclass FileAppender to dynamically look up log file destinations at run time (e.g. by looking up locations dynamically via properties, JNDI, etc.) This option is pretty flexible, but forces at least one developer to become knowledgeable about the Log4j API. This is not a big deal at all – subclassing FileAppender is very straightforward – but it does mean that you’re not just writing application code, but framework extension code too. You also now need extra configuration steps, possibly dealing with classloader issues. And there’s the (unlikely) possibility that the Log4j API will change and break your subclass.

Build the configuration file at build time

The last option is the one I tend to use. In addition to providing file location flexibility, it also allows developer-specific or deployment-specific configuration. For example, during debugging it’s common to change the log level of classes and/or categories to generate more or less log output. Log4j configuration files are often source-controlled, so when you make local changes you run the risk of unintentionally committing temporary changes, or forgetting to commit real changes because you’re so used to ignoring the modified local file.

To implement this, I create a template log4j.xml file with placeholders for things that can change, such as file locations and category levels. I also leave in a placeholder for including arbitrary xml snippets, but this is entirely optional. Then I use token filtering in Ant‘s copy task to generate the final log4j.xml file.

Here’s a sample log4j.xml template file (I typically put this in APP_ROOT/resources/log4j.xml.template and add it to source control):

<?xml version='1.0' encoding='UTF-8' ?>

<!DOCTYPE log4j:configuration SYSTEM 'log4j.dtd'>

<log4j:configuration xmlns:log4j='http://jakarta.apache.org/log4j/' debug='false'>

  <appender name='stdout' class='org.apache.log4j.ConsoleAppender'>
    <layout class='org.apache.log4j.PatternLayout'>
      <param name='ConversionPattern' value='%-10r %-5p %-20c{1} %x - %m%n' />
    </layout>
  </appender>

  <appender name='file' class='org.apache.log4j.FileAppender'>
    <param name='File' value='@LOG_DIR@/my_app.log' />
    <param name='Append' value='true' />
    <param name='DatePattern' value="'.'yyyy-MM-dd"/>
    <layout class='org.apache.log4j.PatternLayout'>
      <param name='ConversionPattern' value='%-10r %-5p %-20c{1} %x - %m%n' />
    </layout>
  </appender>

  <appender name='smtp' class='org.apache.log4j.net.SMTPAppender'>
    <param name='from' value='@ERROR_EMAIL_FROM@' />
    <param name='to' value='@ERROR_EMAIL_TO@' />
    <param name='subject' value='My_App error' />
    <param name='SMTPHost' value='@SMTP_SERVER@' />
    <layout class='org.apache.log4j.PatternLayout'>
      <param name='ConversionPattern' value='%-10r %-5p %-20c{1} %x - %m%n' />
    </layout>
  </appender>

  <category name='com.my_app' additivity='false'>
    <priority value='@LOG_LEVEL_APP@' />
    <appender-ref ref='file' />
  </category>

  <category name='org.apache.jsp' additivity='false'>
    <priority value='@LOG_LEVEL_JSP@' />
    <appender-ref ref='file' />
  </category>

  @USER_CATEGORIES@

  <root>
    <priority value='@LOG_LEVEL_ROOT@' />
    <appender-ref ref='file' />
    <appender-ref ref='smtp' />
  </root>

</log4j:configuration>

In this example I’ve configured a console appender (not used), a file appender, and an email appender.

The file appender depends on having the LOG_DIR token replaced with the user- or deployment-specific location. The email appender depends on the ERROR_EMAIL_FROM, ERROR_EMAIL_TO, SMTP_SERVER tokens.

There are 3 tokens to control the log level of the application root category, JSPs, and the Log4j root category, LOG_LEVEL_APP, LOG_LEVEL_JSP, and LOG_LEVEL_ROOT respectively.

And finally there’s the USER_CATEGORIES token, which will be replace with the contents of the file user.log4j.xml if it exists. This allows the developer to include arbitary XML, but would typically be used to temporarily specify log levels for specific categories or classes.

Here’s the build.xml code to create the file. In this example, the properties user.log.dir, error.email.from, and error.email.to must be specified in a file named build.properties – the build will fail otherwise as there are no default values. However, log.level.jsp, log.level.app, and log.level.root do have default values, but they can be overridden by specifying values in build.properties – Ant’s property resolution guarantees that once the value is set it won’t be changed, so the order matters in build.xml.

.
.
.
<property file='build.properties' />
.
.
.
<target name='generate.log4j'>
  <fail unless='user.log.dir' message='Set user.log.dir in build.properties'/>
  <fail unless='error.email.from' message='Set error.email.from in build.properties'/>
  <fail unless='error.email.to' message='Set error.email.to in build.properties'/>

  <!-- default values in case not set in build.properties -->
  <property name='log.level.jsp' value='WARN'/>
  <property name='log.level.app' value='WARN'/>
  <property name='log.level.root' value='WARN'/>

  <!-- optionally load user log4j categories -->
  <loadfile property='log4j.user' srcFile='user.log4j.xml' failonerror='false' />
  <property name='log4j.user' value=''/>

  <copy file='resources/log4j.xml.template' tofile='build/classes/log4j.xml'
    filtering='true' overwrite='true'>

    <filterset>
      <filter token='LOG_DIR' value='${user.log.dir}'/>
      <filter token='LOG_LEVEL_JSP' value='${log.level.jsp}'/>
      <filter token='LOG_LEVEL_APP' value='${log.level.app}'/>
      <filter token='LOG_LEVEL_ROOT' value='${log.level.root}'/>
      <filter token='ERROR_EMAIL_FROM' value='${error.email.from}'/>
      <filter token='ERROR_EMAIL_TO' value='${error.email.to}'/>
      <filter token='SMTP_SERVER' value='${smtp.server}'/>
      <filter token='USER_CATEGORIES' value='${log4j.user}'/>
    </filterset>
  </copy>
</target>

If your application is simple, you may just be able to use one of the simpler approaches, such as hard-coding the locations and levels or using system properties. But if you’re working in a team with several developers, or are in an environment with different deployment configurations, this approach gives you the flexibility to specify locations and other logging attributes without editing source-controlled files. Common configuration information is stored and versioned in source control, and anything that can change is set via tokens during build time, using a combination of default property values and user-specified values.

3 Responses to “Where Are Your Log4j Log Files?”

  1. Martin says:

    Last week I have been searching for how other projects accomplish to specify the location of their log4j files. I must say, that configuration via build script is the most powerful one I have seen until now, when one wants to avoid the programmatical approach.

    I will drop a comment when I find an even better possibility, but I assume there will be none. In the meantime, I’ll be extending my build files.

    Thank you very much.

  2. Burt says:

    That would be great – this post was something of a brain dump, listing the options that I’m aware of, so I’d be interested in hearing about other approaches.

Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 License.