Log4J Architecture and Configuration

The fundamental principle of Log4j is "A Logger writes a message into an Appender with a Level and then formatted by Layout and Pattern before writing into Destination/Output".

The following steps explain in simple terms from an application perspective.

1. An Application (JEE or client/server etc..) reads and loads the Log4j configuration file from a desired location into JVM.
2.From an Application , a Logger (a class from java package) writes a message with Level (debug, info, warn, error, fatal etc..).

3. An Appender (Console, File, Rolling File, SMTP, JDBC or JMS etc..) is configured to filter (debug, info, warn, error, fatal etc..) received messages before writing them with a Layout (Pattern, HTML, XML etc..) and Pattern (conversion) into a destination (File, Database, Email, Message Queue/Topic or Terminal Console etc..).


The Log4J configuration is maintained either in .xml or . properties file for an application.
It's highly recommended to have only one configuration per application, though multiple configurations are also supported.
In a standard scenario, A JEE application uses or depends on following types of libraries.
1. Libraries from Java Standard Edition platform (J2SE1.4.x, Java5 or Java6 etc..).
2. Libraries from Java Enterprise Edition platform (J2EE1.4 or JEE1.5 etc..), provided by container- vendor.
3. Libraries from 3rd Party providers (JDBC drivers, Apache Commons, Framework {spring, struts, hibernate}, JMS etc..).
4. Libraries from enterprise/corporate (LDAP access, utils, etc...),  reusable components for other enterprise applications.
5. Libraries created for specific application.
6. Classes created for application but not reusable in other components.


The sample enterprise application (sales-app-1.3.ear), shown in below image, consists of one web application (app1.war) and 3 EJB components (com-abc-ejb1-1.2.jar, com-abc-ejb2-2.1.jar and com-abc-ejb3-4.2.jar).   The web application uses 3rd party framework (Spring, Struts and Hibernate) libraries, a few enterprise (abc*) libraries and application (app1) specific libraries. As the enterprise application runs inside the confinements of JEE container, it will have to depend on  libraries provided by Vendor.
The application will also use libraries form Java Standard Edition platform.
The application may also use other 3rd party vendor-libraries (Ex. Oracle, mysql JDBC drivers..etc..) configured in container as shared libraries or packaged within application.

Lo4J Configuration for sample JEE application


1. Create a Log4j.xml ( I prefer .xml configuration over .properties for readability and maintainability reasons). 
   Note: Keeping the Log4j.xml file outside of application (.war) file provides flexibility of tweaking the log output based on diagnosing problem in an environment.
2. Create a ConsoleAppender for all DEBUG level Messages with no/default layout.

<!--Create a Console Appender that can have messages only level DEBUG, INFO, WARN, ERROR and FATAL -->
  <appender name="STDOUT_APPENDER" class="org.apache.log4j.ConsoleAppender">
    <layout class="org.apache.log4j.PatternLayout">
          <param name="ConversionPattern" value="%c %d{ISO8601} -- %p -- %m%n"/>
    </layout>
<filter class="org.apache.log4j.varia.LevelRangeFilter">
      <param name="LevelMin" value="DEBUG" />
    </filter>
  </appender>
3. Create a RollingFileAppender for all DEBUG level messages with Pattern Layout.

<!--Create a Rolling File Appender that can have messages only level DEBUG, INFO, WARN, ERROR, FATAL -->

  <appender name="DEBUG_APPENDER" class="org.apache.log4j.RollingFileAppender">
     <param name="File" value="/var/log/sales/app1/ALL_DEBUG.log"/>
     <param name="Append" value="true"/>
     <param name="MaxFileSize" value="1000KB"/>
     <param name="MaxBackupIndex" value="2"/>
    <layout class="org.apache.log4j.PatternLayout">
          <param name="ConversionPattern" value="%c %d{ISO8601} -- %p -- %m%n"/>
    </layout>
  </appender>


4. Create a RollingFileAppender for all INFO level messages with Pattern Layout.

<!--Create a Rolling File Appender that can have messages only level above INFO, WARN, ERROR, FATAL -->

  <appender name="INFO_APPENDER" class="org.apache.log4j.RollingFileAppender">
     <param name="File" value="/var/log/sales/app1/ALL_INFO.log"/>
     <param name="Append" value="true"/>
     <param name="MaxFileSize" value="1000KB"/>
     <param name="MaxBackupIndex" value="2"/>
    <layout class="org.apache.log4j.PatternLayout">
          <param name="ConversionPattern" value="%c %d{ISO8601} -- %p -- %m%n"/>
    </layout>
    <filter class="org.apache.log4j.varia.LevelRangeFilter">
      <param name="LevelMin" value="INFO" />
    </filter>
  </appender>



5. Create a RollingFileAppender for all WARN messages.
<!--Create a Rolling File Appender that can have messages only level above  WARN, ERROR, FATAL -->

  <appender name="WARN_APPENDER" class="org.apache.log4j.RollingFileAppender">

     <param name="File" value="/var/log/sales/app1/ALL_WARN.log"/>

     <param name="Append" value="true"/>

     <param name="MaxFileSize" value="1000KB"/>

     <param name="MaxBackupIndex" value="2"/>

    <layout class="org.apache.log4j.PatternLayout">

          <param name="ConversionPattern" value="%c %d{ISO8601} -- %p -- %m%n"/>

    </layout>

    <filter class="org.apache.log4j.varia.LevelRangeFilter">

      <param name="LevelMin" value="WARN" />

    </filter>

  </appender>

6. Create a RollingFileAppender for all ERROR messages.


<!--Create a Rolling File Appender that can have messages only level above  ERROR, FATAL -->
  <appender name="ERROR_APPENDER" class="org.apache.log4j.RollingFileAppender">
     <param name="File" value="/var/log/sales/app1/ALL_ERROR.log"/>
     <param name="Append" value="true"/>
     <param name="MaxFileSize" value="1000KB"/>
     <param name="MaxBackupIndex" value="2"/>
    <layout class="org.apache.log4j.PatternLayout">
          <param name="ConversionPattern" value="%c %d{ISO8601} -- %p -- %m%n"/>
    </layout>
    <filter class="org.apache.log4j.varia.LevelRangeFilter">
      <param name="LevelMin" value="ERROR" />
    </filter>
  </appender>



7. Configure ROOT logger to write into APPENDERS with DEBUG priority.

  <!-- Register a rootlogger with DEBUG priority and register appenders. 
  This conifguration allows APPLICATION to write ALL messages from all LOGGERS(application, enterprise, framework classes and 3rd Party) into registered APPENDERs.
  Each APPENDER is configured filter only messages of certain LEVEL -->
  <root>
    <priority value="debug"/>
    <appender-ref ref="STDOUT_APPENDER"/>
    <appender-ref ref="DEBUG_APPENDER"/>
    <appender-ref ref="INFO_APPENDER"/>
    <appender-ref ref="WARN_APPENDER"/>
    <appender-ref ref="ERROR_APPENDER"/>
  </root>

8. Create a RollingFileAppender for messages from application loggers (classes in com.abc.app1 package). 

<!--Create a Rolling File Appender for DEBUG level messages only from  loggers/classes in 'com.abc.app1' -->

  <appender name="APP1_APPENDER" class="org.apache.log4j.RollingFileAppender">

     <param name="File" value="/var/log/sales/app1/APP1-DEBUG.log"/>

     <param name="Append" value="true"/>

     <param name="MaxFileSize" value="10000KB"/>

     <param name="MaxBackupIndex" value="10"/>

    <layout class="org.apache.log4j.PatternLayout">

          <param name="ConversionPattern" value="%c %d{ISO8601} -- %p -- %m%n"/>

    </layout>

</appender>

9. Configure APPLICATION specific Logger:  Log all DEBUG messages from loggers whose package is com.abc.app1 into 'APP1_APPENDER'

<logger name="com.abc.app1">
<level value="debug"/>  
<appender-ref ref="APP1_APPENDER" /> 
  </logger> 


10. Configure a component (Servlet, Action or a bean in applicationContext.xml of spring DI) to load this log4j.xml file into JVM for application before any other custom component is loaded.
11. Make ONLY one version of log4j.jar available to application. Remove all additional and external configurations to avoid inconsistencies.


A sample Log4j.xml is provided for reference. Tuning and Tweaking is required based on environment. Download from https://docs.google.com/leaf?id=0B0YbuPQCG3jtZmMzYmE1OGEtM2YyYi00ZTA3LTg3ZmEtMzNjOTUyZGJkNGUx&hl=en&authkey=CKfngsgK


Deploy your application into JEE container verify if the messages are logged with different levels into destination/output as per configuration.


I've addressed basic and core concepts of Log4J that are good enough to configure medium to large scale applications. If you are looking for more features then browse to http://logging.apache.org/index.html .


I've noticed that behavior of Log4J is NOT very consistent across different J2EE servers as their internal class loading mechanisms are different. Please google it in such cases. 


If log4j.x.x jar is loaded from outside of application then it's behavior is very inconsistent, one good and quick fix is to remove all versions of log4j jars from ALL classpath entries and ship only one version of jar within application. This ensures there is only one lo4j.x.x.jar loaded for application.


Have happy logging!

WAS (Websphere Application Server) throws 404 (Resource not found) error though a resource is available in application

Problem: 
The WAS (Websphere Application Server) does not recognize the request for a resource though it exists in Web application (WAR file).


Solution: The WAS, by default, does not invoke the servlet filter for every resource until it is explicitly configured in server.
When a request comes for a URL resource, the WAS throws 404 error as if the requested resource is not available in application though it exists.


Follow the below steps to configure in WAS
a. Login to WAS Admin console application (  http(s)://MYHOST:9443/ibm/console  ).
b. Expand 'Servers' node in left pane and click on 'Application Servers'.
c. Select the <'application server instance> from available application servers
d. Click on 'Web Container' under 'Container Settings' header.
e. Select 'custom properties' under 'Additional Properties' header.
f. Add or update the value to 'true' for property (com.ibm.ws.webcontainer.invokefilterscompatibility).
  This helps WAS to invoke Filter compatibility for resources.
g. Save and restart the app server instance.


Please see below image for reference.

Maven support for Grails application in SpringSource Tool Suite (STS) IDE

System requirements.
JDK1.5 or 1.6, Maven2, SpringSource Tools Suite and internet connection.

1. Download and Install JDK (1.5 or 1.6.x), set JAVA_HOME variable in system.
2. Download and Install Maven2.x and set installed location to a system variable (M2_HOME)
a. Create a system variable (MAVEN_OPTS) with the following value.
-Xmx1024m -XX:MaxPermSize=192m

3. Download & Install SpringSource Tool Suite (STS) from SpringSource site (http://www.springsource.com/developer/sts). You may have to register to  download it.
4. Download and Install following extensions for STS. Navigate to 'Dashboard' from 'Help' menu. Select 'Extensions' tab. Select the below 2 extensions and click on "Install" button. Configure proxy in IDE if required.

   a. "Grails (Current Production Release)"
   b. "Grails Support" under Language and Framework Tooling' header
  

5. Restart the STS IDE when requested to do so.
6. Set 'Grails' installed directory (ex.C:\springsource\grails-1.3.6) to a system variable (GRAILS_HOME)
7. Select 'Grails' perspective in IDE.
8. Open 'atchetype-catalog.xml', available in User home (Ex. C:\Documents and Settings\USER1\.m2), add following lines between archetypes elements. If file does not exist create one.

<archetypes>
<archetype>
<groupId>org.grails</groupId>
<artifactId>grails-maven-archetype</artifactId>
<version>1.3.6</version>
<description>archetype-grails-1.3.6-webapp</description>
</archetype>
</archetypes>


Assuming the settings.xml (maven configuration information) is also available in same (C:\Documents and Settings\USER1\.m2) directory and configured with intranet/local or internet maven repositories.

9. Create a new Maven Project using the above archetype.
a. select File ->New --> Other from menubar.
b. type/search for Maven in input box (refer below image) and select "Maven Project"

c. Click 'Next' button to select 'grails-maven-archetype'. If not found then see if your machine is configured to connect a maven respository and download the 'grails-maven-archetype' plugin.
You may have to restart IDE.

d. Click 'Next' to enter maven coordinates for your sample (MyFirstGrailsProj) grails project.

e. Hit 'Finish' button. The IDE takes a few minutes to create a project with Grails environment. Please observe the outpont 'Maven Console' from 'Console' view.

10. Apply 'Grails nature or capabilities' to 'MyFirstGrailsProj'

IDE takes a few minutes to apply the Grails/Groovy behavior/settings to project. Observe output on Console tab/window.

11. Verify if Step 10 is successful. Please refer to below images.

12. Clean the project.
Select 'Clean' from 'Project' in Menubar

12. Select 'Run As' -> 'maven clean' from context menu of project. Observe 'BUILD SUCCESSFUL' in output of 'Maven Console' and 'Console' windows.
Please fix errors if there are any.

13. Select 'Run As' -> 'maven install' from context menu of project. Observe 'BUILD SUCCESSFUL' in output of 'Maven Console' and 'Console' windows.
Please fix errors if there are any.

14. 'Refresh' project once in a while, as sometimes IDE does not get synced with File System.

15. Run commands of Grails on project by either using menu options or keyboard.
a. Using Menu option.

b. Select the project hit ALT + CTRL+ SHIFT + G keys to see a small popup window then type required Grails command (runapp) and hit Enter key.

Please observe the following output in Cosole window.
Server running. Browse to http://localhost:8080/MyFirstGrailsProj
Clicking on the link will display a home page of deployed application..