Grails SQL Logging part 2 – groovy.sql.Sql

Wednesday, October 31st, 2012 10:25pm

I discussed options for logging Hibernate-generated SQL in an earlier post but today I was trying to figure out how to see the SQL from groovy.sql.Sql and didn’t have much luck at first. The core problem is that the Sql class uses a java.util.logging.Logger (JUL) while the rest of the world uses a Log4j logger (often with a Commons Logging or SLF4J wrapper). I assumed that since I am using the Grails support for JUL -> Log4j bridging (enabled with the grails.logging.jul.usebridge = true setting in Config.groovy) that all I needed to do was add the class name to my log4j DSL block:

log4j = {
   error 'org.codehaus.groovy.grails',
         'org.springframework',
         'org.hibernate',
         'net.sf.ehcache.hibernate'
   debug 'groovy.sql.Sql'
}

but that didn’t work. Some googling led to this mailing list discussion which has a solution involving a custom java.util.logging.Handler to pipe JUL log messages for the 'groovy.sql.Sql' logger to Log4j. That seemed like overkill to me since theoretically that’s exactly what grails.logging.jul.usebridge = true already does. I realized I had no idea how the bridging worked, so I started looking at the implementation of this feature.

It turns out that this is handled by the Grails “logging” plugin (org.codehaus.groovy.grails.plugins.log4j.LoggingGrailsPlugin) which calls org.slf4j.bridge.SLF4JBridgeHandler.install(). This essentially registers a listener that receives every JUL log message and pipes it to the corresponding SLF4J logger (typically wrapping a Log4j logger) with a sensible mapping of the different log levels (e.g. FINEST -> TRACE, FINER -> DEBUG, etc.)

So what’s the problem then? While grails.logging.jul.usebridge = true does configure message routing, it doesn’t apply level settings from the log4j block to the corresponding JUL loggers. So although I set the level of 'groovy.sql.Sql' to debug, the JUL logger level is still at the default level (INFO). So all I need to do is programmatically set the logger’s level to DEBUG (or TRACE to see everything) once, e.g. in BootStrap.groovy

import groovy.sql.Sql
import java.util.logging.Level

class BootStrap {

   def init = { servletContext ->
      Sql.LOG.level = Level.FINE
   }
}

This Week in Grails (2012-43)

Monday, October 29th, 2012 11:17pm

The Grails48 hackathon is coming up soon, Nov 9-11th, 2012. Follow @grails48 for more info. All of the cool kids will be there – will you?

The call for papers open for Greach 2.0 (Jan 25/26 2013, Madrid Spain) is open

I wrote a blog post this week: Autodiscovery of JPA-annotated domain classes in Grails


If you want to keep up with these “This Week in Grails” posts you can access them directly via their category link or in an RSS reader with the feed for just these posts.


Translations of this post:



Plugins

There were no new plugins released but 8 updated plugins:

  • bootstrap-crumbs version 1.0.1. Provide simple breadcrumb functionality using the twitter bootstrap library
  • burning-image version 0.5.1. Easily attach images to any domain class via an annotation. You can also configure the plugin to scale images and perform other operation
  • cxf version 1.0.6. Expose Grails services as SOAP web services via CXF
  • cxf-client version 1.4.7. Use existing (or new) Apache CXF wsdl2java generated content to invoke SOAP services
  • kickstart-with-bootstrap version 0.8.3. Start your project with a good looking frontend, with adapted scaffolding templates for standard CRUD pages using Twitter Bootstrap
  • oauth version 2.1.0. Provides easy interaction with OAuth service providers
  • spring-security-taobao version 1.0.12. Integrates the Taobao Open API Authentication with the Spring Security Core plugin
  • vaadin version 1.7.0-beta5.2. Adds Vaadin (http://vaadin.com/) integration

Interesting Tweets

User groups and Conferences


Autodiscovery of JPA-annotated domain classes in Grails

Wednesday, October 24th, 2012 10:04pm

There are some issues to be fixed with the support for adding JPA annotations (for example @Entity) to Groovy classes in grails-app/domain in 2.0. This is due to the changes made to adding most GORM methods to the domain class bytecode with AST transformations instead of adding them to the metaclass at runtime with metaprogramming. There is a workaround – put the classes in src/groovy (or write them in Java and put them in src/java).

This adds a maintenance headache though because by being in grails-app/domain the classes are automatically discovered, but there’s no scanning of src/groovy or src/java for annotated classes so they must be explicitly listed in grails-app/conf/hibernate/hibernate.cfg.xml. We do support something similar with the ability to annotate Groovy and Java classes with Spring bean annotations like @Component and there is an optional property grails.spring.bean.packages in Config.groovy that can contain one or more packages names to search. We configure a Spring scanner that looks for annotated classes and automatically registers them as beans. So that’s what we need for JPA-annotated src/groovy and src/java classes.

It turns out that there is a Spring class that does this, org.springframework.orm.hibernate3.annotation.AnnotationSessionFactoryBean. It extends the standard SessionFactory factory bean class org.springframework.orm.hibernate3.LocalSessionFactoryBean and adds support for an explicit list of class names to use and also a list of packages to scan. Unfortunately the Grails factory bean class org.codehaus.groovy.grails.orm.hibernate.ConfigurableLocalSessionFactoryBean also extends LocalSessionFactoryBean so if you configure your application to use AnnotationSessionFactoryBean you’ll lose a lot of important functionality from ConfigurableLocalSessionFactoryBean. So here’s a subclass of ConfigurableLocalSessionFactoryBean that borrows the useful annotation support from AnnotationSessionFactoryBean and can be used in a Grails application:

package com.burtbeckwith.grails.jpa;

import java.io.IOException;

import javax.persistence.Embeddable;
import javax.persistence.Entity;
import javax.persistence.MappedSuperclass;

import org.codehaus.groovy.grails.orm.hibernate.ConfigurableLocalSessionFactoryBean;
import org.codehaus.groovy.grails.orm.hibernate.cfg.GrailsAnnotationConfiguration;
import org.hibernate.HibernateException;
import org.hibernate.MappingException;
import org.hibernate.cfg.Configuration;
import org.springframework.context.ResourceLoaderAware;
import org.springframework.core.io.Resource;
import org.springframework.core.io.ResourceLoader;
import org.springframework.core.io.support.PathMatchingResourcePatternResolver;
import org.springframework.core.io.support.ResourcePatternResolver;
import org.springframework.core.io.support.ResourcePatternUtils;
import org.springframework.core.type.classreading.CachingMetadataReaderFactory;
import org.springframework.core.type.classreading.MetadataReader;
import org.springframework.core.type.classreading.MetadataReaderFactory;
import org.springframework.core.type.filter.AnnotationTypeFilter;
import org.springframework.core.type.filter.TypeFilter;
import org.springframework.util.ClassUtils;

/**
 * Based on org.springframework.orm.hibernate3.annotation.AnnotationSessionFactoryBean.
 * @author Burt Beckwith
 */
public class AnnotationConfigurableLocalSessionFactoryBean extends ConfigurableLocalSessionFactoryBean implements ResourceLoaderAware {

   private static final String RESOURCE_PATTERN = "/**/*.class";

   private Class<?>[] annotatedClasses;
   private String[] annotatedPackages;
   private String[] packagesToScan;

   private TypeFilter[] entityTypeFilters = new TypeFilter[] {
         new AnnotationTypeFilter(Entity.class, false),
         new AnnotationTypeFilter(Embeddable.class, false),
         new AnnotationTypeFilter(MappedSuperclass.class, false),
         new AnnotationTypeFilter(org.hibernate.annotations.Entity.class, false)};

   private ResourcePatternResolver resourcePatternResolver = new PathMatchingResourcePatternResolver();

   public AnnotationConfigurableLocalSessionFactoryBean() {
      setConfigurationClass(GrailsAnnotationConfiguration.class);
   }

   public void setAnnotatedClasses(Class<?>[] annotatedClasses) {
      this.annotatedClasses = annotatedClasses;
   }

   public void setAnnotatedPackages(String[] annotatedPackages) {
      this.annotatedPackages = annotatedPackages;
   }

   public void setPackagesToScan(String[] packagesToScan) {
      this.packagesToScan = packagesToScan;
   }

   public void setEntityTypeFilters(TypeFilter[] entityTypeFilters) {
      this.entityTypeFilters = entityTypeFilters;
   }

   public void setResourceLoader(ResourceLoader resourceLoader) {
      this.resourcePatternResolver = ResourcePatternUtils.getResourcePatternResolver(resourceLoader);
   }

   @Override
   protected void postProcessMappings(Configuration config) throws HibernateException {
      GrailsAnnotationConfiguration annConfig = (GrailsAnnotationConfiguration)config;
      if (annotatedClasses != null) {
         for (Class<?> annotatedClass : annotatedClasses) {
            annConfig.addAnnotatedClass(annotatedClass);
         }
      }
      if (annotatedPackages != null) {
         for (String annotatedPackage : annotatedPackages) {
            annConfig.addPackage(annotatedPackage);
         }
      }
      scanPackages(annConfig);
   }

   protected void scanPackages(GrailsAnnotationConfiguration config) {
      if (packagesToScan == null) {
         return;
      }

      try {
         for (String pkg : packagesToScan) {
            logger.debug("Scanning package '" + pkg + "'");
            String pattern = ResourcePatternResolver.CLASSPATH_ALL_URL_PREFIX +
                  ClassUtils.convertClassNameToResourcePath(pkg) + RESOURCE_PATTERN;
            Resource[] resources = resourcePatternResolver.getResources(pattern);
            MetadataReaderFactory readerFactory = new CachingMetadataReaderFactory(resourcePatternResolver);
            for (Resource resource : resources) {
               if (resource.isReadable()) {
                  MetadataReader reader = readerFactory.getMetadataReader(resource);
                  String className = reader.getClassMetadata().getClassName();
                  if (matchesFilter(reader, readerFactory)) {
                     config.addAnnotatedClass(resourcePatternResolver.getClassLoader().loadClass(className));
                     logger.debug("Adding annotated class '" + className + "'");
                  }
               }
            }
         }
      }
      catch (IOException ex) {
         throw new MappingException("Failed to scan classpath for unlisted classes", ex);
      }
      catch (ClassNotFoundException ex) {
         throw new MappingException("Failed to load annotated classes from classpath", ex);
      }
   }

   private boolean matchesFilter(MetadataReader reader, MetadataReaderFactory readerFactory) throws IOException {
      if (entityTypeFilters != null) {
         for (TypeFilter filter : entityTypeFilters) {
            if (filter.match(reader, readerFactory)) {
               return true;
            }
         }
      }
      return false;
   }
}

You can replace the Grails SessionFactory bean in your application’s grails-app/conf/spring/resources.groovy by using the same name as the one Grails registers:

import com.burtbeckwith.grails.jpa.AnnotationConfigurableLocalSessionFactoryBean

beans = {
   sessionFactory(AnnotationConfigurableLocalSessionFactoryBean) { bean ->
      bean.parent = 'abstractSessionFactoryBeanConfig'
      packagesToScan = ['com.mycompany.myapp.entity']
   }
}

Here I’ve listed one package name in the packagesToScan property but you can list as many as you want. You can also explicitly list classes with the annotatedClasses property. Note that this is for the “default” DataSource; if you’re using multiple datasources you will need to do this for each one.

So this means we can define this class in src/groovy/com/mycompany/myapp/entity/Person.groovy:

package com.mycompany.myapp.entity

import javax.persistence.Column
import javax.persistence.Entity
import javax.persistence.GeneratedValue
import javax.persistence.Id
import javax.persistence.Version

@Entity
class Person {

   @Id @GeneratedValue
   Long id

   @Version
   @Column(nullable=false)
   Long version

   @Column(name='first', nullable=false)
   String firstName

   @Column(name='last', nullable=false)
   String lastName

   @Column(nullable=true)
   String initial

   @Column(nullable=false, unique=true, length=200)
   String email
}

It will be detected as a domain class and if you run the schema-export script the table DDL will be there in target/ddl.sql.


There are a few issues to be aware of however, mostly around constraints. You can’t define a constraints or mapping block in the class – they will be ignored. The mappings that you would have added just need to go in the annotations. For example I have overridden the default names for the firstName and lastName properties in the example above. But nullable=true is the default for JPA and it’s the opposite in Grails – properties are required by default. So while the annotations will affect the database schema, Grails doesn’t use the constraints from the annotations and you will get a validation error for this class if you fail to provide a value for the initial property.

You can address this by creating a constraints file in src/java; see the docs for more details. So in this case I would create src/java/com/mycompany/myapp/entity/PersonConstraints.groovy with a non-static constraints property, e.g.

constraints = {
   initial(nullable: true)
   email unique: true, length: 200)
}

This way the Grails constraints and the database constraints are in sync; without this I would be able to create an instance of the domain class that has an email with more than 200 characters and it would validate, but cause a database constraint exception when inserting the row.

This also has the benefit of letting you use the Grails constraints that don’t correspond to a JPA constraint such as email and blank.

This Week in Grails (2012-42)

Wednesday, October 24th, 2012 10:01pm

The big news last week was SpringOne 2GX in Washington, DC. It was a lot of fun as usual, and apparently 300 of the over 1000 attendees were there for the Groovy and Grails content, so that’s a great indicator of interest in the technologies – perhaps in another few years we’ll be over 50% 🙂 There was as usual a lot of Twitter activity during the conference (search using the #sg2x hashtag):

Also check out Ryan Vanderwerf’s SpringOne2GX Wrap up. I’m sure there will be more like this in next week’s post too.

I did a blog post this week on Logging Hibernate SQL based on a question at one of my 2GX talks.


If you want to keep up with these “This Week in Grails” posts you can access them directly via their category link or in an RSS reader with the feed for just these posts.


Translations of this post:



Plugins

There were 4 new plugins released:

  • android-gcm version 0.2. Provides a service to easily access the Google Cloud Messaging (GCM) services
  • app-forty-two-paas version 0.1. Develop engaging and connected Mobile, Web, Social, Enterprise and SaaS Apps using ShepHertz App42 PaaS Cloud and Backend as a Service Platform
  • cassandra-orm version 0.2.6. Provides GORM-like dynamic methods for persisting Groovy objects into Cassandra (but does not implement the GORM API)
  • wkhtmltopdf version 0.1.7. Provides a wrapper for wkhtmltopdf, a simple shell utility to convert html to pdf using the webkit rendering engine, and qt

and 16 updated plugins:

  • airbrake version 0.8.1. Notifier plugin for integrating apps with Airbrake
  • build-info-tag version 0.3.1. Puts a build.info file in the generated WAR file and provides a GSP tag to display the information in it
  • cassandra-astyanax version 0.2.6. Exposes the Astyanax Cassandra client as a Grails service and adds dynamic methods
  • cors version 1.0.3. Installs a servlet filter to set Access-Control-Allow-Origin and other CORS related headers to enable cross site AJAX requests to your Grails application
  • ember-templates-precompiler version 0.2. Precompiles EmberJS powered Handlebars templates
  • functional-test-development version 0.9.3. Installs a script, develop-functional-tests, that you can use to develop your functional tests more conveniently
  • google-visualization version 0.5.6. Provides a taglib for the interactive charts of the Google Visualization API
  • gvps version 0.3. Host, manage and display video assets, and convert standard movie formats to the Flash movie format FLV
  • kickstart-with-bootstrap version 0.7.2. Start your project with a good looking frontend, with adapted scaffolding templates for standard CRUD pages using Twitter Bootstrap
  • mysql-connectorj version 5.1.22.1. MySQL Connector/J
  • neo4j version 1.0.0.M3. GORM for Neo4j
  • plastic-criteria version 0.4. Mock Grails Criteria for Unit Tests
  • platform-core version 1.0.M6.1. Provides functionality for plugins to use to achieve greater integration with each other and with applications
  • social-sharing version 1.0. Provides a tag library for inserting ‘Sexy Bookmarks’ into your application
  • vaadin version 1.7.0-beta5.1. Adds Vaadin (http://vaadin.com/) integration
  • webhook version 0.9.1.6. Easily register and associate a webhook with services provided by individualized controllers

Interesting Tweets

User groups and Conferences


Logging Hibernate SQL

Thursday, October 18th, 2012 2:54pm

There are two well-known ways to log Hibernate SQL in Grails; one is to add logSql = true in DataSource.groovy (either in the top-level block for all environments or per-environment)

dataSource {
   dbCreate = ...
   url = ...
   ...
   logSql = true
}

and the other is to use a Log4j logging configuration:

log4j = {
   ...
   debug 'org.hibernate.SQL'
}

The problem with logSql is that it’s too simple – it just dumps the SQL to stdout and there is no option to see the values that are being set for the positional ? parameters. The logging approach is far more configurable since you can log to the console if you want but you can configure logging to a file, to a file just for these messages, or any destination of your choice by using an Appender.

But the logging approach is problematic too – by enabling a second Log4j category

log4j = {
   ...
   debug 'org.hibernate.SQL'
   trace 'org.hibernate.type'
}

we can see variable values, but you see them both for PreparedStatement sets and for ResultSet gets, and the gets can result in massive log files full of useless statements. This works because the “Type” classes that Hibernate uses to store and load Java class values to database columns (for example LongType, StringType, etc.) are in the org.hibernate.type package and extend (indirectly) org.hibernate.type.NullableType which does the logging in its nullSafeSet and nullSafeGet methods.

So if you have a GORM domain class

class Person {
   String name
}

and you save an instance

new Person(name: 'me').save()

you’ll see output like this:

DEBUG hibernate.SQL  - insert into person (id, version, name) values (null, ?, ?)
TRACE type.LongType  - binding '0' to parameter: 1
TRACE type.StringType  - binding 'me' to parameter: 2
DEBUG hibernate.SQL  - call identity()

When you later run a query to get one or more instances

def allPeople = Person.list()

you’ll see output like this

DEBUG hibernate.SQL  - select this_.id as id0_0_, this_.version as version0_0_, this_.name as name0_0_ from person this_
TRACE type.LongType  - returning '1' as column: id0_0_
TRACE type.LongType  - returning '0' as column: version0_0_
TRACE type.StringType  - returning 'me' as column: name0_0_

This isn’t bad for one instance but if there were multiple results then you’d have a block for each result containing a line for each column.

I was talking about this yesterday at my Hibernate talk at SpringOne 2GX and realized that it should be possible to create a custom Appender that inspects log statements for these classes and ignores the statements resulting from ResultSet gets. To my surprise it turns out that everything has changed in Grails 2.x because we upgraded from Hibernate 3.3 to 3.6 and this problem has already been addressed in Hibernate.

The output above is actually from a 1.3.9 project that I created after I got unexpected output in a 2.1.1 application. Here’s what I saw in 2.1.1:

DEBUG hibernate.SQL  - 
    /* insert Person
        */ insert 
        into
            person
            (id, version, name) 
        values
            (null, ?, ?)

TRACE sql.BasicBinder  - binding parameter [1] as [BIGINT] - 0

TRACE sql.BasicBinder  - binding parameter [2] as [VARCHAR] - asd

and

DEBUG hibernate.SQL  -
    /* load Author */ select
        author0_.id as id1_0_,
        author0_.version as version1_0_,
        author0_.name as name1_0_
    from
        author author0_
    where
        author0_.id=?

TRACE sql.BasicBinder  - binding parameter [1] as [BIGINT] - 1

TRACE sql.BasicExtractor  - found [0] as column [version1_0_]

TRACE sql.BasicExtractor  - found [asd] as column [name1_0_]

So now instead of doing all of the logging from the types’ base class, it’s been reworked to delegate to org.hibernate.type.descriptor.sql.BasicBinder and org.hibernate.type.descriptor.sql.BasicExtractor. This is great because now we can change the Log4j configuration to

log4j = {
   ...
   debug 'org.hibernate.SQL'
   trace 'org.hibernate.type.descriptor.sql.BasicBinder'
}

and have our cake and eat it too; the SQL is logged to a configurable Log4j destination and only the PreparedStatement sets are logged.

Note that the SQL looks different in the second examples not because of a change in Grails or Hibernate but because I always enable SQL formatting (with format_sql) and comments (with use_sql_comments) in test apps so when I do enable logging it ends up being more readable, and I forgot to do that for the 1.3 app:

hibernate {
   cache.use_second_level_cache = true
   cache.use_query_cache = false
   cache.region.factory_class = 'net.sf.ehcache.hibernate.EhCacheRegionFactory'
   format_sql = true
   use_sql_comments = true
}

This Week in Grails (2012-41)

Monday, October 15th, 2012 9:17am

I didn’t have time to do a post last week since I was getting ready for SpringOne 2GX 2012 which will be this week. Should be another great conference this year, I’m sure there will be a lot of buzz on Twitter.

Groovy 2.0.5 was released this week.

Groovy project tech lead Jochen “blackdrag” Theodorou has published two blog posts this week, Owner, Delegate and (implicit) this in an Open Block and Open Blocks and MOP 2.

Be sure to vote for your favorite web framework in the InfoQ Top 20 Web Frameworks for the JVM poll.

Spring Tool Suite and Groovy/Grails Tool Suite 3.1.0 were released this week.

Spock 0.7 was released this week. Looks like a lot of cool new features.

Check out this new Grails job site, http://findgrailsjobs.com/.


If you want to keep up with these “This Week in Grails” posts you can access them directly via their category link or in an RSS reader with the feed for just these posts.


Translations of this post:



Plugins

There were 10 new plugins released:

  • app-forty-two-paas version 0.1. Develop engaging and connected Mobile, Web, Social, Enterprise and SaaS Apps using ShepHertz App42 PaaS Cloud and Backend as a Service Platform
  • backbonejs version 0.9.2.2. Provides resources for Backbone.js http://backbonejs.org/
  • bruteforce-defender version 1.0. Adds functionality of blocking user account after a configured number of failed login, thus countering brute-force attacks
  • closure-compiler version 0.4. Compiles/optimizes your javascript resources with the Google Closure Compiler
  • ember-templates-precompiler version 0.1. Precompiles EmberJS powered Handlebars templates
  • foursquare version 0.1. Integrates the Foursquare APIs
  • glickr version 0.1. Integrates the Flickr API
  • gvps version 0.2. Host, manage and display video assets, and convert standard movie formats to the Flash movie format FLV
  • rabbitmq-tasks version 0.5.2. Run background tasks using RabbitMQ to queue them
  • webhook version 0.9.1.1. Easily register and associate a webhook with services provided by individualized controllers

and 22 updated plugins:

  • asynchronous-mail version 0.7. Send email asynchronously by storing them in the database and sending with a Quartz job
  • attachmentable version 0.3.0. Provides a generic way to add and manage attachments
  • aws-sdk version 1.3.22. Use the Amazon Web Services infrastructure services
  • cors version 1.0.1. Installs a servlet filter to set Access-Control-Allow-Origin and other CORS related headers to enable cross site AJAX requests to your Grails application
  • cucumber version 0.6.2. Test your Grails apps with Cucumber
  • cxf version 1.0.5. Expose Grails services as SOAP web services via CXF
  • cxf-client version 1.4.6. Use existing (or new) Apache CXF wsdl2java generated content to invoke SOAP services
  • dustjs-resources version 0.9.2-BETA2. Supports server-side compilation of .dust template files to their .js counterparts
  • ext-proc version 0.3. Provides easy access to external processes
  • facebook-sdk version 0.4.2. Allows your application to use the Facebook Platform and develop Facebook apps on Facebook.com or on web sites (with Facebook Connect)
  • font-awesome-resources version 2.0.1. Integrates the Font Awesome icon set
  • functional-test-development version 0.9.2. Installs a script, develop-functional-tests, that you can use to develop your functional tests more conveniently
  • grom version 0.2.5. Sends notifications on Windows, Linux, and Mac
  • guard version 1.0.7. Provides a way to run integration tests without having to repeatedly bootstrap the environment
  • jasper version 1.6.1. Enables use of JasperReports
  • jquery-ui version 1.8.24. Supplies jQuery UI resources, and depends on the jQuery plugin to include the core jquery libraries
  • plastic-criteria version 0.3. Mock Grails Criteria for Unit Tests
  • remote-control version 1.3. Execute code inside a remote Grails application
  • struts1 version 1.3.11. Lets you use Struts 1 as a the controller/view layer
  • underscore version 1.4.2. Simple plugin wrapper for useful Underscore.js library
  • uploadr version 0.6.0.1. HTML5 Drag and Drop file uploader
  • vaadin version 1.5.5. Adds Vaadin (http://vaadin.com/) integration

Interesting Tweets

Jobs



User groups and Conferences


This Week in Grails (2012-39)

Wednesday, October 03rd, 2012 8:50am

SpringSource is giving away a free seat in the Groovy and Grails course – all you have to do is sign up for a newsletter.

I did a few “real” blog posts this week:

According to this blog post the code AUTHD will get you 50% off Programming Grails.


If you want to keep up with these “This Week in Grails” posts you can access them directly via their category link or in an RSS reader with the feed for just these posts.


Translations of this post:



Plugins

There were 3 new plugins released:

  • akka version 0.5. Akka actors integration from Groovy and Java, in a Servlet 3.x environment
  • cors version 1.0.0. Installs a servlet filter to set Access-Control-Allow-Origin and other CORS related headers to enable cross site AJAX requests to your Grails application
  • mandrill version 0.1. A simple wrapper for the Mandrill REST API – http://www.mandrillapp.com

and 16 updated plugins:

  • airbrake version 0.7.2. Notifier plugin for integrating apps with Airbrake
  • aws-sdk version 1.3.21.1. Use the Amazon Web Services infrastructure services
  • build-info-tag version 0.2. Puts a build.info file in the generated WAR file and provides a GSP tag to display the information in it
  • dustjs-resources version 0.9. Supports server-side compilation of .dust template files to their .js counterparts
  • errors version 0.8. Sets up some basic error handling for your application
  • events-push version 1.0.M3. A client-side event bus based on the portable push library Atmosphere that propagates events from the server-side event bus to the browser
  • events-si version 1.0.M3. Standard Events system for Grails implementation; it is a Spring Integration implementation and uses its artefacts to map listeners, senders and events messages
  • facebook-sdk version 0.4.1. Allows your application to use the Facebook Platform and develop Facebook apps on Facebook.com or on web sites (with Facebook Connect)
  • faker version 0.7. A port of Data::Faker from Perl, is used to easily generate fake data: names, addresses, phone numbers, etc.
  • federated-grails version 0.3.1. Uses Shiro and Shibboleth to integrate into federated authentication
  • jxl version 0.54. Export data to Excel using the JXL library
  • portlets-gatein version 0.3. Provides a simple way of deploying Grails portlets to JBoss GateIN 3.1 Portal
  • remote-pagination version 0.3.1. Provides tags for pagination and to sort columns without page refresh using Ajax and loads only the list of objects needed
  • underscore version 1.4.0. Simple plugin wrapper for useful Underscore.js library
  • yammer-metrics version 2.1.2-3. Packages Coda Hale’s yammer metrics jars
  • zipped-resources version 1.0.1. Integrates with Grails’ resources framework to automatically gzip static files

Interesting Tweets

User groups and Conferences


Keeping Grails namedQueries DRY

Thursday, September 27th, 2012 1:11am

The Named queries support in Grails is cool feature where you define a partial criteria query and can call it directly, or compose it with additional criteria to create complex queries. For example you might want to query for instances created before a specified date:

class Foo {
   String name
   Date dateCreated
   Integer bar

   static namedQueries = {
      createdBefore { date ->
         le 'dateCreated', date
      }
   }
}

Using this we can use just the named query:

def moreThanTwoDaysOld = Foo.createdBefore(new Date() - 2).list()

or further refine the query:

def moreThanTwoDaysOld = Foo.createdBefore(new Date() - 2) {
   lt 'bar', 20
}

But what if you have common named queries across various domain classes? It turns out it’s not hard to keep things DRY, but it’s a bit tricky.


To see how the solution works, it’s important to understand how the definion of the named queries is implemented. It’s a static Closure in a domain class, which contains methods whose names will end up being the names of the queries. This will be more clear if we add in the optional parentheses that were omitted above:

static namedQueries = {
   createdBefore({ date ->
      le 'dateCreated', date
   })
}

So it should hopefully be clear that this is an invocation of the createdBefore method, and its only argument is a Closure containing the various criteria statements. The Grails builder that parses the namedQueries blocks has method-missing support since there’s obviously no createdBefore method, and the method name and argument are used to build a named query. If we’re going to change where the named queries are defined, we need to continue to call these missing methods.

The approach I suggest is to define static methods in a helper class in src/groovy which return the Closures that define the queries. The method names don’t matter, but for consistency it’s best to use the same method name in the utility class as the named query’s name in the domain class:

package com.burtbeckwith.blog

class NamedQueries {

   static createdBefore() {{ date ->
      le 'dateCreated', date
   }}
}

The syntax is a bit weird since the method returns a Closure, so you end up with double braces.

Now update the domain class so the argument of the missing createdBefore method isn’t an inline Closure, but the one that the NamedQueries.createdBefore() method returns:

import com.burtbeckwith.blog.NamedQueries

class Foo {
   String name
   Date dateCreated
   Integer bar

   static namedQueries = {
      createdBefore NamedQueries.createdBefore()
   }
}

Now this named query can be reused in any domain class. The code that uses the queries doesn’t change at all, and if you later decide to modify the logic, you only need to change it in one place instead of every location that you copy/pasted it.


This post is a continuation of my earler Stuff I Learned Consulting post since I used this on that consulting engagement, but I thought that this topic was big enough that it deserved its own post.

Stuff I Learned Consulting

Wednesday, September 26th, 2012 11:55pm

I don’t do much Grails consulting since I work for the Engineering group, and we have an excellent group of support engineers that usually work directly with clients. I do occasionally teach the 3-day Groovy and Grails course but I’ve only been on two onsite consulting gigs so far, and one was a two-week engagement that ended last week. As is often the case when you teach something or help someone else out, I learned a lot and was reminded of a lot of stuff I’d forgotten about, so I thought it would be good to write some of that down for future reference.


SQL Logging

There are two ways to view SQL output from queries; adding logSql = true in DataSource.groovy and configuring Log4j loggers. The Log4j approach is a lot more flexible since it doesn’t just dump to stdout, and can be routed to a file or other appender and conveniently enabled and disabled. But it turns out it’s easy to toggle logSql SQL console logging. Get a reference to the sessionFactory bean (e.g. using dependency injection with def sessionFactory) and turn it on with

sessionFactory.settings.sqlStatementLogger.logToStdout = true

and off with

sessionFactory.settings.sqlStatementLogger.logToStdout = false

stacktrace.log

The stacktrace.log file was getting very large and they wanted to configure it to use a rolling file appender. Seemed simple enough, but it took a lot longer than I expected. The trick is to create an appender with the name 'stacktrace'; the Grails logic that parses the Log4j DSL looks for an existing appender and uses it, and only configures the default one if there isn’t one already configured. So here’s one that configures a RollingFileAppender with a maximum of 10 files, each a maximum of 10MB in size, and with the standard layout pattern. In addition it includes logic to determine if it’s deployed in Tomcat so it can write to the Tomcat logs folder, or the target folder if you’re using run-app.

If you’re deploying to a different container, adjust the log directory calculation appropriately.

appenders {
   String logDir = grails.util.Environment.warDeployed ?
                       System.getProperty('catalina.home') + '/logs' :
                       'target'
   rollingFile name: 'stacktrace',
               maximumFileSize: 10 * 1024 * 1024,
               file: "$logDir/stacktrace.log",
               layout: pattern(
                   conversionPattern: "'%d [%t] %-5p %c{2} %x - %m%n'"),
               maxBackupIndex: 10
}

Dynamic fooId property

In a many-to-one where you have a Foo foo field (or static belongsTo = [foo: Foo] which triggers adding a ‘foo’ field) you can access its foreign key with the dynamic fooId property. This can be used in a few ways. Since references like this are lazy by default, checking if a nullable reference exists using foo != null involves loading the entire instance from the database. But checking fooId != null involves no database access.

Other queries or updates that really only need the foreign key will be cheaper using fooId. For example, to set a reference in another instance you would typically use code like this:

bar2.foo = bar1.foo
bar2.save()

But you can use the load method

bar2.foo = bar1.fooId ? Foo.load(bar1.fooId) : null
bar2.save()

and avoid loading the Foo instance just to set its foreign key in the second instance and then discard it.

Deleting by id is less expensive too; ordinarily you use get to load an instance and call its delete method, but retrieving the entire instance isn’t needed. You can do this instead:

Foo.load(bar.fooId).delete()

DRY constraints

You can use the importFrom method inside a constraints block in a domain class to avoid repeating constraints. You can import all constraints from another domain class:

static constraints = {
   someProperty nullable: true
   ...
   importFrom SomeOtherDomainClass
}

and optionally use the include and/or exclude properties to use a subset:

static constraints = {
   someProperty nullable: true
   ...
   importFrom SomeOtherDomainClass, exclude: ['foo', 'bar']
}

Flush event listener

They were seeing some strange behavior where collections that weren’t explicitly modified were being changed and saved, causing StaleObjectStateExceptions. It wasn’t clear what was triggering this behavior, so I suggested registering a Hibernate FlushEventListener to log the state of the dirty instances and collections during each flush:

package com.burtbeckwith.blog

import org.hibernate.HibernateException
import org.hibernate.collection.PersistentCollection
import org.hibernate.engine.EntityEntry
import org.hibernate.engine.PersistenceContext
import org.hibernate.event.FlushEvent
import org.hibernate.event.FlushEventListener

class LoggingFlushEventListener implements FlushEventListener {

   void onFlush(FlushEvent event) throws HibernateException {
      PersistenceContext pc = event.session.persistenceContext

      pc.entityEntries.each { instance, EntityEntry value ->
         if (instance.dirty) {
            println "Flushing instance $instance"
         }
      }

      pc.collectionEntries.each { PersistentCollection collection, value ->
         if (collection.dirty) {
            println "Flushing collection '$collection.role' $collection"
         }
      }
   }
}

It’s not sufficient in this case to use the standard hibernateEventListeners map (described in the docs here) since that approach adds your listeners to the end of the list, and this listener needs to be at the beginning. So instead use this code in BootStrap.groovy to register it:

import org.hibernate.event.FlushEventListener
import com.burtbeckwith.blog.LoggingFlushEventListener

class BootStrap {

  def sessionFactory

  def init = { servletContext ->

    def listeners = [new LoggingFlushEventListener()]
    def currentListeners = sessionFactory.eventListeners.flushEventListeners
    if (currentListeners) {
      listeners.addAll(currentListeners as List)
    }
    sessionFactory.eventListeners.flushEventListeners =
            listeners as FlushEventListener[]
  }
}

“Read only” objects and Sessions

The read method was added to Grails a while back, and it works like get except that it marks the instance as read-only in the Hibernate Session. It’s not really read-only, but if it is modified it won’t be a candidate for auto-flushing using dirty detection. But you can explicitly call save() or delete() and the action will succeed.

This can be useful in a lot of ways, and in particular it is more efficient if you won’t be changing the instance since Hibernate will not maintain a copy of the original database data for dirty checking during the flush, so each instance will use about half of the memory that it would otherwise.

One limitation of the read method is that it only works for instances loaded individually by id. But there are other approaches that affect multiple instances. One is to make the entire session read-only:

session.defaultReadOnly = true

Now all loaded instances will default to read-only, for example instances from criteria queries and finders.

A convenient way to access the session is the withSession method on an arbitrary domain class:

SomeDomainClass.withSession { session ->
   session.defaultReadOnly = true
}

It’s rare that an entire session will be read-only though. You can set the results of individual criteria query to be read-only with the setReadOnly method:

def c = Account.createCriteria()
def results = c {
   between("balance", 500, 1000)
   eq("branch", "London")
   maxResults(10)
   setReadOnly true
}

One significant limitation of this technique is that attached collections are not affected by the read-only status of the owning instance (and there doesn’t seem to be a way to configure collection to ignore changes on a per-instance basis).

Read more about this in the Hibernate documentation

Updates for “Delayed SessionFactory Creation in Grails”

Wednesday, September 26th, 2012 1:50pm

Back in the beginning of 2010 I did a post on how to delay creating the SessionFactory based on a discussion on the User mailing list. This has come up again and I thought I’d look and see if things had changed for Grails 2.

The general problem is the same as it was; Grails and Hibernate create database connections during startup to help with configuration, so the information that is auto-discovered has to be explicitly specified. In addition any eager initialization that can wait should wait.

One such configuration item is the lobHandlerDetector bean. This hasn’t changed from before, so the approach involves specifying the bean yourself (and it’s different depending on whether you’re using Oracle or another database). Since it’s the same I won’t include the details here; see the previous post.

Another is the Dialect. Again, this is the same as before – just specify it in DataSource.groovy. This is a good idea in general since there might be particular features you need in a non-default Dialect class, and specifying org.hibernate.dialect.MySQL5InnoDBDialect for MySQL guarantees that you’ll be using transactional InnoDB tables instead of non-transactional MyISAM tables.

The remaining issues have to do with eager initialization. I started down the path of reworking how to lazily initialize the SessionFactory since using a Spring bean post-processor is significantly less involved (and brittle) than the approach I had previously used. But it turns out that the more recent version of Hibernate that we’re now using supports a flag that avoids database access during SessionFactory initialization, hibernate.temp.use_jdbc_metadata_defaults. So add this to the hibernate block in DataSource.groovy:

hibernate {
   ...
   temp.use_jdbc_metadata_defaults = false
}

And the last issue is the DataSource itself. Up to this point all of the changes will avoid getting a connection, but the pool might pre-create connections at startup. The default implementation in Grails is org.apache.commons.dbcp.BasicDataSource and its initial size is 0, so you’re ok if you haven’t configured a different implementation. If you have, be sure to set its initial size to 0 (this isn’t part of the DataSource so the setter method is implementation-specific if it even exists).


If you’re using multiple datasources, you can delay their database connectivity too. There is a lobHandlerDetector bean for each datasource, so for example if you have a second one with the name “ds2”, configure a lobHandlerDetector_ds2 bean in resources.groovy. Likewise for the Dialect; specify it in the dataSource_ds2 block in DataSource.groovy. Set the use_jdbc_metadata_defaults option in the hibernate_ds2 block:

hibernate_ds2 {
   ...
   temp.use_jdbc_metadata_defaults = false
}

And finally, as for the single-datasource case, if you’ve reconfigured secondary datasource beans’ pool implementations, set their initial sizes to 0.

Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 License.