Archive for September 26th, 2012

Stuff I Learned Consulting

Wednesday, September 26th, 2012

I don’t do much Grails consulting since I work for the Engineering group, and we have an excellent group of support engineers that usually work directly with clients. I do occasionally teach the 3-day Groovy and Grails course but I’ve only been on two onsite consulting gigs so far, and one was a two-week engagement that ended last week. As is often the case when you teach something or help someone else out, I learned a lot and was reminded of a lot of stuff I’d forgotten about, so I thought it would be good to write some of that down for future reference.


SQL Logging

There are two ways to view SQL output from queries; adding logSql = true in DataSource.groovy and configuring Log4j loggers. The Log4j approach is a lot more flexible since it doesn’t just dump to stdout, and can be routed to a file or other appender and conveniently enabled and disabled. But it turns out it’s easy to toggle logSql SQL console logging. Get a reference to the sessionFactory bean (e.g. using dependency injection with def sessionFactory) and turn it on with

sessionFactory.settings.sqlStatementLogger.logToStdout = true

and off with

sessionFactory.settings.sqlStatementLogger.logToStdout = false

stacktrace.log

The stacktrace.log file was getting very large and they wanted to configure it to use a rolling file appender. Seemed simple enough, but it took a lot longer than I expected. The trick is to create an appender with the name 'stacktrace'; the Grails logic that parses the Log4j DSL looks for an existing appender and uses it, and only configures the default one if there isn’t one already configured. So here’s one that configures a RollingFileAppender with a maximum of 10 files, each a maximum of 10MB in size, and with the standard layout pattern. In addition it includes logic to determine if it’s deployed in Tomcat so it can write to the Tomcat logs folder, or the target folder if you’re using run-app.

If you’re deploying to a different container, adjust the log directory calculation appropriately.

appenders {
   String logDir = grails.util.Environment.warDeployed ?
                       System.getProperty('catalina.home') + '/logs' :
                       'target'
   rollingFile name: 'stacktrace',
               maximumFileSize: 10 * 1024 * 1024,
               file: "$logDir/stacktrace.log",
               layout: pattern(
                   conversionPattern: "'%d [%t] %-5p %c{2} %x - %m%n'"),
               maxBackupIndex: 10
}

Dynamic fooId property

In a many-to-one where you have a Foo foo field (or static belongsTo = [foo: Foo] which triggers adding a ‘foo’ field) you can access its foreign key with the dynamic fooId property. This can be used in a few ways. Since references like this are lazy by default, checking if a nullable reference exists using foo != null involves loading the entire instance from the database. But checking fooId != null involves no database access.

Other queries or updates that really only need the foreign key will be cheaper using fooId. For example, to set a reference in another instance you would typically use code like this:

bar2.foo = bar1.foo
bar2.save()

But you can use the load method

bar2.foo = bar1.fooId ? Foo.load(bar1.fooId) : null
bar2.save()

and avoid loading the Foo instance just to set its foreign key in the second instance and then discard it.

Deleting by id is less expensive too; ordinarily you use get to load an instance and call its delete method, but retrieving the entire instance isn’t needed. You can do this instead:

Foo.load(bar.fooId).delete()

DRY constraints

You can use the importFrom method inside a constraints block in a domain class to avoid repeating constraints. You can import all constraints from another domain class:

static constraints = {
   someProperty nullable: true
   ...
   importFrom SomeOtherDomainClass
}

and optionally use the include and/or exclude properties to use a subset:

static constraints = {
   someProperty nullable: true
   ...
   importFrom SomeOtherDomainClass, exclude: ['foo', 'bar']
}

Flush event listener

They were seeing some strange behavior where collections that weren’t explicitly modified were being changed and saved, causing StaleObjectStateExceptions. It wasn’t clear what was triggering this behavior, so I suggested registering a Hibernate FlushEventListener to log the state of the dirty instances and collections during each flush:

package com.burtbeckwith.blog

import org.hibernate.HibernateException
import org.hibernate.collection.PersistentCollection
import org.hibernate.engine.EntityEntry
import org.hibernate.engine.PersistenceContext
import org.hibernate.event.FlushEvent
import org.hibernate.event.FlushEventListener

class LoggingFlushEventListener implements FlushEventListener {

   void onFlush(FlushEvent event) throws HibernateException {
      PersistenceContext pc = event.session.persistenceContext

      pc.entityEntries.each { instance, EntityEntry value ->
         if (instance.dirty) {
            println "Flushing instance $instance"
         }
      }

      pc.collectionEntries.each { PersistentCollection collection, value ->
         if (collection.dirty) {
            println "Flushing collection '$collection.role' $collection"
         }
      }
   }
}

It’s not sufficient in this case to use the standard hibernateEventListeners map (described in the docs here) since that approach adds your listeners to the end of the list, and this listener needs to be at the beginning. So instead use this code in BootStrap.groovy to register it:

import org.hibernate.event.FlushEventListener
import com.burtbeckwith.blog.LoggingFlushEventListener

class BootStrap {

  def sessionFactory

  def init = { servletContext ->

    def listeners = [new LoggingFlushEventListener()]
    def currentListeners = sessionFactory.eventListeners.flushEventListeners
    if (currentListeners) {
      listeners.addAll(currentListeners as List)
    }
    sessionFactory.eventListeners.flushEventListeners =
            listeners as FlushEventListener[]
  }
}

“Read only” objects and Sessions

The read method was added to Grails a while back, and it works like get except that it marks the instance as read-only in the Hibernate Session. It’s not really read-only, but if it is modified it won’t be a candidate for auto-flushing using dirty detection. But you can explicitly call save() or delete() and the action will succeed.

This can be useful in a lot of ways, and in particular it is more efficient if you won’t be changing the instance since Hibernate will not maintain a copy of the original database data for dirty checking during the flush, so each instance will use about half of the memory that it would otherwise.

One limitation of the read method is that it only works for instances loaded individually by id. But there are other approaches that affect multiple instances. One is to make the entire session read-only:

session.defaultReadOnly = true

Now all loaded instances will default to read-only, for example instances from criteria queries and finders.

A convenient way to access the session is the withSession method on an arbitrary domain class:

SomeDomainClass.withSession { session ->
   session.defaultReadOnly = true
}

It’s rare that an entire session will be read-only though. You can set the results of individual criteria query to be read-only with the setReadOnly method:

def c = Account.createCriteria()
def results = c {
   between("balance", 500, 1000)
   eq("branch", "London")
   maxResults(10)
   setReadOnly true
}

One significant limitation of this technique is that attached collections are not affected by the read-only status of the owning instance (and there doesn’t seem to be a way to configure collection to ignore changes on a per-instance basis).

Read more about this in the Hibernate documentation

Updates for “Delayed SessionFactory Creation in Grails”

Wednesday, September 26th, 2012

Back in the beginning of 2010 I did a post on how to delay creating the SessionFactory based on a discussion on the User mailing list. This has come up again and I thought I’d look and see if things had changed for Grails 2.

The general problem is the same as it was; Grails and Hibernate create database connections during startup to help with configuration, so the information that is auto-discovered has to be explicitly specified. In addition any eager initialization that can wait should wait.

One such configuration item is the lobHandlerDetector bean. This hasn’t changed from before, so the approach involves specifying the bean yourself (and it’s different depending on whether you’re using Oracle or another database). Since it’s the same I won’t include the details here; see the previous post.

Another is the Dialect. Again, this is the same as before – just specify it in DataSource.groovy. This is a good idea in general since there might be particular features you need in a non-default Dialect class, and specifying org.hibernate.dialect.MySQL5InnoDBDialect for MySQL guarantees that you’ll be using transactional InnoDB tables instead of non-transactional MyISAM tables.

The remaining issues have to do with eager initialization. I started down the path of reworking how to lazily initialize the SessionFactory since using a Spring bean post-processor is significantly less involved (and brittle) than the approach I had previously used. But it turns out that the more recent version of Hibernate that we’re now using supports a flag that avoids database access during SessionFactory initialization, hibernate.temp.use_jdbc_metadata_defaults. So add this to the hibernate block in DataSource.groovy:

hibernate {
   ...
   temp.use_jdbc_metadata_defaults = false
}

And the last issue is the DataSource itself. Up to this point all of the changes will avoid getting a connection, but the pool might pre-create connections at startup. The default implementation in Grails is org.apache.commons.dbcp.BasicDataSource and its initial size is 0, so you’re ok if you haven’t configured a different implementation. If you have, be sure to set its initial size to 0 (this isn’t part of the DataSource so the setter method is implementation-specific if it even exists).


If you’re using multiple datasources, you can delay their database connectivity too. There is a lobHandlerDetector bean for each datasource, so for example if you have a second one with the name “ds2”, configure a lobHandlerDetector_ds2 bean in resources.groovy. Likewise for the Dialect; specify it in the dataSource_ds2 block in DataSource.groovy. Set the use_jdbc_metadata_defaults option in the hibernate_ds2 block:

hibernate_ds2 {
   ...
   temp.use_jdbc_metadata_defaults = false
}

And finally, as for the single-datasource case, if you’ve reconfigured secondary datasource beans’ pool implementations, set their initial sizes to 0.

Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 License.