Archive for September, 2012

Keeping Grails namedQueries DRY

Thursday, September 27th, 2012

The Named queries support in Grails is cool feature where you define a partial criteria query and can call it directly, or compose it with additional criteria to create complex queries. For example you might want to query for instances created before a specified date:

class Foo {
   String name
   Date dateCreated
   Integer bar

   static namedQueries = {
      createdBefore { date ->
         le 'dateCreated', date
      }
   }
}

Using this we can use just the named query:

def moreThanTwoDaysOld = Foo.createdBefore(new Date() - 2).list()

or further refine the query:

def moreThanTwoDaysOld = Foo.createdBefore(new Date() - 2) {
   lt 'bar', 20
}

But what if you have common named queries across various domain classes? It turns out it’s not hard to keep things DRY, but it’s a bit tricky.


To see how the solution works, it’s important to understand how the definion of the named queries is implemented. It’s a static Closure in a domain class, which contains methods whose names will end up being the names of the queries. This will be more clear if we add in the optional parentheses that were omitted above:

static namedQueries = {
   createdBefore({ date ->
      le 'dateCreated', date
   })
}

So it should hopefully be clear that this is an invocation of the createdBefore method, and its only argument is a Closure containing the various criteria statements. The Grails builder that parses the namedQueries blocks has method-missing support since there’s obviously no createdBefore method, and the method name and argument are used to build a named query. If we’re going to change where the named queries are defined, we need to continue to call these missing methods.

The approach I suggest is to define static methods in a helper class in src/groovy which return the Closures that define the queries. The method names don’t matter, but for consistency it’s best to use the same method name in the utility class as the named query’s name in the domain class:

package com.burtbeckwith.blog

class NamedQueries {

   static createdBefore() {{ date ->
      le 'dateCreated', date
   }}
}

The syntax is a bit weird since the method returns a Closure, so you end up with double braces.

Now update the domain class so the argument of the missing createdBefore method isn’t an inline Closure, but the one that the NamedQueries.createdBefore() method returns:

import com.burtbeckwith.blog.NamedQueries

class Foo {
   String name
   Date dateCreated
   Integer bar

   static namedQueries = {
      createdBefore NamedQueries.createdBefore()
   }
}

Now this named query can be reused in any domain class. The code that uses the queries doesn’t change at all, and if you later decide to modify the logic, you only need to change it in one place instead of every location that you copy/pasted it.


This post is a continuation of my earler Stuff I Learned Consulting post since I used this on that consulting engagement, but I thought that this topic was big enough that it deserved its own post.

Stuff I Learned Consulting

Wednesday, September 26th, 2012

I don’t do much Grails consulting since I work for the Engineering group, and we have an excellent group of support engineers that usually work directly with clients. I do occasionally teach the 3-day Groovy and Grails course but I’ve only been on two onsite consulting gigs so far, and one was a two-week engagement that ended last week. As is often the case when you teach something or help someone else out, I learned a lot and was reminded of a lot of stuff I’d forgotten about, so I thought it would be good to write some of that down for future reference.


SQL Logging

There are two ways to view SQL output from queries; adding logSql = true in DataSource.groovy and configuring Log4j loggers. The Log4j approach is a lot more flexible since it doesn’t just dump to stdout, and can be routed to a file or other appender and conveniently enabled and disabled. But it turns out it’s easy to toggle logSql SQL console logging. Get a reference to the sessionFactory bean (e.g. using dependency injection with def sessionFactory) and turn it on with

sessionFactory.settings.sqlStatementLogger.logToStdout = true

and off with

sessionFactory.settings.sqlStatementLogger.logToStdout = false

stacktrace.log

The stacktrace.log file was getting very large and they wanted to configure it to use a rolling file appender. Seemed simple enough, but it took a lot longer than I expected. The trick is to create an appender with the name 'stacktrace'; the Grails logic that parses the Log4j DSL looks for an existing appender and uses it, and only configures the default one if there isn’t one already configured. So here’s one that configures a RollingFileAppender with a maximum of 10 files, each a maximum of 10MB in size, and with the standard layout pattern. In addition it includes logic to determine if it’s deployed in Tomcat so it can write to the Tomcat logs folder, or the target folder if you’re using run-app.

If you’re deploying to a different container, adjust the log directory calculation appropriately.

appenders {
   String logDir = grails.util.Environment.warDeployed ?
                       System.getProperty('catalina.home') + '/logs' :
                       'target'
   rollingFile name: 'stacktrace',
               maximumFileSize: 10 * 1024 * 1024,
               file: "$logDir/stacktrace.log",
               layout: pattern(
                   conversionPattern: "'%d [%t] %-5p %c{2} %x - %m%n'"),
               maxBackupIndex: 10
}

Dynamic fooId property

In a many-to-one where you have a Foo foo field (or static belongsTo = [foo: Foo] which triggers adding a ‘foo’ field) you can access its foreign key with the dynamic fooId property. This can be used in a few ways. Since references like this are lazy by default, checking if a nullable reference exists using foo != null involves loading the entire instance from the database. But checking fooId != null involves no database access.

Other queries or updates that really only need the foreign key will be cheaper using fooId. For example, to set a reference in another instance you would typically use code like this:

bar2.foo = bar1.foo
bar2.save()

But you can use the load method

bar2.foo = bar1.fooId ? Foo.load(bar1.fooId) : null
bar2.save()

and avoid loading the Foo instance just to set its foreign key in the second instance and then discard it.

Deleting by id is less expensive too; ordinarily you use get to load an instance and call its delete method, but retrieving the entire instance isn’t needed. You can do this instead:

Foo.load(bar.fooId).delete()

DRY constraints

You can use the importFrom method inside a constraints block in a domain class to avoid repeating constraints. You can import all constraints from another domain class:

static constraints = {
   someProperty nullable: true
   ...
   importFrom SomeOtherDomainClass
}

and optionally use the include and/or exclude properties to use a subset:

static constraints = {
   someProperty nullable: true
   ...
   importFrom SomeOtherDomainClass, exclude: ['foo', 'bar']
}

Flush event listener

They were seeing some strange behavior where collections that weren’t explicitly modified were being changed and saved, causing StaleObjectStateExceptions. It wasn’t clear what was triggering this behavior, so I suggested registering a Hibernate FlushEventListener to log the state of the dirty instances and collections during each flush:

package com.burtbeckwith.blog

import org.hibernate.HibernateException
import org.hibernate.collection.PersistentCollection
import org.hibernate.engine.EntityEntry
import org.hibernate.engine.PersistenceContext
import org.hibernate.event.FlushEvent
import org.hibernate.event.FlushEventListener

class LoggingFlushEventListener implements FlushEventListener {

   void onFlush(FlushEvent event) throws HibernateException {
      PersistenceContext pc = event.session.persistenceContext

      pc.entityEntries.each { instance, EntityEntry value ->
         if (instance.dirty) {
            println "Flushing instance $instance"
         }
      }

      pc.collectionEntries.each { PersistentCollection collection, value ->
         if (collection.dirty) {
            println "Flushing collection '$collection.role' $collection"
         }
      }
   }
}

It’s not sufficient in this case to use the standard hibernateEventListeners map (described in the docs here) since that approach adds your listeners to the end of the list, and this listener needs to be at the beginning. So instead use this code in BootStrap.groovy to register it:

import org.hibernate.event.FlushEventListener
import com.burtbeckwith.blog.LoggingFlushEventListener

class BootStrap {

  def sessionFactory

  def init = { servletContext ->

    def listeners = [new LoggingFlushEventListener()]
    def currentListeners = sessionFactory.eventListeners.flushEventListeners
    if (currentListeners) {
      listeners.addAll(currentListeners as List)
    }
    sessionFactory.eventListeners.flushEventListeners =
            listeners as FlushEventListener[]
  }
}

“Read only” objects and Sessions

The read method was added to Grails a while back, and it works like get except that it marks the instance as read-only in the Hibernate Session. It’s not really read-only, but if it is modified it won’t be a candidate for auto-flushing using dirty detection. But you can explicitly call save() or delete() and the action will succeed.

This can be useful in a lot of ways, and in particular it is more efficient if you won’t be changing the instance since Hibernate will not maintain a copy of the original database data for dirty checking during the flush, so each instance will use about half of the memory that it would otherwise.

One limitation of the read method is that it only works for instances loaded individually by id. But there are other approaches that affect multiple instances. One is to make the entire session read-only:

session.defaultReadOnly = true

Now all loaded instances will default to read-only, for example instances from criteria queries and finders.

A convenient way to access the session is the withSession method on an arbitrary domain class:

SomeDomainClass.withSession { session ->
   session.defaultReadOnly = true
}

It’s rare that an entire session will be read-only though. You can set the results of individual criteria query to be read-only with the setReadOnly method:

def c = Account.createCriteria()
def results = c {
   between("balance", 500, 1000)
   eq("branch", "London")
   maxResults(10)
   setReadOnly true
}

One significant limitation of this technique is that attached collections are not affected by the read-only status of the owning instance (and there doesn’t seem to be a way to configure collection to ignore changes on a per-instance basis).

Read more about this in the Hibernate documentation

Updates for “Delayed SessionFactory Creation in Grails”

Wednesday, September 26th, 2012

Back in the beginning of 2010 I did a post on how to delay creating the SessionFactory based on a discussion on the User mailing list. This has come up again and I thought I’d look and see if things had changed for Grails 2.

The general problem is the same as it was; Grails and Hibernate create database connections during startup to help with configuration, so the information that is auto-discovered has to be explicitly specified. In addition any eager initialization that can wait should wait.

One such configuration item is the lobHandlerDetector bean. This hasn’t changed from before, so the approach involves specifying the bean yourself (and it’s different depending on whether you’re using Oracle or another database). Since it’s the same I won’t include the details here; see the previous post.

Another is the Dialect. Again, this is the same as before – just specify it in DataSource.groovy. This is a good idea in general since there might be particular features you need in a non-default Dialect class, and specifying org.hibernate.dialect.MySQL5InnoDBDialect for MySQL guarantees that you’ll be using transactional InnoDB tables instead of non-transactional MyISAM tables.

The remaining issues have to do with eager initialization. I started down the path of reworking how to lazily initialize the SessionFactory since using a Spring bean post-processor is significantly less involved (and brittle) than the approach I had previously used. But it turns out that the more recent version of Hibernate that we’re now using supports a flag that avoids database access during SessionFactory initialization, hibernate.temp.use_jdbc_metadata_defaults. So add this to the hibernate block in DataSource.groovy:

hibernate {
   ...
   temp.use_jdbc_metadata_defaults = false
}

And the last issue is the DataSource itself. Up to this point all of the changes will avoid getting a connection, but the pool might pre-create connections at startup. The default implementation in Grails is org.apache.commons.dbcp.BasicDataSource and its initial size is 0, so you’re ok if you haven’t configured a different implementation. If you have, be sure to set its initial size to 0 (this isn’t part of the DataSource so the setter method is implementation-specific if it even exists).


If you’re using multiple datasources, you can delay their database connectivity too. There is a lobHandlerDetector bean for each datasource, so for example if you have a second one with the name “ds2”, configure a lobHandlerDetector_ds2 bean in resources.groovy. Likewise for the Dialect; specify it in the dataSource_ds2 block in DataSource.groovy. Set the use_jdbc_metadata_defaults option in the hibernate_ds2 block:

hibernate_ds2 {
   ...
   temp.use_jdbc_metadata_defaults = false
}

And finally, as for the single-datasource case, if you’ve reconfigured secondary datasource beans’ pool implementations, set their initial sizes to 0.

This Week in Grails (2012-38)

Tuesday, September 25th, 2012

We released Grails 2.2 RC1 this week. It has initial support for artifact namespaces, and uses Groovy 2.0. Please start looking now to make sure that your applications and plugins work with Grails 2.2 and Groovy 2.0 so we can identify and fix issues before the final release. Peter Lebrook pointed out that some plugin code is failing to compile in Groovy 2.0 due to it being stricter in certain ways.

The Groovy team released version 2.0.4 to address some issues in 2.0.3.

There has been a lot of interest in the upcoming Grails Hackathon November 9th-11th. It will be coordinated online but some local user groups are getting together to work in-person.

The dates for the 2013 Greach conference in Madrid have been announced – January 25th-26th.

The second pre-release version of Programming Grails is now available; there are four new chapters (for a total of seven). Download the updated version from your O’Reilly account.


If you want to keep up with these “This Week in Grails” posts you can access them directly via their category link or in an RSS reader with the feed for just these posts.


Translations of this post:



Plugins

There were no new plugins released but 12 updated plugins:

  • airbrake version 0.7.1. Notifier plugin for integrating apps with Airbrake
  • ckeditor version 3.6.4.0. Implements the integration layer between Grails and the CKEditor web rich text editor.
  • countries version 0.4. A way to deal with continents and countries in Grails applications
  • federated-grails version 0.3. Uses Shiro and Shibboleth to integrate into federated authentication
  • jasper version 1.6.0. Enables use of JasperReports
  • modernizr version 2.6.2. Provides the Modernizr Javascript library resource files from http://www.modernizr.com/
  • newrelic version 0.2. Adds the NewRelic Real User Monitoring feature to your GSP pages
  • plastic-criteria version 0.2. Mock Grails Criteria for Unit Tests
  • searchable version 0.6.4. Adds rich search functionality to Grails domain models
  • shiro-openid version 0.7. Adds OpenID authentication to the Shiro plugin with a set of installable Shiro domain class and openid4java view templates
  • zk version 2.1.0.M2. Adds ZK Ajax framework (www.zkoss.org) support to Grails applications
  • zkui version 0.5.4. Seamlessly integrates ZK with Grails’ infrastructures; uses the Grails’ infrastructures such as GSP, controllers rather than zk’s zul as in ZKGrails plugin

Interesting Tweets

User groups and Conferences


This Week in Grails (2012-37)

Wednesday, September 19th, 2012

Lots of stuff for this week’s post since I was busy last week doing some onsite consulting and didn’t have time to get that one out.

We’ve released version 2.1.1 of Grails, and the Groovy team released versions versions 2.0.2 and 1.8.8. Not to be outdone, the Gradle team released version 1.2 and are planning some cool stuff for 1.3. And the STS team have released version 3.1.0.M1 of STS and GGTS.

If you have an idea for a talk at the 2012 Groovy & Grails eXchange in London, submit it here.


If you want to keep up with these “This Week in Grails” posts you can access them directly via their category link or in an RSS reader with the feed for just these posts.


Translations of this post:



Plugins

There were 4 new plugins released:

  • airbrake version 0.4. Notifier plugin for integrating apps with Airbrake
  • retina version 1.0.1. Simple tag for adding inline Retina images to your GSP pages
  • cassandra-astyanax version 0.2.0. Exposes the Astyanax Cassandra client as a Grails service and adds dynamic methods
  • trimmer version 0.1. Trims all form submission inputs

and 18 updated plugins:

  • angularjs-resources version 1.0.2. Adds AngularJS resources to an application
  • app-info version 1.0.2. Provides a UI for inspecting and altering various aspects of the application’s configuration
  • artefact-messaging version 0.3. Adds the message function just as in controllers to services or other arbitrarily defined artefacts
  • bean-fields version 1.0. Provides a suite of tags for rendering form fields for domain and command objects
  • cached-resources version 1.1. Provides a “hash and cache” mapper for the resources framework, automatically creating safe unique filenames for your resources and setting them to eternally cache in the browser
  • concordion version 0.1.2. Provides a convenient integration between the Concordion framework —an open source tool for writing automated acceptance tests in Java— and Grails applications
  • cucumber version 0.6.1. Test your Grails apps with Cucumber
  • email-confirmation version 2.0.6. Sends an email to a user with a link to click to confirm site registration
  • font-awesome-resources version 2.0. Integrates the Font Awesome icon set
  • handlebars-resources version 0.3.1. Supports using Handlebars.js templates with the Grails Resources Plugin
  • jetty version 2.0.1. Makes Jetty the development time container
  • localizations version 1.4.4.5. Store i18n strings in a database
  • mongodb-morphia version 0.8.0. Alternative MongoDB GORM based on the Morphia library (former gorm-mongodb)
  • neo4j version 1.0.0.SNAPSHOT. GORM for Neo4j
  • remoting version 1.3. Makes it easy to expose your Grails services to remote clients via RMI, Hessian, Burlap and Spring’s HttpInvoker protocol, and also easily access remote services via the same set of protocols
  • resources version 1.2.RC2. A resource management and processing framework
  • twitter-bootstrap version 2.1.1. Twitter Bootstrap CSS framework resource files
  • zk version 2.0.4. Adds ZK Ajax framework (www.zkoss.org) support to Grails applications

Interesting Tweets

Jobs



User groups and Conferences


This Week in Grails (2012-35)

Wednesday, September 05th, 2012

If you want to keep up with these “This Week in Grails” posts you can access them directly via their category link or in an RSS reader with the feed for just these posts.


Translations of this post:



Plugins

There were 2 new plugins released:

  • debian-packager version 0.1. Creates Debian packages from a Grails application
  • handlebars version 1.0.0. Server side rendering of Handlebars.js templates

and 8 updated plugins:

  • facebook-sdk version 0.3.6. Allows your application to use the Facebook Platform and develop Facebook apps on Facebook.com or on web sites (with Facebook Connect)
  • geb version 0.7.2. Geb is a library for headless web browsing on the JVM, suitable for automation and functional web testing
  • jmx version 0.7.2. Adds JMX support and provides the ability to expose services and other Spring beans as MBeans
  • jquery-validation-ui version 1.4. Client Side Validation without writing JavaScript
  • rabbitmq version 1.0.0.RC2. Integrates with Rabbit MQ messaging
  • spring-security-taobao version 1.0.9. Integrates the Taobao Open API Authentication with the Spring Security Core plugin
  • xwiki-rendering version 1.0-RC1. Convert texts using XWiki Rendering Framework
  • zk version 2.0.2. Adds ZK Ajax framework (www.zkoss.org) support to Grails applications

Interesting Tweets

Jobs



User groups and Conferences


Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 License.