Archive for the Category 'groovy'

SpringOne/2GX

Saturday, October 24th, 2009

I got back from Spring One/2GX in New Orleans a couple of days ago and wanted to write up some thoughts about what a great time it was. This was my first SpringOne – my first attempt at getting sent by my employer didn’t go so well and I didn’t go to the Groovy/Grails Experience in Reston, VA in 2008. But earlier this year I noticed Guillaume Laforge’s tweet announcing the presentation request page so I submitted 5 Grails proposals, and luckily 4 were selected.

I got into New Orleans around 1:30 on Monday and took a cab to the hotel. When I checked in the clerk smiled and told me I’d been upgraded to a very nice room. She wasn’t kidding, it was huge. As a geek all I need is a bed and a shower (and of course wifi and power for the laptop) but it was cool that the organizers got the speakers such nice rooms.

I went to registration to get my speaker badge and schedule and met Dave Klein, the first of many famous GR8 folks that I met this week. After that I went out wandering and ended up on Bourbon St. Even at 3pm on a Monday it was pretty wild, lots of people and great music. Later on was dinner and then Rod Johnson’s keynote. Of course the highlight for us Grails folks was Graeme’s demo of STS 2.2 with Grails support. After the keynote a bunch of us went to Bourbon St. and had some beers.


My first session of the conference was my Clustering a Grails Application for Scalability and Availability talk. They’d assigned talks to the various rooms based on expected attendance but based on responses from the attendee questionnaire they moved some talks and mine got moved to a larger room, so there was a little confusion there. I’d given an earlier version of this talk at the Boston Grails meetup so it was a good one to do first. It went pretty well and the audience asked lots of great questions. Unfortunately although I discussed two Quartz bugs that need to be fixed, I only fixed one, so the demo crashed with an error that wasn’t very obvious, so I wasn’t able to get that fixed. Of course 5 minutes after the talk I realized what I’d forgotten.

Next up was Hamlet D’Arcy’s Groovy Compiler Metaprogramming and AST Transformations. AST transforms are a powerful addition to Groovy but it’s a complex topic and Hamlet did a great job explaining the concepts. The AST viewer that he wrote is pretty slick. I’ve got a couple of potential uses for ASTs that I’ll be trying out in the next few weeks.

After lunch was my UI Performance – Maximizing Page Load Efficiency talk. This one surprised me. I’m not a UI guy, but I am a performance nut and I’m psyched about what the UI Performance plugin provides to Grails applications. But I was concerned about running under for this talk since this isn’t really an area of expertise for me. However we had a great interactive discussion with lots of great audience questions and this talk was ironically probably the best of the four that I did.

After that was Andy Clement’s Eclipse Groovy Tooling. I’m a hardcore Eclipse user and had been using the v1 Eclipse Groovy plugin. The work that Andy Clement and Andrew Eisenberg have done with the v2 plugin has been fantastic. I’m subscribed to the
eclipse-plugin-user and eclipse-plugin-dev mailing lists so I’m aware of a lot of what’s happened along the way but it was great to hear the stories and challenges from the source.

The last session of the day was Ken Sipe’s Grails and the JVM Memory Management. I’m comfortable with garbage collection and the various heap generations so I didn’t get much out of the first part of Ken’s talk but the demos of VisualVM and BTrace were great. It’s always good to add new tools to the toolkit and these look great.

I got Thai takeout for dinner and skipped Adrian Colyer’s technical keynote to do prep for the next day’s talk but went to the Grails BOF. It was a lively discussion with Graeme and Jeff – you can listen to a recording that Colin Harrington tweeted here.


The first talk Wednesday was Andres Almiray’s Flying with Griffon. I’m a big fan of Griffon, having done a bunch of Swing development. Pretty much everything I do currently is web-based but I’m looking for an excuse to write a desktop app in Griffon and after seeing Andres’ talk I’m even more psyched. The highlight of the talk was finding out that the slick presentation that he’d switched to from his PowerPoint slides was actually a Griffon app: Griffon Talk. btw – I highly recommend Griffon in Action – it’s available in Manning’s early access program and is already packed with great information.

Next was Matt Taylor’s GrailsUI Primer. Being a bad UI developer I need good tools for UI work and GrailsUI is very good. Matt also suggested that it’d be good to integrate GrailsUI with the UI Performance plugin, so that should be interesting.

After lunch was my Advanced GORM – Performance, Customization and Monitoring talk. When I did an earlier version of this talk at the Boston Grails meetup it ran long and I didn’t get to the demo of the graphics and monitoring part, so I rushed this one a bit. But overall the audience seemed to find the content interesting. I even had some people offer to help making the monitoring stuff into a plugin – please email me if you were one of them.

Next was my one Spring talk, Chris Richardson’s Running Java and Grails applications on the Amazon Elastic Compute Cloud. I’ll be working on a project that will deploy to EC2 so this is useful info, but to be honest I was more than a little distracted since they had just announced that STS 2.2 was released so I spent a lot of the talk trying that out.

The final talk of the day was Paul King’s Groovy and Concurrency talk. This was an interesting discussion of the various options for multithreading in Groovy, from Ant’s parallel task to various wrappers of the java.util.concurrent classes like Jetlang and GPars, and the new Fork/Join stuff that’ll be in Java 7.

That night I had dinner with Calen and Chelle from Orange and Bronze. I’ll be working on an outsourcing project with them and they had a booth at the conference, so it was a good opportunity to talk with them in person. The restaurant was at the other end of the French Quarter so we had plenty of time to talk – if the rest of the company is like Calen and Chelle we’ll be in good hands.


I did some prep work for the next day’s Demystifying Spring Security in Grails talk and got to bed reasonably early, but for some reason didn’t set an alarm. Luckily a friend who noticed that I wasn’t available on IM called me at 8am so I at least had 1/2 hour to take a shower and get ready. I didn’t have any caffeine or breakfast and was pretty annoyed at myself for nearly screwing up in a major way, but like the rest of the talks I got plenty of great questions and participation from the audience so except for yet another partially failed demo things went well.

I had another celebrity sighting on the way to the talk: Luke Taylor, who’s currently doing most of the work on Spring Security now that Ben Alex is working on Roo. We talked briefly about the breaking changes in Spring Security 3 that are prompting me to write a new plugin (this will be a great opportunity to start from scratch and also modularize the plugin btw) and he offered to answer questions, so I’ll definitely be taking him up on that offer. Later I ran into Luke and Ben in the hotel coffee shop when I was finally able to get caught up on caffeine and I talked to Ben about Roo – he’s clearly excited about the enthusiasm that people have shown for Roo so far. I doubt I’ll do much with Roo since I’ve spent so much time working with Grails, but I can see how Java developers who like the idea of Convention over Configuration but aren’t ready to make the leap to the dynamic language-based Grails would find Roo attractive.

I was going to go to Ramnivas Laddad’s AOP talk next, but I needed to check out and hadn’t had a chance to pack since I’d overslept, so I did that and spent some time catching up on email. I got a takeout falafel wrap from a funky little Middle Eastern place near the hotel and spent some time working on parallelizing Grails integration tests (inspired by a thread on the Grails user mailing list and Paul King’s talk). It’s mostly finished; once I get it working I’ll post the code and a writeup – it should help speed up test time on multicore machines.

My last talk of the conference was Scott Vlaminck’s AOP in Grails. I had heard about Scott’s Circuit Breaker Plugin but didn’t read enough about it to see how it’d be useful. So I was glad I went to the talk – this plugin is a great idea. We have a Log4j SMTP appender configured on a project and on two occasions recently have generated over 2000 error emails in under an hour, so a circuit breaker would have come in very handy there. Most of the time in Grails we can just use MOP for before/around/after advice, but “real” AOP is necessary when Java is calling Java since there isn’t a hook for MOP.


So it was a fantastic 3 1/2 days. It was cool to meet so many people that I had previously only known online, both the big names in the Groovy/Grails/Griffon communities but also developers, plugin authors, and other users and fans of the GR8 stack.

I’ll be posting the presentations and sample apps for the talks that I did when I get a chance so people can try them out and hopefully take advantage of some of the code.

I’m definitely psyched for next year, and I’m also looking forward to the next GR8 event, the Groovy & Grails Exchange in London in December. After that I’ll be headed to the Philippines in January to work with Orange and Bronze and speak at a Spring and Grails conference that they’re sponsoring, so it’ll be an interesting few months.

Patching a Running Server Using Groovy MOP

Monday, June 15th, 2009

We found a bug recently on the site (a Grails app we're building) and quickly fixed it in SVN, but we're not due for a release until next week. We're in early beta and have only a few users, so it's not that urgent that we fix this bug. But every time the bug is triggered we get several error emails thanks to the Log4j SMTP error appender we have configured, so it's an annoying bug.

The problem is due to the assumption that all purchases use a credit card, but we currently only have "online affiliate" purchases. We have CreditCard and OnlinePaymentSource domain classes to model the payment sources; CreditCard has a type attribute (e.g. Visa, MasterCard, etc.) but OnlinePaymentSource doesn't and the bug is in a GSP which calls getType() on an OnlinePaymentSource. The fix was simply to write a custom tag that renders a description of the card or online payment - the 'type' and last 4 digits of the card, or 'Online' for online purchases.

My colleague Kris suggested half-jokingly that we could use MOP to fix the problem before next week's update:

OnlinePaymentSource.metaClass.getType = { -> 'Online' }

We have a web-based Groovy console that's based on the standard Swing console (and similar to the Grails Console plugin but we don't use the plugin since it's 1.1-only and we're still on 1.0.4 (and we'd already written our own)) that we use for site admin. So we decided to patch the code while running since we have so little traffic and a quick server restart would revert the changes if there's a problem.

Once we made the fix I used the "switch user" feature of the Spring Security plugin to assume the identity of a user that had made a purchase and was able to view the page without causing an error page or annoying emails :)

We probably won't be making changes like this often, but it's nice knowing that it's an option, especially for a more serious bug with a similarly straightforward fix. Of course there are changes that cannot be made using this approach - for example you can't add a new value to an Enum, and MetaClass method overrides are ignored if the method is in a Java class and is called from another Java class. But it's a great tool to have in the toolbox.

Grails “Lite” on Google AppEngine

Monday, April 13th, 2009

This week Google announced Java support for AppEngine. I had signed up when they originally announced Python support, hoping to use it as an excuse to learn Python, but never did anything with it, mostly because I've been spending so much time with Grails. So when they announced Java support I immediately signed up and was approved in about an hour.

Guillaume Laforge blogged about Groovy support - they'd worked with Google to ensure compatibility and released v1.6.1 with the required fixes. Of course with Groovy support, lots of people were asking if Grails would work. Unfortunately it looks like it'll take some work for that. So far it looks like there are problems with threads, XPathFactory, and SiteMesh, and of course GORM will have to be enhanced to work with the AppEngine datastore.

Lots of work is being done to get frameworks working in AppEngine. Shay Banon updated Compass and posted a screencast demonstrating its usage. Thomas Mueller updated H2 database, so in-memory JDBC is possible, and he's planning on writing a JDBC driver wrapping the datastore API which would make porting apps a lot more straightforward. In addition work is being done to make Tapestry, Struts, JSF, and other frameworks compatible.

The SDK supports both JPA and JDO, but JDO is currently much better documented. I have no experience with JDO so I started playing with the "jdoexamples" demo project that ships with the SDK. Like most of the examples it uses Servlets and JSPs, which after using SpringMVC and Grails seemed painfully dated. And the database code is very repetitive with lots of boilerplate. So I thought I'd play with using Groovy to add Grails-like functionality to reduce some of the clutter. This also gave me an excuse to use the new MVC features in Spring 2.5, especially the annotation support.


I reworked the jdoexamples project as a Spring MVC application, along with a bunch of Groovy MetaClass magic. There's no BeanBuilder or resources.groovy/resources.xml; instead the "core" applicationContext.xml includes two other context files, one that's auto-generated during the Ant "war" target containing services, and one that is application-specific in src/java/applicationContext_user.xml.

The domain classes aren't changed much except that they take advantage of being written in Groovy, and static inner classes were reworked as top-level classes since Groovy doesn't support inner classes. The "Utils" classes were reworked as Services. The Servlets were reworked as Controllers, and all of the scriptlets in the JSPs were moved to the controllers or reworked using JSTL tags. I didn't improve the UI (it's pretty sparse) since I was more focused on the code and not the look and feel.

The code expects a directory structure similar to a Grails project: domain classes in grails-app/domain, services in grails-app/services, controllers in grails-app/controllers, Java source in src/java, and Groovy helper classes in src/groovy:

In addition there's the framework classes (in src/framework, there's no separate library for these) and core applicationContext.xml, some application-specific config files in src/java (which will end up in WEB-INF/classes), application jars in lib, web content in war, web config files in war/WEB-INF, JSPs in war/WEB-INF/jsp, and Groovlets in war/WEB-INF/groovy (these aren't auto-discovered, they need to be manually registered in web.xml as Servlets).

During the build, a file is generated (WEB-INF/artifacts.config) containing the names of the domain, controller, and service classes; this way the framework code knows which MOP methods to add when the app starts up.

The added domain class MetaClass methods are:

  • save
  • get
  • delete
  • findAll
  • withTransaction
  • withPersistenceManager (to execute code with access to the PersistenceManager)
  • isJdoPersistent, isJdoDeleted, isJdoDetached, isJdoDirty, isJdoNew, isJdoTransactional, getJdoTransactionalObjectId, getJdoVersion, getJdoObjectId, and jdoMakeDirty corresponding to the JDOHelper methods isPersistent, isDeleted, isDetached, isDirty, isNew, isTransactional, getTransactionalObjectId, getVersion, getObjectId, and makeDirty

Controllers have these attributes added to their MetaClass:

  • params (with Request attributes like in Grails)
  • session
  • request
  • response
  • servletContext

These attributes provide controllers access to the standard servlet classes but they aren't as useful as in Grails since the approach taken is to use the newer MVC features in Spring 2.5 like declaring typed (auto-converted) parameters using annotations in controller methods. But they're available in case they're needed.

All artifacts have a Commons Logging "log" (wrapping a java.util.Logger) wired up, although for some reason this doesn't work in Services. The only thing I can think of is that it's because they're Spring beans, but that shouldn't matter since they're not proxied.

Controllers are singletons. I appreciate the convenience of having stateful per-instance (prototype scope) Controllers in Grails but I tend not to take advantage of that and prefer the more traditional stateless singleton scope approach.

There are no Spring samples in the downloadable SDK demos, but after I'd written most of the demo app code I saw on the mailing list that the autoshoppe demo had been checked into the trunk (along with a few others). I should have expected that Spring would have a JdoTemplate - I'll need to rework some of the code to use that since I reinvented a few wheels.

There's a lot of missing functionality. No taglibs, scripts, converters, or GSPs. Also no SiteMesh, no BeanBuilder, and obviously no plugins :) But it mostly works and definitely makes Spring/AppEngine development easier.

In addition I was thinking that I'd integrate Compass, plus it shouldn't be much work to add Spring Security - it should only need a JDO-based UserDetailsService implementation.


You can download the sample code here and view it in action here.

Note that to use Ant to build the app, you need to copy groovy-all-1.6.1.jar to $ANT_HOME/lib. You also need a build.properties with an "appengine.sdk" property pointing to the root directory of your AppEngine SDK directory (to load the shared Ant targets). And you'll need to change the app name ("bb-grails-lite") in build.xml, web.xml, and appengine-web.xml. You'll also need to change the Controller package name in spring-servlet.xml from com.burtbeckwith.gae.guestbook.controller to whatever your Controller package is.

There are two issues with the online demo. One is that the "friends" demo displays an error page - there are error messages in the console logs complaining about a missing index (this works fine locally). The other is that the "addressbook" demo mostly works, but it doesn't save the state or phoneNumber fields correctly. This means that you can't search by state, only by last name, and it displays the state value instead of the phoneNumber value. As you can see from the screenshot of the admin console, it thinks that there's a metaClass field instead of state, and isn't aware of phoneNumber at all. This seems highly likely to be due to Groovy, but it'll have to be investigated. If the domain classes can't be written in Groovy, there's a lot of lost functionality without being able to use MOP.

Grails Acegi (Spring Security) Plugin v0.5 Released

Monday, January 12th, 2009

Version 0.5 went out yesterday. This was an interesting one to work on. There were a few relatively minor bug fixes, but there are four new features/enhancements and the plugin now works with Grails 1.1.


Probably the coolest new feature is being able to define security rules directly in the Controllers. Previously there were two ways to define the URL->Role mappings, in a static string (the standard Spring Security approach) or using Requestmap entries in the database. Now there's a third - using annotations in controllers.

I'd written about using annotations previously and I used that approach. Annotations can be ugly, but I think here they're great here since they'll only ever have a fairly short list of values (roles plus special tokens like IS_AUTHENTICATED_FULLY, etc.). You can annotate individual actions and/or put an annotation at the class level and then all of the actions in the controller share those mappings, and if needed you can override that for individual actions. And they're inheritable, so if you use controller base classes you can define them there and share rules throughout the hierarchy. For example, if you have an AbstractAdminController that all administrative controllers extend you could annotate just the base class with @Secured(['ROLE_ADMIN']) and restrict access to the entire admin hierarchy.

There are some notes here describing the new approach and the original two.

To better support stuff like administrative sections of a site, I added in an IP address filter so you can specify IP ranges (using Ant patterns or masks) for URLs. This way you can restrict access for your admin area to '10.**' or '192.168.**' in addition to requiring specific roles to ensure that only users in the intranet or VPN have access.

I also added in support for ChannelProcessingFilter to allow defining which URLs require HTTPS and which require HTTP. With this addition there aren't many Spring Security filters left that haven't been mapped in the plugin - just ConcurrentSessionFilter, SessionFixationProtectionFilter, X509PreAuthenticatedProcessingFilter, and RequestHeaderPreAuthenticatedProcessingFilter.

Someone on the Grails user mailing list was talking about Facebook logins, and I'd been wanting an excuse to play with their API so I wired up support for that. It's of limited use (like the OpenID support) since you don't get much information from the authentication, so there would be a lot of manual work required in the app when setting up user information in the database. For example, you don't even get the login name/email, only the numeric ID, so that has to be the username attribute in the User table.

Getting things going with Grails 1.1 was interesting. There was one real bug that I'd introduced in AuthenticatedVetoableDecisionManager - I think it's a Groovy bug but I worked around it. I had named both a variable and a method deny, and it was trying to invoke deny() on the variable instead of the method. I changed it to denyCount which is more self-documenting and it was fine.

The plugin generates its own controllers and CRUD pages, so I needed to update those to use new 1.1 features. The allowedMethods map is now static so I changed those to avoid warnings at startup. Optimistic locking is now checked in controllers by adding an <input type='hidden'> form element in the GSP with the current domain instance version and checking it against the database after submitting updates, so I wired that up so the User/Role/Requestmap management pages work like non-plugin pages.

I also had to rework my automated testing due to the changes in the way plugins are installed in 1.1. I have an Ant task that creates a test project, installs the plugin, and configures the app for testing. Then I run a suite of automated tests using the WebTest plugin. Plugins are now global by default, so installing a plugin for a second app will share the plugin that the first app installed, which will save a bunch of space both locally and in source control. But I need the plugin to be local to the test app, and this is possible by overriding the default behavior and reverting to the 1.0 approach by setting grails.plugins.dir='plugins' in grails-app/conf/BuildConfig.groovy. And this doesn't affect 1.0 since BuildConfig.groovy is ignored.

So check out the new release. There are no backwards compatibility issues, so upgrading is simple. Check out the fixes and enhancements here and the docs here.

Spring Bean Aliases in Grails

Tuesday, January 06th, 2009

I was looking at the source code of a Spring application recently and saw an applicationContext.xml element that I hadn't seen before: <alias>. It turns out this has been around for quite a while, since v1.2 (2005). It sure would have been convenient several months ago.

My first Grails app was a Live Chat app for tech support. I used Comet, so I needed to access Grails beans from Java classes. I did this by having my Services implement an interface, and used that as the type for the Spring bean dependency injection.

But the problem with that was that in Grails you don't get to choose the bean names of artifacts - there's a strict convention, so a Service named ChatService is registered as chatService. But the convention I've always used is to name my interface ChatService and name the implementation class ChatServiceImpl. But then the Spring bean name would be chatServiceImpl.

In a regular Spring app I'd implicitly alias when I added the bean element to the applicationContext.xml file, e.g.

<bean id='chatService' class='com.foo.bar.ChatServiceImpl' />

So instead I chose an option that I really dislike - I named the interface IChatService and the Grails Service ChatService.

But with the <alias> tag, I should be able to do what I want - name the interface ChatService and the Grails Service ChatServiceImpl, and add

<alias name='chatServiceImpl' alias='chatService' />

to resources.xml. The original bean name will still be there, but I can use the more natural bean name for dependency injection.

This doesn't cause an error, but it doesn't work either. I spent some time in a debugger and the alias is registered, but the application context that's available when the app is running is different from the one that the alias was added to.

But it turns out that you can add programatically add aliases at any time, so I added the alias call to BootStrap.groovy:

class BootStrap {

  def grailsApplication

  def init = { servletContext ->
    grailsApplication.mainContext.registerAlias('chatServiceImpl', 'chatService')
  }

  def destroy = {}
}

and now I can use the chatService alias as the bean name.


I noticed though that since aliases are stored in a Map and not directly in the bean registry, that ctx.getBeanDefinitionNames() only returns 'real' bean names and no aliases. But it turns out that it's pretty simple to get all bean names and aliases:

def allNames = [] as Set
ctx.beanDefinitionNames.each { name ->
    allNames <<name
    ctx.getAliases(name).each { alias -> allNames <<alias }
}

A Grails Plugin for Twitter

Monday, January 05th, 2009

I've recently started using Twitter, late to the party as usual :) So when somebody asked a question this weekend on the Grails User mailing list about using the JTwitter API, I took a look at it and being the easily-distracted person that I am, started playing with it.1

This seemed like something that people might want to have available as a plugin, so I created a simple implementation, basically just a Service to wrap the JTwitter API and a TagLib to help with rendering, plus a sample Controller and GSPs.

Check out the plugin's wiki page for installation instructions, usage, etc. If you find this useful and discover bugs or want to suggest new features, send an email to the Grails User mailing list or create an issue in JIRA.

  1. Actually I have the attention span of a gnat - someone else asked this weekend about integrating Facebook logins with a Grails app so I spent a few hours adding that to the Acegi (Spring Security) plugin[back]

An internal error occurred while showing an internal error

Friday, January 02nd, 2009

I get this a lot using the Groovy/Grails Eclipse plugin at work:

An internal error occurred while showing an internal error

This reminds me of that line from Brazil - "My complications had complications."

Yes I've tried Netbeans and IntelliJ - they're nice but I don't want to re-learn an IDE, I've spent a lot of time with Eclipse and other than the Groovy plugin, it does what I need and very well. And yes I've played with memory settings in eclipse.ini.

I have several Grails apps in my workspace, so I try to keep open only the ones that I'm actively using to avoid memory issues. But eventually memory gets screwy, and I get to restart Eclipse again. Yay!

I'm definitely looking forward to SpringSource allocating resources to improving the plugin now that they've aquired G2One - should help tremendously.

An Alternative Approach for Grails Integration Tests

Tuesday, December 30th, 2008

I like the approach that Grails takes for integration (and unit) testing; it's similar to the testing strategy that I've used in non-Grails apps. Using HSQLDB to create an in-memory test database significantly reduces the time required to run database tests, and Hibernate's database independence means you don't need to pollute application code to make it testable.

Thomas Mueller originally wrote Hypersonic SQL but he ended up discontinuing it, and then other developers picked up where he left off, creating HSQLDB. For the past few years Mueller has been developing H2 from scratch and it has benefited from hindsight and lessons learned from Hypersonic and HSQLDB. According to the HSQLDB website there are some cool new features coming in 1.9 and 2.0, but H2 is already significantly more advanced. So I've chosen to use H2 here, but it would be pretty much the same with HSQLDB - just change the configuration properties.


There are many options for database testing; the approach Grails takes is to start a transaction for each test and roll it back afterwards. The approach I'd previously used was to rebuild the database between tests. Since the database is in-memory this is very fast and allows what I think is a more natural coding style. Since each test starts with a guaranteed clean database, there's no need for a single wrapper transaction - I run everything in autocommit mode, similar to the way application code works in the app. If transactional behavior is required, just group multiple statements in a transaction - Grails makes it simple with withTransaction blocks and with transactional Services.

The key to this approach is a base test class that extends GroovyTestCase and which all Domain class test classes (or any test that accesses the database) extend:

import groovy.sql.Sql

import org.apache.log4j.Logger
import org.codehaus.groovy.grails.orm.hibernate.cfg.DefaultGrailsDomainConfiguration
import org.hibernate.cfg.Configuration

import org.hibernate.tool.hbm2ddl.SchemaExport

/**
 * Abstract base class for H2-based integration tests.
 * Rebuilds an in-memory database for each test.
 */

abstract class AbstractIntegrationTestCase extends GroovyTestCase {

  private static Configuration _configuration

  /** Disable Grails transaction around the tests. */
  def transactional = false

  def dataSource
  def grailsApplication
  def messageSource
  def sessionFactory

  /**
   * Replace the base class logger with one that logs using the actual class name.
   */

  protected final Logger log = Logger.getLogger(getClass())

  /**
   * {@inheritDoc}
   * @see junit.framework.TestCase#setUp()
   */

  @Override
  protected void setUp() {
    super.setUp()

    if (!_configuration) {
      // 1-time creation of the configuration

      Properties properties = new Properties()
      properties.setProperty 'hibernate.connection.driver_class', 'org.h2.Driver'
      properties.setProperty 'hibernate.connection.username', 'sa'
      properties.setProperty 'hibernate.connection.password', ''
      properties.setProperty 'hibernate.connection.url', 'jdbc:h2:mem:test'
      properties.setProperty 'hibernate.dialect', 'org.hibernate.dialect.H2Dialect'

      _configuration = new DefaultGrailsDomainConfiguration(
        grailsApplication: grailsApplication,
        properties: properties)
    }

    // rebuild the database before each test.
    new SchemaExport(_configuration).create(false, true)
    loadTestData dataSource

    clearSession()
  }

  protected void loadTestData(dataSource) {
    // override as necessary; by default run GORM create statements,
    // but could also load sql script(s)
  }

  /**
   * Run a JDBC query independent of Hibernate to verify the table count.
   * @param table  the table to query
   * @param where  optional WHERE clause
   * @return  the count
   */

  protected int findTableCount(String table, String where = null)  {

    String sqlSelect = "SELECT COUNT(0) FROM " + table
    if (where) {
      sqlSelect += " WHERE " + where
    }

    Sql.newInstance(dataSource).firstRow(sqlSelect)[0]
  }

  /**
   * Save or create the bean if it validates, and log validation errors otherwise.
   * @param bean  the bean
   * @param flush  if <code>true</code>, flush on save/update
   * @param throwException  if <code>true</code>, throw an exception if
   *    validation fails instead of returning null
   * @return  the saved/updated instance or <code>null</code>
   *    if there was a problem
   */

  protected def saveOrUpdate(bean, flush = false, throwException = true) {
    bean.validate()
    if (bean.hasErrors()) {
      println "problem creating/updating bean: ${bean}"
      def locale = Locale.getDefault()
      bean.errors.each { fieldErrors ->
        fieldErrors.allErrors.each { error ->
          println messageSource.getMessage(error, locale)
        }
      }
      if (throwException) {
        fail 'validation failed in saveOrUpdate'
      }
      return null
    }

    return bean.save(flush: flush)
  }

  protected void clearSession() {
    sessionFactory.currentSession.clear()
  }

  protected def getDomainClass() {
    return grailsApplication.getArtefact('Domain', getDomainClassName())
  }

  protected String getDomainClassName() {
    return getClass().name - 'Tests'
  }
}

It sets transactional = false to keep Grails from starting a transaction around each test. There's a static variable to ensure that the one-time Configuration creation runs just once for all tests, and it runs SchemaExport.create() to run the drop and create DDL statements to create new empty tables for each test. A protected loadTestData() method is used to populate test data. Individual tests might need specific data but often there's a core set of data that all tests use - this is good place to populate that shared data and to override as necessary.

Other utility methods include findTableCount() (with an optional WHERE clause) to check table row counts independent of Hibernate, saveOrUpdate() to save or update Domain class instances and log validation errors to help debug data population code issues, and clearSession() to clear the Hibernate Session's 1st-level cache (to force reloading saved instances).

Here's some sample test code from a test class:

void testWhatever() {

   Thing.withTransaction { status ->
      saveOrUpdate new Thing(...)
      saveOrUpdate new OtherThing(...)
      ...
   }

   clearSession()

   assertEquals ...
   assertTrue ...
}

but transactions aren't required:

void testSomethingElse() {

   saveOrUpdate new Thing(...)
   assertEquals 5, Thing.count()
   assertTrue ...
}

For the most part I find that the test code I write using this isn't much different from what I'd write in a regular Grails app. But it is definitely more flexible, and I find it's more intuitive.

Performance issues with GString logging

Sunday, December 28th, 2008

Everybody knows about wrapping Log4j log.debug() statements in if (log.isDebugEnabled()) {} blocks to make sure that the message being sent to the log method isn't evaluated unnecessarily if the DEBUG level isn't enabled for the logger. This is true in regular Java apps as well as Groovy/Grails apps.

For example, suppose you have large Collection foo and you log it with a logger whose level is set to INFO:

log.debug("foo is " + foo);

This will concatenate "foo is " with foo.toString() but then discard the combined string since the logger isn't at the DEBUG level. If there are a lot of these debug statements in your code then you'll waste a lot of cycles concatenating strings, especially if you're logging in loops. This is trivially fixed with this though:

if (log.isDebugEnabled()) {
   log.debug("foo is " + foo);
}

But if there's no string concatenation and evaluation, then you don't need to wrap the call - the cost of a discarded log() invocation is about the same as the isDebugEnabled() test;

log.debug("just a simple string");

incurs next to zero overhead.


Enter Groovy. What happens if you use a GString, e.g.:

log.debug "foo is ${foo}"

It turns out that this needs to be wrapped also, since although GStrings are lazy evaluated, they must be converted to a String for the debug() invocation, so you're right back where you started. To test this, try out this code in a Groovy/Grails console.

Note that to test whether toString() is called there's a println statement in toString():

import org.apache.log4j.Level
import org.apache.log4j.Logger

class Thing {
  String toString() {
    println "calling thing tostring"
    return "a thing"
  }
}

def thing = new Thing()

def log = Logger.getLogger('foo')
log.level = Level.INFO
log.debug "Debug: Thing: $thing"
log.error "Error: Thing: $thing"

When running this the output to the shell is

[2344] foo Error: Thing: a thing

and in the console is

calling thing tostring

so it didn't log the debug message, but it did log the error message since it's above the INFO level that the logger is set at. But it only printed the message from the toString() method once, meaning it didn't evaluate the GString in the debug message. Hmmm.

Turns out I was wrong in an earlier assumption - the signatures of the various Logger methods take an Object, not a String, e.g.

public void debug(Object message)

and

public void debug(Object message, Throwable t)

So there's nothing to evaluate, and the GString is discarded. Cool - Groovy to the rescue, no performance issues with logging GStrings.

I wondered though what it would take to coerce a GString to a String:

This doesn't; nothing gets printed:

log.debug "Debug: Thing: $thing" + " now"

This also doesn't, since it's concatenating two GStrings:

log.debug "Debug: Thing: $thing at " + "${System.currentTimeMillis()}"

This does trigger toString() since it's no longer concatenating strings:

log.debug "Debug: Thing: $thing at " + System.currentTimeMillis()

and of course

String message = "Debug: Thing: $thing"
log.debug message

triggers toString() since it explicitly creates a String.

So it turns out that you'd pretty much have to work at causing a performance issue with logging GStrings.

Annotations in Grails Controllers

Sunday, December 28th, 2008

Grails has support for enforcing that a controller method is called via GET or POST using the 'allowedMethods' map, e.g.

def allowedMethods = [delete: 'POST', save: 'POST', update: 'POST']

This works well but is somewhat restrictive. Grails decides how to handle a caller making a GET request when POST is required and vice versa - it sends a 405 error.

But if you want some other behavior in this case, or want to check other preconditions (other than security checks, those are much better left to proper security frameworks) then you need to implement the checks yourself, and Grails filters make the most sense.

That leaves the question of how to declare the rules. The "Grails Way"tm would be to define a Map, List, or a DSL block in the controller class that contains all of the rules for the class, but I think that annotations are the way to go here. Annotations have limitations; they can get way out of control. I converted a fairly complicated Spring/Hibernate application that used hbm files to be annotation-based, and ended up with about as much code in the annotations as I replaced in the XML - not much of a reduction.


Recently I was working on creating a site map and some automated security testing so I went through all of the controllers and noted their urls. I tested each one with a browser, issuing a GET request for each. Many failed, assuming that they'd be called via POST with required parameters that were missing, or failed less badly assuming they're be called via Ajax and only sending a partial page or JSON response. So I created some annotations, RequiresPost and RequiresAjax to guard these. I'm not sure yet what the proper handling of using the wrong request method or type should be, but this conveniently puts that handling code in one place (and separates it from the controllers as a cross-cutting concern).

In this case the annotations are pretty simple, and are declared either at the method or class level. And since they're declared where they're used, they're highly self-documenting - it's clear looking at the method which rules apply, or that none apply.

Here are the annotations:

package com.foo.annotation

import java.lang.annotation.Documented;
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

/**
 * Indicates that a service method (or entire controller) requires
 * being called using POST.
 */

@Target({ElementType.FIELD, ElementType.TYPE})
@Retention(RetentionPolicy.RUNTIME)
@Documented
public @interface RequiresPost {

  /**
   * Whether to display an error or be silent (default).
   * @return  <code>true</code> if an error should be shown
   */

  boolean error() default false;
}

package com.foo.annotation

import java.lang.annotation.Documented;
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

/**
 * Indicates that a service method (or entire controller) requires
 * being called by Ajax.
 */

@Target({ElementType.FIELD, ElementType.TYPE})
@Retention(RetentionPolicy.RUNTIME)
@Documented
public @interface RequiresAjax {

  /**
   * Whether to display an error or be silent (default).
   * @return  <code>true</code> if an error should be shown
   */

  boolean error() default false;
}

and a couple of sample controllers using them. First, one that requires that all actions be called via Ajax:

@RequiresAjax
class PopupController {

  def viewFoo = {
    def foo = Foo.get(params.id)
    render foo as JSON
  }

  def viewThing = {
    def thing = Thing.get(params.id)
    [thing: thing]
  }
}

and another that requires Post or Ajax on a per-action basis:

class SignupController {

  def index = {
    ...
  }

  /**
   * Ajax call to monitor upload status.
   */

  @RequiresAjax
  def uploadInfo = {
    def data = ...
    render data as JSON
  }

  /**
   * Ajax post to perform search.
   */

  @RequiresAjax @RequiresPost
  def searchResults = {
    def results = ...
    render results as JSON
  }
}

So how do we enforce the behavior specified by the annotations? With a Grails filter:

import com.foo.annotation.ControllerAnnotationHelper

class SiteFilters {

  def filters = {

    ajaxAndPostRequired(controller: '*', action:'*') {
      before = {
        boolean ok = true

        if (!request.xhr &&
            ControllerAnnotationHelper.requiresAjax(controllerName, actionName)) {
          ok = false
        }

        if (ok) {
          if (!request.post &&
              ControllerAnnotationHelper.requiresPost(controllerName, actionName)) {
            ok = false
          }
        }

        if (!ok) {
          render text: '', template: null, contentType: 'text/plain'
          return false
        }

        return true
      }
    }
  }
}

This checks in the before() closure whether POST is required and used and whether Ajax is required and used. If either condition isn't met, it renders a blank page and returns false, which indicates that no further processing should happen.

This isn't necessarily the final implementation of this check, but changing site-wide behavior only requires changing this single file. Also note that the annotations support an error attribute indicating whether an error should be reported; this is currently not implemented.


One other missing piece is how to detect Ajax. Most of the popular Ajax frameworks have standardized on indicating that a request was made via Ajax by setting the X-Requested-With request header to XMLHttpRequest, so we can add an isXhr() method to the HttpServletRequest metaclass:

if (!HttpServletRequest.metaClass.hasMetaMethod('isXhr')) {
  HttpServletRequest.metaClass.isXhr = { ->
    'XMLHttpRequest' == delegate.getHeader('X-Requested-With')
  }
}

in BootStrap.groovy or in a plugin.


Finally, here's the helper class that scans the controllers looking for annotations. This would be best in a plugin, but we currently have it implemented as a standalone class that's triggered in BootStrap.groovy, calling init() when the application starts up. I'm intentionally using generics in type declarations, even though Groovy ignores them, to document the collection types. I'm getting increasingly frustrated with the Grails/Groovy "everything is an Object" approach that often makes it hard to figure out what parameters to send and what method return values are, but that's another blog post ...

package com.foo.annotation

import org.apache.commons.lang.WordUtils

import org.codehaus.groovy.grails.commons.ApplicationHolder as AH

class ControllerAnnotationHelper {

  private static Map<String, Map<String, List<Class>>> _actionMap = [:]
  private static Map<String, Class> _controllerAnnotationMap = [:]

  /**
   * Find controller annotation information. Called by BootStrap.init().
   */

  static void init() {

    AH.application.controllerClasses.each { controllerClass ->
      def clazz = controllerClass.clazz
      String controllerName = WordUtils.uncapitalize(controllerClass.name)
      mapClassAnnotation clazz, RequiresAjax, controllerName
      mapClassAnnotation clazz, RequiresPost, controllerName

      Map<String, List<Class>> annotatedClosures = findAnnotatedClosures(
          clazz, RequiresAjax, RequiresPost)
      if (annotatedClosures) {
        _actionMap[controllerName] = annotatedClosures
      }
    }
  }

  // for testing
  static void reset() {
    _actionMap.clear()
    _controllerAnnotationMap.clear()
  }

  // for testing
  static Map<String, Map<String, List<Class>>> getActionMap() {
    return _actionMap
  }

  // for testing
  static Map<String, Class> getControllerAnnotationMap() {
    return _controllerAnnotationMap
  }

  private static void mapClassAnnotation(clazz, annotationClass, controllerName) {
    if (clazz.isAnnotationPresent(annotationClass)) {
      def list = _controllerAnnotationMap[controllerName] ?: []
      list <<annotationClass
      _controllerAnnotationMap[controllerName] = list
    }
  }

  /**
   * Check if the specified controller action requires Ajax.
   * @param controllerName  the controller name
   * @param actionName  the action name (closure name)
   */

  static boolean requiresAjax(String controllerName, String actionName) {
    return requiresAnnotation(RequiresAjax, controllerName, actionName)
  }

  /**
   * Check if the specified controller action requires POST.
   * @param controllerName  the controller name
   * @param actionName  the action name (closure name)
   */

  static boolean requiresPost(String controllerName, String actionName) {
    return requiresAnnotation(RequiresPost, controllerName, actionName)
  }

  private static boolean requiresAnnotation(Class annotationClass,
      String controllerName, String actionName) {

    // see if the controller has the annotation
    def annotations = _controllerAnnotationMap[controllerName]
    if (annotations && annotations.contains(annotationClass)) {
      return true
    }

    // otherwise check the action
    Map<String, List<Class>> controllerClosureAnnotations =
            _actionMap[controllerName] ?: [:]
    List<Class> annotationClasses = controllerClosureAnnotations[actionName]
    return annotationClasses && annotationClasses.contains(annotationClass)
  }

  private static Map<String, List<Class>> findAnnotatedClosures(
      Class clazz, Class... annotationClasses) {

    // since action closures are defined as "def foo = ..." they're
    // fields, but they end up private
    def map = [:]
    for (field in clazz.declaredFields) {
      def fieldAnnotations = []
      for (annotationClass in annotationClasses) {
        if (field.isAnnotationPresent(annotationClass)) {
          fieldAnnotations <<annotationClass
        }
      }
      if (fieldAnnotations) {
        map[field.name] = fieldAnnotations
      }
    }

    return map
  }
}

This approach can help centralize request method handling, but more generically also provides some template code for defining and processing other annotations in a Grails app.

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 2.5 License.