Patching a Running Server Using Groovy MOP

Monday, June 15th, 2009 10:23pm

We found a bug recently on the site (a Grails app we're building) and quickly fixed it in SVN, but we're not due for a release until next week. We're in early beta and have only a few users, so it's not that urgent that we fix this bug. But every time the bug is triggered we get several error emails thanks to the Log4j SMTP error appender we have configured, so it's an annoying bug.

The problem is due to the assumption that all purchases use a credit card, but we currently only have "online affiliate" purchases. We have CreditCard and OnlinePaymentSource domain classes to model the payment sources; CreditCard has a type attribute (e.g. Visa, MasterCard, etc.) but OnlinePaymentSource doesn't and the bug is in a GSP which calls getType() on an OnlinePaymentSource. The fix was simply to write a custom tag that renders a description of the card or online payment - the 'type' and last 4 digits of the card, or 'Online' for online purchases.

My colleague Kris suggested half-jokingly that we could use MOP to fix the problem before next week's update:

OnlinePaymentSource.metaClass.getType = { -> 'Online' }

We have a web-based Groovy console that's based on the standard Swing console (and similar to the Grails Console plugin but we don't use the plugin since it's 1.1-only and we're still on 1.0.4 (and we'd already written our own)) that we use for site admin. So we decided to patch the code while running since we have so little traffic and a quick server restart would revert the changes if there's a problem.

Once we made the fix I used the "switch user" feature of the Spring Security plugin to assume the identity of a user that had made a purchase and was able to view the page without causing an error page or annoying emails :)

We probably won't be making changes like this often, but it's nice knowing that it's an option, especially for a more serious bug with a similarly straightforward fix. Of course there are changes that cannot be made using this approach - for example you can't add a new value to an Enum, and MetaClass method overrides are ignored if the method is in a Java class and is called from another Java class. But it's a great tool to have in the toolbox.

links for 2009-04-15

Wednesday, April 15th, 2009 3:37pm

Grails “Lite” on Google AppEngine

Monday, April 13th, 2009 12:00am

This week Google announced Java support for AppEngine. I had signed up when they originally announced Python support, hoping to use it as an excuse to learn Python, but never did anything with it, mostly because I've been spending so much time with Grails. So when they announced Java support I immediately signed up and was approved in about an hour.

Guillaume Laforge blogged about Groovy support - they'd worked with Google to ensure compatibility and released v1.6.1 with the required fixes. Of course with Groovy support, lots of people were asking if Grails would work. Unfortunately it looks like it'll take some work for that. So far it looks like there are problems with threads, XPathFactory, and SiteMesh, and of course GORM will have to be enhanced to work with the AppEngine datastore.

Lots of work is being done to get frameworks working in AppEngine. Shay Banon updated Compass and posted a screencast demonstrating its usage. Thomas Mueller updated H2 database, so in-memory JDBC is possible, and he's planning on writing a JDBC driver wrapping the datastore API which would make porting apps a lot more straightforward. In addition work is being done to make Tapestry, Struts, JSF, and other frameworks compatible.

The SDK supports both JPA and JDO, but JDO is currently much better documented. I have no experience with JDO so I started playing with the "jdoexamples" demo project that ships with the SDK. Like most of the examples it uses Servlets and JSPs, which after using SpringMVC and Grails seemed painfully dated. And the database code is very repetitive with lots of boilerplate. So I thought I'd play with using Groovy to add Grails-like functionality to reduce some of the clutter. This also gave me an excuse to use the new MVC features in Spring 2.5, especially the annotation support.


I reworked the jdoexamples project as a Spring MVC application, along with a bunch of Groovy MetaClass magic. There's no BeanBuilder or resources.groovy/resources.xml; instead the "core" applicationContext.xml includes two other context files, one that's auto-generated during the Ant "war" target containing services, and one that is application-specific in src/java/applicationContext_user.xml.

The domain classes aren't changed much except that they take advantage of being written in Groovy, and static inner classes were reworked as top-level classes since Groovy doesn't support inner classes. The "Utils" classes were reworked as Services. The Servlets were reworked as Controllers, and all of the scriptlets in the JSPs were moved to the controllers or reworked using JSTL tags. I didn't improve the UI (it's pretty sparse) since I was more focused on the code and not the look and feel.

The code expects a directory structure similar to a Grails project: domain classes in grails-app/domain, services in grails-app/services, controllers in grails-app/controllers, Java source in src/java, and Groovy helper classes in src/groovy:

In addition there's the framework classes (in src/framework, there's no separate library for these) and core applicationContext.xml, some application-specific config files in src/java (which will end up in WEB-INF/classes), application jars in lib, web content in war, web config files in war/WEB-INF, JSPs in war/WEB-INF/jsp, and Groovlets in war/WEB-INF/groovy (these aren't auto-discovered, they need to be manually registered in web.xml as Servlets).

During the build, a file is generated (WEB-INF/artifacts.config) containing the names of the domain, controller, and service classes; this way the framework code knows which MOP methods to add when the app starts up.

The added domain class MetaClass methods are:

  • save
  • get
  • delete
  • findAll
  • withTransaction
  • withPersistenceManager (to execute code with access to the PersistenceManager)
  • isJdoPersistent, isJdoDeleted, isJdoDetached, isJdoDirty, isJdoNew, isJdoTransactional, getJdoTransactionalObjectId, getJdoVersion, getJdoObjectId, and jdoMakeDirty corresponding to the JDOHelper methods isPersistent, isDeleted, isDetached, isDirty, isNew, isTransactional, getTransactionalObjectId, getVersion, getObjectId, and makeDirty

Controllers have these attributes added to their MetaClass:

  • params (with Request attributes like in Grails)
  • session
  • request
  • response
  • servletContext

These attributes provide controllers access to the standard servlet classes but they aren't as useful as in Grails since the approach taken is to use the newer MVC features in Spring 2.5 like declaring typed (auto-converted) parameters using annotations in controller methods. But they're available in case they're needed.

All artifacts have a Commons Logging "log" (wrapping a java.util.Logger) wired up, although for some reason this doesn't work in Services. The only thing I can think of is that it's because they're Spring beans, but that shouldn't matter since they're not proxied.

Controllers are singletons. I appreciate the convenience of having stateful per-instance (prototype scope) Controllers in Grails but I tend not to take advantage of that and prefer the more traditional stateless singleton scope approach.

There are no Spring samples in the downloadable SDK demos, but after I'd written most of the demo app code I saw on the mailing list that the autoshoppe demo had been checked into the trunk (along with a few others). I should have expected that Spring would have a JdoTemplate - I'll need to rework some of the code to use that since I reinvented a few wheels.

There's a lot of missing functionality. No taglibs, scripts, converters, or GSPs. Also no SiteMesh, no BeanBuilder, and obviously no plugins :) But it mostly works and definitely makes Spring/AppEngine development easier.

In addition I was thinking that I'd integrate Compass, plus it shouldn't be much work to add Spring Security - it should only need a JDO-based UserDetailsService implementation.


You can download the sample code here and view it in action here.

Note that to use Ant to build the app, you need to copy groovy-all-1.6.1.jar to $ANT_HOME/lib. You also need a build.properties with an "appengine.sdk" property pointing to the root directory of your AppEngine SDK directory (to load the shared Ant targets). And you'll need to change the app name ("bb-grails-lite") in build.xml, web.xml, and appengine-web.xml. You'll also need to change the Controller package name in spring-servlet.xml from com.burtbeckwith.gae.guestbook.controller to whatever your Controller package is.

There are two issues with the online demo. One is that the "friends" demo displays an error page - there are error messages in the console logs complaining about a missing index (this works fine locally). The other is that the "addressbook" demo mostly works, but it doesn't save the state or phoneNumber fields correctly. This means that you can't search by state, only by last name, and it displays the state value instead of the phoneNumber value. As you can see from the screenshot of the admin console, it thinks that there's a metaClass field instead of state, and isn't aware of phoneNumber at all. This seems highly likely to be due to Groovy, but it'll have to be investigated. If the domain classes can't be written in Groovy, there's a lot of lost functionality without being able to use MOP.

links for 2009-04-08

Wednesday, April 08th, 2009 10:01am

links for 2009-03-25

Wednesday, March 25th, 2009 10:02am

links for 2009-03-24

Tuesday, March 24th, 2009 10:02am

Fixing Grails 1.0 Many-to-Many Mappings in 1.1

Sunday, March 01st, 2009 10:25pm

One of the more frequently asked questions about Grails involves its "backwards" Many-to-Many mapping. Given domain classes Foo and Bar, the join table foo_bar that's generated for them will have a 'foo' column that points to the bar table and a 'bar' column that points to the foo table. In the comments of this bug it's explained that this was intentional. But in 1.1 the Grails team decided to listen to the users and rework the approach.

Unfortunately this is a stealth fix that's listed in the release notes along with many other changes but not in the breaking changes and it is very much a breaking change - unless you make changes your many-to-many relationships will not work after a 1.0 -> 1.1 upgrade. This is of particular interest to me as a developer on the Spring Security (Acegi) plugin since Role < -> User relationships are modeled as a many-to-many.

The plugin uses three domain classes, User, Role, and Requestmap. You can use whatever names you want but I'll assume those names here; note that Requestmap isn't affected by this issue. In Grails 1.0 your database would have two tables, user and role, and a many-to-many join table role_user, with backwards mappings, i.e. people_id refers to role and authorities_id refers to user:

CREATE TABLE role (
   id BIGINT NOT NULL AUTO_INCREMENT,
   version BIGINT NOT NULL,
   authority VARCHAR(255) NOT NULL UNIQUE,
   description VARCHAR(255) NOT NULL,
   PRIMARY KEY (id)
) ENGINE=InnoDB;

CREATE TABLE user (
   id BIGINT NOT NULL AUTO_INCREMENT,
   version BIGINT NOT NULL,
   description VARCHAR(255) NOT NULL,
   email VARCHAR(255) NOT NULL,
   email_show bit NOT NULL,
   enabled bit NOT NULL,
   passwd VARCHAR(255) NOT NULL,
   user_real_name VARCHAR(255) NOT NULL,
   username VARCHAR(255) NOT NULL UNIQUE,
   PRIMARY KEY (id)
) ENGINE=InnoDB;

CREATE TABLE role_user (
   people_id BIGINT NOT NULL,
   authorities_id BIGINT NOT NULL,
   PRIMARY KEY (people_id, authorities_id)
) ENGINE=InnoDB;

ALTER TABLE role_user ADD INDEX FK1407FDF48F01F561 (people_id),
ADD CONSTRAINT FK1407FDF48F01F561 FOREIGN KEY (people_id) REFERENCES role (id);

ALTER TABLE role_user ADD INDEX FK1407FDF4CF6CDEE4 (authorities_id),
ADD CONSTRAINT FK1407FDF4CF6CDEE4 FOREIGN KEY (authorities_id) REFERENCES user (id);

Here the name of the mapping table is the owning end ('Role') followed by the owned end ('User'). Note that the foreign key and index names ('FK1407FDF48F01F561', etc.) will probably be different for your tables.

However in Grails 1.1 the join table looks like this:

CREATE TABLE role_people (
    role_id BIGINT NOT NULL,
    user_id BIGINT NOT NULL,
    PRIMARY KEY (role_id, user_id)
) ENGINE=InnoDB;

ALTER TABLE role_people ADD INDEX FK28B75E7852388A1A (role_id),
ADD CONSTRAINT FK28B75E7852388A1A FOREIGN KEY (role_id) REFERENCES role (id);

ALTER TABLE role_people ADD INDEX FK28B75E78F7634DFA (user_id),
ADD CONSTRAINT FK28B75E78F7634DFA FOREIGN KEY (user_id) REFERENCES user (id);

and the mappings are correct - role_id references role and user_id references user. The name is the owning end ('role') followed by the collection name of the owned end ('people').

This means that your many-to-many relationships are completely broken, and in particular it means that you won't be able to log in to your application if you're using the Spring Security plugin. You'll see errors like this in your logs:

ERROR springsecurity.GrailsDaoImpl  - User [foo] has no GrantedAuthority

So there are two options - migrate your data to the new table, or use the GORM mapping closure to get Grails to work in '1.0 mode'. The first option is cleaner, but if you're accessing your join tables explicitly, e.g. for reporting queries using SQL, then this approach will break those.

For MySQL, the migration command is:

INSERT INTO role_people (role_id, user_id)
SELECT people_id, authorities_id FROM role_user;

and you'll want to create the missing foreign keys and associated indexes:

ALTER TABLE role_people ADD INDEX FK28B75E7852388A1A (role_id),
ADD CONSTRAINT FK28B75E7852388A1A FOREIGN KEY (role_id) REFERENCES role (id);

ALTER TABLE role_people ADD INDEX FK28B75E78F7634DFA (user_id),
ADD CONSTRAINT FK28B75E78F7634DFA FOREIGN KEY (user_id) REFERENCES user (id);

Once you verify that this worked, you should drop role_user and its foreign keys:

ALTER TABLE role_user DROP FOREIGN KEY FK1407FDF48F01F561;
ALTER TABLE role_user DROP FOREIGN KEY FK1407FDF4CF6CDEE4;
DROP TABLE role_user;

Alternatively, to continue to use "1.0 mode", you just need to add a mapping closure to each of your domain classes:

In Role.groovy:

static mapping = {
   people column: 'people_id', joinTable: 'role_user'
}

and in User.groovy:

static mapping = {
   authorities column: 'authorities_id', joinTable: 'role_user'
}

and your app will continue to work - and still be backwards ;)

links for 2009-02-08

Sunday, February 08th, 2009 10:03am

links for 2009-02-05

Thursday, February 05th, 2009 10:01am

links for 2009-01-25

Sunday, January 25th, 2009 10:02am

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 2.5 License.