Dynamic GORM Domain Classes

A recent discussion on the Grails Dev mailing list about creating a dynamic form builder involved needing to compile new domain classes at runtime. The consensus seemed to be that it’s not possible and/or not advisable, but I’ve thought a lot about this topic and had done similar work when creating the Dynamic Controller plugin, so I started playing with it.

The solution I came up with isn’t pretty but seems to work. There were a few issues to tackle. One is that Grails does quite a bit of work to convert your relatively simple domain classes into full GORM classes, registered with Hibernate and wired up with validation, convenience MetaClass methods, etc. There’s also the issue of automatically compiling in an id and version field, a default toString() method, and collections corresponding to hasMany declarations. In addition there are four Spring beans created for each domain class. There’s a lot being done under the hood that we tend to take for granted.

But the big hurdle is registering the new entity with Hibernate. It’s expected that this is done at startup and never changed, so the data fields in SessionFactoryImpl are mostly private and in two cases final. So the solution is rather hackish and involves brute force reflection. It just so happens that when using reflection, final fields are only mostly final. So I create a whole new SessionFactoryImpl (it’d be convenient to create a small one with just the new domain class, but then you couldn’t reference other domain classes) and replace the real SessionFactoryImpl‘s data with the data from the new one. I can’t replace the SessionFactoryImpl since other classes will have a reference to the previous one.

The primary class is DynamicDomainService, although this could certainly be in a helper class in src/groovy:

package com.burtbeckwith.dynamicdomain

import org.codehaus.groovy.control.CompilerConfiguration
import org.codehaus.groovy.grails.commons.ApplicationHolder as AH
import org.codehaus.groovy.grails.commons.DomainClassArtefactHandler
import org.codehaus.groovy.grails.commons.GrailsDomainClass
import org.codehaus.groovy.grails.compiler.injection.ClassInjector
import org.codehaus.groovy.grails.compiler.injection.DefaultGrailsDomainClassInjector
import org.codehaus.groovy.grails.compiler.injection.GrailsAwareClassLoader
import org.codehaus.groovy.grails.orm.hibernate.ConfigurableLocalSessionFactoryBean
import org.codehaus.groovy.grails.plugins.DomainClassGrailsPlugin
import org.codehaus.groovy.grails.plugins.orm.hibernate.HibernatePluginSupport
import org.codehaus.groovy.grails.validation.GrailsDomainClassValidator

import org.springframework.beans.factory.config.MethodInvokingFactoryBean
import org.springframework.beans.factory.config.RuntimeBeanReference
import org.springframework.beans.factory.support.AbstractBeanDefinition
import org.springframework.beans.factory.support.GenericBeanDefinition
import org.springframework.util.ReflectionUtils

/**
 * Compiles and registers domain classes at runtime.
 *
 * @author <a href='mailto:burt@burtbeckwith.com'>Burt Beckwith</a>
 */
class DynamicDomainService {

   static transactional = false

   void registerNewDomainClass(String code) {
      def application = AH.application
   
      def clazz = compile(code, application)

      // register it as if it was a class under grails-app/domain
      GrailsDomainClass dc = application.addArtefact(
            DomainClassArtefactHandler.TYPE, clazz)

      def ctx = application.mainContext

      registerBeans ctx, dc

      wireMetaclass ctx, dc

      updateSessionFactory ctx
   }

   private Class compile(String code, application) {
      Class clazz = new DynamicClassLoader().parseClass(code)
      application.classLoader.setClassCacheEntry clazz // TODO hack
      clazz
   }

   // this is typically done in DomainClassGrailsPlugin.doWithSpring
   private void registerBeans(ctx, GrailsDomainClass dc) {

      ctx.registerBeanDefinition dc.fullName,
         new GenericBeanDefinition(
            beanClass: dc.clazz,
            scope: AbstractBeanDefinition.SCOPE_PROTOTYPE,
            autowireMode:AbstractBeanDefinition.AUTOWIRE_BY_NAME)

      GenericBeanDefinition beanDef = new GenericBeanDefinition(
         beanClass: MethodInvokingFactoryBean,
         lazyInit: true)
      setBeanProperty beanDef, 'targetObject',
            new RuntimeBeanReference('grailsApplication', true)
      setBeanProperty beanDef, 'targetMethod', 'getArtefact'
      setBeanProperty beanDef, 'arguments',
            [DomainClassArtefactHandler.TYPE, dc.fullName]
      ctx.registerBeanDefinition "${dc.fullName}DomainClass", beanDef

      beanDef = new GenericBeanDefinition(
         beanClass: MethodInvokingFactoryBean,
         lazyInit: true)
      setBeanProperty beanDef, 'targetObject',
            new RuntimeBeanReference("${dc.fullName}DomainClass")
      setBeanProperty beanDef, 'targetMethod', 'getClazz'
      ctx.registerBeanDefinition "${dc.fullName}PersistentClass", beanDef

      beanDef = new GenericBeanDefinition(
         beanClass: GrailsDomainClassValidator,
         lazyInit: true)
      setBeanProperty beanDef, 'messageSource',
            new RuntimeBeanReference('messageSource')
      setBeanProperty beanDef, 'domainClass',
            new RuntimeBeanReference("${dc.fullName}DomainClass")
      setBeanProperty beanDef, 'grailsApplication',
            new RuntimeBeanReference('grailsApplication', true)
      ctx.registerBeanDefinition "${dc.fullName}Validator", beanDef
   }

   private void setBeanProperty(GenericBeanDefinition bean,
            String name, value) {
      bean.propertyValues.addPropertyValue name, value
   }

   private void wireMetaclass(ctx, GrailsDomainClass dc) {
      def fakeApplication = new FakeApplication(dc)
      DomainClassGrailsPlugin.enhanceDomainClasses(
            fakeApplication, ctx)
      HibernatePluginSupport.enhanceSessionFactory(
            ctx.sessionFactory, fakeApplication, ctx)
   }

   // creates a new session factory so new classes can
   // reference existing, and then replaces the data in
   // the original session factory with the new combined data
   private void updateSessionFactory(ctx) {
      def sessionFactoryBean = ctx.getBean('&sessionFactory')
      def newSessionFactoryFactory = new ConfigurableLocalSessionFactoryBean(
         dataSource: ctx.dataSource,
         configLocations: getFieldValue(
               sessionFactoryBean, 'configLocations'),
         configClass: getFieldValue(sessionFactoryBean, 'configClass'),
         hibernateProperties: getFieldValue(
               sessionFactoryBean, 'hibernateProperties'),
         grailsApplication: ctx.grailsApplication,
         lobHandler: getFieldValue(sessionFactoryBean, 'lobHandler'),
         entityInterceptor: getFieldValue(
               sessionFactoryBean, 'entityInterceptor'))

      newSessionFactoryFactory.afterPropertiesSet()

      def newSessionFactory = newSessionFactoryFactory.object
      
      ['entityPersisters', 'collectionPersisters', 'identifierGenerators',
       'namedQueries', 'namedSqlQueries', 'sqlResultSetMappings',
       'imports', 'collectionRolesByEntityParticipant',
       'classMetadata', 'collectionMetadata'].each { fieldName ->
         def field = ReflectionUtils.findField(
               ctx.sessionFactory.getClass(), fieldName)
         field.accessible = true
         field.set ctx.sessionFactory, new HashMap(
               field.get(newSessionFactory))
      }
   }

   private getFieldValue(sessionFactoryBean, String fieldName) {
      def field = ReflectionUtils.findField(
               sessionFactoryBean.getClass(), fieldName)
      field.accessible = true
      field.get sessionFactoryBean
   }
}

This depends on a helper class that extends DefaultGrailsApplication and is used to ensure that only the new class gets MetaClass methods wired up:

package com.burtbeckwith.dynamicdomain

import org.codehaus.groovy.grails.commons.ApplicationHolder as AH
import org.codehaus.groovy.grails.commons.DefaultGrailsApplication
import org.codehaus.groovy.grails.commons.GrailsDomainClass

/**
 * This is needed when calling DomainClassGrailsPlugin.enhanceDomainClasses()
 * and HibernatePluginSupport.enhanceSessionFactory() so they only act
 * on the new class.
 *
 * @author Burt
 */
class FakeApplication extends DefaultGrailsApplication {

   final domainClasses

   FakeApplication(GrailsDomainClass dc) {
      super([dc.clazz] as Class[], AH.application.classLoader)
      domainClasses = [dc]
   }
}

We also need a custom GrailsDomainClassInjector since the standard mechanism doesn’t recognize the new classes as domain classes:

package com.burtbeckwith.dynamicdomain

import org.codehaus.groovy.ast.ClassNode
import org.codehaus.groovy.classgen.GeneratorContext
import org.codehaus.groovy.control.SourceUnit
import org.codehaus.groovy.grails.compiler.injection.DefaultGrailsDomainClassInjector

/**
 * Works around the fact that the class is dynamically
 * compiled and not from a file.
 *
 * @author <a href='mailto:burt@burtbeckwith.com'>Burt Beckwith</a>
 */
class DynamicDomainClassInjector extends DefaultGrailsDomainClassInjector {

   // always true since we're only compiling dynamic domain classes
   @Override
   boolean shouldInject(URL url) { true }

   // always true since we're only compiling dynamic domain classes
   @Override
   protected boolean isDomainClass(ClassNode cn, SourceUnit su) { true }
}

Finally we need a custom classloader that uses the custom injector:

package com.burtbeckwith.dynamicdomain

import java.security.CodeSource

import org.codehaus.groovy.control.CompilationUnit
import org.codehaus.groovy.control.CompilerConfiguration
import org.codehaus.groovy.control.Phases
import org.codehaus.groovy.grails.commons.ApplicationHolder
import org.codehaus.groovy.grails.compiler.injection.ClassInjector
import org.codehaus.groovy.grails.compiler.injection.GrailsAwareClassLoader
import org.codehaus.groovy.grails.compiler.injection.GrailsAwareInjectionOperation

/**
 * Uses a custom injector.
 *
 * @author <a href='mailto:burt@burtbeckwith.com'>Burt Beckwith</a>
 */
class DynamicClassLoader extends GrailsAwareClassLoader {

   private final ClassInjector[] _classInjectors = [
         new DynamicDomainClassInjector()]

   DynamicClassLoader() {
      super(ApplicationHolder.application.classLoader,
            CompilerConfiguration.DEFAULT)
      classInjectors = _classInjectors
   }

   @Override
   protected CompilationUnit createCompilationUnit(
            CompilerConfiguration config, CodeSource source) {
       CompilationUnit cu = super.createCompilationUnit(config, source)
       cu.addPhaseOperation(new GrailsAwareInjectionOperation(
          getResourceLoader(), _classInjectors), Phases.CANONICALIZATION)
       cu
   }
}

Here’s an example demonstrating usage. First compile the Book class:

String bookCode = """
package com.foo.testapp.book

class Book {
   String title
}
"""

dynamicDomainService.registerNewDomainClass bookCode

and then the Author class:

String authorCode = """
package com.foo.testapp.author

import com.foo.testapp.book.Book

class Author {

   static hasMany = [books: Book]

   String name
}
"""

dynamicDomainService.registerNewDomainClass authorCode

Then you can load the classes dynamically and persist new instances:

def Book = grailsApplication.getClassForName('com.foo.testapp.book.Book')
def Author = grailsApplication.getClassForName('com.foo.testapp.author.Author')

def author = Author.newInstance(name: 'Stephen King')
author.addToBooks(Book.newInstance(title: 'The Shining'))
author.addToBooks(Book.newInstance(title: 'Rose Madder'))
author.save(failOnError: true)

Note that you can’t use new since the classes are compiled dynamically, so use newInstance instead.


The reason I’ve been interested in this idea is that I’d like to make domain class reloading during development more friendly. Currently if you edit a domain class while running in dev mode with run-app, the application restarts. This is frustrating if you only made a change in a helper method that has no impact on persistence though. So I’m hoping to be able to diff the previous persistence metadata with the data after recompiling, and if there’s no difference, just reload the class like a service or controller.

On the other end of the spectrum, if you do make a change affecting persistence but you’re running in create-drop mode, it should be straightforward to rebuild the session factory and reload the class, plus re-export the schema, all without restarting the application. I’m not sure yet what to do with the other cases, e.g. when running in update mode or with no dbCreate set at all. Hopefully this will make it into Grails 1.4 or 2.0.


You can download the referenced code here.

19 Responses to “Dynamic GORM Domain Classes”

  1. Jorge says:

    Pretty amazing exercise… and it exposes some interesting wires of Grails, too.

    I just happen to like your reason (allowing domain-class reloading) much more than supporting the dynamic form builder 😉

  2. Tomas Lin says:

    Very cool stuff. How do you envision this being generalized to support the noSql engines like the Redis plugin. Would it be simpler?

  3. […] información sobre Dynamic GORM Domain Classes (traducido al […]

  4. Hi Burt,

    great research and write-up!

    Dynamic reloading of (domain) classes is something I would like to incorporate into the OSGi plugin sometimes in the future. In an optimal case, domain classes could be split into several bundles and loaded dynamically. Again, would be nice to see something like this in future Grails versions!

    Regards,

    Wolfgang

  5. Roshan Shrestha says:

    “Note that you can use new since the classes are compiled dynamically, so use newInstance instead.”

    I think “you can” should be “you cannot”.

  6. Robert Kasanicky says:

    “So I’m hoping to be able to diff the previous persistence metadata with the data after recompiling, and if there’s no difference, just reload the class like a service or controller.”

    This would be a *huge* improvement for Grails IMHO. Being a DDD fan I like to give the domain classes rich behavior and the application restarts are an enormous productivity killer then.

    It would be awesome to see this fixed, but I couldn’t find anything related on the roadmap in jira – is there a jira issue (or whatever else) I can watch to stay up to date on this?

    • Burt says:

      @Robert no, there’s no JIRA or roadmap item because this is just something that I’ve been thinking about implementing.

  7. Lim Chee Kin says:

    Hi,

    I would like to announce the Grails Dynamic Domain Class Plugin 0.1 Released.

    The Dynamic Domain Class plugin enabled Grails application to create domain class dynamically when application is running. The plugin is created based on the works of Burt Beckwith posted here. Thanks to Burt for his kindness to contribute the initial code base of the plugin. It is hardly imagine the possibility of creating the plugin without his contribution.

    Known Issues:
    * The plugin is not production-ready. It doesn’t even working with Tomcat as it is facing groovy.lang.MissingMethodException: No signature of method: org.apache.catalina.loader.WebappClassLoader.setClassCacheEntry() due to the application.classLoader.setClassCacheEntry clazz hack at line 76 of org.grails.dynamicdomain.DynamicDomainService at http://code.google.com/p/grails-dynamic-domain-class-plugin/source/browse/trunk/src/groovy/org/grails/dynamicdomain/DynamicDomainService.groovy.

    Hi Burt, any idea how to solve the known issue above?

    Find Out More:
    * Project Site and Documentation: http://code.google.com/p/grails-dynamic-domain-class-plugin/
    * Support: http://code.google.com/p/grails-dynamic-domain-class-plugin/issues/list
    * Discussion Forum: http://groups.google.com/group/grails-dynamic-domain-class-plugin

    Wish to hear from you soon!

    Regards,
    Chee Kin

  8. Lim Chee Kin says:

    Hi there,

    I would like to announce the Grails Dynamic Domain Class Plugin 0.2 Released. Please see the announcement news at http://www.grails.org/blog/view/limcheekin/Grails+Dynamic+Domain+Class+Plugin+0.2+Released+-+Create+domain+class+on-the-fly

    Regards,
    Chee Kin

  9. Ryan V says:

    Any chance of making this work with the gorm nosql stuff like MongoDB or Redis? I tried it but it seems to mostly fail (I can do a list, but not a count, and can’t save, etc.)

  10. chuyen nguyen says:

    Thanks, that’s great ,but i have question can you please help me

    After create domain from your code if i want add or remove some fields in domain,that we call update domain , how can I do it?

    How can update domain , and it’s still keep data in db?

  11. Justin says:

    Burt,

    I am working on a project where I need to create domain classes on the fly. I implemented your above code after removing the database stuff. My application has no database. As a matter of fact i just commented out hibernate and added nothing back. the “database” for my app is really a device similar to a corporate router. So you can create interfaces delete interfaces and so forth from the device. you can think of it as the database. Any way my domain classes will be derived from xml. I was completely able to do this for domain classes. The next issue is I need to create a controller for that class. I tried to modify the domain code but after calling addArtifact all controllers even the ones that are actual files cannot be seen. When trying to navigate to any action (list() show() and so forth) on any controller gives me a server error saying that it cant be found. I was hoping you could give me some insight on this. As well as what beans I need to register for the controller. Here is my code which is just your above code modified slightly.

    void registerNewControllerClass(String code) {
    def application = AH.application

    def clazz = compile(code, application)

    // register it as if it was a class under grails-app/controller
    GrailsControllerClass cc = application.addArtefact(
    ControllerArtefactHandler.TYPE, clazz)

    def ctx = application.mainContext

    registerControllerBeans ctx, cc

    // wireMetaclass ctx, dc
    //
    // updateSessionFactory ctx
    }

    hopefully you can read that well enough…as you see I commented out the last two helper methods. I didn’t need those for the domain class either. it throws an exception because I am not using hibernate.

    Thanks,
    Justin

  12. Justin says:

    here is the actual error I get when I try to navigate to an action in any controller ERROR filter.UrlMappingsFilter – Error when matching URL mapping [/(*)/(*)?/(*)?]:null

  13. Rakesh Kumar says:

    Burt,

    Nice post. As it turns out it is really pertinent to our project where we need to generate quite a few domain classes for viewing reports each with a similar looking selection form. The legacy code uses database to store the fields and corresponding types. It will be a sheer waste of time to build each and every report MVC by hand and a slick tool like this will really be a time saver. Given that your post is couple years old has it already made to the grails trunk?

    thanks in advance
    –rakesh

  14. Aziz says:

    hi
    i have install the plugin on grails 2.0.1,but i have some problem in:

    getFieldValue(sessionFactoryBean, String fieldName)

    Class
    java.lang.NullPointerException
    Message
    Cannot set property ‘accessible’ on null object

    coul you please help me

  15. Bilel says:

    hi all,
    very interesting idea it’s an open door to a large number of ideas, it is really amazing.

    but do you think that i can use this plugin in production, and before that is it usable? because i saw in google code that there is no evolution in the project, is it abandoned ??

Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 License.