Adding a “lite” Groovy web console to a Grails war

Thursday, December 07th, 2017 8:23am

Suppose you have a Grails application deployed to a server – how would you go about finding out how the application was configured? If you have the source then you can view Config.groovy, BuildConfig.groovy, etc. (in this case I’m talking about a Grails 2 app but these ideas are generalizable to Grails 3+) but that’s often not enough.

Grails 2 supports external configuration files, which can be in various places and get merged into the final configuration. But just having what you think is the correct source and configuration files isn’t enough since changes could have been made that didn’t make it into source control. And you can’t easily get information from those files in a WAR since they’re compiled into classes.

My preference for digging into a running Grails application is the console plugin, but to use that you would need to add it to BuildConfig.groovy and build and deploy a new WAR, but again that’s not necessarily going to have the same configuration as the previous deployment.

I have a situation like this at work, so I came up with a lightweight way to add a web-based console similar to the console plugin to a WAR. Originally it was a servlet which generated the HTML for a simple form containing a textarea for Groovy code and a submit button to post the code to be run on the server, and the logic (mostly borrowed from the console plugin) to execute the code and return the results to the browser. I compiled it in the same project that the WAR was built from to ensure that it’s compatible with the versions of Groovy, Grails, Spring, etc. and copied the .class file to WEB-INF/classes in the exploded directory in Tomcat’s webapps folder, and manually edited WEB-APP/web.xml to add the required <servlet> and <servlet-mapping> elements, and everything worked great in my small test app.

But when I tried it in the real application I couldn’t access it because of Spring Security. In this particular case I could have worked around that because the app stores Requestmap instances in the database, but I didn’t want to make changes that I might forget to undo, and there’s the chicken-and-egg problem that I don’t necessarily know what the database settings are for this deployment. So instead I converted the servlet to a servlet filter, and made sure to add the filter before the Spring Security filter chain in web.xml and it worked as expected after restarting the server.

I made the changes in the exploded war directory, but it’s also possible to make the changes in the WAR file itself. Since WAR files are ZIP files, you can unzip the WAR, make the changes, and re-zip.

Here’s the source for the filter:

package com.burtbeckwith.hack

import groovy.transform.CompileStatic
import groovy.util.logging.Slf4j
import org.codehaus.groovy.grails.commons.GrailsApplication
import org.springframework.context.ApplicationContext
import org.springframework.web.context.support.WebApplicationContextUtils

import javax.servlet.Filter
import javax.servlet.FilterChain
import javax.servlet.FilterConfig
import javax.servlet.ServletException
import javax.servlet.ServletRequest
import javax.servlet.ServletResponse
import javax.servlet.http.HttpServletRequest
import javax.servlet.http.HttpServletResponse

@CompileStatic
@Slf4j
class HackFilter implements Filter {

   private ApplicationContext applicationContext
   private GrailsApplication grailsApplication

   void init(FilterConfig fc) throws ServletException {
      applicationContext = WebApplicationContextUtils.getRequiredWebApplicationContext(fc.servletContext)
      grailsApplication = applicationContext.getBean(GrailsApplication)
   }

   void doFilter(ServletRequest req, ServletResponse res, FilterChain chain) throws IOException, ServletException {
      HttpServletRequest request = (HttpServletRequest) req
      HttpServletResponse response = (HttpServletResponse) res

      if ('GET' == request.method) {
         doGet request, response
      }
      else {
         // assume POST
         doPost request, response
      }
   }

   void destroy() {}

   private void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
      response.writer.write html(request.contextPath)
   }

   private void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
      long startTime = System.currentTimeMillis()

      String code = request.getParameter('code')

      ByteArrayOutputStream baos = new ByteArrayOutputStream()
      PrintStream out = new PrintStream(baos)
      PrintStream systemOut = System.out

      Throwable e
      String result = ''
      try {
         System.out = out
         result = new GroovyShell(grailsApplication.classLoader, new Binding(
               config: grailsApplication.config,
               ctx: applicationContext,
               grailsApplication: grailsApplication,
               out: out,
               request: request,
               session: request.session)).evaluate(code)
      }
      catch (Throwable t) {
         e = t
      }
      finally {
         System.out = systemOut
      }

      if (e) {
         StringWriter sw = new StringWriter()
         e.printStackTrace new PrintWriter(sw)
         result = sw.toString().replace('\t', '   ').replace(System.getProperty('line.separator'), '<br/>\n')
      }

      response.writer << html(request.contextPath, code, """\
Total time: ${System.currentTimeMillis() - startTime}ms

Stdout:
${baos.toString('UTF8')}

${e ? 'Exception' : 'Result'}:
$result""")
   }

   private String html(String contextPath, String code = '', String results = '') {
      """\
<html>
<head>
<title>Hack</title>
</head>
<body>
   <form action="$contextPath/hack" method="POST">
      <span>Code: (binding vars include <i>config</i>, <i>ctx</i>, <i>grailsApplication</i>, <i>out</i>, <i>request</i>, <i>session</i>)</span><br/>
      <textarea name="code" cols="120" rows="25">$code</textarea><br/>
      <input type="submit" value="Execute" name="execute" /><br/>
      <span>Results:</span><br/>
      <textarea name="results" cols="120" rows="25" disabled="disabled">$results</textarea>
   </form>
</body>
</html>
"""
   }
}

and these are the corresponding <filter> and <filter-mapping> elements for web.xml:

<filter>
   <filter-name>hack</filter-name>
   <filter-class>com.burtbeckwith.hack.HackFilter</filter-class>
</filter>
<filter-mapping>
   <filter-name>hack</filter-name>
   <url-pattern>/hack</url-pattern>
</filter-mapping>

To access the console, navigate to http://server:port/contextPath/hack. As in the console plugin you can run arbitrary Groovy code (including service method calls, working with domain classes, etc.), and there are several objects in the Binding that you can use – config, ctx, grailsApplication, out, request, and session.

To change the uri from /hack to something else, be sure to update both the <url-pattern> tag in web.xml and the action attribute in the generated form in the filter class.

Building GORM Criteria Queries Dynamically

Monday, March 21st, 2016 4:11am

I originally wrote most of the queries in the spring-security-ui plugin using HQL because I find it more intuitive than criteria queries, but HQL only works with Hibernate and relational databases. A pull request updated the queries to use criteria queries to allow the plugin to be used with NoSQL datastores, but one query didn’t fit the programming style that I was using. That wasn’t a big deal, but since a lot of the controller code is basically CRUD code and very similar to the others, I’ve tried to keep the code generic and push shared logic into the base classes.

The original HQL included this

hql.append " AND e.aclObjectIdentity.aclClass.id=:aclClass"

and the converted criteria code was

aclObjectIdentity {
   aclClass {
      eq 'id', params.long('aclClass')
   }
}

with the whole query being similar to this:

def results = lookupClass().createCriteria().list(max: max, offset: offset) {
   // other standard criteria method calls

   if (params.aclClass) {
      aclObjectIdentity {
         aclClass {
            eq 'id', params.long('aclClass')
         }
      }
   }
}

That got me thinking about creating a way to represent that two-level projection and criterion generically.

If we restore the omitted optional parentheses the code becomes

aclObjectIdentity({
   aclClass({
      eq('id', params.long('aclClass'))
   })
})

So it should be more clear that this is a sequence of method calls; calling aclObjectIdentity with a closure argument, then aclClass with a closure argument, and finally eq with a String and a long argument. Splitting out the closures as local variables makes it more clear, first as

def aclClassClosure = {
   eq('id', params.long('aclClass'))
}

aclObjectIdentity({
   aclClass(aclClassClosure)
})

and then

def aclClassClosure = {
   eq 'id', params.long('aclClass')
}

def aclObjectIdentityClosure = {
   aclClass(aclClassClosure)
}

aclObjectIdentity(aclObjectIdentityClosure)

To make this a bit more concrete, lets say we have three domain classes;

Department:

class Department {
   String name
}

Manager:

class Manager {
   String name
   Department department
}

and Employee:

class Employee {
   String name
   Manager manager
}

We create some instances:

Department d = new Department(name: 'department1').save()
Manager m = new Manager(name: 'manager1', department: d).save()
Employee e1 = new Employee(name: 'employee1', manager: m).save()
Employee e2 = new Employee(name: 'employee2', manager: m).save()

and later want to run a query:

Employee.createCriteria().list(max: 10, offset: 0) {
   eq 'name', 'employee1'

   manager {
      department {
         eq 'name', 'department1'
      }
   }
}

My goal is to represent this query with only some helper methods and without any closures (or as few as possible). Splitting that out like above we have

def departmentClosure = {
   eq 'name', 'department1'
}

def managerClosure = {
   department(departmentClosure)
}

def criteriaClosure = {
   eq 'name', 'employee1'

   manager(managerClosure)
}

Employee.createCriteria().list([max: 10, offset: 0], criteriaClosure)

When the query is run, the delegate of criteriaClosure is set to an instance of HibernateCriteriaBuilder when using Hibernate, or an analogous builder for MongoDB or whatever other GORM implementation you’re using. The builder has defined methods for eq, like, between, etc., so when you make those calls in your criteria closure they’re run on the builder.

It turns out that it works the same way if you split the closure into multiple closures and call them with the builder as the delegate for each. So a method like this works:

def runCriteria(Class clazz, List<Closure> criterias, Map paginateParams) {
   clazz.createCriteria().list(paginateParams) {
      for (Closure criteria in criterias) {
         criteria.delegate = delegate
         criteria()
      }
   }
}

and that means that we can split

Employee.createCriteria().list(max: 10, offset: 0) {
   eq 'name', 'employee1'

   manager {
      department {
         eq 'name', 'department1'
      }
   }
}

into

def closure1 = {
   eq 'name', 'employee1'
}

def closure2 = {
   manager {
      department {
         eq 'name', 'department1'
      }
   }
}

and run it as

runCriteria Employee, [closure1, closure2], [max: 10, offset: 0]

But how can we make that projection generic? It’s an inner method call, wrapped in one or more closures that project down to another domain class.

What I ultimately want is to be able to specify a projection with an inner criteria call without closures:

def projection = buildProjection('manager.department',
                                 'eq', ['name', 'department1'])
runCriteria Employee, [closure1, projection], [max: 10, offset: 0]

Here’s the buildProjection method that does this:

Closure buildProjection(String path, String criterionMethod, List args) {

   def invoker = { String projectionName, Closure subcriteria ->
      delegate."$projectionName"(subcriteria)
   }

   def closure = { ->
      delegate."$criterionMethod"(args)
   }

   for (String projectionName in (path.split('\\.').reverse())) {
      closure = invoker.clone().curry(projectionName, closure)
   }

   closure
}

To understand how this works, look again at the innermost closure:

department {
   eq 'name', 'department1'
}

This will be invoked as a method call on the delegate, in effect

delegate.department({
   eq 'name', 'department1'
})

Groovy lets us call methods dynamically using GStrings, so this is the same as

String methodName = 'department'

delegate."$methodName"({
   eq 'name', 'department1'
})

So we can represent the nested closures as an inner closure invoked as the closure argument of its containing closure, and that invoked as the closure argument of its containing closure, and so on until we run out of levels.

And we can build a closure that calls eq 'name', 'department1' (or any criterion method with arguments, this is just a simplified example), as

def closure = { ->
   delegate."$criterionMethod"(args)
}

So to represent the nested closures, start with an ‘invoker’ closure:

def invoker = { String projectionName, Closure subcriteria ->
   delegate."$projectionName"(subcriteria)
}

and successively clone it at each nesting level, and curry it to embed the projection name and its inner closure since the criteria builder doesn’t expect any closure arguments, working from the inside out:

for (String projectionName in (path.split('\\.').reverse())) {
   closure = invoker.clone().curry(projectionName, closure)
}

So, finally we can run the decomposed query as one or more ‘core’ criteria closures with standard criterion method calls, plus zero or more derived projection closures:

def criteria = {
   eq 'name', 'employee1'
}
def projection = buildProjection('manager.department',
                                 'eq', ['name', 'department1'])

runCriteria Employee, [criteria, projection], [max: 10, offset: 0]

I doubt there’s a lot of reuse potential here to be honest, but working through this helped me to better understand how GORM runs criteria queries. I’ll be talking about this and some other GORM topics at Greach next month, so if you find this interesting be sure to check out the recording of that talk.

Hacking Delhi

Tuesday, January 19th, 2016 1:10am

I did a talk at GR8Conf India on Saturday called “Fun With Spring Security” where I presented a few sample applications demonstrating various non-standard techniques for authentication, restricting access to URLs, etc. The test applications are available at GitHub and each has a README.adoc but I wanted to summarize the applications in a combined blog post.

All of the apps use Grails 3 and spring-security-core 3.x but would be easily converted to Grails 2 and version 2.x of the plugin since nothing depends on the newer versions.

Some items that are common to all or most of the apps:

  • when you start the app the index.gsp page has some links to test the various actions
  • debug/trace logging for the plugin and Spring Security is configured but commented out in logback.groovy
  • most of the applications are intentionally stripped-down:
    • no static resources
    • the GSPs are very minimal
    • unused attributes were removed from the grails.plugin.springsecurity block in application.groovy

autorole


The first example is the “autorole” application. It demonstrates how to infer one or more roles from persisted user information without explicitly storing roles in the traditional many-to-many format. In this example there isn’t even a Role or UserRole domain class, only the User class; all of the work is done in autorole.AutoRoleUserDetailsService, the custom UserDetailsService implementation.

A more realistic implementation would probably use a hybrid approach, storing some role information in the database and inferring the rest, with the UserDetailsService merging the “real” and “virtual” roles as needed.

Items of note:

  • test.User is the user domain class generated by the s2-quickstart script with one modification, adding a boolean admin property
  • the Role and UserRole classes generated by the s2-quickstart script were deleted since they’re not used
  • two users are created in BootStrap.groovy; user “admin” (password “password”) has the admin boolean set to true and will be auto-granted ROLE_ADMIN, and user “user” (password “password”) has the default value for admin (false) which will result in a grant of ROLE_USER
  • autorole.AutoRoleUserDetailsService is registered in grails-app/conf/spring/resources.groovy as the userDetailsService bean
  • secured.SecureController has two annotated actions; /secure requires ROLE_USER (or ROLE_ADMIN since hierarchical roles are configured) and /secure/admin requires ROLE_ADMIN

noroles


The next example is the “noroles” application. It shows how to use expressions to guard access when access rules are simple and roles aren’t required. In this example there isn’t even a Role or UserRole domain class, only the User class.

Items of note:

  • test.User is the user domain class generated by the s2-quickstart script with a few modifications:
    • a final authorities = [] property so GormUserDetailsService works correctly, but doesn’t grant any roles since there aren’t any (for demo purposes here, since there is a custom UserDetailsService)
    • a UserType userType property
    • a String businessUnit property
    • a boolean developer property
  • the Role and UserRole classes generated by the s2-quickstart script were deleted since they’re not used
  • four users are created in BootStrap.groovy, all with password “password”:
    • admin1 has UserType admin, businessUnit: ‘group1’
    • admin2 has UserType admin, businessUnit: ‘group2’
    • salesdude has UserType sales, businessUnit: ‘group1’
    • codemonkey has UserType other, businessUnit: ‘it’, developer true
  • secured.SecureController has several annotated actions using expressions to guard access
  • a custom UserDetailsService creates an extended UserDetails instance to cache nonstandard user properties for use in expressions

hacking_newdelhi


This is an update of the demo app from Greach 2015 (“hacking_madrid”) which was an update of the demo app from GGX 2011 (“hacking_london”). I updated it to Grails 3 and spring-security-core 3.0.3.

It adds an “organization” drop-down to the login page in addition to username and password, and a customized Authentication class, servlet filter, and AuthenticationProvider.

Items of note:

  • grails-app/views/login/auth.gsp is the same as auth.gsp from the core plugin, with the addition of a to select the user’s Organization during login
  • hacking.extralogin.OrganizationAuthentication extends UsernamePasswordAuthenticationToken to add a String organizationName property
  • hacking.extralogin.ui.OrganizationFilter extends the core plugin’s GrailsUsernamePasswordAuthenticationFilter and is registered as the authenticationProcessingFilter bean to process form logins; it creates OrganizationAuthentication instances from POST parameters for authentication
  • hacking.extralogin.auth.OrganizationAuthenticationProvider uses the data in OrganizationAuthentication to authenticate
    • It’s based on DaoAuthenticationProvider but directly accesses the database using GORM instead of delegating to a UserDetailsService
  • two new domain classes, Organization and OrgUser are used to persist the user/organization relationship
    • OrgUser is the many-to-many join class which uses two 1-to-many relationships instead of the traditional GORM static hasMany mapped collections
  • BootStrap.groovy creates test data:
    • two Organization instances, “Org1” and “Org2”
    • a user with ROLE_ADMIN (“admin”/”password”) in “Org1”
    • a user with ROLE_USER (“user”/”password”) in “Org2”
    • a user with ROLE_USER and enabled set to false (“disabled”/”password”) in “Org1”
  • rather than copying and pasting the entire bean definitions into resources.groovy to override the bean class for the authenticationProcessingFilter and daoAuthenticationProvider beans, hacking.HackingBeanFactoryPostProcessor is registered in resources.groovy and updates the bean class in the bean definition. This approach retains all of the dependency injections and configuration updates and helps prevent the app from breaking if updated to a newer version of the plugin that has different dependencies and/or config options for the beans
  • NoStackBadCredentialsException is thrown as needed instead of BadCredentialsException; it’s similar to the core plugin’s NoStackUsernameNotFoundException which avoids filling in the stack trace to reduce creation cost
  • secured.SecureController has two annotated actions; /secure requires ROLE_USER (or ROLE_ADMIN since hierarchical roles are configured) and /secure/admin requires ROLE_ADMIN

lockout


This application shows how to use events to lock a user account after a fixed number of failed login attempts.

Items of note:

  • lockout.FailureEventListener is registered to listen for AuthenticationFailureBadCredentialsEvent
  • lockout.SuccessEventListener is registered to listen for AuthenticationSuccessEvent
  • the User domain class has int badCredentialsCount to track failed logins
  • UserService increments badCredentialsCount for failures and resets to 0 on success
  • secured.SecureController has two annotated actions; /secure requires ROLE_USER (or ROLE_ADMIN since hierarchical roles are configured) and /secure/admin requires ROLE_ADMIN

x509


This application showing how to use X.509 browser certificates to authenticate.

Items of note:

  • X.509 is enabled by adding useX509 = true in application.groovy
  • two users (“dianne” and “scott”) are created in BootStrap.groovy, both with password “not_used” since it’s unused with certificate authentication
  • add the dianne.p12 and/or scott.p12 certificate to your browser to authenticate as that person
  • you must use SSL with X.509 authentication; I tested by building a WAR file and deploying it to Tomcat 8, and configuring run-app similarly is left as an exercise for the reader
    • To test, run `grails war` and copy build/libs/x509-0.1.war to the Tomcat webapps folder, renaming the war to ROOT.war so it uses the default context
    • be sure to access the application with SSL URLs, e.g. https://localhost:8443/secure/index
  • configure server.jks as the keystore and truststore; server.xml is an example Tomcat 8 config file that does this, expecting that server.jks is in the conf directory
  • secured.SecureController has two annotated actions; /secure requires ROLE_USER (or ROLE_ADMIN since hierarchical roles are configured) and /secure/admin requires ROLE_ADMIN

x509chained


This last application shows how to use X.509 browser certificates and a second authentication provider to authenticate.

Note that this has been only lightly tested and should be used with caution. I have no idea if there are gaps in the implementation. Test anything based on this approach and/or code extensively before using in a real application. If you find problems with the approach or code let me know so I can update the code.

Items of note:

  • X.509 is enabled by adding useX509 = true in application.groovy
  • two users (“dianne” and “scott”) are created in BootStrap.groovy, both with password “password” since the password is needed for the second form-auth phase
  • add the dianne.p12 and/or scott.p12 certificate to your browser to authenticate as that person
  • you must use SSL with X.509 authentication; I tested by building a WAR file and deploying it to Tomcat 8, and configuring run-app similarly is left as an exercise for the reader
    • To test, run `grails war` and copy build/libs/x509chained-0.1.war to the Tomcat webapps folder, renaming the war to ROOT.war so it uses the default context
    • be sure to access the application with SSL URLs, e.g. https://localhost:8443/secure/index
  • configure server.jks as the keystore and truststore; server.xml is an example Tomcat 8 config file that does this, expecting that server.jks is in the conf directory
  • x509chained.LoginController extends the plugin’s LoginController to not redirect to successHandler.defaultTargetUrl if authenticated. This is needed because the chained authentication happens in two requests with a redirect. If the first phase (X.509) succeeds, there will be an active authentication, but it’s incomplete and cannot be used yet. Filter chain processing must be allowed to happen to allow the second authentication phase to run.
  • x509chained.ChainedX509AuthenticationFilter extends the standard X509AuthenticationFilter to replace the Authentication in successfulAuthentication with one with all of the real roles replaced with ROLE_CHAINED_X509 as a marker to indicate that the first authentication phase succeeded. The second authentication phase will create a standard Authentication with the real roles.
  • x509chained.ChainedAuthenticationProcessingFilter extends the plugin’s form authentication filter (GrailsUsernamePasswordAuthenticationFilter). It detects that the X.509 phase has occurred and redirects to the login page, replacing the credentials (since they’re unused by X.509) with a marker string so downstream processing is aware of the current state in the workflow.
  • secured.SecureController has two annotated actions; /secure requires ROLE_USER (or ROLE_ADMIN since hierarchical roles are configured) and /secure/admin requires ROLE_ADMIN

Some thoughts on the Grails and Groovy communities

Sunday, September 20th, 2015 10:41pm

If you’re a Grails user then you’re probably aware of some recent events online (Twitter, the Grails Slack channels, etc.) that are unfortunately still ongoing as I write this, and the whole situation is quite a mess. I’m not going to link to any of it or discuss it directly, but it got me thinking again about the Grails and Groovy communities, and the other “GR8” communities in the Groovy ecosystem (Ratpack, Gradle, Griffon, Geb, GPars, etc.)

When I start working in a new area of tech I usually sign up for a mailing list or forum, or whatever they tend to use for discussions about usage, upcoming releases, future plans, etc. I rarely get involved though. When I’m new at the tech I usually find that questions I would have asked have been asked and answered, and once I get good enough at it I tend to rely less on those resources. So I tend to lurk in general and don’t actively participate.

My tendency toward being a lurker is also affected by not wanting to embarrass myself online. When I was a graduate student I discovered the TeX typesetting software and thought that it would be great to use for papers, so I started collecting software and resources. Windows 95 had recently been released1 and Windows options weren’t great then. Someone asked a question on one of the local Usenet groups and being an expert I replied. Except that I was far from an expert and gave rather bad advice, and a few others who actually knew what they were talking about jumped in and provided useful information. I tend to have to touch the pretty glowing orange thing on the stove to learn that it’s very hot and a painful thing to do, so this was a good experience for me. It continues to be a reminder to try to be humble and to help where possible, but be willing to say “I don’t know” when that’s the case, and stay out of the way if I don’t have anything to contribute.

When I discovered Grails I signed up for the user and dev mailing lists. There was a lot to learn, and many of the emails didn’t make sense but I saved a bunch that seemed like they’d be useful later. I quickly learned a lot thanks to the helpful developers and users on the lists. Eventually I got good enough that there were questions that I could answer, but often others would answer before I did. Occasionally a more obscure question would go unanswered, and those would nag at me from my inbox. Sometimes when I was stuck on something for work I would take a break and look at some of those questions and try to see what’s going on, either because it did or might affect what I was working on, or just because it was interesting. If I didn’t get anywhere, then I would let it go, but if I felt like I figured it out or that I at least could contribute useful insights, I would reply with what I found.

Getting a reply on the list from a frustrated user saying that I’ve helped them get past an annoying issue is a pretty cool thing, and a big motivator to continue. Much like teaching, helping others with tech support issues is usually beneficial for both sides since it forces you to understand what’s going on under the hood much more than when you just use the feature. In retrospect, although I was reluctant to start giving back, it didn’t make sense not to. The lists were active with lots of generally respectful discussions, and although people have come and gone, there remains an overall tone and it’s profoundly positive.

When I started going to Grails conferences, the subject of how helpful the communities are was (and still is) a common topic in hallway and mealtime discussions. This was weird to me because I hadn’t really noticed, mostly since I hadn’t really seen communities that weren’t. Of course I’ve become aware of some of what they were referring to 🙂

So the big question is: why are these communities so helpful and friendly? I can’t know for sure, but it obviously has a lot to do with the project leads and big names in the various techs. From the beginning, their personalities and temperaments set the tone that continues today. Why we were lucky to have these people driving the technologies in the first place is a topic for a future blog post, or at least something to discuss over beers at future conferences and meetups.

I’ve learned so much more than I could have expected by participating in all of this, and I’ve been lucky to meet a large number of smart, funny, inquisitive, amazing people. Thank you all, and keep up the great work. It’s easy to get distracted by the frustrating bits and the negative aspects, but weeks like this one are, if we’re able to see past them, a good reminder of how much we have and tend to take for granted.

As a side note, I cringed a bit when the Greach organizers announced that they had put up a code of conduct for this year’s conference. After some high-profile incidents at Python- and Ruby-related conferences (and elsewhere) many conferences did the same, but it didn’t make sense to me to do this at a Groovy-related conference since it’s not needed. It didn’t take long to realize that this was naive, and while we are a friendly bunch, there have likely been incidents where attendees were harassed or made uncomfortable and I wasn’t aware of them. And even if not, there’s nothing wrong with putting core beliefs and expectations in writing. My version would be a lot shorter though: “Don’t be a jerk”.

I rarely say this but it’s pretty much always the case – these are my thoughts and opinions and I don’t claim to speak for anyone or any company or organization I’ve worked with in the past, present or future.

Also, all comments on this blog are moderated and I will not accept any that aren’t constructive, and I reserve the right to disable comments entirely on this post if necessary.

  1. I pre-ordered my copy and it was delivered by FedEX the day it was released – very exciting![back]

One more approach for diagnosing spring-security-core login errors

Thursday, January 30th, 2014 10:16pm

I wrote up a few techniques to help determine what is happening when you are unable to authenticate with use the Grails spring-security-core plugin here and since then I’ve had to use them myself a couple of times. I have everything configured to easily debug the issues and often that ends up being the best approach, but not everyone is comfortable with a debugger, or not excited about the prospect of configuring their IDE with JAR sources and the other steps needed to properly debug. This and this Stack Overflow question prompted me to document another approach.

This one is a rather dangerous and should be removed as soon as the issue is resolved because it prints cleartext passwords to the console, which is a huge security risk.

There are a handful of places where the cleartext password is available, but I think the most convenient is in the additionalAuthenticationChecks method of DaoAuthenticationProvider. The arguments to the method are the UserDetails instance, which will have the hashed password from the database, and the UsernamePasswordAuthenticationToken which will have the cleartext password.

To access this, create a subclass of DaoAuthenticationProvider:

package deletethisasap

import org.springframework.security.authentication.UsernamePasswordAuthenticationToken as UPAT
import org.springframework.security.authentication.dao.DaoAuthenticationProvider
import org.springframework.security.core.Authentication
import org.springframework.security.core.AuthenticationException
import org.springframework.security.core.userdetails.UserDetails

class MyDaoAuthenticationProvider extends DaoAuthenticationProvider {

  protected void additionalAuthenticationChecks(UserDetails ud, UPAT auth) throws AuthenticationException {
    String hashedPassword = passwordEncoder.encodePassword(
         auth.credentials, null)
    boolean ok = passwordEncoder.isPasswordValid(
         ud.password, auth.credentials, null)
    println "Cleartext: '$auth.credentials' from DB: '$ud.password' hashed '$hashedPassword' Valid: $ok"
    super.additionalAuthenticationChecks ud, auth
  }
}

and register it with the same bean name in grails-app/conf/spring/resources.groovy:

import deletethisasap.MyDaoAuthenticationProvider

beans = {
  daoAuthenticationProvider(MyDaoAuthenticationProvider) {
    userDetailsService = ref('userDetailsService')
    passwordEncoder = ref('passwordEncoder')
    userCache = ref('userCache')
    saltSource = ref('saltSource')
    preAuthenticationChecks = ref('preAuthenticationChecks')
    postAuthenticationChecks = ref('postAuthenticationChecks')
    authoritiesMapper = ref('authoritiesMapper')
    hideUserNotFoundExceptions = true
  }
}

It prints the hashed password and the cleartext password, and the cleartext password hashed for comparison. This won’t necessarily be the same as the value from the database, for example when using bcrypt, but the encoder has an isPasswordValid method to verify that they’re equivalent, so that is also printed.

This code assumes that you’re using standard form-based authentication (i.e. DaoAuthenticationProvider) and that you’re not using a salt. If you are using salted passwords, dependency-inject the saltSource bean and use it in the encodePassword and isPasswordValid methods.

Automatically converting password hashes in Grails spring-security-core

Friday, December 20th, 2013 5:13pm

I was looking at this Stack Overflow question about converting password hashes and realized that it’s possible and rather convenient when using the spring-security-core plugin to automate the process.

To start, we’ll need a PasswordEncoder that can work with both algorithms. Here I’m assuming that you’ll be converting from SHA-256 (optionally with a salt) to bcrypt, but the general approach is mostly independent of the algorithms. Sha256ToBCryptPasswordEncoder will always hash new passwords using bcrypt, but can detect the difference between hashes from SHA-256 and bcrypt in isPasswordValid:

package com.burtbeckwith.grails.security;

import grails.plugin.springsecurity.authentication.encoding.BCryptPasswordEncoder;

import org.springframework.security.authentication.encoding.MessageDigestPasswordEncoder;

public class Sha256ToBCryptPasswordEncoder
       implements org.springframework.security.authentication.encoding.PasswordEncoder {

   protected MessageDigestPasswordEncoder sha256PasswordEncoder;
   protected BCryptPasswordEncoder bcryptPasswordEncoder;

   public String encodePassword(String rawPass, Object salt) {
      return bcryptPasswordEncoder.encodePassword(rawPass, null);
   }

   public boolean isPasswordValid(String encPass,
            String rawPass, Object salt) {
      if (encPass.startsWith("$2a$10$") && encPass.length() == 60) {
         // already bcrypt
         return bcryptPasswordEncoder.isPasswordValid(
                    encPass, rawPass, null);
      }

      if (encPass.length() == 64) {
         return sha256PasswordEncoder.isPasswordValid(
                    encPass, rawPass, salt);
      }

      // TODO
      return false;
   }

   /**
    * Dependency injection for the bcrypt password encoder
    * @param encoder the encoder
    */
   public void setBcryptPasswordEncoder(BCryptPasswordEncoder encoder) {
      bcryptPasswordEncoder = encoder;
   }

   /**
    * Dependency injection for the SHA-256 password encoder
    * @param encoder the encoder
    */
   public void setSha256PasswordEncoder(
           MessageDigestPasswordEncoder encoder) {
      sha256PasswordEncoder = encoder;
   }
}

This needs dependency injections for properly configured SHA-256 and bcrypt encoders, and we’ll see that in a bit.

Sha256ToBCryptPasswordEncoder cannot make any changes as only password information is available, so we’ll subclass DaoAuthenticationProvider and do this work in additionalAuthenticationChecks:

package com.burtbeckwith.grails.security

import grails.plugin.springsecurity.SpringSecurityUtils
import grails.plugin.springsecurity.userdetails.GrailsUser

import org.springframework.security.authentication.UsernamePasswordAuthenticationToken
import org.springframework.security.authentication.dao.DaoAuthenticationProvider
import org.springframework.security.core.AuthenticationException
import org.springframework.security.core.userdetails.UserDetails

class PasswordFixingDaoAuthenticationProvider
extends DaoAuthenticationProvider {

   def grailsApplication

   protected void additionalAuthenticationChecks(
         UserDetails userDetails,
         UsernamePasswordAuthenticationToken authentication)
            throws AuthenticationException {
      super.additionalAuthenticationChecks userDetails, authentication

      // if we got this far the password was ok

      String oldHashedPassword = userDetails.getPassword()
      if (oldHashedPassword.startsWith('$2a$10$') &&
            oldHashedPassword.length() == 60) {
         // already bcrypt
         return
      }

      if (oldHashedPassword.length() != 64) {
         // TODO
         return
      }

      String bcryptPassword = getPasswordEncoder().encodePassword(
               authentication.credentials, null) 

      // use HQL to update the password in the database directly

      def conf = SpringSecurityUtils.securityConfig
      String userClassName = conf.userLookup.userDomainClassName
      Class<?> User = grailsApplication.getDomainClass(userClassName).clazz

      def args = [p: bcryptPassword]
      String hql = 'update ' + User.name + ' u set u.password=:p where '
      if (userDetails instanceof GrailsUser) {
         hql += 'u.id=:id'
         args.id = userDetails.id
      }
      else {
         hql += 'u.' + conf.userLookup.usernamePropertyName + '=:un'
         args.un = userDetails.username
      }

      User.withNewSession {
         User.withTransaction {
            User.executeUpdate hql, args
         }
      }
   }
}

Calling super.additionalAuthenticationChecks() will ensure that a password was provided and it will be verified using either SHA-256 or bcrypt by Sha256ToBCryptPasswordEncoder, so if there is no exception thrown it’s safe to update the password. Note that the update code is generic and can be made more compact by hard-coding your class and property names.

We register Sha256ToBCryptPasswordEncoder as the passwordEncoder bean, and create bcryptPasswordEncoder and sha256PasswordEncoder beans, configured with the SHA-256 settings that were being used, and the bcrypt settings that will be used (configure those in Config.groovy as described in the docs). Also configure the bean override of daoAuthenticationProvider to be a PasswordFixingDaoAuthenticationProvider with the same configuration as is done in SpringSecurityCoreGrailsPlugin.groovy with the addition of the grailsApplication reference:

import grails.plugin.springsecurity.SpringSecurityUtils
import grails.plugin.springsecurity.authentication.encoding.BCryptPasswordEncoder

import org.springframework.security.authentication.encoding.MessageDigestPasswordEncoder

import com.burtbeckwith.grails.security.PasswordFixingDaoAuthenticationProvider
import com.burtbeckwith.grails.security.Sha256ToBCryptPasswordEncoder

beans = {

   def conf = SpringSecurityUtils.securityConfig

   bcryptPasswordEncoder(BCryptPasswordEncoder, conf.password.bcrypt.logrounds) // 10

   sha256PasswordEncoder(MessageDigestPasswordEncoder, conf.password.algorithm) {
      encodeHashAsBase64 = conf.password.encodeHashAsBase64 // false
      iterations = conf.password.hash.iterations // 10000
   }

   passwordEncoder(Sha256ToBCryptPasswordEncoder) {
      bcryptPasswordEncoder = ref('bcryptPasswordEncoder')
      sha256PasswordEncoder = ref('sha256PasswordEncoder')
   }

   daoAuthenticationProvider(PasswordFixingDaoAuthenticationProvider) {
      userDetailsService = ref('userDetailsService')
      passwordEncoder = ref('passwordEncoder')
      userCache = ref('userCache')
      saltSource = ref('saltSource')
      preAuthenticationChecks = ref('preAuthenticationChecks')
      postAuthenticationChecks = ref('postAuthenticationChecks')
      authoritiesMapper = ref('authoritiesMapper')
      hideUserNotFoundExceptions = conf.dao.hideUserNotFoundExceptions // true
      grailsApplication = ref('grailsApplication')
   }
}

With this configuration, new users’ passwords will be hashed with bcrypt, and valid existing users’ passwords will be converted to bcrypt using the plaintext passwords used during login. Once your users are converted, back out these changes and convert to the standard bcrypt approach. This would involve deleting the grails.plugin.springsecurity.password.algorithm attribute and all salt configuration since bcrypt doesn’t support a salt, deleting Sha256ToBCryptPasswordEncoder and PasswordFixingDaoAuthenticationProvider, and removing the bcryptPasswordEncoder and sha256PasswordEncoder bean definitions and passwordEncoder and daoAuthenticationProvider overrides from resources.groovy since the beans configured by the plugin using Config.groovy settings will be sufficient. Also if you had added salt to the User class encodePassword method, e.g.

protected void encodePassword() {
   password = springSecurityService.encodePassword(password, username)
}

convert it back to the default without a salt:

protected void encodePassword() {
   password = springSecurityService.encodePassword(password)
}

Some approaches for diagnosing spring-security-core login errors

Wednesday, December 11th, 2013 10:08pm

If you use the Grails spring-security-core plugin and you’re unable to authenticate, it can be hard to know where to start looking for the cause. Often it’s due to confusion from older blog posts that tell you to explicitly hash the password when creating a user, e.g.

def user = new User(username: 'me', enabled: true,
     password: springSecurityService.encodePassword('password')).save()

But the generated User domain class auto-hashes your password for you, so you would just do this (I omitted setting enabled since in 2.0 it defaults to true):

def user = new User(username: 'me', password: 'password').save()

If that isn’t the problem, Spring Security logs a lot at the debug level, so enable that in the log4j block in Config.groovy:

log4j = {
   ...
   debug 'org.springframework.security'
}

With any luck there will be a useful signal in the noise and it’ll point you in the right direction.

If that doesn’t do it, you can configure a debugger and set a breakpoint in org.springframework.security.authentication.ProviderManager in the authenticate method. That’s where each registered AuthenticationProvider gets a chance to attempt an authentication if it supports the current Authentication.

Not everyone likes to use a debugger though. Another option is to register an event listener and look at the failure events. This is described in the plugin docs here. Here’s a simple configuration that will print information to the console for all failure events:

grails.plugin.springsecurity.useSecurityEventListener = true
grails.plugin.springsecurity.onAbstractAuthenticationFailureEvent = { e, appCtx ->
   println "\nERROR auth failed for user $e.authentication.name: $e.exception.message\n"
}

Using MongoDB With Version 2.x of the Grails Spring Security Core Plugin

Sunday, December 01st, 2013 10:32pm

With a few customization steps it’s easy to use MongoDB to store user and role information for the spring-security-core plugin instead of using Hibernate, and after seeing this Stack Overflow question I thought I’d write up some notes on how to do this with the current plugins. Note that much of this is based on this blog post.

I created a demo application using Grails 2.3.3; it’s available on GitHub. The general steps were:

  • $ grails create-app mongoSpringSecurity
  • add the plugins to BuildConfig.groovy
  • $ grails s2-quickstart auth User Role
  • update DataSource.groovy to use MongoDB
  • create a custom UserDetailsService and register it in resources.groovy
  • create a test role and a user in BootStrap.groovy
  • customize the domain classes to use MongoDB
  • add tags to index.gsp to add a login link if you’re not logged in, and show that you’re logged in if you are

One difference between what I do here and what was done in the original blog post is that the custom UserDetailsService is not a Grails service – it’s in src/groovy and not in grails-app/services. It wasn’t necessary to be a real service then and isn’t now; it’s a coincidence that the Spring Security interface name ends in “Service”. See the plugin documentation for general information about customizing this bean.

You can see the source for the custom bean here. By embedding the authorities in the user domain class, the many-to-many relationship is not needed and the model is a lot simpler, so the class implementation is also – for example there’s no need for a withTransaction block to avoid lazy loading exceptions.

The changes for the User class are fairly minor. You need static mapWith = 'mongo' if you have both the Hibernate and MongoDB plugins; in this case it’s unnecessary but harmless to leave it in. The id field should be an ObjectId, and I retained the other customizations from the earlier blog post (the embedded roles, the addition of the email field, extra constraints, etc.). The Role class changes are similar.

Since we’re using a custom UserDetailsService, we can delete the userLookup.userDomainClassName, userLookup.authorityJoinClassName, and authority.className properties from Config.groovy, and since the roles are embedded in the user class we can delete the generated UserRole class.

You should be able to clone the repo and start the application (assuming you have MongoDB and Grails 2.3.3 already). Click the login link on the start page and after you successfully authenticate, the link should be replaced by a message.

Converting Grails Applications to Plugins and vice versa

Monday, July 22nd, 2013 2:57pm

I was in London last week on vacation with my family and was lucky that there was a London GGUG meetup during my visit. David Dawson discussed modularizing Grails applications by refactoring them into multiple plugins (you can see the video of his talk here). One thing that he mentioned was the idea of plugins that can be run like standalone applications since the structures of Grails applications and plugins are so similar, and I thought it would be interesting to look at the process of converting an application to a plugin or a plugin to an application.

The approaches I’ll discuss here include a lot of bias on my part about how I like to create apps and plugins, so YMMV. I also won’t go into partial conversions since David covered this well in his talk.

So to convert an application to a plugin, the general workflow would be something like

  • Create the plugin descriptor, FooGrailsPlugin.groovy. The easiest way to do this is to run grails create-plugin pluginname and copy the generated file from there
  • delete everything from application.properties except the app.grails.version property
  • if you have jars in the lib directory that are available in a Maven repo, delete them and replace with BuildConfig.groovy dependencies
  • change any plugin and jar dependencies that are needed for development and testing but not when the plugin is installed to not be exported, by adding export = false
  • If you need the _Install.groovy, _Uninstall.groovy, or _Upgrade.groovy scripts (you probably don’t) grab those from the dummy plugin from step 1 (but delete any you don’t need, they’re all optional)
  • delete ApplicationResources.groovy if you aren’t using it and don’t depend on resources plugin
  • move code from BootStrap.groovy init() to FooGrailsPlugin.doWithApplicationContext and/or FooGrailsPlugin.doWithDynamicMethods and destroy() to FooGrailsPlugin.onShutdown, and delete BootStrap.groovy
  • add a dependency for the release plugin in BuildConfig.groovy
  • delete everything but the log4j configuration from Config.groovy
  • delete UrlMappings.groovy unless you have exported mappings; only keep the added ones
  • move bean definitions from resources.groovy to FooGrailsPlugin.doWithSpring and delete resources.groovy
  • delete grails-app/i18n message bundle files unless you added messages; only keep the added ones
  • delete everything from grails-app/views that you don’t use (in particular error.gsp, index.gsp, and layouts/main.gsp)
  • delete everything from web-app that you don’t use (including WEB-INF xml and tld files)
  • now would be a great time to write those tests you’ve been meaning to get to
  • create one or more test applications to install the plugin into to ensure that it works as a plugin; consider scripting this
  • write documentation for how to use the plugin; at a minimum a README file, but Grails gdoc files would be much better (run grails doc --init to get started)

Converting a plugin to an application is similar, except for the most part reversed:

  • Create a dummy application with grails create-app appname to copy missing files from
    • Move BootStrap.groovy, Config.groovy, UrlMappings.groovy, and ApplicationResources.groovy into grails-app/conf, merging if needed
    • Move resources.groovy into grails-app/conf/spring
    • Move message bundle files to grails-app/i18n, merging if needed
    • Move missing GSPs to grails-app/views
    • Move static resources from web-app
    • Move xml and tld files from web-app/WEB-INF
  • Move code from FooGrailsPlugin.doWithApplicationContext and FooGrailsPlugin.doWithDynamicMethods to BootStrap.groovy init(), and code from FooGrailsPlugin.onShutdown to destroy()
  • Move bean definitions from FooGrailsPlugin.doWithSpring to resources.groovy
  • Delete the plugin descriptor
  • Restore missing properties in application.properties
  • Delete the _Install.groovy, _Uninstall.groovy, and _Upgrade.groovy scripts
  • Remove the dependency for the release plugin from BuildConfig.groovy
  • Now would be a great time to write those tests you’ve been meaning to get to
  • If you make changes in the plugin’s doWithWebDescriptor, run grails install-templates and add them to src/templates/war/web.xml
  • If you add custom artifacts, or were supporting development-environment code or config changes in onChange and/or onConfigChange, these aren’t directly doable in an application. Use the pluginator plugin for these, and also for dynamic web.xml changes that require runtime logic

  • I’ve probably skipped a few steps and definitely omitted a lot of detail, but this should cover most of the process for each type of conversion.

This Week in Grails (2013-20)

Saturday, May 25th, 2013 7:15pm

The big news is that Grails 2.3 M1 was released. 2.3 will be a big update with many fixes, enhancements, and new features. M1 only has a subset of what’s coming and it’s already packed with cool new stuff. Graeme did some writeups on what’s coming, Grails 2.3 M1 and Upcoming M2 and Road to Grails 2.3: RESTful URL Mappings and URL Mapping Reports.

Be sure to check out this cool video on Grails that the folks at Orange & Bronze created.

I released two plugins, standalone-tomcat-memcached and standalone-tomcat-redis. These are like the database-session plugin in that they reconfigure the web server to store session data in a datastore instead of the traditional approach, but these only work with Tomcat since they work directly with the Tomcat APIs instead of intercepting and replacing the real sessions. I’ve been working on reconfiguring how Grails applications deploy to Heroku, changing from using an embedded Jetty instance to using the standalone plugin (either with Tomcat or Jetty). Along the way since Heroku doesn’t support sticky sessions I created these plugins to get things working smoothly with the heroku plugin. More on this later.


If you want to keep up with these “This Week in Grails” posts you can access them directly via their category link or in an RSS reader with the feed for just these posts.


Translations of this post:



Plugins

There were 7 new plugins released:

  • apidocs version 0.1. Generates REST API documentation
  • criteria-js version 0.3. GORM criteria for JavaScript
  • db-helper-rest version 0.3. Provides methods for dropping, creating and initialization of database schema from the project bootstrap
  • httplogger version 1.0. Logs HTTP traffic
  • required-parameters version 1.0.0. Enables verifying required parameters for controller actions using simple annotations
  • standalone-tomcat-memcached version 0.1. Uses Memcached as the Tomcat session manager
  • standalone-tomcat-redis version 0.1. Uses Redis as the Tomcat session manager

and 26 updated plugins:

  • goodform version 1.0.2. Create extremely usable complex forms with rules based flow that work well in browsers
  • airbrake version 0.9.3. Notifier plugin for integrating apps with Airbrake
  • asynchronous-mail version 1.0-RC5. Send email asynchronously by storing them in the database and sending with a Quartz job
  • aws-sdk version 1.4.3. Use the Amazon Web Services infrastructure services
  • closure-templates-resources version 0.2.3. Supports server-side compilation of .soy template files to JavaScript files
  • easyui version 1.3. Supplies jQuery EasyUI resources and taglibs
  • external-config-reload version 1.3.0. Polls for changes to external configuration files (files added to grails.config.locations), reloads the configuration when a change has occurred, and notifies specified plugins by firing the onConfigChange event in each
  • font-awesome-resources version 3.1. Integrates the Font Awesome icon set
  • google-visualization version 0.6.2. Provides a taglib for the interactive charts of the Google Visualization API
  • grails-melody version 1.44. Integrates the JavaMelody system monitoring tool
  • hibernate version 3.6.10.M3. Provides integration between Grails and Hibernate through GORM
  • i18n-fields version 0.6.4. Provides a declarative way of localizing domain classes’ fields for different languages
  • jrimum-bopepo version 0.3. Allows you to create Boletos Bancarios for Banks of Brazil using the Jrimum Bopepo library
  • neo4j version 1.0.0.RC2. GORM for Neo4j
  • newrelic version 1.0-2.18.0. Adds the NewRelic Real User Monitoring feature to your GSP pages
  • pusher version 0.5. Wrapper for pusher.com REST api
  • recaptcha version 0.6.5. Integrates ReCaptcha and Mailhide services
  • remote-pagination version 0.4.4. Provides tags for pagination and to sort columns without page refresh using Ajax and loads only the list of objects needed
  • segmentio version 0.2.0. Use Segment.io to send your analytics data to any service you want, without you having to integrate with each one individually
  • spring-security-mock version 1.0.2. Mock authentication support for Spring Security
  • standalone version 1.2.1. Runs a Grails application as a JAR file with an embedded Tomcat server
  • vaadin version 1.7.5. Adds Vaadin (http://vaadin.com/) integration
  • validate-config version 0.4.2. Adds methods to ConfigObject for validating expected and required properties
  • war-exec version 1.0.3. Makes the war files generated by Grails executable (java -jar myapp.war) by embedding Jetty. Jetty can be configured using properties in Config.groovy
  • webdriver version 0.4.2. Integrates Webdriver with Grails to allow functional testing in both HtmlUnit and real browsers
  • xwiki-rendering version 1.0-RC3. Convert texts using XWiki Rendering Framework

Interesting Tweets

Jobs



User groups and Conferences

Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 License.