Config.groovy
, BuildConfig.groovy
, etc. (in this case I’m talking about a Grails 2 app but these ideas are generalizable to Grails 3+) but that’s often not enough.
Grails 2 supports external configuration files, which can be in various places and get merged into the final configuration. But just having what you think is the correct source and configuration files isn’t enough since changes could have been made that didn’t make it into source control. And you can’t easily get information from those files in a WAR since they’re compiled into classes.
My preference for digging into a running Grails application is the console plugin, but to use that you would need to add it to BuildConfig.groovy
and build and deploy a new WAR, but again that’s not necessarily going to have the same configuration as the previous deployment.
I have a situation like this at work, so I came up with a lightweight way to add a web-based console similar to the console plugin to a WAR. Originally it was a servlet which generated the HTML for a simple form containing a textarea for Groovy code and a submit button to post the code to be run on the server, and the logic (mostly borrowed from the console plugin) to execute the code and return the results to the browser. I compiled it in the same project that the WAR was built from to ensure that it’s compatible with the versions of Groovy, Grails, Spring, etc. and copied the .class file to WEB-INF/classes
in the exploded directory in Tomcat’s webapps
folder, and manually edited WEB-APP/web.xml
to add the required <servlet>
and <servlet-mapping>
elements, and everything worked great in my small test app.
But when I tried it in the real application I couldn’t access it because of Spring Security. In this particular case I could have worked around that because the app stores Requestmap
instances in the database, but I didn’t want to make changes that I might forget to undo, and there’s the chicken-and-egg problem that I don’t necessarily know what the database settings are for this deployment. So instead I converted the servlet to a servlet filter, and made sure to add the filter before the Spring Security filter chain in web.xml
and it worked as expected after restarting the server.
I made the changes in the exploded war directory, but it’s also possible to make the changes in the WAR file itself. Since WAR files are ZIP files, you can unzip the WAR, make the changes, and re-zip.
Here’s the source for the filter:
package com.burtbeckwith.hack import groovy.transform.CompileStatic import groovy.util.logging.Slf4j import org.codehaus.groovy.grails.commons.GrailsApplication import org.springframework.context.ApplicationContext import org.springframework.web.context.support.WebApplicationContextUtils import javax.servlet.Filter import javax.servlet.FilterChain import javax.servlet.FilterConfig import javax.servlet.ServletException import javax.servlet.ServletRequest import javax.servlet.ServletResponse import javax.servlet.http.HttpServletRequest import javax.servlet.http.HttpServletResponse @CompileStatic @Slf4j class HackFilter implements Filter { private ApplicationContext applicationContext private GrailsApplication grailsApplication void init(FilterConfig fc) throws ServletException { applicationContext = WebApplicationContextUtils.getRequiredWebApplicationContext(fc.servletContext) grailsApplication = applicationContext.getBean(GrailsApplication) } void doFilter(ServletRequest req, ServletResponse res, FilterChain chain) throws IOException, ServletException { HttpServletRequest request = (HttpServletRequest) req HttpServletResponse response = (HttpServletResponse) res if ('GET' == request.method) { doGet request, response } else { // assume POST doPost request, response } } void destroy() {} private void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { response.writer.write html(request.contextPath) } private void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { long startTime = System.currentTimeMillis() String code = request.getParameter('code') ByteArrayOutputStream baos = new ByteArrayOutputStream() PrintStream out = new PrintStream(baos) PrintStream systemOut = System.out Throwable e String result = '' try { System.out = out result = new GroovyShell(grailsApplication.classLoader, new Binding( config: grailsApplication.config, ctx: applicationContext, grailsApplication: grailsApplication, out: out, request: request, session: request.session)).evaluate(code) } catch (Throwable t) { e = t } finally { System.out = systemOut } if (e) { StringWriter sw = new StringWriter() e.printStackTrace new PrintWriter(sw) result = sw.toString().replace('\t', ' ').replace(System.getProperty('line.separator'), '<br/>\n') } response.writer << html(request.contextPath, code, """\ Total time: ${System.currentTimeMillis() - startTime}ms Stdout: ${baos.toString('UTF8')} ${e ? 'Exception' : 'Result'}: $result""") } private String html(String contextPath, String code = '', String results = '') { """\ <html> <head> <title>Hack</title> </head> <body> <form action="$contextPath/hack" method="POST"> <span>Code: (binding vars include <i>config</i>, <i>ctx</i>, <i>grailsApplication</i>, <i>out</i>, <i>request</i>, <i>session</i>)</span><br/> <textarea name="code" cols="120" rows="25">$code</textarea><br/> <input type="submit" value="Execute" name="execute" /><br/> <span>Results:</span><br/> <textarea name="results" cols="120" rows="25" disabled="disabled">$results</textarea> </form> </body> </html> """ } }
and these are the corresponding <filter>
and <filter-mapping> elements for web.xml
:
<filter> <filter-name>hack</filter-name> <filter-class>com.burtbeckwith.hack.HackFilter</filter-class> </filter> <filter-mapping> <filter-name>hack</filter-name> <url-pattern>/hack</url-pattern> </filter-mapping>
To access the console, navigate to http://server:port/contextPath/hack. As in the console plugin you can run arbitrary Groovy code (including service method calls, working with domain classes, etc.), and there are several objects in the Binding that you can use – config
, ctx
, grailsApplication
, out
, request
, and session
.
To change the uri from /hack to something else, be sure to update both the <url-pattern>
tag in web.xml
and the action
attribute in the generated form in the filter class.
The original HQL included this
hql.append " AND e.aclObjectIdentity.aclClass.id=:aclClass"
and the converted criteria code was
aclObjectIdentity { aclClass { eq 'id', params.long('aclClass') } }
with the whole query being similar to this:
def results = lookupClass().createCriteria().list(max: max, offset: offset) { // other standard criteria method calls if (params.aclClass) { aclObjectIdentity { aclClass { eq 'id', params.long('aclClass') } } } }
That got me thinking about creating a way to represent that two-level projection and criterion generically.
If we restore the omitted optional parentheses the code becomes
aclObjectIdentity({ aclClass({ eq('id', params.long('aclClass')) }) })
So it should be more clear that this is a sequence of method calls; calling aclObjectIdentity
with a closure argument, then aclClass
with a closure argument, and finally eq
with a String
and a long
argument. Splitting out the closures as local variables makes it more clear, first as
def aclClassClosure = { eq('id', params.long('aclClass')) } aclObjectIdentity({ aclClass(aclClassClosure) })
and then
def aclClassClosure = { eq 'id', params.long('aclClass') } def aclObjectIdentityClosure = { aclClass(aclClassClosure) } aclObjectIdentity(aclObjectIdentityClosure)
To make this a bit more concrete, lets say we have three domain classes;
Department
:
class Department { String name }
Manager
:
class Manager { String name Department department }
and Employee
:
class Employee { String name Manager manager }
We create some instances:
Department d = new Department(name: 'department1').save() Manager m = new Manager(name: 'manager1', department: d).save() Employee e1 = new Employee(name: 'employee1', manager: m).save() Employee e2 = new Employee(name: 'employee2', manager: m).save()
and later want to run a query:
Employee.createCriteria().list(max: 10, offset: 0) { eq 'name', 'employee1' manager { department { eq 'name', 'department1' } } }
My goal is to represent this query with only some helper methods and without any closures (or as few as possible). Splitting that out like above we have
def departmentClosure = { eq 'name', 'department1' } def managerClosure = { department(departmentClosure) } def criteriaClosure = { eq 'name', 'employee1' manager(managerClosure) } Employee.createCriteria().list([max: 10, offset: 0], criteriaClosure)
When the query is run, the delegate of criteriaClosure
is set to an instance of HibernateCriteriaBuilder
when using Hibernate, or an analogous builder for MongoDB or whatever other GORM implementation you’re using. The builder has defined methods for eq
, like
, between
, etc., so when you make those calls in your criteria closure they’re run on the builder.
It turns out that it works the same way if you split the closure into multiple closures and call them with the builder as the delegate for each. So a method like this works:
def runCriteria(Class clazz, List<Closure> criterias, Map paginateParams) { clazz.createCriteria().list(paginateParams) { for (Closure criteria in criterias) { criteria.delegate = delegate criteria() } } }
and that means that we can split
Employee.createCriteria().list(max: 10, offset: 0) { eq 'name', 'employee1' manager { department { eq 'name', 'department1' } } }
into
def closure1 = { eq 'name', 'employee1' } def closure2 = { manager { department { eq 'name', 'department1' } } }
and run it as
runCriteria Employee, [closure1, closure2], [max: 10, offset: 0]
But how can we make that projection generic? It’s an inner method call, wrapped in one or more closures that project down to another domain class.
What I ultimately want is to be able to specify a projection with an inner criteria call without closures:
def projection = buildProjection('manager.department', 'eq', ['name', 'department1']) runCriteria Employee, [closure1, projection], [max: 10, offset: 0]
Here’s the buildProjection
method that does this:
Closure buildProjection(String path, String criterionMethod, List args) { def invoker = { String projectionName, Closure subcriteria -> delegate."$projectionName"(subcriteria) } def closure = { -> delegate."$criterionMethod"(args) } for (String projectionName in (path.split('\\.').reverse())) { closure = invoker.clone().curry(projectionName, closure) } closure }
To understand how this works, look again at the innermost closure:
department { eq 'name', 'department1' }
This will be invoked as a method call on the delegate, in effect
delegate.department({ eq 'name', 'department1' })
Groovy lets us call methods dynamically using GString
s, so this is the same as
String methodName = 'department' delegate."$methodName"({ eq 'name', 'department1' })
So we can represent the nested closures as an inner closure invoked as the closure argument of its containing closure, and that invoked as the closure argument of its containing closure, and so on until we run out of levels.
And we can build a closure that calls eq 'name', 'department1'
(or any criterion method with arguments, this is just a simplified example), as
def closure = { -> delegate."$criterionMethod"(args) }
So to represent the nested closures, start with an ‘invoker’ closure:
def invoker = { String projectionName, Closure subcriteria -> delegate."$projectionName"(subcriteria) }
and successively clone it at each nesting level, and curry it to embed the projection name and its inner closure since the criteria builder doesn’t expect any closure arguments, working from the inside out:
for (String projectionName in (path.split('\\.').reverse())) { closure = invoker.clone().curry(projectionName, closure) }
So, finally we can run the decomposed query as one or more ‘core’ criteria closures with standard criterion method calls, plus zero or more derived projection closures:
def criteria = { eq 'name', 'employee1' } def projection = buildProjection('manager.department', 'eq', ['name', 'department1']) runCriteria Employee, [criteria, projection], [max: 10, offset: 0]
I doubt there’s a lot of reuse potential here to be honest, but working through this helped me to better understand how GORM runs criteria queries. I’ll be talking about this and some other GORM topics at Greach next month, so if you find this interesting be sure to check out the recording of that talk.
]]>README.adoc
but I wanted to summarize the applications in a combined blog post.
All of the apps use Grails 3 and spring-security-core 3.x but would be easily converted to Grails 2 and version 2.x of the plugin since nothing depends on the newer versions.
Some items that are common to all or most of the apps:
logback.groovy
grails.plugin.springsecurity
block in application.groovy
The first example is the “autorole” application. It demonstrates how to infer one or more roles from persisted user information without explicitly storing roles in the traditional many-to-many format. In this example there isn’t even a Role or UserRole domain class, only the User class; all of the work is done in autorole.AutoRoleUserDetailsService
, the custom UserDetailsService
implementation.
A more realistic implementation would probably use a hybrid approach, storing some role information in the database and inferring the rest, with the UserDetailsService
merging the “real” and “virtual” roles as needed.
Items of note:
test.User
is the user domain class generated by the s2-quickstart
script with one modification, adding a boolean admin
propertyRole
and UserRole
classes generated by the s2-quickstart
script were deleted since they’re not usedBootStrap.groovy
; user “admin” (password “password”) has the admin
boolean set to true
and will be auto-granted ROLE_ADMIN
, and user “user” (password “password”) has the default value for admin
(false
) which will result in a grant of ROLE_USER
autorole.AutoRoleUserDetailsService
is registered in grails-app/conf/spring/resources.groovy as the userDetailsService
beansecured.SecureController
has two annotated actions; /secure
requires ROLE_USER
(or ROLE_ADMIN
since hierarchical roles are configured) and /secure/admin
requires ROLE_ADMIN
The next example is the “noroles” application. It shows how to use expressions to guard access when access rules are simple and roles aren’t required. In this example there isn’t even a Role or UserRole domain class, only the User class.
Items of note:
test.User
is the user domain class generated by the s2-quickstart
script with a few modifications:
final authorities = []
property so GormUserDetailsService
works correctly, but doesn’t grant any roles since there aren’t any (for demo purposes here, since there is a custom UserDetailsService
)UserType userType
propertyString businessUnit
propertyboolean developer
propertys2-quickstart
script were deleted since they’re not used
BootStrap.groovy
, all with password “password”:
admin
, businessUnit: ‘group1’admin
, businessUnit: ‘group2’sales
, businessUnit: ‘group1’other
, businessUnit: ‘it’, developer true
secured.SecureController
has several annotated actions using expressions to guard accessUserDetailsService
creates an extended UserDetails
instance to cache nonstandard user properties for use in expressions
This is an update of the demo app from Greach 2015 (“hacking_madrid”) which was an update of the demo app from GGX 2011 (“hacking_london”). I updated it to Grails 3 and spring-security-core 3.0.3.
It adds an “organization” drop-down to the login page in addition to username and password, and a customized Authentication
class, servlet filter, and AuthenticationProvider
.
Items of note:
grails-app/views/login/auth.gsp
is the same as auth.gsp
from the core plugin, with the addition of a
to select the user’s Organization during loginhacking.extralogin.OrganizationAuthentication
extends UsernamePasswordAuthenticationToken
to add a String organizationName
propertyhacking.extralogin.ui.OrganizationFilter
extends the core plugin’s GrailsUsernamePasswordAuthenticationFilter
and is registered as the authenticationProcessingFilter
bean to process form logins; it creates OrganizationAuthentication
instances from POST parameters for authenticationhacking.extralogin.auth.OrganizationAuthenticationProvider
uses the data in OrganizationAuthentication
to authenticate
DaoAuthenticationProvider
but directly accesses the database using GORM instead of delegating to a UserDetailsService
Organization
and OrgUser
are used to persist the user/organization relationship
static hasMany
mapped collectionsOrganization
instances, “Org1” and “Org2”ROLE_ADMIN
(“admin”/”password”) in “Org1”ROLE_USER
(“user”/”password”) in “Org2”ROLE_USER
and enabled
set to false (“disabled”/”password”) in “Org1”resources.groovy
to override the bean class for the authenticationProcessingFilter
and daoAuthenticationProvider
beans, hacking.HackingBeanFactoryPostProcessor
is registered in resources.groovy
and updates the bean class in the bean definition. This approach retains all of the dependency injections and configuration updates and helps prevent the app from breaking if updated to a newer version of the plugin that has different dependencies and/or config options for the beansNoStackBadCredentialsException
is thrown as needed instead of BadCredentialsException
; it’s similar to the core plugin’s NoStackUsernameNotFoundException
which avoids filling in the stack trace to reduce creation costsecured.SecureController
has two annotated actions; /secure
requires ROLE_USER
(or ROLE_ADMIN
since hierarchical roles are configured) and /secure/admin
requires ROLE_ADMIN
This application shows how to use events to lock a user account after a fixed number of failed login attempts.
Items of note:
lockout.FailureEventListener
is registered to listen for AuthenticationFailureBadCredentialsEvent
lockout.SuccessEventListener
is registered to listen for AuthenticationSuccessEvent
int badCredentialsCount
to track failed loginsUserService
increments badCredentialsCount for failures and resets to 0 on successsecured.SecureController
has two annotated actions; /secure
requires ROLE_USER
(or ROLE_ADMIN
since hierarchical roles are configured) and /secure/admin
requires ROLE_ADMIN
This application showing how to use X.509 browser certificates to authenticate.
Items of note:
useX509 = true
in application.groovy
BootStrap.groovy
, both with password “not_used” since it’s unused with certificate authenticationdianne.p12
and/or scott.p12
certificate to your browser to authenticate as that personrun-app
similarly is left as an exercise for the reader
server.jks
as the keystore and truststore; server.xml
is an example Tomcat 8 config file that does this, expecting that server.jks
is in the conf
directorysecured.SecureController
has two annotated actions; /secure
requires ROLE_USER
(or ROLE_ADMIN
since hierarchical roles are configured) and /secure/admin
requires ROLE_ADMIN
This last application shows how to use X.509 browser certificates and a second authentication provider to authenticate.
Note that this has been only lightly tested and should be used with caution. I have no idea if there are gaps in the implementation. Test anything based on this approach and/or code extensively before using in a real application. If you find problems with the approach or code let me know so I can update the code.
Items of note:
useX509 = true
in application.groovy
BootStrap.groovy
, both with password “password” since the password is needed for the second form-auth phasedianne.p12
and/or scott.p12
certificate to your browser to authenticate as that personrun-app
similarly is left as an exercise for the reader
server.jks
as the keystore and truststore; server.xml
is an example Tomcat 8 config file that does this, expecting that server.jks
is in the conf
directoryx509chained.LoginController
extends the plugin’s LoginController
to not redirect to successHandler.defaultTargetUrl
if authenticated. This is needed because the chained authentication happens in two requests with a redirect. If the first phase (X.509) succeeds, there will be an active authentication, but it’s incomplete and cannot be used yet. Filter chain processing must be allowed to happen to allow the second authentication phase to run.x509chained.ChainedX509AuthenticationFilter
extends the standard X509AuthenticationFilter
to replace the Authentication
in successfulAuthentication
with one with all of the real roles replaced with ROLE_CHAINED_X509
as a marker to indicate that the first authentication phase succeeded. The second authentication phase will create a standard Authentication
with the real roles.x509chained.ChainedAuthenticationProcessingFilter
extends the plugin’s form authentication filter (GrailsUsernamePasswordAuthenticationFilter
). It detects that the X.509 phase has occurred and redirects to the login page, replacing the credentials (since they’re unused by X.509) with a marker string so downstream processing is aware of the current state in the workflow.secured.SecureController
has two annotated actions; /secure
requires ROLE_USER
(or ROLE_ADMIN
since hierarchical roles are configured) and /secure/admin
requires ROLE_ADMIN
When I start working in a new area of tech I usually sign up for a mailing list or forum, or whatever they tend to use for discussions about usage, upcoming releases, future plans, etc. I rarely get involved though. When I’m new at the tech I usually find that questions I would have asked have been asked and answered, and once I get good enough at it I tend to rely less on those resources. So I tend to lurk in general and don’t actively participate.
My tendency toward being a lurker is also affected by not wanting to embarrass myself online. When I was a graduate student I discovered the TeX typesetting software and thought that it would be great to use for papers, so I started collecting software and resources. Windows 95 had recently been released1 and Windows options weren’t great then. Someone asked a question on one of the local Usenet groups and being an expert I replied. Except that I was far from an expert and gave rather bad advice, and a few others who actually knew what they were talking about jumped in and provided useful information. I tend to have to touch the pretty glowing orange thing on the stove to learn that it’s very hot and a painful thing to do, so this was a good experience for me. It continues to be a reminder to try to be humble and to help where possible, but be willing to say “I don’t know” when that’s the case, and stay out of the way if I don’t have anything to contribute.
When I discovered Grails I signed up for the user and dev mailing lists. There was a lot to learn, and many of the emails didn’t make sense but I saved a bunch that seemed like they’d be useful later. I quickly learned a lot thanks to the helpful developers and users on the lists. Eventually I got good enough that there were questions that I could answer, but often others would answer before I did. Occasionally a more obscure question would go unanswered, and those would nag at me from my inbox. Sometimes when I was stuck on something for work I would take a break and look at some of those questions and try to see what’s going on, either because it did or might affect what I was working on, or just because it was interesting. If I didn’t get anywhere, then I would let it go, but if I felt like I figured it out or that I at least could contribute useful insights, I would reply with what I found.
Getting a reply on the list from a frustrated user saying that I’ve helped them get past an annoying issue is a pretty cool thing, and a big motivator to continue. Much like teaching, helping others with tech support issues is usually beneficial for both sides since it forces you to understand what’s going on under the hood much more than when you just use the feature. In retrospect, although I was reluctant to start giving back, it didn’t make sense not to. The lists were active with lots of generally respectful discussions, and although people have come and gone, there remains an overall tone and it’s profoundly positive.
When I started going to Grails conferences, the subject of how helpful the communities are was (and still is) a common topic in hallway and mealtime discussions. This was weird to me because I hadn’t really noticed, mostly since I hadn’t really seen communities that weren’t. Of course I’ve become aware of some of what they were referring to
So the big question is: why are these communities so helpful and friendly? I can’t know for sure, but it obviously has a lot to do with the project leads and big names in the various techs. From the beginning, their personalities and temperaments set the tone that continues today. Why we were lucky to have these people driving the technologies in the first place is a topic for a future blog post, or at least something to discuss over beers at future conferences and meetups.
I’ve learned so much more than I could have expected by participating in all of this, and I’ve been lucky to meet a large number of smart, funny, inquisitive, amazing people. Thank you all, and keep up the great work. It’s easy to get distracted by the frustrating bits and the negative aspects, but weeks like this one are, if we’re able to see past them, a good reminder of how much we have and tend to take for granted.
As a side note, I cringed a bit when the Greach organizers announced that they had put up a code of conduct for this year’s conference. After some high-profile incidents at Python- and Ruby-related conferences (and elsewhere) many conferences did the same, but it didn’t make sense to me to do this at a Groovy-related conference since it’s not needed. It didn’t take long to realize that this was naive, and while we are a friendly bunch, there have likely been incidents where attendees were harassed or made uncomfortable and I wasn’t aware of them. And even if not, there’s nothing wrong with putting core beliefs and expectations in writing. My version would be a lot shorter though: “Don’t be a jerk”.
I rarely say this but it’s pretty much always the case – these are my thoughts and opinions and I don’t claim to speak for anyone or any company or organization I’ve worked with in the past, present or future.
Also, all comments on this blog are moderated and I will not accept any that aren’t constructive, and I reserve the right to disable comments entirely on this post if necessary.