Grails Envers plugin update

Quite awhile ago I wrote a plugin for Hibernate Envers. You can find the original blog post here: http://www.lucasward.net/2011/04/grails-envers-plugin.html

After that I got insanely busy with my startup. The Grails releases started to pile up and you can see the comments of the original blog post asking about when new releases of Grails would be supported. The great thing about open source (and especially github) is that others could take what I did and build on it easily. After my startup sold a few months back, I peeked my head back up to see what had happened with the plugin. Tomasz Kalkosinski summarized what had happened with forks by lots of people: http://refaktor.blogspot.com/2012/08/hibernate-envers-with-grails-210.html

From following the chain it looks like the following group all provided individual contribution to the plugin:

• Jay Hogan
• Tomasz Kalkosiński
• Damir Murat
• Matija Folnovic
• Alexander Abdugafarov

If I have missed anyone, I sincerely apologize. That's at least who I could find in various commit logs.

I took the various commits from everyone and turned it into three versions of the plugin that you can find in the Grails central plugin repository:

• 1.0.0
• 2.0.0
• 2.1.0

- 1.0.0 -> This is my original version of the plugin that works for 1.3.6. I made a separate release because I know others that are still suck on an older version of grails. So, having a version they can download without having to pull down the old zip is helpful.

- 2.0.0 -> This is the version mostly with work from Jay Hogan. It provides support for Grails 2.0.x and cleans up a few things.

-2.1.0 -> The latest release I could find. It appears to support Grails 2.1.x, although I had no issues with it working on 2.2.1. It may be what was pushed to the grails repo as 0.4.4.

You can see all the versions in the Grails maven repo: http://repo.grails.org/grails/plugins/org/grails/plugins/envers/

 Please feel free to submit any pull requests to: https://github.com/lucaslward/grails-envers-plugin

Fundspire has been Acquired

This is definitely a bit belated, however, I wanted to announce on my blog that my startup, Fundspire, was acquired by Evestment last November. Nothing much changes for me day to day, and I'm hopeful that it will allow more time to blog and work on my open source projects. I still very much need to get the grails envers plugin in the official repo and upgraded to support the latest versions of grails.

How to figure out what the Grails meta-methods do

Since I code in Grails everyday, I periodically have friends and acquaintances that are learning the framework asking me questions. By far the most common one is something like: "where are all of these magic methods coming from!?!?!?!"

Grails heavily uses Groovy meta programming across every tier, but especially in controllers and domain objects. One example would be domainObject.save(), another would be the render method on controllers. You can see a list of all these dynamic methods in the grails documentation: http://grails.org/doc/latest/ I can't find a good way to link to just the dynamic methods, but on the right side of the documentation there is a quick reference section that contains a listing of all the available methods and brief descriptions if you click through.

While it's definitely the best way to figure out what meta-methods are available to you, it doesn't help you figure out exactly what those methods are doing. Furthermore, the documentation is generally pretty brief around how to use them. In my experience the best way to see all the configuration options and understand what's happening is to look at the actual methods that are added to your controllers and domain objects. However, because they're added to the 'metaClass' of these classes when Grails bootstraps it can be difficult to figure out where and how they're added.

Fortunately, it's not as bad as you think. Grails follows it's own recommendations well and as a result all of these methods are added in the exact same fashion as they would be in a normal plugin. So, what you need to look for are the plugins that add them. I'll start first with the Domain classes because they're the most complicated and also the ones you will most likely need to understand. (More information on Grails plugins can be found here)

GORM Dynamic methods

The first thing to note when examining the GORM Methods is that it was designed to be plug and play from the get go. By default GORM is a hibernate implementation connecting to a standard RDBMS. However, that can be changed to use something like Mongo. To keep things simple I'll assume the default.

In IntelliJ the hibernate plugin is included with all other project plugins

As you can see, just like any other plugin, the hibernate plugin adds the dynamic methods in doWithDynamicMethods. In this case it's delegating to another class, likely to allow for easier testing. I'm not going to go through the entire class, but it's fairly straightforward from here, as most of the dynamic methods are added directly by HibernatePluginSupport.

Controller Dynamic Methods

The controller methods are a little bit harder to find. Like the Hibernate plugin above, they are added via a plugin. However, you won't find it with the normal plugin list, as it's 'core'. (Unless something massive has changed in 2.0) 

You'll find when you look in this class that there's a pretty natural separation of concerns, there's a class for adding just the render methods, etc. In general, you can find most dynamic methods by searching for *GrailsPlugin.groovy.

Hopefully this helps you understand better how the Grails 'magic' works. Anyone with experience in Spring MVC or Hibernate will recognize most of what these methods are doing, but if not I suggest you do a quick look through those products to help you understand what's happening.

Using Spring Security Concurrent Session controls with Grails

While it isn't a problem for most website, many have a requirement to control how many users can be connected concurrently. This is usually related to sites that sell 'per user' licenses, such as my current company. Unfortunately, this isn't very straightforward in Grails.

First thing to note: Grails Spring Security plugin doesn't support concurrent session. Spring security does have the feature, and the plugin doesn't prevent it from being used, but it doesn't help either. And it's a bit harder because you don't have the advantage of the spring security namespace to help with some of this. Most of the information I found on how to set this up came from this blog post: http://blog.block-consult.com/2012/01/restricting-concurrent-user-sessions-in-grails-2-using-spring-security-core-plugin/. Using that as a guide, There are three things that have to be added to resources.groovy:

sessionRegistry(SessionRegistryImpl)

concurrencyFilter(ConcurrentSessionFilter) {
    sessionRegistry = sessionRegistry
    logoutHandlers = [ref("rememberMeServices"), ref("securityContextLogoutHandler")]
    expiredUrl='/login/invalidated'
}


concurrentSessionControlStrategy(ConcurrentSessionControlStrategy, sessionRegistry) {
    alwaysCreateSession = true
    exceptionIfMaximumExceeded = true
    maximumSessions = 1
}

The SessionRegistry is just a mechanism with a thread-safe map to register new sessions. It's stores them both by session id and by principal. It uses a class called SessionInformation. This object contains some information, such as the last time anything happened on this session, and if it's expired. In general, sessions are only registered by the ConcurrentSessionControlStrategy. This strategy actually extends the Spring SessionFixationStragey, which is a whole other ball of wax I won't go into. Either way, this strategy is generally used by the UsernamePasswordAuthenticationFilter. Normally I imagine this is setup using the spring namespace. However, in grails you have to deal with it manually. The way I found for it to work is an entry in boostrap.groovy:

authenticationProcessingFilter.sessionAuthenticationStrategy = concurrentSessionControlStrategy

What this strategy does is enforce *at authentication time* various rules about concurrent sessions. In our case, we set the Maximum sessions to 1. I also set it to throw an exception if it's exceeded. It basically stops the authentication. I do this because I would prefer to tell the user what will happen if they log in: the other session will be expired. This is done by the strategy by looping through the SessionInformation objects tied to the given principal and calling expireNow() on the least recently used one.

Which brings me to the filter.

The filter shown above is inserted into the standard spring security authentication filter chain. From the blog post above, it can be added to the filter chain using the following code in bootstrap.groovy:

SpringSecurityUtils.clientRegisterFilter('concurrencyFilter', SecurityFilterPosition.CONCURRENT_SESSION_FILTER)

Basically, what the filter does is call the SessionRegistry for the given session id. If there is a session information and it is marked as expired, it forces a redirect to the provided 'expiredUrl'.

Once this is done, it generally works as you expect. If you prevent a user from logging in, it's fairly simple to mark other session information as 'expired' and the filter will then redirect them to the 'expiredURL'.

Unless you're using Remember me, in which case everything is screwed.

If you notice in the config of the filter, there's a couple of log out handlers, and one of them is the remember me service. Which removes the remember me token when the filter 'expires' the session. But there's still a gigantic problem: RememberMe tokens are processed by a completely different filter that is oblivious to the concurrent session control. This is the RememberMeAuthenticationFilter. Keep in mind that the username and password filter is only called for a specific url and is usually posted to by some login page. So, it only touches a request if the url of the request matches the one it's looking for.

So, here's the scenario: User A logs in with a remember me token. The normal UsernamePasswordAuthenticationFilter is called. Because there's the remember me option enabled. (easily setup by the grails plugin) It tells the RememberMeServices to add the cookie to the response. Everything is fine after that. So, let's say that user sits idle for a bit and their session expires. The SessionRegistryImpl is also setup to receive HttpSession change events. This is setup by turning on the session event publisher via an option in config.groovy. Something like: grails.plugins.springsecurity.useHttpSessionEventPublisher = true. Which, is also necessary for concurrent session handling to work.

The container expiring the session causes the registry to remove the SessionInformation for this session from the registry. Now, let's say another user logs in with the same credentials (the same principal in Spring Security terms) That user is logged in and now has a session tied to that principal in the registry. Now, let's say that the first user with the remember me cookie starts using the application again. This creates a request that doesn't have a security context, which would normally punt you back to login. However, the RememberMeAuthenticationFilter sees the cookie and authenticates the user. In this case, it's actually a slightly different Authentication object, but the user is still authenticated. The RememberMe filter never looks at the registry or strategy, so as far as the SessionRegistry is concerned, there is still only one user for this principal. Thus the user with the token can use it all day long without running afoul of the concurrent session limits. What this basically means is that anyone using Spring Security with remember me can bypass concurrent session limits by using a remember me token and letting their session expire.

My first assumption was that there was something that needs to be setup that wasn't being done. Maybe something the namespace was doing. But I can't find any hook in either the remember me filter of the service for anything that would handle this. Even looking at newer versions of spring security, the only changes I see are for the registry to use java.util.concurrent, which was probably done after 3.1 when they could assume Java 5.

There are some pretty reasonable extension points to the filter though. I extended it and overrode the method onSuccessfulAuthentication, thinking I could do some of the same things the strategy does. I could register a new information and mark others as expired. But it gets a bit complicated. In this case, I think the users with the token coming back from idle should be the one bounced. Which isn't super hard, as long as the remember me filter is before the ConcurrentSessionFilter. However, there's all kinds of potential threading issues. Especially if an application uses ajax. It's really non-deterministic. I honestly think that the registry would need to be upgraded to understand the RememberMe cookie.

Usually, when working with an established open source project, I assume that I'm doing something wrong. However, when looking through the code, I can't find anything to handle this properly.

So, my current bottom line is this: Concurrent Session can be used with the Grails plugin, however, not if you allow your users to use a 'remember me' token.

Update

I had passed on this issue to some friends at Vmware, and it looks like I was correct: Concurrent Session doesn't work with RememberMe. Here's the ticket that was created: https://jira.springsource.org/browse/SEC-2028

On Git Command Line Usability

Today I was reading about a new tool to use Git on a Mac called GitBox. I generally prefer the command-line, but as it was on sale and mentioned as one of the best clients for Mac, I figured I would at least take a look. When looking through the comments though, one part of a particular review stuck out to me:

This app generally is a relieve compared to the terribly complicated git command line

It's a common refrain from even a seasoned developer that has been using other tools for awhile, even those familiar with subversion on the command-line. To simplify the discussion, let's take one part of the obvious usability issues out of the general discussion of git usability: DVCS in general. If using something like Hg or Git after using a more centralized VCS, it's going to be more complicated by nature. With DVCS you're effectively managing your own repo and merging with another common repo shared with the rest of your team. With a more centralized system like SVN you're really just pushing change lists to a repo managed by someone else. I'm sure there's some power users at every company, but I'm betting a lot of SVN users aren't doing a lot of branch merging. Even looking at GitBox, which does look like a more simplified view of Git, the overall actions you're performing are more complicated than say, something like tortoiseSVN.

DVCS has it's advantages and disadvantages, but given that the overall model is more complex in nature for individual developers committing code, let's shelve that and focus solely on the question of whether the git command line itself is overly complicated. (as the review described)

My gut response to this question is 'it depends'. Perhaps I was a consultant for too long. Git to me seems Unixy. That's not really a word and doesn't mean much though. I suppose another way of saying it is: if you're used to working in unix like environments, it feels comfortable and familiar to you, and if you're not, it's never going to feel like anything other than overly complicated. If you've written a lot of bash scripts, or regularly tweaked your command line (i.e. bash settings), git feels natural. If you're not that kind of command-line user, git seems complex.

I would like to strenuously point out that I am in no way trying to offend users who don't like unix (or unix like operating system) or using the command-line in general. I have some strong personal feelings about it, but at the end of the day, it's personal preference. If you don't like using the command-line, then you don't like it. As an example, when talking through how to us a rest API with someone a month or so ago, I mentioned that they could just toss the URL into curl. I'm so used to reaching for the command-line as my first instinct, that I didn't even think that for someone not used to the command-line, it might seem complex. It's just curl -XGET http://someurl afterall.

But it's never that simple is it? The next question is, ok, how do I post data?

Hmm.....well, something like this:

curl -XPOST http://someurl -d "{"jsonstuff":"value"}"

Except, that won't work. You can't use double quotes like that. It has to be:

curl -XPOST http://someurl -d '{"jsonstuff":"value"}'

That will work. Well, maybe. You might need to set a json content header. If you're used to using a unix command, you just type man curl, look up the option and do:

curl -XPOST http://someurl -H "ContentType:application/json" -d '{"jsonstuff":"value"}'

But reading man pages can sometimes be an acquired skill. For those of us that are used to it, we might quickly notice the quote issue, or quickly find the option we're looking for with a quick man page search. However, if you just wanted to get something done quickly and you're not used to these types of tools, it makes a simple test of a rest API turn into an epic undertaking leaving you doing a google search for rest thick clients.

As another example, I recently wiped my mac machine. I hadn't used it in awhile, as I normally code on an Ubuntu machine. However, the stock git that came with the fresh install of OS X Lion was jarring. There was no colors set, and I didn't have my customery indicator in bash telling me what branch I was on. I needed to copy some of my bash config over from my other machine.

It's worth repeating, to effectively use git on the command line, I needed to modify my bash profile. (or bashrc, whatever you want to call it, depending on how you have things setup and distro, etc etc) For a regular command line user, it's not a problem. But I remember when I first was learning to use linux, and failing miserably to even add things to my PATH the first few times.

So I guess that's my main point. Git to me fits in well with all my other gnu/unix style command line tools. It feels familiar and as it follows these conventions, I can work with it. But as I have introduced it and other command line tools to fellow programmers, it becomes obvious to me: If you're not familiar with these conventions and with how to work with this type of command line, it's going to seem extremely complex.

Why developers sometimes do things 'the hard way'

I don't like using the a word. I never want to catch myself saying anything that could be construed as: 'methodology X says we should do it this way'. Especially when it comes to engineering practices. These kinds of things should stand on their own really. In fact, I'm going to try and steer clear of any specifics in this post and speak to a larger trend.

When working through an issue or a new feature, regardless of what environment you're coding in or what language you're using, there is probably a 'hard way' to work through the issue/feature. I don't mean this in terms of level of effort. I mean it takes longer for whatever reason. It may even be an uglier solution. I'll give an example. Let's say you're adding a new feature to a web application. It's not a big feature, maybe just adding the ability to search off of a new set of values. The easier way is to find some kind of quick feedback. A way to implement the new search and find out quickly if there's an issue. I'll stop short of saying 'write a test' for the reasons mentioned above. However, instead you make the change and fire up the application server. You try the simple feature and it doesn't work. You forgot something simple. You fix it, bounce the server, and try it again. Still doesn't work. Rinse and repeat ad infinitum.

To the outside observer, this certainly is 'the hard way' to do it. Because of the amount of time it takes to restart the web server and click through to test the feature. It takes more effort than some other means with a shorter feedback cycle. But to the person using this approach is it really 'the hard way'? The way they are doing it requires less thinking, and while they're waiting on the server to bounce back up they can check reddit, failblog, or have an impromptu sword fight. To put it another way, it takes longer but requires less effort.

Of course, at this point someone could start throwing out the word 'lazy'. But let's take a more sympathetic approach. Why are they doing it this way? Why do they want to think less? Maybe they're burnt out? Maybe they feel the project is out of their control and there is no reason to put in a lot of effort. That its better to save their brain power for the new programming language or framework they're learning in their spare time. Maybe they aren't getting enough sleep. They probably know deep down that its not really the most efficient way to do it, but they're tired. Emotionally, physically, or even subconsciously. I've certainly felt that way before.

Maybe the approach isn't convincing your developers to do things the more efficient way. Maybe its better to be more sympathetic and find out why they're doing it the easy way.

Open Symphony, RIP (2000 - 2011)

It looks like the folks over at open symphony have declared the project dead. The main projects that originated there and are still in use have their own separate sites now, but the rest hadn't been updated in awhile. I remember using OSCache awhile ago, but it seemed to have been completely replaced by EHCache.

The question in my mind is, do organization like OpenSymphony need to exist anymore? (perhaps the same could be said for something like CodeHaus as well) Given sites like github and bitbucket, and the ease of getting a project website up and running via various mechanisms, perhaps they aren't needed anymore? The one exception to that of course is community built around a particular project. Like plugins for Grails, or some of the Spring sub projects. They're tied to a particular solution and can benefit from some of the success of the solution as well. Although, in the case of Grails plugins, they're all hosted on GitHub, as is the code for grails itself.