Thursday, March 24, 2011

Excluding jars from a Grails generated war file

Yesterday, I posted about how to upgrade hibernate versions in grails. This solution worked great for me locally. (i.e. using run-app) But we were still getting weird issues in our staging environment. It's not uncommon for this kind of thing to happen, and I used my local tomcat setup to try and nail down the problem. When I cracked open the WAR, I found two version of hibernate, 3.3.1 and 3.3.2. Upon closer inspection of the problem, it turned out to be the same original issue I had upgraded Hibernate to fix, somewhat hidden by some additional 'noise' in the log file. I tried about 10 different things to get the war to generate correctly, until I stumbled onto this blog post:

http://www.anyware.co.uk/2005/2009/01/21/excluding-files-from-a-war-with-grails-the-right-way/

Grails gives you the option to declare a closure to modify the war resources. I added the following to the top of my buildConfig.groovy, and it fixed the issue:


grails.war.resources = { stagingDir ->
delete(file:"${stagingDir}/WEB-INF/lib/hibernate-core-3.3.1.GA.jar")
}

Wednesday, March 23, 2011

Upgrading Grails 1.3.x to use Hibernate 3.3.2

While using Hibernate Envers, I ran into an issue that was fixed in Hibernate 3.3.2. Unfortunately, Grails is using Hibernate 3.3.1. Luckily, it's fairly straightforward to exclude and declare a new dependency in your BuildConfig.groovy:


inherits("global") {
excludes 'hibernate'
}
dependencies {
compile ('org.hibernate:hibernate-core:3.3.2.GA'){
excludes 'ehcache', 'xml-apis', 'commons-logging'
}
}
Update: turns out it's not as straightforward as I thought. The above will work fine with 'grails run-app' but the 3.3.1 version will still be in the War file: http://www.lucasward.net/2011/03/excluding-jars-from-grails-generated.html You can fix that by adding the following to your BuildConfig.groovy:


grails.war.resources = { stagingDir ->
delete(file:"${stagingDir}/WEB-INF/lib/hibernate-core-3.3.1.GA.jar")
}


Unfortunately, in the 3.3.2 release, "“org.hibernate.dialect.DialectFactory” was moved to “org.hibernate.dialect.resolve.DialectFactory”. Which was part of the following issue: http://opensource.atlassian.com/projects/hibernate/browse/HHH-2933

This is apparently an issue many have been seeing: http://grails.1312388.n4.nabble.com/update-hibernate-version-td3002449.html

While it may seem crazy that the Hibernate team moved something to a new package in a point release. I'm pretty sure they consider this class to be internal and not part of their API. The issue the change was made in was to expose to users a way to resolve their own dialects programmatically. Which, if you look at where the Grails Hibernate plugin is creating the offending spring bean, is the same thing Grails is doing:


if (ds && ds.dialect) {
if (ds.dialect instanceof Class) {
hibProps."hibernate.dialect" = ds.dialect.name
}
else {
hibProps."hibernate.dialect" = ds.dialect.toString()
}
}
else {
dialectDetector(HibernateDialectDetectorFactoryBean) {
dataSource = ref("dataSource")
vendorNameDialectMappings = vendorToDialect
}
hibProps."hibernate.dialect" = dialectDetector
}


The above is taken from HibernatePluginSupport.groovy, line 128. (At least in Grails 1.3.6) Essentially they've written a Spring bean to programmatically determine the dialect by calling the DialectFactory themselves. You can bypass this issue completely by specifying your dialect, rather than having it detected. If you're using mysql, it looks something like:


dataSource {
driverClassName = "com.mysql.jdbc.Driver"
dialect = "org.hibernate.dialect.MySQLDialect"
}

Thursday, March 17, 2011

The Programmer's Time Paradox

At the beginning of this week I started work on a new feature. For the purposes of this post it doesn't matter what the feature was, but I started in on it on Monday and things went extremely well. I was in the zone. It's hard to describe, but I think if you're a programmer you'll understand the feeling. I started out with just the right amount of testing in the right places, built something simple and added on top of it, while refactoring periodically as I discovered new angles on the problem. Everytime I moved forward, I was able to run my tests and tell immediately that my last fix had broken something. I was moving forward in small steps and it was all falling together nicely.

And then I got greedy. I suppose it makes me a dork to admit it, but I was having fun. I had been somewhat behind on the feature because I was debating on a couple different paths, but now that I had chosen, I was moving forward with wreckless abandon. It's somewhat intoxicating to be in the zone. So, I pushed myself hard. I worked from around 9 AM until 1 or 2 in the morning every night for 3 days straight. (I work from home, so it's a bit easier to work later like that)

This morning I woke up begrudgingly. The comforter was very, very heavy. It was the weird programmer kind of fatigue, where you've done nothing but sit on your but all day, but your mind has been going too fast for too long, so you feel mentally exhausted but physically fine. When you hit that point, no amount of coffee is going to make it better. Now, I've been on some pretty hellish death marches before. Some of them spanned months, so it's not like working hard for 3 days is all that terrible. But the programming gods are vengeful. I might have been tired, but I was excited to get the feature working. I was in the home stretch. My sister is coming to town this weekend and it's her first time in Chicago since turning 21, so I wanted to get this thing done and party like it was...2003?

So I got sloppy. I ran into a weird bug. It was a NullPointerException with no stack trace. Which sometimes interchanged itself with an InovcationTargetException. When you see something like that happen, it's time to stop the production line. I should have stopped, immediately, and attempted to recreate the conditions in my testing setup. I had a perfect, isolated, test suite setup where I could have recreated this thing in a controlled environment, but I didn't do that. Instead, I fiddled with idea after idea. They were all well informed, reasonable things to be trying. But when you're doing this kind of thing inside your application, there can be too many factors. Production code is complicated, and there's just too many variables.

At around 6PM I stopped myself, and did what I should have done in the first place. In an integration test, I slowly recreated the problem, piece by piece. Within 15 minutes I had found the problem. I forgot to put an annotation on a particular class.

I. forgot. an. annotation.....$#$%#$%#$!!!!!!!!!!!

I suppose I could get mad at the guy who wrote the open source framework I was using. But I would be one heck of a hypocrite if I did. I can't imagine how many people have been in the same situation with Spring Batch, and probably felt the same way about me. I hit a scenario he didn't think of, so there wasn't specific error handling for it, and the results were nondeterministic.

No, I was tired. I pushed myself too hard and I made a stupid mistake, and the funny thing is, I lost all of the time I had saved by working so much! Awesome.

And when has that not been the case? I can't think of time where I worked too many hours and this exact same thing hasn't happened. I was tired and made a stupid mistake. A mistake that I would have easily caught if fully rested. And that mistake had cost all the time I had spent working late. And it's compounded by the number of people overworking themselves. The more people doing it, the worse the effect is. Because your stupid mistake made while trying to fix someone else's stupid mistake quickly snowballs.

But sometimes, it's just hard to grok. I don't think the human mind is fully capable of thinking that way. If you're running somewhere, running faster gets you there quicker, baring the limits of endurance. But overall, if you can run faster, running faster will get you there sooner. Or at least is seems that way when you're coding at 1 in the morning on an interesting problem. And I know all about sustainable pace, and I agree with it fully. And I wasn't under the spell of an evil project manager either. It's a small company, and I'm the CTO. The only drive I have is to get features done quicker to support our sales efforts. And I know, from personal experience, what happens when programmers work unsustainable paces.

You know what it's like? It's like white castle. At least once or twice a year I'll decide eating at White Castle is a good idea. I may or may not be sober when I make this decision, but I make it anyway. And I pay for it. The next day you wake up and you *know* you made a terrible mistake. And for days, maybe even weeks, you'll swear to never eat at white castle again. But then you'll be at your cousin's wedding and there just so happens to be a white castle within walking distance, and the terrible cycle repeats itself.

Wednesday, March 9, 2011

Grails Testing issue when rendering as 'text/json'

I ran into this issue while unit testing my controllers (many of which only return JSON) in Grails. I've reported the issue to grails: http://jira.codehaus.org/browse/GRAILS-7339 but it's also worth noting here, so anyone else left scratching their head as to why the rendering doesn't appear to be working in their unit tests can find the workaround as well:

If you are writing a unit or integration test in grails and mocking the controller, which can be done either by extending ControllerUnitTestCase (or using mockController) you might run into this issue. As you probably know, there are multiple ways to render JSON in grails. One is to take a normal map and render as JSON:


def jsonMap = [success:true]
render jsonMap as JSON


This will work fine everywhere, because you're effectively passing it in as a converter. However, if you use the more verbose (and in many ways more powerful) rendering, you will run into issues:


render(contentType: "text/json") {
success(true)
results {
result.each {
def period = it.getPeriod()
b(id: it.id,name: it.name)
}
}
}


In my case, my test looks like this:


assert JSON.parse(this.controller.response.contentAsString)?.success == true


This won't work, because the content on the mock response will be null. With a little digging into how mockController() works, you will find the problem lies in MockUtils. Specifically, in the method that handles a map with render options and a closure:


clazz.metaClass.render = {Map map, Closure c ->
renderArgs.putAll(map)

switch(map["contentType"]) {
case null:
break

case "application/xml":
case "text/xml":
def b = new StreamingMarkupBuilder()
if (map["encoding"]) b.encoding = map["encoding"]

def writable = b.bind(c)
delegate.response.outputStream << writable
break

default:
println "Nothing"
break
}
}


As you can see, the case of 'text/json' isn't handled. I was able to workaround this issue by creating a custom unit test case class to extend from the Grails ControllerUnitTestCase:


class CustomControllerTestCase extends ControllerUnitTestCase{

protected void setUp() {
super.setUp()
controller.class.metaClass.render = {Map map, Closure c ->
renderArgs.putAll(map)

switch(map["contentType"]) {
case null:
break

case "application/xml":
case "text/xml":
def b = new StreamingMarkupBuilder()
if (map["encoding"]) b.encoding = map["encoding"]

def writable = b.bind(c)
delegate.response.outputStream << writable
break

case "text/json":
new JSonBuilder(delegate.response).json(c)
break
default:
println "Nothing"
break
}
}
}

}


Since it's groovy, you could pretty much replace this anywhere by overriding your mock controller's meta class.

Disabling Spring Security for grails integration testing

One of the great things about Grails is how easy it is to do integration testing. When running the integration tests, the grails container will be started up, including an in-memory version of hsqldb. (and it looks to go to h2 in 1.4) When you combine that with controller mocking, you can easily test from you controller down to the database. However, because you're getting your entire system, minus only the web server, it can take a while to startup. It's not too bad for a CI build, but can get annoying when you're writing and running tests in your development environment. One of the big culprits in terms of time is spring security. I'm not sure why that is, but it is possible to disable it in your integration test environment. The first step is to set the active configuration property to false in Config.groovy:


test {
grails.plugins.springsecurity.active = false
}


Now, when you run 'grails test-app integration:' you should see something like this:


grails test-app integration:
Welcome to Grails 1.3.6 - http://grails.org/
Licensed under Apache Standard License 2.0
Resolving dependencies...
Dependencies resolved in 5123ms.
Environment set to test
Starting integration test phase ...
Spring Security is disabled, not loading


The important thing to notice is the last line: Spring Security is disabled, not loading

Of course, there's now a new problem, any code you have that is referencing springSecurityService will now fail. In general, the most likely method to cause an issue is getCurrentUser(). However, it's simple to create a quick stub for it:


class StubSpringSecurityService {

def currentUser

Object getCurrentUser() {
return currentUser
}

String encodePassword(String password, salt = null) {
return password
}

}


In my case, I only needed to stub getCurrentUser() and encodePassword, however, you may need to add more, and the general approach can be applied to any additional methods you may need. It is also worth noting that I debated about using something like stubFor, but decided it was more straightforward to make a simple stub.

Now that you have a stub, you need to make sure that it's being set in the test environment. You can do this by adding the following to resources.groovy:


if (GrailsUtil.environment == "test"){
springSecurityService(StubSpringSecurityService)
}


Now for the last step, setting up the current user in your bootstrap:


if (GrailsUtil.environment == "test") {
def testUser = new User()
testUser.save(failOnError: true)
springSecurityService.currentUser = testUser;
}


Any code running during an integration test, will now get back the saved user if it calls getCurrentUser()