On Git Command Line Usability

Today I was reading about a new tool to use Git on a Mac called GitBox. I generally prefer the command-line, but as it was on sale and mentioned as one of the best clients for Mac, I figured I would at least take a look. When looking through the comments though, one part of a particular review stuck out to me:

This app generally is a relieve compared to the terribly complicated git command line

It's a common refrain from even a seasoned developer that has been using other tools for awhile, even those familiar with subversion on the command-line. To simplify the discussion, let's take one part of the obvious usability issues out of the general discussion of git usability: DVCS in general. If using something like Hg or Git after using a more centralized VCS, it's going to be more complicated by nature. With DVCS you're effectively managing your own repo and merging with another common repo shared with the rest of your team. With a more centralized system like SVN you're really just pushing change lists to a repo managed by someone else. I'm sure there's some power users at every company, but I'm betting a lot of SVN users aren't doing a lot of branch merging. Even looking at GitBox, which does look like a more simplified view of Git, the overall actions you're performing are more complicated than say, something like tortoiseSVN.

DVCS has it's advantages and disadvantages, but given that the overall model is more complex in nature for individual developers committing code, let's shelve that and focus solely on the question of whether the git command line itself is overly complicated. (as the review described)

My gut response to this question is 'it depends'. Perhaps I was a consultant for too long. Git to me seems Unixy. That's not really a word and doesn't mean much though. I suppose another way of saying it is: if you're used to working in unix like environments, it feels comfortable and familiar to you, and if you're not, it's never going to feel like anything other than overly complicated. If you've written a lot of bash scripts, or regularly tweaked your command line (i.e. bash settings), git feels natural. If you're not that kind of command-line user, git seems complex.

I would like to strenuously point out that I am in no way trying to offend users who don't like unix (or unix like operating system) or using the command-line in general. I have some strong personal feelings about it, but at the end of the day, it's personal preference. If you don't like using the command-line, then you don't like it. As an example, when talking through how to us a rest API with someone a month or so ago, I mentioned that they could just toss the URL into curl. I'm so used to reaching for the command-line as my first instinct, that I didn't even think that for someone not used to the command-line, it might seem complex. It's just curl -XGET http://someurl afterall.

But it's never that simple is it? The next question is, ok, how do I post data?

Hmm.....well, something like this:

curl -XPOST http://someurl -d "{"jsonstuff":"value"}"

Except, that won't work. You can't use double quotes like that. It has to be:

curl -XPOST http://someurl -d '{"jsonstuff":"value"}'

That will work. Well, maybe. You might need to set a json content header. If you're used to using a unix command, you just type man curl, look up the option and do:

curl -XPOST http://someurl -H "ContentType:application/json" -d '{"jsonstuff":"value"}'

But reading man pages can sometimes be an acquired skill. For those of us that are used to it, we might quickly notice the quote issue, or quickly find the option we're looking for with a quick man page search. However, if you just wanted to get something done quickly and you're not used to these types of tools, it makes a simple test of a rest API turn into an epic undertaking leaving you doing a google search for rest thick clients.

As another example, I recently wiped my mac machine. I hadn't used it in awhile, as I normally code on an Ubuntu machine. However, the stock git that came with the fresh install of OS X Lion was jarring. There was no colors set, and I didn't have my customery indicator in bash telling me what branch I was on. I needed to copy some of my bash config over from my other machine.

It's worth repeating, to effectively use git on the command line, I needed to modify my bash profile. (or bashrc, whatever you want to call it, depending on how you have things setup and distro, etc etc) For a regular command line user, it's not a problem. But I remember when I first was learning to use linux, and failing miserably to even add things to my PATH the first few times.

So I guess that's my main point. Git to me fits in well with all my other gnu/unix style command line tools. It feels familiar and as it follows these conventions, I can work with it. But as I have introduced it and other command line tools to fellow programmers, it becomes obvious to me: If you're not familiar with these conventions and with how to work with this type of command line, it's going to seem extremely complex.

Why developers sometimes do things 'the hard way'

I don't like using the a word. I never want to catch myself saying anything that could be construed as: 'methodology X says we should do it this way'. Especially when it comes to engineering practices. These kinds of things should stand on their own really. In fact, I'm going to try and steer clear of any specifics in this post and speak to a larger trend.

When working through an issue or a new feature, regardless of what environment you're coding in or what language you're using, there is probably a 'hard way' to work through the issue/feature. I don't mean this in terms of level of effort. I mean it takes longer for whatever reason. It may even be an uglier solution. I'll give an example. Let's say you're adding a new feature to a web application. It's not a big feature, maybe just adding the ability to search off of a new set of values. The easier way is to find some kind of quick feedback. A way to implement the new search and find out quickly if there's an issue. I'll stop short of saying 'write a test' for the reasons mentioned above. However, instead you make the change and fire up the application server. You try the simple feature and it doesn't work. You forgot something simple. You fix it, bounce the server, and try it again. Still doesn't work. Rinse and repeat ad infinitum.

To the outside observer, this certainly is 'the hard way' to do it. Because of the amount of time it takes to restart the web server and click through to test the feature. It takes more effort than some other means with a shorter feedback cycle. But to the person using this approach is it really 'the hard way'? The way they are doing it requires less thinking, and while they're waiting on the server to bounce back up they can check reddit, failblog, or have an impromptu sword fight. To put it another way, it takes longer but requires less effort.

Of course, at this point someone could start throwing out the word 'lazy'. But let's take a more sympathetic approach. Why are they doing it this way? Why do they want to think less? Maybe they're burnt out? Maybe they feel the project is out of their control and there is no reason to put in a lot of effort. That its better to save their brain power for the new programming language or framework they're learning in their spare time. Maybe they aren't getting enough sleep. They probably know deep down that its not really the most efficient way to do it, but they're tired. Emotionally, physically, or even subconsciously. I've certainly felt that way before.

Maybe the approach isn't convincing your developers to do things the more efficient way. Maybe its better to be more sympathetic and find out why they're doing it the easy way.

Open Symphony, RIP (2000 - 2011)

It looks like the folks over at open symphony have declared the project dead. The main projects that originated there and are still in use have their own separate sites now, but the rest hadn't been updated in awhile. I remember using OSCache awhile ago, but it seemed to have been completely replaced by EHCache.

The question in my mind is, do organization like OpenSymphony need to exist anymore? (perhaps the same could be said for something like CodeHaus as well) Given sites like github and bitbucket, and the ease of getting a project website up and running via various mechanisms, perhaps they aren't needed anymore? The one exception to that of course is community built around a particular project. Like plugins for Grails, or some of the Spring sub projects. They're tied to a particular solution and can benefit from some of the success of the solution as well. Although, in the case of Grails plugins, they're all hosted on GitHub, as is the code for grails itself.

Grails testing improvements

Peter Ledbrook put up a new blog post detailing some of the improvements to come 1.4 for unit testing grails: http://blog.springsource.com/2011/06/07/countdown-to-grails-1-4-unit-testing/

There's some awesome stuff in there. I had read a bit about the new features in milestone 1. But it didn't show the kind of details that Peter has. I'm especially excited about these two features:

1. Full Gorm testing. It looks like they're using some kind of noSQL database in-memory. I'm interested to see if it will slow down the unit test at all, but the method of testing is much more intuitive. The whole mockDomain(Class,listOfInstances) felt a bit dirty, and it was kind of brittle if you used any kind of inheritance at all.
2. Improved JSON response testing. Grails now does the work in the mock response of converting the JSON response to a proper object hierarchy. In my own testing, I could get halfway there by converting the responseText to JSONObjects, but they were a bit painful to work with at times. (especially arrays)
   

Those two fixes represent for me the only real issues I have with grails unit testing. The annotations are nice, but I had given up on the test hierarchy long ago. You can use MockUtils easily in the same way that GrailsUnitTest does.

That leaves only one major complaint for me regarding testing and Grails: Integration testing. Its nice that they found ways to allow for using unit testing more often, but it doesn't solve the fundamental problem. Integration testing takes too long because you have to start the world. Lately I've been working a lot with ElasticSearch, and I refused to put it in the Integration test folder and rely on the bootstrap. In the test setup I brought up an in-memory ElasticSearch instance and test against it. Its definitely an integration test, but I can run it in about 6 seconds. It takes longer than that for Grails to resolve dependencies. (Which is something I hear they're addressing in 1.4) Its all perfectly fine for CI, but you can't use it to test on the fly. Its far too tempting to do a run-war and fiddle with it that way. It seems like it wouldn't be too difficult to bring up a persistent integration testing environment. In truth, it would probably still be some kind of headless container that you could keep running integration tests in without having to restart the container. I've seen a few other Grails bloggers mention it though, so perhaps its on the roadmap somewhere? Maybe post 1.4?

I'm speaking at Chicago Groovy User Group on June 14th

Details can be found here: http://cgug.org/

And you can sign up here: http://cgug.eventbrite.com/?ref=enivtefor?amp;utm_source=eb_email&utm_media=email&utm_compaign=invitefor&utm_term=readmore

Fundspire is hiring

The title says it all. Things are going very well here at Fundspire, and we're looking to expand. You can find more information here: http://www.fundspire.com/careers/

Just because I can be a bit more narrative here. We're using the following technologies: Grails, Ext Js, Spring Batch running on Amazon EC2. Of course, it would be great to find someone with experience in all of those technologies, but it never works out that way. A solid Java developer experienced with Spring and Hibernate would also be great. And of course the most important skill of all, a developer that tests the code they write, not with a debugger, but by writing tests.

If you think you might be interested (or know someone who might be) please apply via the link above, or send your resume to jobs@fundspire.com

Moving the Mysql Data Directory? Beware of AppArmor

Recently, my 8 year old linux server I was running mysql on died. My new computer is more than capable of running mysql in the background, but I didn't want to mess with moving everything over. Now I'm forced to. It's extremely easy on Ubuntu to install mysql. However, I run my operating system and all development related applications off of a 64GB SSD. There isn't room, or a need to run it on the SSD. I have a second platter based hard drive as well. It should be fairly straightforward to store MySql data there. Most instructions for doing so will tell you to do the following:

1. Copy the current data directory to your new partition. (Current directory is usually /var/lib/mysql on Ubuntu)
2. Make sure it is still owned by the mysql user. 'chown -R mysql:mysql newdatadir'
3. Change the mysql configuration to point to the new directory. Inside of my.cnf is a value datadir, which is what needs to be changed to the new directory.

Sounds easy, right? It is, for the most part. If you try these steps on ubuntu, it won't work. MySql won't start up. If you're like me, the first thing you'll look is syslog, where you'll find something like:

[1064644.473299] type=1400 audit(1305502269.192:364):
apparmor="DENIED" operation="open" parent=1 profile="/usr/sbin/mysqld" 
name="/media/data1/mysql-data/" pid=24578 comm="mysqld" requested_mask="r" 
denied_mask="r"fsuid=1001 ouid=1001

Your first instinct is to check the permissions, but they're all fine. So, what's going on? AppArmor is preventing access. If you type 'ls -la /etc/apparmor.d' you'll see the following:

drwxr-xr-x   8 root root  4096 2011-05-15 19:18 ./
drwxr-xr-x 141 root root 12288 2011-05-15 18:20 ../
drwxr-xr-x   3 root root  4096 2011-04-11 18:38 abstractions/
drwxr-xr-x   2 root root  4096 2011-05-15 17:38 cache/
drwxr-xr-x   2 root root  4096 2010-10-07 11:06 disable/
drwxr-xr-x   2 root root  4096 2010-08-06 23:19 force-complain/
-rw-r--r--   1 root root   986 2010-09-13 03:07 gdm-guest-session
drwxr-xr-x   2 root root  4096 2011-05-15 17:38 local/
-rw-r--r--   1 root root  2052 2010-08-06 23:18 sbin.dhclient3
drwxr-xr-x   3 root root  4096 2011-04-11 18:38 tunables/
-rw-r--r--   1 root root  2052 2010-09-27 17:58 usr.bin.evince
-rw-r--r--   1 root root  4006 2011-04-29 04:00 usr.bin.firefox
-rw-r--r--   1 root root  4158 2010-10-01 04:58 usr.sbin.cupsd
-rw-r--r--   1 root root   989 2010-11-10 00:51 usr.sbin.mysqld
-rw-r--r--   1 root root  1172 2010-08-06 12:45 usr.sbin.tcpdump

If you open up usr.sbin.mysqld, and add the new directory to the list, your problems will be solved. Something like:

 /mysql-data/ r,
/mysql-data/** rwk,