Thursday, May 27, 2010

Oracle pushes first version of Lambda/Closures implementation

Test cases may not be the best source of coding gems, as they're probably testing the weirder stuff.  But you can now see some test cases, and therefore the evolving syntax here.

Oracle has a tough task in adding closures to a language that has checked exceptions, minimal type inference, and without breaking compatibility of zillions of lines of existing code.  Go Snoracle!

Monday, May 17, 2010

Some JUnit tricks for easier and better test cases

Whenever I start a new project, the first thing I'll write is a stack trace filterer/clarifier.  The second would be several enhancements to JUnit.  For test driven development to be effective, you need good coverage and lots of tests.  For that to happen, tests need to be dead-simple to write, easy to maintain, and fast.  The following enhancements have really helped me towards these goals.

1. Easy Fixtures - Many bugs/regressions will involve a set of complicated data relationships.  Create an XML/JSON/whatever format to easily import large sets of data.  For example, let's say bug #535 comes in: "Exception is thrown when trying to delete users when they are in a group".  You can quickly go out, and create 535.xml:

Then, your test case could be:

Without an extremely easy way to create test fixtures and import them, 99% of us will write far less tests.

2. Automatic transaction handling and application bootstrapping.  Ideally, the example test method I gave above should be all it takes to run the test.  No explicit setUp, tearDown for bootstrapping your environment, setting up transactions, etc.  It is worth investing your time in some means (maybe a AbstractTest superclass?)  to set these things up automatically for the 95% of test cases that require no special environmental setup.  Don't Repeat Yourself.

3. More powerful Set and List comparison - When I do assertEquals(expected, actual) where "expected" and "actual" are Collections with dozens of elements, the last thing I want to hear that they are not equal, maybe with a toString of both.  So I always implement a assertSetsEqual and assertListsEqual.  assertSetsEqual compare the two collections, where order doesn't matter.   Here's a sample of assertSetsEquals:

The List comparison, where order matters, requires more work to produce a nice message, but it's worth it:

Expected: [a, b, c, d, e, f, g, h, i, j, k]
Actual:   [a, b, c, e, f, g, h, i, j, k, l]
Missing 'd' @ 3
Unexpected l @ 10

4. Type-checking assertEquals - Due to limitations of Java's type system, assertEquals takes two parameters of type Object.  There can be no compile-time checking that the two arguments are actually of types that can actually have equal objects.  For example:

I discovered org.junit.Assert as I wrote this, much better than the junit.framework.  Why is there two?  Just printing the types goes a long way.  I'm surprised how often an assertEquals LOOKS right, but it happily returns false because you are passing incompatible types in.

5. An Eclipse template for creating a new test case.

Just laziness mostly, but I do think it's important to have that fail("No Asserts") in there from the start.  Too often I'll write a test case as I'm exploring a problem, but then I get distracted and abandon it before I really nail down the asserts.  Without asserts, your test case is mostly just slowing things down.

These little changes seem like overkill in the context of writing a silly little test case.  But if your goal is to write hundreds, I found these tricks to be really worth the investment.

Update #1: In the comments, Crias points out that junit.framework is for backwards compatibity with JUnit 3 and earlier and should be avoided if possible.  I think a custom assertEquals which uses reflection determine if the two arguments even can be equal is worthwhile,  but the new org.junit goes a long way just by printing class names.

Update #2: In the comments, David Karr correctly points out the the test runner will print any exception that bubbles out a test method created with my template.  The reason I took this weird approach is the the Eclipse JUnit runner does not output the trace to the console but to the JUnit View, which I find is a goofy place to inspect the exception.  More importantly, the Eclipse runner won't filter my stack traces, which I'm addicted to.

Update #3: Tym The Enchanter points out that Hamcrest helps with my #3 and #4 points about better asserts.  I know very little about Hamcrest, but here's my naive reaction:

  1. is/equalTo for safer equal to comparisons.  These basically force the type of the "actual" to be a subclass (or the same class) as the "expected".  This goes a long way to reducing dumb-dumb comparisons like new Integer(2) and "2".  In my experience, these are exactly the kinds of comparison errors that slow you down.  It's not perfect, however - it's quite possible that the compile time type of your expected is not a subclass of the actual, but they ARE assignable - see the "Garfield" example below.  Of course you can fall back to more traditional assertEquals if it bites you, and I'm unsure how often I'd see this in real life.  
  2. Set/List comparisons - For sure, Hamcrest beats JUnit asserts here.  However, these comparisons basically fail fast - they only tell you the first problem they find - which could lead you to running the test several times before you get it right.  Tell me all of the problems right off the bat! 

Thanks for the comments!