Bushwhacker to help team members debug unit test failures

Right now I'm working on a project with a few people who have never worked in Java before. There have been times where they'll experience some problem that they can't easily figure out. They send me the stack trace and instantly I know what the problem is. I wanted a simple way to automate this.

So I created Bushwhacker a simple library to help provide more helpful messages at development time.  Senior members of the team edit and maintain a simple xml file that describes rules to detect particular exceptions and either replace or enhance the exception message in some way.  The rules are fairly sophisticated -- allowing you to, for example, only detect IllegalArgumentException if it was throw from a stack frame from MyClassWhatever.  Since the xml rules file is read off of the classpath, you can version the XML file along with your source code (in src/test/resources for example).  And you can even maintain multiple rules files for different modules and compose them all together.  So if there are common pitfalls in your corporate "commons" module you can keep the bushwhacker rules for it there, and put your team's additional rules in your module.

It's on Maven Central Repo so its easy to add to your project. Check out usage instructions here: https://github.com/steveash/bushwhacker

Here's a few of the rules that we're using right now:

Single record UPSERT in MSSQL 2008+

I guess I just don't use SQL's MERGE statement enough to remember its syntax off the top of my head. When I look at the Microsoft documentation for it, I can't find just a simple example to do a single record UPSERT. When I google, I don't get any brief blog posts that show it. So here's a brief one to add to the pile:

Table:

DATA_VERSION
datakey varchar(16) not null primary key,
seqno bigint not null,
last_updated datetime not null

The table just keeps track of sequence numbers for things identified by 'datakey'.  The first time we want to increment a value for a key we need to insert a new record with starting with seqno = 1 then every time after that we just want to increment the existing row.

MSSQL:

Spring Test Context Caching + AspectJ @Transactional + Ehcache pain

Are you using AspectJ @Transactionals and Spring? Do you have multiple SessionFactory's maybe one for an embedded database for unit testing and one for the real database for integration testing? Are you getting one of these exceptions?

org.springframework.transaction.CannotCreateTransactionException: Could not open Hibernate Session for transaction; nested exception is org.hibernate.service.UnknownServiceException: Unknown service requested 

or

java.lang.NullPointerException at net.sf.ehcache.Cache.isKeyInCache(Cache.java:3068) at org.hibernate.cache.ehcache.internal.regions.EhcacheDataRegion.contains(EhcacheDataRegion.java:223)

Then you are running in to a problem where multiple, cached application contexts are stepping on each other. This blog post will describe some strategies to deal with the problems that we have encountered.

Background

Spring's Text Context framework by default tries to minimize the number of times the spring container has to start by caching the containers. If you are running multiple tests that all use the same configuration, then you will only have to create the container once for all the tests instead of creating it before each test. If you have 1000's of tests and the container takes 10-15 seconds to startup, this makes a real difference in build/test time.

This only works if everyone (you and all of the libraries that you use) avoid static fields (global state), and unfortunately there are places where this is difficult/impossible to avoid -- even spring violates this! A few places that have caused us problems:

• Spring AspectJ @Transactional support
• EhCache cache managers

Aspects are singletons by design. Spring uses this to place a reference to the BeanFactory as well as the PlatformTransactionManager. If you have multiple containers each with their "own" AnnotationTransactionAspect, they are in fact sharing the AnnotationTransactionAspect and whichever container starts up last is the "winner" causing all kinds of unexpected hard to debug problems. 

Ehcache is also a pain here. The ehcache library maints a static list of all of the cache managers that it has created in the VM. So if you want to use multiple containers, they will all share a reference to the same cache. Spring Test gives you a mechanism to indicate that this test has "dirtied" the container and that it needs to be created. This translates to destroying the container after the test class is finished. This is fine, but if your container has objects that are shared by the other containers, then destroying that shared object breaks the other containers.

Solutions

The easiest solution is to basically disable the application context caching entirely. This can be done simply by placing @DirtiesContext on every test or (better) you probably should use super classes ("abstract test fixtures") to organize your tests anyways, in which case just add @DirtiesContext on the base class. Unfortunately you also lose all of the benefit of caching and your build times will increase. 

There is no general mechanism for the spring container to "clean itself up", because this sharing of state across container is certainly an anti-pattern. The fact that they themselves do it (AnnotationTransactionAspect, EhCacheManagerFactoryBean.setShared(true), etc.) is an indication that perhaps they should add some support. If you want to keep caching, then step 1 is making sure that you don't use any "static field" singletons in your code. Also make sure that any external resources that you write to are separated so that multiple containers can co-exist in the same JVM.

To address the AspectJ problem, the best solution that I have found is to create a TestExecutionListener that "resets" the AnnotationTransactionAspect to point to the correct bean factory and PTM before test execution. The code for such a listener is in this gist.

To then use the listener you put @TestListeners on your base class test fixture so that all tests run with the new listener. Note that when you use the @TestListeners annotation you then have to specify all of the execution listeners, including the existing Spring ones. There is an example in the gist.

The workaround for Ehcache is to not allow CacheManager instances to be shared between containers. To do this, you have to ensure that the cache managers all have unique names. This is actually pretty easy to configure.

Related Issues

Here are some links to spring jira issues covering this problem:
https://jira.spring.io/browse/SPR-6121
https://jira.spring.io/browse/SPR-6353