Dean Wampler's Blog

I decided to move my website for Contract4J from contract4j.org to polyglotprogramming.com/contract4j.

By the way, I’m looking for someone who would like to take over the project. While it’s mostly feature complete, there are some major performance improvements that could be made. Let me know if you’re interested.

This is looong overdue, but you can now manage dependencies on Contract4J5 using Maven. The details are here. Also, I moved the code from a private subversion repository to GitHub.

Contract4J5 is my Design by Contract library for Java. I haven’t worked on it in a while, but I hope to return to it sometime this year to solve the one issue that keeps it from being practical for many teams: performance. Currently, the runtime overhead slows down tests considerably. It is already feature complete and stable.

The Maven bundle includes the all the sources and javadocs, as well as the runtime jar file. However, it is still built with Ant. If you want to work with the source code, clone or fork the GitHub repo.

I just released V0.8.0 of Contract4J5. This release deprecates support for JRuby (for reasons discussed in the README) and provides general robustness improvements, especially in the area of test expression evaluation. My plan is to complete all the remaining "1.0" features by the 0.9 release towards the end of the year. I hope to (finally) release 1.0 early next year. Please visit the Contract4J web site for more information.

A Contract4J5 user reported performance problems using Groovy. After some investigation, it appears that using the Bean Scripting Framework (BSF) increases the overhead of script evaluation by a factor of 3 or so! The overhead for script evaluation is already very high (roughly a factor of 7-10 times slower compared to no contracts). Scripting is expensive if used heavily, as it is in C4J!! I'm not sure why BSF's overhead is so high, but I'm going to implement support for Groovy and JRuby that bypasses BSF. This may take a while to complete, as I have a busy few weeks ahead. The older non-BSF integration with Jexl is still in the distribution (although it's currently marked as deprecated). If you want to use it, put the following code in your main or similar initialization methods:

1 2 3 4

Contract4J c4j = Contract4J.getInstance(); ContractEnforcer ce = c4j.getContractEnforcer(); ce.setExpressionInterpreter (new JexlExpressionInterpreter());

Correction: You can also use the properties file configurator or the Spring configuration file options to specify Jexl without the BSF wrapper. You'll get warnings that JexlExpressionInterpreter is deprecated. You can safely ignore them. I'll remove that javadoc tag in the next minor update. Longer term, I'm trying to improve the use of caching to further reduce the overhead of C4J.

I just released v0.7.0 of Contract4J5, which now supports Groovy and JRuby as alternative scripting languages, in addition to Jexl, the original language used. Along the way, I collected some performance numbers for the three alternatives.

The following discussion is adapted from the README for the v0.7.0 release.

I ran the JUnit test suite with all three languages. The numbers below show the performance when using binary weaving and load-time weaving (LTW) and also the performance using the JVMs in JDK 5 and JDK 6.

JDK 5 (sec.)						JDK 6 (sec.)
Jexl		Goovy		JRuby		Jexl		Goovy		JRuby
Binary	LTW	Binary	LTW	Binary	LTW	Binary	LTW	Binary	LTW	Binary	LTW
21.7	98.6	54.9	324.4	79.6	189.4	17.7	62.9	44.8	133.5	63.2	108.4

These times are approximate "user times", averaged over a few runs, measured on a ThinkPad T42 running Ubuntu Linux. There are slightly different build activities and I/O overhead involved in the LTW numbers, adding a few percentage points to the numbers. The tests do some different manipulations depending on which interpreter is used. Hence, the results are rough, at best. Most likely, the numbers reflect the relative amounts of overhead to load the interpreters and to set up the "environments" for evaluating the scripts, which happens frequently in the test suite. Since the scripts are always very small, evaluation is probably not a large percentage of the execution time. (However, this is speculation!) Your mileage may vary...

Note that the JDK 6 performance is significantly better. The code was compiled with the JDK 5 javac and executed using the JDK 6 JVM. Ironically, the performance was slightly slower when compiled with the JDK 6 javac. (This is not reflected in the numbers)

The Jexl test runs are roughly twice as fast as the Groovy and JRuby runs, probably because Jexl is a more "minimalist" environment, incurring less startup and execution overhead.

While not shown, I observed that the memory requirements for Groovy and JRuby are also higher, as you would expect. I didn't actually measure memory usage, but I observed this because it was necessary to increase the maximum heap size for the LTW JUnit run when using Groovy or JRuby. Also, when using JRuby, I had to increase the heap size for the binary-weaving JUnit runs in Eclipse.

The LTW test runs take significantly longer, but the results are somewhat hard to interpret. Under JDK 5, the Jexl and Groovy runs are five to six times longer, while the JRuby runs are only about 2.5 times longer. Under JDK 6, the Jexl and Groovy runs are a little more than three times longer, while the JRuby run is under twice as long. It is not clear why JRuby LTW configurations perform so much better under both JDKs. However, it is clear that, for all the interpreters, the startup byte-code instrumentation required for LTW is much better under JDK 6.

In general, the longer LTW times occur because a weaving step happens at the beginning of each TestCase, as the class is loaded. The JDK 6 speed-up is a good argument for running this VM, at least for your tests!

LTW is the most convenient way to adopt Contract4J5 (true for other aspects, as well), but if the startup time is too long for your needs, then binary weaving of compiled code provides better performance with only a modest change in your build process.

I just released v0.7.0 of Contract4J5. This release adds support for using Groovy or JRuby instead of Jexl as the scripting engine for the test scripts embedded in annotations. In fact, Groovy is now the default.

The advantage of Groovy or JRuby is that they provide a richer environment for more sophisticated testing. They are also better suited for planned, long-term enhancements of Contract4J5 to support user-configurable annotations and behaviors, which will take the tool beyond Design by Contract support.

The only major disadvantage over Jexl is lower performance. I'll blog about those results next. You can also find details in the README.

I was pleased to find that all but a few of the existing Jexl-based unit tests worked without modification for Groovy. With a few hacks, I got them to work for JRuby as well. The differences for Groovy include slight differences in how Goovy vs. Jexl handle protected access (private, protected, etc.). Also, Groovy does not force you to provide a public accessor to evaluate tests on member fields, which Jexl required.

For JRuby, I found that almost all Ruby vs. Java differences could be handled if I did a few simple replacements of the most common Java idioms in scripts with the corresponding Ruby idioms:

• Replace equals(...) with eql?(...)
• Replace compareTo(...) with <=>(...)
• Replace null with nil

That is, Contract4J5, when using JRuby, will make these substitutions in your test expressions so they work, most of the time, even though they are written in Java syntax. This tends to work because the test expressions tend to be relatively simple and they often consist of comparisons to null or calls to String comparison methods.

After a longer-than-expected effort, I'm pleased to announce the v0.6.0 release of Contract4J5. The API for writing contract tests has not changed, but the internals have been restructured considerably to improve the code quality and make it easier to configure Contract4J5 using property files and Spring Framework Dependency Injection. The release also provides examples of using Contract4J5 with binary weaving or load-time weaving. The latter is now the preferred way to introduce Contract4J5 into a "pure-Java" environment. For more details, see the Release Notes in the README or browse to http://www.contract4j.org.