Groovy Development

About this Article

Unlike many of my previous articles, I decided not to make this a click-along. There is so much to talk about in the Groovy language that I did not want to get too sidetracked in explaining the SQL database setup, or creating a framework to demonstrate it all.

I can tell you, however, that this information is based on my own experience constructing such a framework for a project at work. We needed a quick and easy way to construct translators (importers and exporters, if you prefer) for getting data in and out of a database schema. In many cases, the export (or import) was to (or from) an XML file.

We could have used any number of pre-written XML tools for this purpose; for example, Oracle (the database in question in this case) has a number of XML processing options to easily get data in and out of a database, and there are many other options that have similar functionality. In the case of Oracle’s own tool, we wanted more database agnosticism. As for the other tools? Well, in the translators for this project, it was not simply case of mapping XML data straight into a waiting database schema. For one thing, the schema did not always resemble the XML data being imported (or exported). For another, there were numerous business rules used to calculate still more database or XML data.

To illustrate, the system constructed was a workflow system that interfaced with external agencies using various XML formats. Using the data in the database, an XML schedule file was formed and sent out to the company members, who would then examine it and provide a response, also in XML. The response had some identifying information relating to the data originally sent out, and then could contain a number of different responses. For one, a problem with the data could be signaled, and it might be specific to a certain series in the outgoing schedule, or even an individual value. For another, our system needed a certain “response code” field to be set based on the response data code in combination with the anomalies signaled.

For example, if the response code is G29, and one of the data series is marked as having inconsistent values, our response code is -210. Otherwise, if G29 is marked, but no data is identified as problematic, set the response code to -200 (really, these are made-up numbers, but hopefully they show the kind of business rule that was needed).

So, we are faced with the possibility of using a simple and fast mapping tool to get the data in, and then doing a separate post-processing step to apply all of the business logic, or of using a more sophisticated language for the whole process, with Java being high on the list because the rest of the application is already written in Java.

What Is Groovy?

Around the same time we were looking at this problem, I was following the development of a new language called Groovy (a name that is the language’s worst enemy, in my opinion, because I was the one who had to pitch the use of something called “Groovy” to write our translation engine and it was a hard sell because of the name alone). Groovy also can be referred to as JSR-241, which is far more boring, but also much more acceptable in a corporate setting.

Groovy is a scripting language that is largely compatible with Java syntax. (And when I say that, I mean that there are very few things you can write in Java that will break Groovy, one curious and notable exception being the for loop that has been simplified.) You can either interpret or compile Groovy, and compiled Groovy code can be packaged up into jars that look just like Java code, but will run slower than Java.

Why Use Groovy?

So, if the language has such similar syntax to Java and is slower, why not just use Java? Well, just because Groovy can understand straight Java code, it doesn’t mean that is all it can understand. Normal Java requires a lot of strong typing and compile time checking that mean that (unless you use reflection) many of the neat shortcuts that dynamic scripting languages are hard or require a lot of extra code (like reflection code).

Although Groovy will compile strongly typed code quite happily, you can also drop the typing, so that something like:

String

myStr = “Hello world”;

can become

myStr = “Hello world”;

Also, a method definition like:

public int myMethod(boolean flag, String greeting, double mult) {

can become

def myMethod(flag, greeting, mult) {

Also, things such as loops have been simplified a great deal:

for(int i=0; i<100; i++) {

becomes

for (i in 0..99) {

Ok, so nothing earth shattering so far, but Groovy doesn’t stop there. For one thing, it has all sorts of shorthand syntax that saves a lot of time. Some examples (but not all by far) include:

• Native list and map syntax
  
• Closures (you can think of a closure very loosely as a method with no name or formal definition—a code block, if you prefer)
  
• Excellent iteration syntax ( list.each(), for (i in 1..10) )
  
• Incredibly concise XML and SQL syntax support, including Markup

All of these features will be covered in this article, but it is this last point that really convinced me to to try Groovy for our translators. As it turned out, it was a very good choice.

Getting Groovy

OK, before going any further, you may be wondering where you can find Groovy.

Groovy’s home is http://groovy.codehaus.org/, and the latest version can be found at http://dist.codehaus.org/groovy/distributions/?M=D (the link is already sorted with most recent at the top).

Installation is easy; just choose the gz or zip file (depending on your preference—note also that you can pick up the src distros if you want to; but to start with, grab one that does not have src in the name) and unpack it onto your machine somewhere. I then recommend adding the bin directory to your PATH environment variable so that the machine can execute the compiler and shells. Try it from the command line by typing groovysh; you should see something like this:

Lets get Groovy!

================

Version: 1.0-jsr-03 JVM: 1.4.2_06-b03

Type ‘exit’ to terminate the shell

Type ‘help’ for command help

Type ‘go’ to execute the statements

Type exit to get out of it—you are ready to go Groovy.

The Console and JEdit

Groovy has a very useful tool, called the Groovy Console, that comes with it. You can start it by typing GroovyConsole at the command line and you should see a window come up with an upper and a lower pane. The upper pane can be used to type in Groovy code to try out, and the bottom pane is the groovy shell and shows execution traces. Note that any output to standard out or standard error will go not to the bottom pane, but to the standard output in the command line window.

The Groovy console is extremely useful. If you are trying to develop some snippet of useful, Groovy code, this is the place to do it. Once you have edited the code you want to run, just go to the Actions menu and select Run. It will either run the code snippet, or explain to you why it couldn’t. It’s very basic compared with a full Java IDE, but it is very useful to just try out some ideas.

In this example, you have some code that could be Java (except from the test part not needing to be enclosed in a public static void main() construct). If you run it, you should see “Hello Dr. Livingstone” written to the standard output in the command line you called GroovyConsole from.

I would also like to mention JEdit here; if you have never used JEdit, I strongly recommend it. You can download it from http://www.jedit.org/ and it has Groovy syntax coloring. Again, this is a far cry from a full IDE environment but those will be improved over time. In the meantime, I tend to use either JEdit or NetBeans with the Groovy plugin available from https://coyote.dev.java.net/ and there is also an Eclipse plugin for Groovy available at http://groovy.codehaus.org/Eclipse+Plugin.

Moving on to the real meat of the article… As a Groovy neophyte, the first problem I found was that the documentation is a bit sparse at the moment. It is for this reason that I want to delve into at least those features of the language that I found useful for the translators I wrote, in the hope that it will provide a starting point for people and give them a bit of a leg up. I am sure there will be holes because I treat Groovy like most other things I deal with on a daily basis—a tool to solve a problem, and often once I have solved the problem, I move on to the next problem and come back to the tool later.

Native List and Map Syntax

Like Jython (my previous favorite JVM-based scripting language), Groovy has some very powerful support for lists and maps built in. For example, instead of having to say:

HashMap myMap = new HashMap();
myMap..put("1","Fred");
myMap.put("2","George");

You can use the much shorter and clearer:

groovyMap = ["1":"Fred", "2":"George"]

Give it a try in the GroovyConsole and run it; you should see the groovyMap contents printed to the command line standard output.

Likewise, list support is built in:

myList = [1,"2","Three",4]

Note: Boxing for the int values in the list is handled for you, so the above is perfectly valid Groovy syntax.

Not much exciting there, but the simplicity continues:

copyList = myList[1..2]

will create a list called copyList with the contents [”2″,”Three”] based on the second and third elements of myList.

This barely scratches the surface of collections in Groovy, but you should move on to the more interesting stuff. Suffice to say that much more information on the collections and the other language features can be found at http://groovy.codehaus.org/Language+Guide.

Closures

The reason I did bring collections up is because they are really useful for demonstrating closures. In a nutshell, a closure is a code block that does not have a name. One typical use for a closure is to run some block of code on every element in a collection.

A closure has enclosing braces, and then a part where parameters are declared, followed by a -> operator, and then the code block. So, for example:

groovyMap = ["Fred":1, "George":3, "Harry":2, "Tom":2]
groovyMap.each() { name, times ->
   for (i in 1..times) {
      System.out.print(name + " ");
   }
   System.out.println();
}

When executed, this will print each of the names the number of times matching the numerical argument associated with each in the map. Closures can also just be stored in a variable; for example:

def clos = { name, times ->
   for (i in 1..times) {
      System.out.print(name + " ");
   }
   System.out.println();
}

clos.call("James",4)

Will print James four times to the standard output. Note that while this might just look like a strange function definition, the closure could be passed in to a Groovy method to be applied to data from inside the method. This is very powerful and yet simple polymorphism. Smalltalk and Lisp programmers will immediately recognize the potential and power here.

Closures can be applied to all sorts of things. I have demonstrated simple collections here, but farther down I will demonstrate them on much more useful things, like XML nodes and SQL results sets.

Other Syntax Enhancements

There are many other syntactical enhancements that make programming Groovy a pleasure. For example: built-in support for regular expressions using the =~ operator, and an improved switch operator that can natively match strings and also perform range checks (is switch value in range 0..12).

Another important one is the Groovy expression language. This lets you insert variables and expressions into strings or other formatted output. For example:

println "hello ${name}"

will print “hello ” followed by the value of the name variable. This comes in useful in the examples below. It can do more than just variable expansion, allowing you to navigate object hierarchies. You even can call methods and the return argument will be inserted into the string.

There are many more than could be covered in this article, so again I would like to point you at the language guide (http://groovy.codehaus.org/Language+Guide) because I really want to move on to the SQL and XML stuff.

Markup

There is one more important Groovy construct before you start laying your hands on XML and SQL. That is the idea of Markup. Markup is very cool. It is the ability to construct something (say, an XML file or Swing GUI) by using structured code that looks kind of like a Java class or method definition. It’s hard to explain, but far easier to demonstrate.

import groovy.xml.MarkupBuilder;
def myXMLDoc = new MarkupBuilder()
myXMLDoc.library {
   section(id:"Programming") {
      book(title:"Java? Groovy!", author:"Duke McCoffee")
      book(title:"The art of hacking code", author:"Uber Hacker")
   }
}
println myXMLDoc

results in the XML snippet:

<library>
   <section id='Programming'>
      <book title='Java? Groovy!' author='Duke McCoffee' />
      <book title='The art of hacking code' author='Uber Hacker' />
   </section>
</library>

Not bad. When was the last time you were able to form an XML document in Java where the code to create it was about as compact as the resulting XML document (ignoring the import and the line to create the markup builder)?

XML Handling

You have just seen the compact way that XML can be output, but face it, although the notation is compact, there is nothing particularly hard about outputting XML, right? The hard part with XML is parsing it in; even the “easy” option of DOM in Java is pretty wordy. Fortunately, Groovy is as strong at parsing XML as it is at writing it. Once again, an example tells you more than any amount of description:

xmlData = """<library>
   <section id='Programming'>
      <book title='Java? Groovy!' author='Duke McCoffee' />
      <book title='The art of hacking code' author='Uber Hacker' />
   </section>
   <section id='Whimsy'>
      <book title='Ooh, la la' author='The Fops' />
   </section>
</library>"""
library = new groovy.util.XmlParser().parseText(xmlData)
library.section.each() { section ->
   println "Section = ${section['@id']}"
   section.book.each() { book ->
      println "Book: ${book['@title']} by ${book['@author']}"
   }
}

The first section is just putting an XML snippet into a string; the triple quotes let the string span several lines and include the newlines, so this is representative of what you might read out of an XML file. The line that reads:

library = new groovy.util.XmlParser().parseText(xmlData)

creates a groovy XML parser node that you can use within the rest of the example. The really interesting stuff happens below that, though.

Firstly, you assign the handle “library” to the node; you know in this case that your document’s outer node is the “library” node, so that is just for readability. However, you also can start referring to the XML tags inside of that node by name.

library.section matches the section child tags inside the library node; the details are not too important, but you can think of it as returning a collection of those section nodes, which you can use a closure on. So, the line:

library.section.each() { section ->

iterates over the section nodes in library and calls the closure for each one, putting the node into the section variable before doing so; you then can refer to it from there. Apply this trick deep enough and you can parse the whole XML document, and equally easily just ignore the sections you are not interested in.

Also, farther down you start to use the XML attributes in the tags. For example, you access the ID attribute in the section tag, and the title and author attributes from the book tag. To do this, you use the xpath notation @attr-name in the context of a list or map lookup. For example:

println "Book: ${book['@title']} by ${book['@author']}"

obtains the title and author attributes out of the book node as though they were named map entries (which, in a sense, they are). Note also that the Groovy expression language is used to good effect here to dump the resulting values directly into a formatted string.

Now, I don’t know about you, but this is among the simplest and best XML handling I have seen in any language, for both input and output. This example is small, but this same few lines of code could intelligently handle a library with hundreds of sections and books. A few more nested closures and it could handle some pretty large and complex XML schemas in a very few lines of code. Likewise, the output, with careful use of loops and closures there mixed in with the XML markup, could produce equally large and complex XML schemas in a similarly compact way.

You have won the first battle for your translators—easy XML handling—now what can you do about the SQL side (remember, these translators are for XML->SQL and SQL->XML)?

SQL Handling

Fortunately, SQL handling is just about as strong as XML handling in Groovy. Because space is limited in this article, I am not going to go into the setup of JDBC drivers, and making the connection to the database. Assuming you already have a JDBC connection, getting a Groovy object to make life easy is as simple as saying:

sql = new groovy.sql.Sql(jdbcConn)

For the remainder of the article, I am going to assume that there are some suitable tables in the database, let’s say:

Table: SECTIONS
Fields: SECTION_ID, SECTION_NAME

and

Table: BOOKS
Fields: BOOK_ID, SECTION_ID, TITLE, AUTHOR

Now, this SQL variable can be used to do some very powerful operations against the database:

Getting Data Out from the DB

Given the SQL variable defined as above, executing a query and getting the results is super easy:

sql.eachRow("SELECT SECTION_NAME, SECTION_ID FROM SECTIONS")
   { section ->

This line will execute the SELECT statement, returning a results set of sections. It then will iterate over these and put the current row in the variable section ready for you to use. You can refer to the fields in the row by name, for example:

section.SECTION_NAME

will give you the section name for the current row.

Now, suppose that you want to get a list of books in a particular section. In the above enclosure, you could write:

sql.eachRow("SELECT TITLE, AUTHOR FROM BOOKS WHERE SECTION_ID=
            '${section.SECTION_ID}'") {
               book ->

Note the use of the expression language in the query, and also the single quotes around the expression (these are present to keep the SQL parser of the database happy), so that the resulting query will be:

SELECT TITLE, AUTHOR FROM BOOKS WHERE SECTION_ID='101'

with the quotes around the number.

So, upi see now that nesting SQL queries that depend on foreign key relationships like the SECTION_ID field is also easy.

Putting Data Into the DB

The same SQL variable can be used to put data back into the database. To do this, you use the execute method:

sql.execute("INSERT INTO BOOKS(BOOK_ID, SECTION_ID, TITLE,
             AUTHOR) VALUES (?,?,?,?)",
           [book_id, section_id, title, author])

Where book_id, section_id, title and author are just variables in scope within Groovy, and a list is created for them on the fly for insertion. You also could use the expression language notation if you wanted to, but separating out the variables using this notational form and using the list means that the implementation of the SQL handler can try and optimize queries by pre-compiling them and binding in the list of variables (this will be a familiar concept to anyone who has done much in the way of JDBC programming—pre-compiled statements with variables bound in at runtime are much more efficient).

Note: When you are putting data back in to the database based on an XML file input, you may have to keep a track of ID fields using counters (for example, book_id and section_id) they they are an invention of the relational model and do not exist in the XML file (where the relationships are done by nesting). For this purpose, you could simply use a variable counter, but it is probably safer to use some kind of database provided sequence.

Time To Put It All Together

OK, now that you can read and write both XML and SQL data pretty easily, it’s time to do an actual translation where you read the DB data and output it to XML. Again, this app assumes you have done the work to have a JDBC connection ready to use (which means setting up a schema, adding the right driver to the classpath, and opening it in Groovy—that part you can use the equivalent Java code for—there should be no work required to get Groovy to accept it).

import groovy.xml.MarkupBuilder;
myXMLDoc = new MarkupBuilder()
sql = new groovy.sql.Sql(jdbcConn)
myXMLDoc.library {
   // find the sections using the database query
   sql.eachRow("SELECT SECTION_NAME, SECTION_ID FROM SECTIONS")
      { sectionRow ->
   // output the section name to the XML file
   myXMLDoc.section(id:"${sectionRow.SECTION_NAME}") {
   // find the books in that section
   sql.eachRow("SELECT * FROM BOOKS
                WHERE SECTION_ID='${sectionRow.SECTION_ID}'") {
                   bookRow ->
         myXMLDoc.book(title:"${bookRow.TITLE}",
                       author:"${bookRow.AUTHOR}")
      }
   }
}
println myXMLDoc

And, that’s it! A complete translator from a database schema to an XML document. Yes this is simple, but it illustrates the potential (the translators I wrote were much more complex and did manipulations on the ordering and grouping of the resulting XML files to meet the customer’s requirements, but that would have gone far beyond the scope of this article and also broken client confidentiality).

So Why Not XSLT, Oracle’s XML System, Castor, and So Forth….

The above example is extremely simplistic, and the same results can be achieved in numerous other ways with similar compactness. For example, XSLT could be employed to create the SQL statements to import data, Oracle has its own XML pipeline system that you might have used, and other options abound for this kind of work, so why did I use Groovy?

The reasons I believe Groovy is a good choice for this kind of task are (in no particular order)

• The XML and SQL notation are about as compact as I have seen anywhere
• The familiarity of syntax and semantics that are very close to Java, thereby removing a learning curve that may exist with other solutions
• Groovy is a full programming language, backed up by a full platform library (in other words, it has access to everything Java does). This means that if you need to go beyond simply translating data, you can do anything that Java can (which is basically anything you like). In practice, this meant that the full business rules logic necessary for successful data transformation was possible, and even easier with Groovy than Java because of some of the excellent language features
• Transparency of use, Groovy (using groovyc) compiles to .class files and can be wrapped as a jar. Furthermore, it has an ant task for compilation, and can inherit from and be inherited by other Java classes. The integration is so tight that, from the outside, another developer will not even know that you have used Groovy (unless you tell them).

But, there are things to be aware of when using Groovy:

• The lack of a debugger means that problems will need to be sorted out with diagnostics and patience. For this reason, keep the scripts as simple and compact as possible. In fact, I wrote the support framework for the translators in Java. The framework handled everything like setting up the DB connections and opening the XML files, and so on. That way, when I got to write the translator, I only had to concentrate on taking data from one source and converting it to the other. This is good practice anyway, but even better when you can move any potentially problematic code into Java (where it can be easily debugged).
• The JSR releases are pretty reliable and I have not hit any big problems, but remember that it is not final yet. In particular, the documentation (or lack thereof) can be the biggest challenge. In practice, this is another reason to keep the scripts small and focused.
• It’s a quirky thing, but the name can be a hard sell with management. I found referring to it as JSR 241 got me a lot further than Groovy (in truth, it is a poor choice of name for easy acceptance into a corporate environment).

Conclusion

While not the answer to everything (at least not yet), used with care Groovy can be an extremely useful tool in a developer’s toolbox. It is not an overstatement to say that it saved me many hours of development time on the translator project (I estimate that the translators took probably a third of the time they would have taken to write in Java, and that is learning the Groovy language and environment along the way, although, in truth, they were very similar to Java).

About the Author

Dick Wall is a Lead Systems Engineer for NewEnergy Associates, A Siemens Company based in Atlanta, GA that provides energy IT and consulting solutions for decision support and energy operations. He can be reached for comment on this and other matters at dick.wall@newenergyassoc.com. He also co-hosts the JavaCast, a podcast devoted to Java news and the Java community, which can be found at http://javacast.thepostmodern.net.