Java Building with Ant: Directory Structure

Building with Ant: Directory Structure

Recap

In the first part of this series,
we established an Ant
build file
with the following targets: init, clean, compile,
test, javadoc, webapp, and war. Now, we will examine the
directory structure in which these tasks do their work.

The Directory Structure

The foundation of any build process is the project directory
structure. A messy project directory, cluttered with files
dropped willy nilly into whatever directory the “file save”
dialog happened to open up with, leads to arcane and bug-prone
build scripts. On the other hand, if you try to organize too
compulsively, you can end up with a byzantine nest of
sub-sub-subdirectories that make it tedious to find the files
you’re looking for.

Here, I sketch what I think is a good foundation or skeleton for
a project directory structure that can scale to large teams
using version control and multiple releases.

Directory Structure:
myproject/

project root directory
(run all tools from here)
 build.xml

Ant build file
 deploy-dev.sh

script to deploy your app to your development server
 deploy-live.sh

script to deploy your app to your production (live) server
 lib/

jar files required for build
 src/

Java source code for servlets and utilities
 web/

Web files (html, jsp, etc.)
 conf/

configuration files
  web-dev.xml

Webapp config file for development server
  web-live.xml

Webapp config file for production (live) server
 build/

files compiled/copied into here
(this directory can be safely deleted)
  webapp/

the build destination for the web application
   WEB-INF/

conf/web.xml etc. will be copied into here during build
    classes/

class file root for servlets and utilities
    lib/

jar files required for running app
(usually a subset of the ones in myproject/lib)
 test/

directory used by unit test classes to create test
files; can be safely deleted after build

In the section below, we discuss the implications of this
directory structure.

Keep It Clean (the build directory)

The most important piece of discipline I recommend is to make a
build directory that is totally separate from your source
directory. That is, your built files must be in a different
place from your source files. This is often overlooked by
novice programmers but is important for many reasons:

  • The all-important “make clean” operation becomes a trivial
    “delete directory,” rather than a complicated file-matching
    operation that is all too likely to either skip deleting build
    files, or delete important source files, or both.

  • You (and your version control system) will not be tempted to
    add .class files, or other build detritus, into version control.

  • It simplifies distribution (see below).

  • It requires you to copy/compile/create all appropriate files
    from your build script. This may sound like a pain, but it
    helps you avoid the problem of copying too many files into
    your build project. It is all too easy to deliver a release
    to the client that contains huge files, or backup tarballs, or
    private information. For example, do you really want your
    your todo.txt file to be downloadable from your Web site?

In your build.xml, add a property named build,
pointing simply to the directory “build”. This simplifies your
build code and enables easier code reuse to future projects. It
also allows you (or your client) to override the setting on the
command line, in order to build the project into a totally
different directory.

ant -Dbuild=/path/to/other/build/directory
    

Inside the build directory, we will have a subdirectory
called classes (for the compiled .class and
resource files), and one called doc (for the built
javadoc documentation). We will also have a subdirectory of
build named webapp which, in turn, will be the root for
all files to be published in the Web application. If you need
to build other files (for instance, for interim compiled files
for tools like ANTLR), you should put them into a subdirectory
of build as well.

My default location for the build directory is inside the
project directory. However, this can clutter things up, and
confuse version control, so there’s no harm in specifying it as
“../build” or “../build/${project}” instead. (Likewise for
naming the test directory “../test” or even
“../build/${project}/test” or “../test/${project}” or …)

If
you do leave the build directory inside your project, you can
create a file called “.cvsignore” that contains at least the
following two lines:

.cvsignore
build
    

The second line tells CVS to ignore the directory (or file) called
“build;” the first line tells CVS to ignore the .cvsignore file
itself. Alternately, you could check in the .cvsignore file
(though it wouldn’t hurt to leave the .cvsignore line in).

web vs. webapp

All source files — including Web resources like HTML and JSP
files — will be stored inside the project directory, and copied
into a separate webapp directory during the build. In order to
reinforce this distinction, I store the HTML etc. source in a
directory named “web,” and build/copy them into a target
directory called “webapp.” Note that only the “build/webapp”
directory has a subdirectory called WEB-INF. This helps
reinforce that WEB-INF is filled with information about
the Web application (its libraries and config files), and thus
its source does not belong in the same place as Web resource
files. It also helps us adjust to the possibility of multiple
web.xml files.

(This structure can come as something of a shock if you’re used
to editing your HTML and JSP files “in-place,” and immediately
clicking “reload” to see the new versions. Essentially, it adds
a compile phase to HTML and JSP (though it’s not really
compiling, just copying). This pain can be reduced somewhat if
you write a script that copies files from web to
build/webapp every few seconds or so — see
webup.sh
for one such script — but in practice, it doesn’t take long to
adjust.

Source Code (the src directory)

Although it is tempting to just start dropping .java source
files into the project directory, you should not yield to
temptation. Instead, make a src directory and start
putting your source files in there. Since Java has a strange
coupling between package names and directory names, this will
keep your project root directory from being cluttered with
directories with names like “com” and “org”. It will also
reduce clutter from throwaway source files in the “default
package” (the package with no name, whose source files are
stored in the source root directory).

Normally, when we use javac to compile our source,
it puts the compiled .class files into the same
directory as the source .java files. However, this
would violate our rule of keeping a separate build directory, so
we must give javac a “destdir” argument.

 <target name="compile" depends="init">
  <javac classpath="${classpath}" 
	srcdir="src"
	destdir="${classes}"
	deprecation="on" />
 </target>
    

Compiled Code (the build/classes and the
build/webapp/WEB-INF/classes directories)

As noted immediately above, the recommended target for building
.class files is the ${classes} property, which
we set to ${build}/classes.

This leaves you with a choice when building your webapp: you can
either copy the build/classes directory directly into
the build/webapp/WEB-INF/classes directory, or you can
JAR your classes up into a ${project}.jar file placed
into build/webapp/WEB-INF/lib. The JAR option can
yield a slightly smaller project on disk, but other than that, I
don’t know of any reason to choose one over the other.

Testing

The test directory is for files that are created or required by
the unit tests. In the present project, in order to test the
file I/O routines, we need a place to store and load files. The
particular unit test code responsible for this assumes there
will be a subdirectory called “build/test” in which it’s free to
play. The build script needs to recognize the possible
existence of this directory:

  • during “init” to create it
  • during “clean” to delete it
  • when creating tar archives, to ignore it

For test cases that require external files (say, sample HTML
files to which to compare program output), you can store these
master files in a directory called test-data (to
distinguish from the test directory which is transient,
under the build directory).

Libraries (the lib directory)

These days, a Java library is usually a JAR file (although some
JDBC drivers, for example, are still shipped as ZIP files), so I
will just use the term “JAR” to refer to a library file.

Some JARs are used only during the build process (for example,
ant.jar and junit.jar), some are used only by the running
application (for example, JDBC drivers), and some are used in
both. However, these different uses need not overwhelm you. I
recommend that you store all JARs in a common “lib” directory,
and sort out these different uses in your build script.

Libraries: Classpath

Usually, you can get away with having a single CLASSPATH for use
during compiling, testing, and javadoc. So make an Ant property
in your init target called classpath. Make
sure to also add the build directory where your .class
files will be compiled (in this example, the property
${classes}, which points to the directory
“build/classes”). Don’t worry about semicolons — Ant will
convert the separator to the correct character of the current
operating system. Here I’m including JARs from both the project
lib, and from a different directory named lib that
happens to be one directory level up.

<property name="classpath" value="${classes}:lib/junit.jar:lib/purple.jar:lib/
servlet.jar:../lib/httpunit.jar:../lib/Tidy.jar:../lib/xerces.jar" />

You can then refer to it from the javac,
javadoc, and junit tasks.

<javac classpath="${classpath}" 
       srcdir="src"
       destdir="${classes}"
       deprecation="on" debug="on" />
    

Another way to do it is to use Ant’s <path> element. This
is a bit more flexible but also more verbose. The above could
be written as follows:

  <path id="classpath.path">
   <pathelement location="${classes}"/>
   <fileset dir="lib">
    <include name="*.jar"/>
   </fileset>
   <pathelement location="../lib/httpunit.jar"/>
   <pathelement location="../lib/Tidy.jar"/>
   <pathelement location="../lib/xerces.jar"/>
  </path>

   <javac srcdir="${src}"
	 destdir="${classes}"
	 deprecation="on"
	 debug="on">
   <classpath><path refid="classpath.path"/></classpath>
  </javac>
    

As with <property>, you can put the <path>
definition outside of any target if you like.

Erik Hatcher, author of Java Development with Ant,
recommends a further level of indirection: naming the individual
JAR files as property variables. This allows you to use a
different version of a particular JAR from the command line,
like this:

ant -Dpurple.jar=purple-new-version.jar
    

Personally, I think this is overkill.

Libraries: Development vs. Production JARs

Many of your libraries will need to be deployed with your Web
application. The Servlet spec defines a directory,
WEB-INF/lib, for this purpose — any JAR files in this
directory will automatically be added to the application’s class
path at runtime. So in your webapp target, you must
copy the JARs you need from myproject/lib into
myproject/build/webapp/WEB-INF/lib.

  <copy todir="${webapp}/WEB-INF/lib" file="lib/purple.jar"/>
    

Note that I am explicitly copying each JAR that I need (in this
case, only one, purple.jar, which contains my

Purple Technology Library

of utility code.). Ant allows you to copy all files in a
directory fairly easily (using the “*” wildcard). However, this
is not recommended in this case, for two reasons:

  • Adding JARs that are used only during development-time (like
    junit.jar, which is used during testing) will bloat
    your Web application.

  • Some development-time JARs (like servlet.jar or some
    XML parsers) will conflict with JARs that are used by your Web
    server. For instance, the Tomcat server uses an XML parser to
    read its own configuration files. If you include an XML
    parser that uses a different version of the DOM interfaces
    than the XML parser used by Tomcat, this can cause horribly
    confusing bugs that only show up at runtime. (Note that
    Tomcat 4 seems to have fixed this particular issue, but it
    required some convoluted ClassLoader acrobatics, and I still
    don’t trust other server vendors to get it right).

A note on servlet.jar: Unfortunately, J2SE does not
include the servlet classes and interfaces. That means that, in
order to compile servlets, you need to put a copy of the servlet
classes into your classpath. There are many places to acquire
these; I prefer the file servlet.jar that comes with
Tomcat. However, even if you put servlet.jar in your
classpath at compile-time, you must make sure not to
install servlet.jar into your
webapp/WEB-INF/lib directory. This can cause a bug if
and when you deploy your application into a server that is using
a new version of servlet.jar, especially if you are
using JSPs.

For instance, the version of Jasper (the JSP compiler) that
ships with Tomcat 4 generates JSP Java code that uses several
methods that are available only in the Servlet Spec version 2.3.
That means that if you happen to have the interfaces for Servlets
version 2.2 in your classpath, your generated JSP source files
will fail to compile — since they’re looking for methods that are
absent from the Servlet classes included in your 2.2
servlet.jar file. Ugh.

What’s Up, Doc?

Isn’t JavaDoc nice? Wouldn’t it be nice to have a JavaDoc build
target, so that you always have access to up-to-the-minute
documentation of your source code? Are these rhetorical
questions? The <javadoc> task will help you
answer them.

 <target name="javadoc" depends="init">
  <javadoc sourcepath="src" destdir="${build}/doc" packagenames="irv.*" classpath="${classpath}" />
 </target>
    

Go (con)figure (the conf directory)

It is useful to store configuration files in the project root
directory, for easy access. However, if you have more than a
few configuration files, you should probably group them together
in a common directory (I prefer to call this conf).

At first, you may only have one file in the conf directory
(web.xml). But soon you may need many copies of
web.xml (see next article), and there may be other configuration
files, so it’s good to have a place for it. “A place for
everything, and everything in its place,” my mom always said.

Pulling It All Together: the webapp target

The webapp target is pretty straightforward. It is
responsible for gathering all the various pieces of your Web
application — classes, JARs, HTML, JSP, web.xml, plus maybe
docs and tarballs for your source code — and sticking them all
in the right place inside the build/webapp directory.
Note that we can also do tricks like copying the recently built
javadoc into a subdirectory of the webapp.

Review the following extract from build.xml and make
sure you understand what each line is doing, and why it’s doing
it.

 <target name="webapp" depends="compile,test,dist,javadoc">

  <!-- all web files -->
  <copy todir="${webapp}">
   <fileset dir="web"/>
  </copy>

  <!-- WEB-INF config directory -->
  <copy todir="${webapp}/WEB-INF/lib" file="lib/purple.jar"/>
  <jar jarfile="${webapp}/WEB-INF/lib/${project}.jar" basedir="${classes}" />
  <copy todir="${webapp}/WEB-INF" file="conf/web.xml"/>

  <!-- copy dist tarball -->
  <copy todir="${webapp}/dev" file="${tarball}.gz" />

  <!-- copy javadoc into webapp -->
  <copy todir="${webapp}/dev/doc">
   <fileset dir="${build}/doc" />
  </copy>

  <!-- also copy the full project - - why not? -->
  <copy todir="${webapp}/dev/${project}">
   <fileset dir="." excludes="build/**,test/**"/>
  </copy>

 </target>
    

Take a Breath

Next time, we discuss some advanced issues, like multiple
deployment targets, versioning, and source distribution.

References

  • Tomcat Application Developer’s Guide by Craig McClanahan –
    http://jakarta.apache.org/tomcat/tomcat-4.0-doc/appdev/index.html
  • jGuru Ant FAQ –
    http://www.jguru.com/faq/Ant
  • jGuru Servlets FAQ –
    http://www.jguru.com/faq/Servlets
  • Eric Siegel’s Computer Science Songs –
    http://www.cs.columbia.edu/~evs/songs/
  • JUnit test framework –
    http://junit.org

About the Author

Alex Chaffee is a leading consultant and trainer specializing in Java. He has been promoting, teaching, and programming in Java since 1995. As the director of software engineering for EarthWeb, he co-created Gamelan, the official directory for the Java community. He has made presentations at numerous users groups and conferences, written articles for several Java magazines and contributed to the book The Official Gamelan Java Directory. He has also published research papers on evolutionary computation (a.k.a., “genetic algorithms”) and on implicit memory in human cognition. You can contact him at [email protected].

Latest Posts

Related Stories