April 16, 2014
Hot Topics:
RSS RSS feed Download our iPhone app

Building with Ant: Directory Structure

  • January 16, 2003
  • By Alex Chaffee
  • Send Email »
  • More Articles »

Recap

In the first part of this series, we established an Ant build file with the following targets: init, clean, compile, test, javadoc, webapp, and war. Now, we will examine the directory structure in which these tasks do their work.

The Directory Structure

The foundation of any build process is the project directory structure. A messy project directory, cluttered with files dropped willy nilly into whatever directory the "file save" dialog happened to open up with, leads to arcane and bug-prone build scripts. On the other hand, if you try to organize too compulsively, you can end up with a byzantine nest of sub-sub-subdirectories that make it tedious to find the files you're looking for.

Here, I sketch what I think is a good foundation or skeleton for a project directory structure that can scale to large teams using version control and multiple releases.

Directory Structure:
myproject/ project root directory (run all tools from here)
 build.xml Ant build file
 deploy-dev.sh script to deploy your app to your development server
 deploy-live.sh script to deploy your app to your production (live) server
 lib/ jar files required for build
 src/ Java source code for servlets and utilities
 web/ Web files (html, jsp, etc.)
 conf/ configuration files
  web-dev.xml Webapp config file for development server
  web-live.xml Webapp config file for production (live) server
 build/ files compiled/copied into here (this directory can be safely deleted)
  webapp/ the build destination for the web application
   WEB-INF/ conf/web.xml etc. will be copied into here during build
    classes/ class file root for servlets and utilities
    lib/ jar files required for running app (usually a subset of the ones in myproject/lib)
 test/ directory used by unit test classes to create test files; can be safely deleted after build

In the section below, we discuss the implications of this directory structure.

Keep It Clean (the build directory)

The most important piece of discipline I recommend is to make a build directory that is totally separate from your source directory. That is, your built files must be in a different place from your source files. This is often overlooked by novice programmers but is important for many reasons:

  • The all-important "make clean" operation becomes a trivial "delete directory," rather than a complicated file-matching operation that is all too likely to either skip deleting build files, or delete important source files, or both.
  • You (and your version control system) will not be tempted to add .class files, or other build detritus, into version control.
  • It simplifies distribution (see below).
  • It requires you to copy/compile/create all appropriate files from your build script. This may sound like a pain, but it helps you avoid the problem of copying too many files into your build project. It is all too easy to deliver a release to the client that contains huge files, or backup tarballs, or private information. For example, do you really want your your todo.txt file to be downloadable from your Web site?

In your build.xml, add a property named build, pointing simply to the directory "build". This simplifies your build code and enables easier code reuse to future projects. It also allows you (or your client) to override the setting on the command line, in order to build the project into a totally different directory.

ant -Dbuild=/path/to/other/build/directory
    

Inside the build directory, we will have a subdirectory called classes (for the compiled .class and resource files), and one called doc (for the built javadoc documentation). We will also have a subdirectory of build named webapp which, in turn, will be the root for all files to be published in the Web application. If you need to build other files (for instance, for interim compiled files for tools like ANTLR), you should put them into a subdirectory of build as well.

My default location for the build directory is inside the project directory. However, this can clutter things up, and confuse version control, so there's no harm in specifying it as "../build" or "../build/${project}" instead. (Likewise for naming the test directory "../test" or even "../build/${project}/test" or "../test/${project}" or ...)

If you do leave the build directory inside your project, you can create a file called ".cvsignore" that contains at least the following two lines:

.cvsignore
build
    

The second line tells CVS to ignore the directory (or file) called "build;" the first line tells CVS to ignore the .cvsignore file itself. Alternately, you could check in the .cvsignore file (though it wouldn't hurt to leave the .cvsignore line in).

web vs. webapp

All source files -- including Web resources like HTML and JSP files -- will be stored inside the project directory, and copied into a separate webapp directory during the build. In order to reinforce this distinction, I store the HTML etc. source in a directory named "web," and build/copy them into a target directory called "webapp." Note that only the "build/webapp" directory has a subdirectory called WEB-INF. This helps reinforce that WEB-INF is filled with information about the Web application (its libraries and config files), and thus its source does not belong in the same place as Web resource files. It also helps us adjust to the possibility of multiple web.xml files.

(This structure can come as something of a shock if you're used to editing your HTML and JSP files "in-place," and immediately clicking "reload" to see the new versions. Essentially, it adds a compile phase to HTML and JSP (though it's not really compiling, just copying). This pain can be reduced somewhat if you write a script that copies files from web to build/webapp every few seconds or so -- see webup.sh for one such script -- but in practice, it doesn't take long to adjust.

Source Code (the src directory)

Although it is tempting to just start dropping .java source files into the project directory, you should not yield to temptation. Instead, make a src directory and start putting your source files in there. Since Java has a strange coupling between package names and directory names, this will keep your project root directory from being cluttered with directories with names like "com" and "org". It will also reduce clutter from throwaway source files in the "default package" (the package with no name, whose source files are stored in the source root directory).

Normally, when we use javac to compile our source, it puts the compiled .class files into the same directory as the source .java files. However, this would violate our rule of keeping a separate build directory, so we must give javac a "destdir" argument.

 <target name="compile" depends="init">
  <javac classpath="${classpath}" 
	srcdir="src"
	destdir="${classes}"
	deprecation="on" />
 </target>
    

Compiled Code (the build/classes and the build/webapp/WEB-INF/classes directories)

As noted immediately above, the recommended target for building .class files is the ${classes} property, which we set to ${build}/classes.

This leaves you with a choice when building your webapp: you can either copy the build/classes directory directly into the build/webapp/WEB-INF/classes directory, or you can JAR your classes up into a ${project}.jar file placed into build/webapp/WEB-INF/lib. The JAR option can yield a slightly smaller project on disk, but other than that, I don't know of any reason to choose one over the other.

Testing

The test directory is for files that are created or required by the unit tests. In the present project, in order to test the file I/O routines, we need a place to store and load files. The particular unit test code responsible for this assumes there will be a subdirectory called "build/test" in which it's free to play. The build script needs to recognize the possible existence of this directory:

  • during "init" to create it
  • during "clean" to delete it
  • when creating tar archives, to ignore it

For test cases that require external files (say, sample HTML files to which to compare program output), you can store these master files in a directory called test-data (to distinguish from the test directory which is transient, under the build directory).

Libraries (the lib directory)

These days, a Java library is usually a JAR file (although some JDBC drivers, for example, are still shipped as ZIP files), so I will just use the term "JAR" to refer to a library file.

Some JARs are used only during the build process (for example, ant.jar and junit.jar), some are used only by the running application (for example, JDBC drivers), and some are used in both. However, these different uses need not overwhelm you. I recommend that you store all JARs in a common "lib" directory, and sort out these different uses in your build script.

Libraries: Classpath

Usually, you can get away with having a single CLASSPATH for use during compiling, testing, and javadoc. So make an Ant property in your init target called classpath. Make sure to also add the build directory where your .class files will be compiled (in this example, the property ${classes}, which points to the directory "build/classes"). Don't worry about semicolons -- Ant will convert the separator to the correct character of the current operating system. Here I'm including JARs from both the project lib, and from a different directory named lib that happens to be one directory level up.

<property name="classpath" value="${classes}:lib/junit.jar:lib/purple.jar:lib/
servlet.jar:../lib/httpunit.jar:../lib/Tidy.jar:../lib/xerces.jar" />

You can then refer to it from the javac, javadoc, and junit tasks.

<javac classpath="${classpath}" 
       srcdir="src"
       destdir="${classes}"
       deprecation="on" debug="on" />
    

Another way to do it is to use Ant's <path> element. This is a bit more flexible but also more verbose. The above could be written as follows:

  <path id="classpath.path">
   <pathelement location="${classes}"/>
   <fileset dir="lib">
    <include name="*.jar"/>
   </fileset>
   <pathelement location="../lib/httpunit.jar"/>
   <pathelement location="../lib/Tidy.jar"/>
   <pathelement location="../lib/xerces.jar"/>
  </path>

   <javac srcdir="${src}"
	 destdir="${classes}"
	 deprecation="on"
	 debug="on">
   <classpath><path refid="classpath.path"/></classpath>
  </javac>
    

As with <property>, you can put the <path> definition outside of any target if you like.

Erik Hatcher, author of Java Development with Ant, recommends a further level of indirection: naming the individual JAR files as property variables. This allows you to use a different version of a particular JAR from the command line, like this:

ant -Dpurple.jar=purple-new-version.jar
    

Personally, I think this is overkill.

Libraries: Development vs. Production JARs

Many of your libraries will need to be deployed with your Web application. The Servlet spec defines a directory, WEB-INF/lib, for this purpose -- any JAR files in this directory will automatically be added to the application's class path at runtime. So in your webapp target, you must copy the JARs you need from myproject/lib into myproject/build/webapp/WEB-INF/lib.

  <copy todir="${webapp}/WEB-INF/lib" file="lib/purple.jar"/>
    

Note that I am explicitly copying each JAR that I need (in this case, only one, purple.jar, which contains my Purple Technology Library of utility code.). Ant allows you to copy all files in a directory fairly easily (using the "*" wildcard). However, this is not recommended in this case, for two reasons:

  • Adding JARs that are used only during development-time (like junit.jar, which is used during testing) will bloat your Web application.
  • Some development-time JARs (like servlet.jar or some XML parsers) will conflict with JARs that are used by your Web server. For instance, the Tomcat server uses an XML parser to read its own configuration files. If you include an XML parser that uses a different version of the DOM interfaces than the XML parser used by Tomcat, this can cause horribly confusing bugs that only show up at runtime. (Note that Tomcat 4 seems to have fixed this particular issue, but it required some convoluted ClassLoader acrobatics, and I still don't trust other server vendors to get it right).

A note on servlet.jar: Unfortunately, J2SE does not include the servlet classes and interfaces. That means that, in order to compile servlets, you need to put a copy of the servlet classes into your classpath. There are many places to acquire these; I prefer the file servlet.jar that comes with Tomcat. However, even if you put servlet.jar in your classpath at compile-time, you must make sure not to install servlet.jar into your webapp/WEB-INF/lib directory. This can cause a bug if and when you deploy your application into a server that is using a new version of servlet.jar, especially if you are using JSPs.

For instance, the version of Jasper (the JSP compiler) that ships with Tomcat 4 generates JSP Java code that uses several methods that are available only in the Servlet Spec version 2.3. That means that if you happen to have the interfaces for Servlets version 2.2 in your classpath, your generated JSP source files will fail to compile -- since they're looking for methods that are absent from the Servlet classes included in your 2.2 servlet.jar file. Ugh.

What's Up, Doc?

Isn't JavaDoc nice? Wouldn't it be nice to have a JavaDoc build target, so that you always have access to up-to-the-minute documentation of your source code? Are these rhetorical questions? The <javadoc> task will help you answer them.

 <target name="javadoc" depends="init">
  <javadoc sourcepath="src" destdir="${build}/doc" packagenames="irv.*" classpath="${classpath}" />
 </target>
    

Go (con)figure (the conf directory)

It is useful to store configuration files in the project root directory, for easy access. However, if you have more than a few configuration files, you should probably group them together in a common directory (I prefer to call this conf).

At first, you may only have one file in the conf directory (web.xml). But soon you may need many copies of web.xml (see next article), and there may be other configuration files, so it's good to have a place for it. "A place for everything, and everything in its place," my mom always said.

Pulling It All Together: the webapp target

The webapp target is pretty straightforward. It is responsible for gathering all the various pieces of your Web application -- classes, JARs, HTML, JSP, web.xml, plus maybe docs and tarballs for your source code -- and sticking them all in the right place inside the build/webapp directory. Note that we can also do tricks like copying the recently built javadoc into a subdirectory of the webapp.

Review the following extract from build.xml and make sure you understand what each line is doing, and why it's doing it.

 <target name="webapp" depends="compile,test,dist,javadoc">

  <!-- all web files -->
  <copy todir="${webapp}">
   <fileset dir="web"/>
  </copy>

  <!-- WEB-INF config directory -->
  <copy todir="${webapp}/WEB-INF/lib" file="lib/purple.jar"/>
  <jar jarfile="${webapp}/WEB-INF/lib/${project}.jar" basedir="${classes}" />
  <copy todir="${webapp}/WEB-INF" file="conf/web.xml"/>

  <!-- copy dist tarball -->
  <copy todir="${webapp}/dev" file="${tarball}.gz" />

  <!-- copy javadoc into webapp -->
  <copy todir="${webapp}/dev/doc">
   <fileset dir="${build}/doc" />
  </copy>

  <!-- also copy the full project - - why not? -->
  <copy todir="${webapp}/dev/${project}">
   <fileset dir="." excludes="build/**,test/**"/>
  </copy>

 </target>
    

Take a Breath

Next time, we discuss some advanced issues, like multiple deployment targets, versioning, and source distribution.

References

  • Tomcat Application Developer's Guide by Craig McClanahan - http://jakarta.apache.org/tomcat/tomcat-4.0-doc/appdev/index.html
  • jGuru Ant FAQ - http://www.jguru.com/faq/Ant
  • jGuru Servlets FAQ - http://www.jguru.com/faq/Servlets
  • Eric Siegel's Computer Science Songs - http://www.cs.columbia.edu/~evs/songs/
  • JUnit test framework - http://junit.org

About the Author

Alex Chaffee is a leading consultant and trainer specializing in Java. He has been promoting, teaching, and programming in Java since 1995. As the director of software engineering for EarthWeb, he co-created Gamelan, the official directory for the Java community. He has made presentations at numerous users groups and conferences, written articles for several Java magazines and contributed to the book The Official Gamelan Java Directory. He has also published research papers on evolutionary computation (a.k.a., "genetic algorithms") and on implicit memory in human cognition. You can contact him at alexc@purpletech.com.






Comment and Contribute

 


(Maximum characters: 1200). You have characters left.

 

 


Sitemap | Contact Us

Rocket Fuel