August 23, 2014
Hot Topics:
RSS RSS feed Download our iPhone app

Building with Ant: Deployment and Distribution

  • February 6, 2003
  • By Alex Chaffee
  • Send Email »
  • More Articles »
"Ah, to build, to build!
That is the noblest art of all the arts."
  -- Henry Wadsworth Longfellow

Review Part 1

Review Part 2

In this installment, we discuss issues of deployment and distribution. We are continuing to use the build.xml file from a real, working Web application -- please download it and follow along.

Deploying Locally

These days, it is trivial to run a servlet container on your workstation. This should be your first line of deployment. Before you push your application to any other servers, you should deploy it locally and look it over. (It is also a good idea to run acceptance tests on the locally running site, using a test framework like HTTPUnit.)

A Web application is just a bunch of files in a directory. At its heart, deployment is simply a matter of copying those files to the right place and letting your servlet container know that you've done so.

In the case of a local servlet container, the first step -- copying the files -- is already done! Remember, our webapp target created a directory inside of build that contains all the files of the Web application. All we need to do now is tell our servlet container where these files are. For Tomcat, that's as easy as adding a single line to TOMCAT_HOME/conf/server.xml:

<Context path="/irv" docBase="c:\home\alex\dev\irv\build\webapp" 
  debug="0" reloadable="true"/>

The "reloadable" attribute tells Tomcat to monitor the file system to see if the .jar or .class files have changed, and if so, to instantiate a new ClassLoader and reload them. Sadly, this feature is still somewhat flaky, so I usually restart the server just to make sure. If all that has changed are HTML or JSP files, then restarting should never be necessary.

Another strategy -- which is possible in Tomcat and may be necessary in other servlet containers -- is to copy your webapp directory into the container's directory space. Tomcat has a directory, TOMCAT_HOME/webapps; each directory inside webapps represents a Web application that is automatically loaded by Tomcat on startup.

 <target name="deploy-local" depends="webapp">
  <property name="tomcat.local" value="c:/jakarta-tomcat-4.0.2" />
  <copy todir="${tomcat.local}/webapps/${project}">
   <fileset dir="${webapp}" />
  </copy>
 
    

Remember, if a developer or workstation on your project has Tomcat installed in a non-standard location (other than c:/jakarta-tomcat-4.0.2), he or she can override the standard value by adding a line to build.properties:

tomcat.local=/path/to/tomcat
    

Finally, some servlet containers may require you to deploy your Web application as a WAR file. Tomcat does not require this, and I do not recommend WAR deployment when you can directly copy the files to the right place. The reason for this is that Tomcat only unpacks a WAR file at server startup, and even this does not function correctly unless you first delete the previously unpacked directory. If you do insist on deploying in WAR form, you can put a copy task inside the deploy-local target.

Deploying to a Remote Server

Deploying to a remote server has all the issues of deploying to a local server, with two additional wrinkles.

Wrinkle 1: How do you transfer the files? There are as many solutions to this problem as there are network admins. Maybe you can use the optional FTP Ant task. Maybe you can <copy> the files to a locally mounted network filesystem.

My preference is to use scp (Secure Copy, part of Secure Shell [SSH]) and rsync. There are scp and rsync clients available for all major platforms (including Windows; see CygWin). The combination of these tools allows for secure password exchange, secure content transfer, and efficient file transfer (since rsync is very clever about transferring only the differences between files, not the entire files; this is great for huge unchanging files like library JARs). There is no Ant task for these yet, but you can use Ant's exec task as follows:

 <target name="deploy-live" depends="webapp">
  <property name="live.server" value="alex@www.purpletech.com"/>
  <property name="live.tomcat" value="/usr/local/java/tomcat"/>
  <exec executable="rsync">
   <arg line="rsync -e ssh -Cavz ${webapp.rsync}/* ${live.server}:${live.tomcat}/webapps/${project}" />
  </exec>
 </target>

 <target name="deploy" depends="deploy-live" />
    

Using SSH in this manner relies on the presence of key files on the server and on the client. Otherwise, SSH would ask you for your password, and since it's being invoked from Ant, not from an interactive console, this would fail. The following incantation may work for you, if you're running bash locally; if not, see your SSH documentation.

ssh-keygen -t dsa
cat ~/.ssh/id_dsa.pub > ( ssh username@www.example.com "cat - >> ~/.ssh/authorized_keys2" )

Note that I am using properties, to allow override from build.properties, in case different workstations have a different idea of what the live server is. Note also that rsync is not happy with DOS file paths (starting with "C:" etc.); therefore I made a new property for webapp.rsync to use value= instead of location=.

Wrinkle 2: How do you restart the remote server? Again, this is a question you will have to answer for yourself. Some servers (e.g., WebLogic) are pretty good about reloading new WAR files. Others require a manual restart. While it is possible to use exec and ssh to tickle the server, I must admit that I usually end up keeping a terminal window open and restarting it manually. Then I'm more certain that the server actually did get restarted.

However you do it, I advise keeping the "deploy" target separate from the "restart" target or script. That way, if you know you've only changed static content, you can "deploy" the HTML files without requiring a server restart (which may lose state, and will certainly take time). If you have created a "restart" target, one slick way to do this is as follows:

<target name="deploy" depends="deploy-live,restart"/>
    

That way, if you know you've only changed static content, you can run "ant deploy-live"; otherwise, run "ant deploy" and the restart will occur as it should.

Deploying on Many Hosts

So much for the simple case.

A Web application will usually have at least two deployment hosts: the workstation you're developing on, and the server you're going to run it "live" on. Also, there may be a staging server, a load-testing server, and so on. While these hosts may be able to run platform-independent Java, they will probably have different configuration parameters -- JDBC source names, directory path locations, security information, and so on. The proper place for this sort of configuration information is usually in the web.xml file, as init parameters.

Unfortunately, the Servlet Spec has room for only a single web.xml file in each WEB-INF directory or .war file, and there's no easy or standard way to configure parameters set inside the web.xml file after it's been deployed. That means that for each deployment host, you will need a separate .war file. Bummer.

Fortunately, Ant can help. You can make one web.xml per host, named local-web.xml, live-web.xml, fred-web.xml, and so on. Each can have its own Init Parameters or whatever. As you deploy your webapp, you can decide which of these files to copy into WEB-INF to become the real web.xml.

 <target name="deploy-live" depends="webapp">
  <!-- first, copy the live version of web.xml -->
  <copy file="${web.xml.live}" tofile="${webapp}/WEB-INF/web.xml" overwrite="yes"/>
  
  <!-- now, copy all the files -->
  <exec executable="${rsync}">
   <arg line=" --rsh=${ssh} -Cavz ${webapp.rsync}/* ${live.user}@${live.server}:${live.tomcat}/webapps/irv" />
  </exec>

  <!-- now recopy the old web.xml so the local version still works -->
  <copy file="${web.xml}" tofile="${webapp}/WEB-INF/web.xml" overwrite="yes" />
 </target>

Another way to do this is to copy web.xml using Ant's copy filter capability. This can be useful to remove duplication among the different web.xml files. In this scenario, you set a copy filter for each variable that must change. This copies the file and makes the appropriate changes on-the-fly, using token substitution. You have one master web.xml template file that contains @token@ references instead of actual variable values. Then in the copy task, you choose the appropriate filter set. See the Ant Documentation on Filtering for more information.

The only real downside to filters is that now you're storing application-specific config information inside your build file. This violates the rule of thumb separating build process metainfo from code.

For still more filter-copying hocus pocus involving web.xml see the Cactus FAQ entry How can I have a web.xml that is valid both for testing and for production ?

version.txt

I find it very useful, as a sanity check, to be able to verify the date and time that a live Web site was actually built and deployed. For this, I use a file named version.txt in the root of the webapp. The trick is that the source for this text file is filled with filter tokens:

@project@ @version@
Built: @now@
    

In the <copy> task that creates the webapp, I enable filtering:

  <copy todir="${webapp}" filtering="on">
   <fileset dir="web" includes="**/*.txt,**/*.html" />
  </copy>

The final version.txt file looks like this:

irv 1.0
Built: 20020325-0949

[In creating this article, I needed to be sure that when I was telling you how to use tokens, that the tokens did not get expanded when I built the site. So instead of putting a literal @ in the source text, I used an HTML escape -- ampersand-pound-64-semicolon, or &#64;. Do a View Source on this page to see how it works -- unless the Gamelan software expands it :-)]

Source Distribution

When you want to deliver your application, assuming you're not simply running it on your own servers, you usually use an archive file format (a ZIP or TAR) so it's easy to download, transfer, or copy. Usually, you want two separate distributions of your project, with different audiences: one containing the functional, built application, for the client (the "release distribution"); and one containing your source and development files, for other developers (the "development" or "source distribution").

With Web application development, the runtime distribution is the WAR file, which we've already seen. So how to go about making a source distribution?

If you followed the advice of making a single build directory, this operation is simple. Make a target named "dist" in your build.xml file that creates the tarball. Since Ant's <tar> command does not have a "compress" option, you must do this in stages, as follows:

 <target name="init">
  <property name="project" value="irv" />
  <tstamp/>
  <property name="now" value="${DSTAMP}-${TSTAMP}" />
  <property name="tarball.tar" value="${project}-${now}.tar" />  
  <property name="tarball.tar.gz" value="${tarball.tar}.gz" />  
...
 <target name="dist" depends="init">
  <tar tarfile="${build}/${tarball.tar}" 
       basedir=".." 
       includes="${project}/**"
       excludes="${project}/build/**, ${project}/test/**" />
  <gzip zipfile="${build}/${tarball.tar.gz}" src="${build}/${tarball.tar}" />
  <delete file="${build}/${tarball.tar}"/>
 </target>
    

Notes:

  • The "project" property should be the same as the name of the directory your project is stored in; we assume that the project name is also what you want to name your tarball and WAR file, as well. If you want any of these to differ, it is simple to declare a new variable (or hardcode the value in a string).
  • We must explicitly exclude the directory "build", since the distribution we're building should contain only the source files. Our client can regenerate the build directory by just running "ant" anyway.
  • Since we want the archive to extract into a single directory, we must specify that the root directory of the tar operation is "..". Then, from the perspective of the parent directory, we will tar up the project directory, which coincidentally has the same name as the project!
  • We are naming the tarball with the current date and time, using Ant's handy <tstamp/> task. This helps avoid version confusion, since it's easy to forget to increment a version number. This way, people reporting bugs or feature requests always know exactly which tarball they downloaded.

Shipping the Source

If you are working on an open source project, you can put the cherry on the top of the sundae by making the webapp target depend on the dist target. You can then copy the dist tarball into your webapp directory, ready for download. You can also copy the newly built javadoc into an appropriate location. These simple steps assure that your site will always be up to date and will never contain outdated source or docs, because new versions will be generated each time you generate your Web application. This is another advantage to separating your Web source files into a "web" directory, whence they are copied into the final "build/webapp" directory.

 <target name="webapp" depends="compile,test,dist,javadoc">
...
  <!-- copy dist tarball -->
  <copy todir="${webapp}/dev" file="${build}/${tarball.tar.gz}" />

  <!-- copy javadoc into webapp -->
  <copy todir="${webapp}/dev/doc">
   <fileset dir="${build}/doc" />
  </copy>
    

Since the name of the tarball changes every time we build it, it might be difficult to put a link to the tarball on the site. Ant's filter capability helps here. In the source code to the download page, we put the text:

Download: <a href="@tarball@">source tarball (@tarball@)</a>

Now when the files in web/ are copied using filtering="on", the name of the current tarball will be set to the current name automatically.

A Rant On Version Names

In my humble opinion, tarballs should be named after the current version, but when they unpack, their contents should not contain the version number. That is, files like foo-1.3.jar or even myproject-1.4.1/ are a bad idea. It totally messes up upgrading if every time you install a new version, you have to go change CLASSPATH or PATH settings in a dozen different config files, and/or remember to update symbolic links. You are welcome to disagree, but then, you'd be wrong :-).

Where to Go from Here?

These three articles have covered a lot of ground. I recommend taking these points one at a time and incorporating them into your existing process. For a new project, you are welcome to use the build.xml file from Jiffy Poll, at http://www.purpletech.com/irv/dev/irv/build.xml. I have tried to make it as generic as possible; if you change a few property settings and preserve the directory structure, it should be relatively painless. I also suggest you study other build.xml files out there. In particular, JDOM's build.xml has some neat ideas in it, but there are many others. Finally, see the References section, and subscribe to my Purple Technology news and announcements list for brief updates on this article and other features.

References

  • Tomcat Application Developer's Guide by Craig McClanahan - http://jakarta.apache.org/tomcat/tomcat-4.0-doc/appdev/index.html
  • jGuru Ant FAQ - http://www.jguru.com/faq/Ant
  • jGuru Servlets FAQ - http://www.jguru.com/faq/Servlets
  • Eric Siegel's Computer Science Songs - http://www.cs.columbia.edu/~evs/songs/
  • JUnit test framework - http://junit.org

About the Author

Alex Chaffee is a leading consultant and trainer specializing in Java. He has been promoting, teaching, and programming in Java since 1995. As the director of software engineering for EarthWeb, he co-created Gamelan, the official directory for the Java community. He has made presentations at numerous users groups and conferences, written articles for several Java magazines and contributed to the book The Official Gamelan Java Directory. He has also published research papers on evolutionary computation (a.k.a., "genetic algorithms") and on implicit memory in human cognition. You can contact him at alexc@purpletech.com.






Comment and Contribute

 


(Maximum characters: 1200). You have characters left.

 

 


Sitemap | Contact Us

Rocket Fuel