Unwrapping Java packages

Packages suck. Which is not to say they don’t have their strong points. They’re a great way to organize your code. They provide a reasonable name space, so your Utilities.class doesn’t get confused with someone else’s Utilities.class. And they allow access to the information hiding and protection features built in to the language (for instance, the protected keyword). If you know how to use them, they’re indispensable for writing large programs, as well as sharing your work with other programmers on your team.

But packages do suck. The main reason they suck is a fairly confusing interaction between (a) the tight coupling between packages and directories, and (b) a design decision by the authors of javac. This article will help clear up this confusion and, hopefully, make you less intimidated, so you can feel comfortable using packages whenever you want.

Let’s look at javac first. Its authors made a very clever design decision. "Wouldn’t it be nice," they thought, "if the compiler automatically reached out and compiled all the source files that had changed, to bring the entire program — the original class and all classes referenced by it — up to date?" Basically, when javac encounters a reference to another class (say, class Student), it looks for a source file with the same name (in this case, Student.java), and compares its last modification date to the modification date of the class file (Student.class). If the source file is newer than the class file, it reaches out and compiles Student.java. It also compiles any classes that Student references, recursively. In other words, javac acts like make [1]. Only, since it’s Java, it’s easy — no makefiles, no scripts, no aliases — just javac [2].

Enter packages. Java enforces a tight coupling between packages and directories. This means that if class Student is in a package named myprog.school, then the file Student.class must be in a directory named school, which in turn must be in a directory named myprog. (See Fig. 1 below.) But let’s say myprog.school.Student makes reference to myprog.school.Grade. If we change directory (cd) to myprog/school and run "javac Student.java", then the compiler will do the following:

1. compile Student.java
2. notice that Student.java refers to myprog.school.Grade
3. look for a directory named myprog, containing a directory named school, containing a file named Grade.class

But we’re already in the directory named school, which does not contain a directory named myprog, so naturally it won’t find it. (The compiler will die with an error, "Class myprog.school.Grade not found in type declaration.") If, instead, we run javac from the high-level directory — using the command line "javac myprog/school/Student.java" — then javac can find the file myprog/school/Grade.java and can make the Grade (so to speak).

There are many more variations on this theme, all revolving around a common problem: programmers are used to thinking of files as containing all the information needed to compile a program. With java, this is no longer true. Instead, the file has to contain a "package" statement, and this well-written file must also be in the correct directory — relative to the current directory.

Fig. 1. Example directory structure

So now you see how packages suck. To solve this problem, I have developed some simple guidelines. If you follow The Rules (no, not those “Rules”), you will successfully navigate the shoals of packages. There are other ways to solve the problem (see below), but I find that these rules are simple enough that you can learn them quickly and apply them easily.

Please walk through each of these rules and convince yourself that they will avoid the situation of javac or java not knowing where a particular package file is.

The Rules: Time-tested secrets for capturing the heart of javac

1. Declare a project root directory. This is just for your sake — there’s no file where you need to enter this or anything — but it’s vitally important that you know what your project’s root is. This could be something like c:ProjectsMyApplet, or ~alex/java/chat. The point is that this directory does not contain any actual Java source. (If it does, then that source is placed into the "default package," which means that it has no package statement.)
2. Put all source files in appropriately named subdirectories of the project root. That is, if class Baz is in package foo.bar, then the file Baz.java must be in a directory named bar, in a directory named foo, in the project root directory.
3. Run all tools from inside the directory. That means javac, java, edit and so forth. To put it another way, never cd out of this directory — when you refer to a file, you’ll have to type its full path name (e.g., javac foo/bar/Baz.java). This is annoying, but trust me, it’s a small price to pay for clarity.
4. Put "." (dot) in your classpath, and do not put the project root in your classpath, or the package directories, or any other directories except for system and third-party libraries.

These rules guarantee that javac will always find your class files when it needs them.

There is another approach that allows you to avoid rule number 3, “never cd out of the project root.” In this scenario, you add the project root directory to your classpath. (For example, in DOS, “SET CLASSPATH=%CLASSPATH%;C:ProjectsMyApplet”.) In that case, you can run javac (and java) from wherever you like and it will find the files. But this is very dangerous if you have multiple projects or multiple versions of the same project. It is very easy to compile a new version of your program but have java find the old version, because the old version is what’s in the classpath. Worse, you might be using some new classes and some old… In brief, to make this work you need a script for each project and each version that sets the environment variables correctly — or that aliases “javac” to “javac -classpath $CLASSPATH:/projects/myproject” — and you need to remember to run that script every time you cd to a different project. What a pain.

Another solution is to use the -d flag. This flag specifies a destination directory for the class files, separate from the source directory. If you add this destination directory to your classpath, then you can run java and javac from wherever you like and it will find the newly compiled versions of your files. However, this technique also requires an alias (of javac, to use the -d flag) and a script (to change CLASSPATH). This makes it more difficult to change to a different project, unless you write a script for each project. But the whole point of the clever javac make facility was to avoid scripts and aliases (and makefiles).

You could also, of course, use make itself, which is a fine solution for a large project. There are several sample makefiles for Java projects available on the Net.

IDEs

• IBM’s Visual Age solves the problem by ignoring it: all source files are slurped into a project database, so they don’t even exist on the file system.
• Symantec Visual Café is a little less thorough. Without getting into the details, let me give you this warning about Café: Always save your project file immediately upon creating it, in the project root directory. In other words, never keep an untitled project. Otherwise, you may find that some of your source files are stored in a temporary directory and that compiling them creates class files in that temporary directory, and these orphan class files are lost forever as far as the compiler is concerned. You also need to be aware and save your source files in the appropriately named subdirectories of your project root, and always save the project file in the project root directory. Apparently among the many engineers who got confused about packages were some working in Cupertino…
• Dispatches from field operatives report that Kawa has integrated support for packages, but we are unable to confirm at this time.
• JDE, the elisp package that turns GNU emacs — for Unix or PC — into a Java IDE, has rudimentary package support. You have to specify the -classpath option for each project. However, it has a nice package browser, called the Speedbar, that lets you browse from the package/directory level down into classes, methods and variables, in a single hierarchical window.
• Does your favorite IDE deal well with packages? Or poorly? Let us know! Contact the author at the address below.

RePackager+

In order to change this, you have to go into each and every source file and change the "import" and "package" statements. For instance, for the com.beanco.FastSort class, you have to change "package com.beanco" to "package com.megabean.util" in the original source code. You then have to go into every source file that references FastSort and change its "import com.beanco.*" to "import com.megabean.util.*". And you need to do this for all files. Talk about carpal tunnel syndrome.

There’s a new utility called RePackager+ (from Wooden Chair Software), with a nice graphical user interface, that purports to solve this problem. The idea is, you import your source files into the app, and RePackager+ presents you with a hierarchical tree view of the names of all your packages and the classes inside them. You then rename or move the packages, and your changes are reflected in a "preview pane" — basically another tree view, this one listing the hierarchical structure of the packages as-they-will-be. When you’re ready, you can export all of the source in the new package structure. This new source tree goes to a different directory, so there’s no danger of overwriting or getting confused with the original code.

As usual with a 1.0 release, there are a few glitches left to iron out. The UI is very pretty, since they used Swing (JFC) to produce a professional-looking application window [3]. But it wasn’t immediately obvious what to do with it all. Turns out you have to click on a tiny button sitting just to the right of the package name in order to rename it. There are also some (known?) bugs with the Swing file selection dialog — for instance, it doesn’t remember the last directory you accessed, and to select a directory you have to click, not double-click, on the directory name and then click "open."

But these are minor complaints. The underlying technology performs as advertised. It’s much easier to click and drag than to manually edit dozens of source files. This technology will come in very handy when it’s time to upgrade to the next version of Swing.

Ah, Swing, the package without a home. In its short life, it has lived in two different packages — com.sun.java.swing and java.awt.swing — before announcing its intention to reside permanently at javax.swing. If you want to use Swing now, then plan to convert all your files later, using a utility like RePackager+. (You might wonder why Sun didn’t provide a utility that does this with the first release of Java … but then again, that’s what third-party products are for.)

Notes

[2] Note that this trick can only work with public classes, which is one reason for the rule "only one public class per source file."

[3] I’m happy to report that, running the latest version of Symantec’s JIT and JRE 1.1.6 on a 200-MHz Pentium Pro in Windows 98, the Swing application is not painfully slow. The Swing menus actually come up noticeably faster than system menus in Windows 98 — if that’s saying anything.

Links on this article

• JFC — Package Name [on changing the Swing package name]
• Wooden Chair Software
• Purple Technology
• JDE, the Java Development Environment for Emacs
• GNU Emacs for Windows
• GNU Make

Alex Chaffee is a leading consultant and trainer specializing in Java. He has been promoting, teaching, and programming in Java since 1995. As the director of software engineering for EarthWeb, he co-created Gamelan, the official directory for the Java community. He has made presentations at numerous user groups and conferences, written articles for several Java magazines and contributed to the book The Official Gamelan Java Directory. He has also published research papers on evolutionary computation (a.k.a., “genetic algorithms”) and on implicit memory in human cognition. You can contact him at alexc@purpletech.com.

Learn Java packages in 21 seconds

A package is a group of related Java classes.

They all share the same first name — e.g., all classes in a package named "com.stinky.util" are named "com.stinky.util.Sorter", "com.stinky.util.FileInfo", and so on.

Classes in the same package can access each other’s protected members.

Yeah, it’s kind of like friends in C++ — you know, "only friends can access your private parts." In Java, classes can access the "protected" and "default" member functions and instance data of other classes in the same package (as well as the "public" members, naturally).

One that doesn’t have any of the following keywords: public, protected, or private.

You just call it by its full name. For instance,

com.stinky.util.Sorter mysorter = new com.stinky.util.Sorter();

Yeah, especially because Java is a strongly typed language.

Seriously, that’s what the "import" keyword is for. If you say "import com.stinky.util.Sorter", then every time you type "Sorter", the compiler will understand that you mean com.stinky.util.Sorter. So you can do:

import com.stinky.util.Sorter;<BR>

class Foo {<BR>

	Sorter mysorter = new Sorter();<BR>

	// etc…<BR>

}<BR>

Yes, the line "import com.stinky.util.*;" will import the names of all classes in the com.stinky.util package. (However, it does not work recursively, so it won’t import the com.stinky.util.net.WackySocket class.)

Not really. #include inserts the text of an entire header file and compiles it as if it were part of the original file. import just tells the compiler that when it sees "Sorter", it should replace it with "com.stinky.util.Sorter".

Three things. One: put a line like "package com.stinky.util;" in the very top of your source file. Two: put the source file in a directory named util, in a directory named stinky, in a directory named com.

The third thing is, make sure the top-level package directory ("com" in my example) is inside a directory that’s in your classpath.

You don’t want to know.

It’s a list of directories, stored in an environment variable named CLASSPATH, that Java searches every time it’s looking for a source or class file. The trick is, the directories in the classpath don’t actually contain the files. They contain the directories that contain the directories that contain the files.

To further confuse matters, the classpath may also contain the names of archive files, specifically ZIP or JAR files. In that case, Java pretends that these archives are actually directories — directories that contain the files and subdirectories in the archive files.

Just make sure "." (dot) is in your classpath and follow The Rules (see main article).

Be careful out there.

Alex Chaffee is a leading consultant and trainer specializing in Java. He has been promoting, teaching, and programming in Java since 1995. As the director of software engineering for EarthWeb, he co-created Gamelan the official directory for the Java community. He has made presentations at numerous users groups and conferences, written articles for several Java magazines and contributed to the book The Official Gamelan Java Directory. He has also published research papers on evolutionary computation (a.k.a., “genetic algorithms”) and on implicit memory in human cognition. You can contact him at alexc@purpletech.com.