March 1, 2021
Hot Topics:

Java 2 security and stack inspection

  • By Gary McGraw
  • Send Email »
  • More Articles »

Research into stack inspection shows that four basic primitives are all that are required to implement a real stack inspection system. In particular, see Dan Wallach's Ph.D. thesis at Princeton [Wallach, 1998] and the paper Understanding Java Stack Inspection [Wallach and Felten, 1998]. Each of the major vendors uses different names for these primitives, but they all boil down to the same four essential operations (all explained more fully in the following discussions):

  • enablePrivilege()
  • disablePrivilege()
  • checkPrivilege()
  • revertPrivilege()

Some resources such as the file system or network sockets need to be protected from use (and possible abuse) by untrusted code. These resources are protected by permissions. Before code (trusted or otherwise) is allowed access to one of these resources, say, R, the system must make sure to call


All three major Java vendors implement a very similar (and simple) stack-inspection algorithm.

Java libraries are set up in such a way that dangerous operations must go through a Security Manager check before they can occur [McGraw and Felten, 1999]. The Java API provides all calls necessary to implement a virtual OS, thus making isolation of all required security checks possible within the API. When a dangerous call is made to the Java API, the Security Manager is queried by the code defining the base classes. The

method is used to help make behind-the-scenes access control decisions in a very similar fashion. To achieve backwards compatibility, the Security Manager can be implemented using the four stack inspection primitives.

When code wants to make use of some resource, R, it must first call

. When this method is invoked, a check of local policy occurs that determines whether the caller is permitted to use R. If the use is permitted, the current stack frame is annotated with an
mark. This allows the code to use the resource normally.

Permission to use the resource does not last forever; if it did, the system would not work. There are two ways in which the privilege annotation is discarded. One way is for the call to return. In this case, the annotation is discarded along with the stack frame. The second way is for the code to make an explicit call to

. The latter call creates a stack annotation that can hide an earlier enabled privilege. The former simply removes annotations from the current stack frame.

All three major Java vendors implement a very similar (and simple) stack-inspection algorithm. A generalization of this algorithm, after Wallach, is shown in the Listing [Wallach and Felten, 1998].

The question is, if and when will Java 2 security mechanisms be adopted in the field?

The algorithm searches stack frames on the caller's stack, in order, from newest to oldest. If the search finds a stack frame with the appropriate enabled-privilege annotation, it terminates, allowing access. If the search finds a stack frame that is forbidden from accessing the target by local policy, or has explicitly disabled its privileges, the search terminates, forbidding access.

It may seem strange that the vendors take different actions when the search reaches the end of the stack without meeting any of the conditions (sometimes called falling off the end of the stack). Netscape denies permission, while both Microsoft and Sun allow permission. This difference has to do with backward compatibility. The Netscape choice causes legacy code to be treated like an old-fashioned applet, and confined to the sandbox. The Microsoft/Sun choice allows a signed Java application to use its privileges, even without explicitly marking its stack frames, thus making it easy to migrate existing applications. Since Netscape did not support applications, they felt no need to follow the Microsoft/Sun approach and, instead, chose the more conservative course of denying permission. For more implementation detail on the three vendors' different code-signing schemes, see Appendix C of Securing Java [McGraw and Felten, 1999].

Listing 1. An algorithm for stack inspection.


// loop, newest to oldest stack frame

foreach stackFrame{

if (local policy forbids access to target 

by class executing in stackFrame)

throw ForbiddenException;

if (stackFrame has enabled privilege for target)

return; // allow access

if (stackFrame has disabled privilege for target)

throw ForbiddenException;


// if we reach here, we have fallen off the end of the stack

if (Netscape)

throw ForbiddenException;

if (Microsoft IE || Sun JDK 1.2)

return; // allow access


Formalizing stack inspection

Members of Princeton University's Secure Internet Programming team (in particular, Dan Wallach and Edward Felten) have created a formal model of Java's stack inspection system in a belief logic known as ABPL (designed by Abadi, Burrows, Lampson, and Plotkin) [Abadi et al., 1993]. Using the model, the Princeton team demonstrates how Java's access control decisions correspond to proving statements in ABPL. Besides putting Java's stack inspection system on solid theoretical footing, the work demonstrates a very efficient way to implement stack inspection systems as pushdown automata using security-passing style. Interested readers should see [Wallach and Felten, 1998], which is available through the Princeton University Web site.

Will Stack Inspection Succeed?

Java 2, with its built-in stack inspection approach to access control, is a very sophisticated security architecture. The question is, if and when will Java 2 security mechanisms be adopted in the field?

Given the inadequate PKI, especially Internet-wide, I predict Java 2 adoption will unfold as follows. (I reserve the right to be completely wrong, of course.) Early Java 2 adopters will institute Java 2 policies inside their organizations using an in-house PKI. This will allow them to sidestep issues surrounding certificate validation chains to multiple issuing authorities, certificate revocation and aging, and other concerns that always crop up when using a PKI in the real world. Only after these organizations have seen some level of success will business-to-business PKIs begin to appear. Business-to-business connectivity will leverage lessons learned from using Virtual Private Networks (VPNs). Finally, Java 2 may find widespread acceptance on the Internet as a whole, but only after a PKI has been established for some other reason (say encrypted e-mail or IPv6).


  • Abadi, M., Burrows, M., Lampson, B., and Plotkin, G. (1993), A calculus for access control in distributed systems. ACM Transactions on Programming Languages and Systems, 15(4):706-734, September 1993.
  • Gary McGraw and Ed Felten (1999), Securing Java: Getting down to business with mobile code, John Wiley & Sons, NY.
  • Dan Wallach and Ed Felten (1998), "Understanding Java Stack Inspection." In Proceedings of the IEEE Symposium on Security and Privacy, May 1998.
  • Wallach, D. (1998), A New Approach to Mobile Code Security. Ph.D. dissertation, Department of Computer Sciences, Princeton University.


Portions of this article are taken by permission from Securing Java: Getting down to business with mobile code (John Wiley & Sons, 1998).

Gary McGraw, Ph.D., is vice president of corporate technology at Reliable Software Technologies. Dr. McGraw is a noted authority on mobile code security and co-authored both Java Security: Hostile Applets, Holes, & Antidotes (John Wiley & Sons, 1996) and Securing Java: Getting down to business with mobile code (Wiley, 1999), with Prof. Ed Felten of Princeton University. Dr. McGraw has written over fifty peer-reviewed technical publications, consults with major e-commerce vendors, including Visa, and researches software assurance technologies.

Page 2 of 2

This article was originally published on May 12, 1999

Enterprise Development Update

Don't miss an article. Subscribe to our newsletter below.

Thanks for your registration, follow us on our social networks to keep up-to-date