March 5, 2021
Hot Topics:

Java 2 security and stack inspection

  • By Gary McGraw
  • Send Email »
  • More Articles »

With the release of Java 2 (a.k.a., JDK 1.2), Sun Microsystems upped the ante on sophisticated security models for mobile code.

In early Java incarnations, untrusted code was constrained to a security sandbox. Code signing was added to the security toolbox in 1997 with the introduction of JDK 1.1. Together, sandboxing and signatures make for a powerful approach to securing untrusted code. (See Sandboxes and signatures: The future of executable content.) However, even with code signing, the JDK 1.1 trust model is black and white; that is, code is either completely trusted or completely untrusted. Java 2 changes all that by allowing fine-grained security policy and access control enforcement. The access control system in Java 2 is built around the concept of stack inspection.

Java 2 code running on the new VMs can be granted special permissions and have its access checked against policy as it runs.

What's the Big Idea?

Everyone agrees that code signing makes the Java security model a lot more complicated, not to mention actually using the new system. Where security is concerned, complexity is bad, because it increases the odds of an error in the system's design or implementation. If we're going to add all of this complexity, what exactly is it that we are gaining? What's the main goal?

The main goal is to gain better control over the security of mobile code. We can achieve this goal by winning the battle on three fronts. By adding code signing and expanding beyond a black-and-white trust model, we hope to gain:

  1. The ability to grant privileges when they're needed.
  2. The ability to have code operate with the minimum necessary privileges.
  3. The ability to closely manage the system's security configuration.

The Java 2 security model addresses these goals by providing a policy-based security enforcement mechanism based on stack inspection.

At its heart, the Java 2 security model has a simple idea: Make all code run under a security policy that grants different amounts of privilege to different programs. While the idea may be simple, in practice, creating a coherent policy is quite difficult.

Java 2 code running on the new Java VMs can be granted special permissions and have its access checked against policy as it runs. The cornerstone of the system is policy (something that will not surprise security practitioners in the least). Policy can be set by the user (usually a bad idea) or by the system administrator, and is represented in the class

. Herein rests the Achilles' Heel of Java 2 security. Setting up a coherent policy at a fine-grained level takes experience and security expertise. Today's harried system administrators are not likely to enjoy this added responsibility. On the other hand, if policy management is left up to users, mistakes are bound to be made. Users have a tendency to prefer "cool" to "secure."

Code can be signed with multiple keys and can potentially match multiple policy entries.

Executable code is categorized based on its URL of origin and the private keys used to sign the code. The security policy maps a set of access permissions to code characterized by particular origin/signature information. Protection domains can be created on demand and are tied to code with particular
properties. If this paragraph confuses you, imagine trying to create and manage a coherent mobile code security policy!

Code can be signed with multiple keys and can potentially match multiple policy entries. In this case, permissions are granted in an additive fashion.

A simple example

An easy example of how this works in practice is helpful. First, imagine a policy representing the statement "code from 'www.rstcorp.com/' applet signed by 'self' is given permission to read and write files in the directory /applet/tmp and connect to any host in the rstcorp.com domain." Next, a class that is signed by "self" and that originates from "www.rstcorp.com/" applet arrives. As the code runs, access control decisions are made based on the permissions defined in the policy. The permissions are stored in permission objects tracked by the Java runtime system. Technically, access control decisions are made with reference to the runtime call stack associated with a thread of computation (more on this below).

Access Control and Stack Inspection

The main business of computer security is controlling access to protected resources. A common approach, taken for decades by security researchers and practitioners, is to set up groups of users and sets of permissions. The idea is to define a logical system in which entities known as principals (often corresponding one-to-one with code owned by users or groups of users) are authorized to access a number of particular protected objects (often system resources such as files).

A good analogy is the notion of user IDs and file permissions found in modern operating systems like Unix and Windows NT. In these systems, logical groupings of users are given particular privileges to read, write, and execute files. These groupings can be used to separate groups so that appropriate boundaries can be placed between them.

Java implements such a system by allowing security-checking code to examine the runtime stack for frames executing untrusted code.

Not surprisingly, Java's language-based approach to security makes use of groupings and permissions. Sometimes a Java application (say, a Web browser) needs to run untrusted code within itself. In this case, Java system libraries need some way of distinguishing between calls originating in untrusted code and calls originating from the trusted application itself. Clearly, the calls originating in untrusted code need to be restricted to prevent hostile activities. By contrast, calls originating in the application itself should be allowed to proceed (as long as they follow any security rules that the operating system mandates). The question is, how can we implement a system that does this?

Java implements such a system by allowing security-checking code to examine the runtime stack for frames executing untrusted code. Each thread of execution has its own runtime stack (see Figure 1). Security decisions can be made with reference to this check. This is called stack inspection [Wallach and Felten, 1998]. All the major vendors have adopted stack inspection to meet the demand for more flexible security policies than those originally allowed under the old sandbox model. Stack inspection is used by Netscape Navigator, Microsoft Internet Explorer, and Sun Microsystems' Java 2. (Interestingly, Java is thus the most widespread use of stack inspection for security ever. You can think of it as a very big security-critical experiment.)

Figure 1: A runtime stack tracks method calls.

Simple stack inspection

Netscape 3.0's stack-inspection-based model (and every other black-and-white security model) is a simple access control system with two principals: system and untrusted. Just to keep things simple, the only privilege available is full.

In this model, every stack frame is labeled with a principal (system if the frame is executing code that is part of the VM or the built-in libraries and untrusted otherwise). Each stack frame also includes a flag that specifies whether privilege is full. A system class can set this flag, thus enabling its privilege. This need only be done when something dangerous must occur - something that not every piece of code should be allowed to do. Untrusted code is not allowed to set the flag. Whenever a stack frame completes its work, its flag (if it has one) disappears.

Every method about to do something potentially dangerous is forced to submit to a stack inspection. The stack inspection is used to decide whether the dangerous activity should be allowed. The stack inspection algorithm searches the frames on the caller's stack in sequence from the newest to the oldest. If the search encounters an untrusted stack frame (which as we know can never get a privilege flag), the search terminates, access is forbidden, and an exception is thrown. The search also terminates if a system stack frame with a privilege flag is encountered. In this case, access is allowed (see Figure 2).

In the example shown in Figure 2, each stack is made of frames with three parts: a privilege flag (where full privilege is denoted by an X), a principal entry (untrusted or system), and a method. In STACK A, an untrusted applet is attempting to use the

method to access a file in the browser's cache. The VM makes a decision regarding whether to set the privilege flag (which it does) by looking at the parameters in the actual method invocation. Since the file in this case is a cache file, access is allowed. In short, a system-level method is doing something potentially dangerous on behalf of the untrusted code. In STACK B, an untrusted applet is also attempting to use the
method; however, in this case, the file argument is not a browser cache file but a normal file in the filesystem. Untrusted code is not allowed to do this, so the privilege flag is not set by the VM and access is denied.

Figure 2: Two examples of simple stack inspection.

Real stack inspection

The simple example of stack inspection just given is only powerful enough to implement black-and-white trust models. Code is either fully trusted (and granted full permission at the same level as the application) or untrusted (and allowed no permission to carry out dangerous operations). However, what we want is the ability to create a shades-of-gray trust model. How can we do that?

It turns out that if we generalize the simple model, we get what we need. The first step is to add the ability to have multiple principals. Then we need to have many more specific permissions than full. These two capabilities allow us to have a complex system in which different principals can have different degrees of permission in (and hence, access to) the system.

Page 1 of 2

This article was originally published on May 12, 1999

Enterprise Development Update

Don't miss an article. Subscribe to our newsletter below.

Thanks for your registration, follow us on our social networks to keep up-to-date