Introduction to the Java Robot Class in Java

Developer.com content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.

Java Programming Notes # 1472

• Preface
• Preview
• Discussion and Sample Code
• Run the Program
• Summary
• What’s Next?
• Complete Program Listings

Programming in Java doesn’t have to be dull and boring.  In fact, it’s possible
to have a lot of fun while programming in Java.  This is the first lesson in a
miniseries that show you how to use the Robot class to write programs
that are both fun and useful.

New features in SDK Version 1.3

One of the new features that was released in SDK Version 1.3 was the
Robot class.  According to Sun,

"This class is used to generate native system input events for the
purposes of test automation, self-running demos, and other applications where
control of the mouse and keyboard is needed. The primary purpose of Robot is to
facilitate automated testing of Java platform implementations."

Different from the system event queue

I have published several tutorial lessons that show you how to create events
and post them to the system event queue.  (For example, see lesson 104
at
http://www.dickbaldwin.com/tocmed.htm.)  However, it is important
to note that using a Robot object to fire events is significantly
different from posting events to the system event queue.

Here is how Sun compares the Robot class and the system event queue.

"Using the class to generate input events differs from posting events to
the AWT event queue or AWT components in that the events are generated in the
platform’s native input queue. For example, Robot.mouseMove will
actually move the mouse cursor instead of just generating mouse move events."

Potential problem areas

Basically, a Robot object makes it possible for your program to
temporarily take over control of the mouse and the keyboard.  However, this
may not be possible on all operating systems.  For example, Sun has this to
say.

"Note that some platforms require special privileges or extensions to
access low-level input control. If the current platform configuration does not
allow input control, an AWTException will be thrown when trying to
construct Robot objects."

Sun goes on to provide some examples of cases where an AWTException
might be thrown.

A word of caution

A runaway Java Robot object has the ability to wrest control away from
the human user, so you need to be a little careful.  For example, if you
allow your Java Robot program to go into an infinite loop, making mouse
moves, clicking the mouse, and entering keystrokes, you may find that the only
practical way to regain control of your computer is to either turn off the power
or press the reset button to force your computer to restart.

Three lessons in the miniseries

According to my current planning, this miniseries on the Robot class
will consist of three lessons.  This first lesson will demonstrate the
low-level nature of the behavior of an object of the Robot class. 
This will be accomplished by showing you how to create a Java robot that can
manipulate other non-Java programs, such as Windows Notepad and Internet
Explorer.

The second lesson in the miniseries will show you how to use a robot to
perform automatic testing on a Java GUI.

The third lesson will show you how to write an animated robot program to
provide a visual demonstration of the use of a Java GUI.

Viewing tip

You may find it useful to open another copy of this lesson in a separate
browser window.  That will make it easier for you to scroll back and
forth among the different listings and figures while you are reading about
them.

Supplementary material

I recommend that you also study the other lessons in my extensive collection
of online Java tutorials.  You will find those lessons published at
Gamelan.com. 
However, as of the date of this writing, Gamelan doesn’t maintain a consolidated
index of my Java tutorial lessons, and sometimes they are difficult to
locate there.  You will find a consolidated index at
www.DickBaldwin.com.

Preview

I will discuss two sample programs named Robot04 and Robot05 in this
lesson.  The sample programs will illustrate:

• How to start a non-Java program running in a separate process.
• How to programmatically invoke mouse clicks on non-Java programs, with
  the result being the same as that, which would be experienced if a human user were to
  produce those same mouse clicks.
• How to programmatically enter text into a non-Java program that is
  running and ready to accept keyboard input.

I will begin with a discussion of the program named Robot04, followed by
a discussion of the program named Robot05.

Description of the program named Robot04

The Robot class makes it possible for your Java program to temporarily take
control of the
mouse and keyboard input functionality at the operating-system level.

(In fact, if you accidentally allow your program to take control of the mouse
and the keyboard in an infinite loop, the program can be very difficult to
terminate.)

This program illustrates the low-level nature of an object of the
Robot class.  In this program, a Robot object is designed to click the Close button
on a non-Java program (such as Internet Explorer) causing that program to terminate.

Should work in Windows 1024 by 768 mode

This program should
run successfully without modification if you are running
Windows with a screen resolution of 1024 by 768 pixels.  If you are running under a different
operating system, or a different screen resolution, you may need to modify the coordinate values in the program
to cause this program to behave as described.  (A comment is provided in
the code to show you where to make the modification.)

Open Internet Explorer in full-screen mode

To see the
intended behavior of this program, start Internet Explorer,
(or some other program with a Close button in its upper right corner) and maximize it to full-screen mode.  This should cause the
Close button with the X for that program to appear in the upper right-hand corner of the screen.

This program will click the Close button

Then run this program at the command prompt.  There is no GUI for this
program.  The
program simply starts running, performs the programmed action, and terminates.

If all goes well, this program will use an object of the
Robot class to position the mouse pointer over the Close button
belonging to the other program.  Then this program will use the Robot
object to click that button.  The result of clicking the button will be the
same as if a human user had clicked the
Close button on that program.

(Frequently, the result of clicking the
Close button on a program’s main GUI is to terminate the program. 
That is the case with Internet Explorer.  However, other programs, such as
Windows Notepad, may bring up a dialog asking the user to confirm something
such as saving a file.  This program is not designed to deal with such a
dialog.)

If
things don’t work as described …

If you don’t get this result, try adjusting the coordinate values in the program to make certain that the mouse pointer is automatically positioned on the
Close button before the mouse click is executed.

(Delays are inserted in the behavior of this program so that you can view the process in slow motion.)

Requires version 1.3 or later

The Robot
class was first released in SDK Version 1.3, so you will need to be running
Version 1.3 or later.  This program was tested on my machine using SDK 1.4.1 under WinXP

The main method
for the program named Robot04

I will discuss this program in fragments.  A complete listing of the
program is shown in Listing 10 near the end of the lesson.

The program begins in Listing 1 where the main
method starts by instantiating a new object of the Robot class.
 

public class Robot04{
  public static void main(String[] args)
                             throws AWTException{
    Robot robot = new Robot();

Listing 1

Mouse and keyboard input under program control

As mentioned earlier, an object of the Robot class makes it possible
for your program to temporarily gain control of the mouse and keyboard.

(The Robot class also provides some other interesting features,
such as the ability to create an image containing pixels read from the screen,
which are not discussed in this lesson.  See Sun’s documentation for a
description of those features.)

Several instance methods are available

The Robot class provides several instance methods, (including the
following), by which your program can produce mouse and keyboard input, just as
though that input were being provided by a human user.

• mouseMove – Moves the mouse pointer to a set of specified absolute
  screen coordinates given in pixels.
• mousePress – Presses one of the buttons on the mouse.
• mouseRelease – Releases one of the buttons on the mouse.
• keyPress – Presses a specified key on the keyboard.
• keyRelease – Releases specified key on the keyboard.

Move the mouse pointer

The code in Listing 2 causes the mouse pointer to move to a location 1005
pixels to the right and 10 pixels down from the upper left-hand corner of the
screen.

(If you watch carefully while the program is running, you will see the
mouse pointer move to this location.

If your program attempts to move the mouse
pointer to a location specified by negative coordinates, it will move to the
0,0 location in the upper-left corner of the screen.  If your program moves
the mouse pointer off
the screen with positive coordinates, the pointer will simply disappear.)

    robot.mouseMove(1005,10);

Listing 2

Should point to the Close button of Internet
Explorer

For a screen resolution of 1024 by 768 with a Windows look and feel, the coordinates 1005, 10 should cause
the mouse pointer to point to the Close button on a full-screen version
of Internet Explorer.  If this is not the case on your system, you may need
to modify these coordinate values to cause the mouse to point to the Close
button.

Insert a delay

The code in Listing 3 inserts a delay in the process by causing the thread to
sleep for two seconds (2000 milliseconds).  This should make it
possible for you to view the pointer in its new location before anything else
happens.

    robot.delay(2000);

Listing 3

What about InterruptedExceptions?

Sometimes when you put a thread to sleep, you may want to be able to catch
InterruptedExceptions that may occur on that thread.  If you have that
need, you should use the sleep method of the Thread class instead
of the delay method of the Robot class.

Press the left mouse button and delay again

The code in Listing 4 invokes the mousePress method on the Robot
object to execute a press of the left mouse button at the current position of
the mouse pointer.
 

    robot.mousePress(InputEvent.BUTTON1_MASK);
    robot.delay(2000);

Listing 4

The mousePress method accepts an incoming int
parameter that specifies which mouse button is pressed.  Suitable values
for specifying the left, center, and right mouse buttons are found as constants
in the InputEvent class as shown below:

• InputEvent.BUTTON1_MASK
• InputEvent.BUTTON2_MASK
• InputEvent.BUTTON3_MASK

(Note that the Robot class was updated in SDK V1.4 to
support a mouse wheel.  According to Sun, the mouseWheel method of
the Robot class "Rotates the scroll wheel on wheel-equipped mice." 
That capability is not illustrated in this lesson.)

Does anything interesting happen?

Whether or not something interesting happens as a result of invoking the
mousePress method in Listing 4 depends on what the mouse pointer is pointing
to when the method is invoked.  The same thing will happen that would
happen if a human user were to press the left mouse button when the mouse
pointer is pointing to the same location on the screen.

If the pointer happens to be pointing at a sensitive location on the
screen, something interesting will happen.  If the mouse pointer happens to be pointing
to a non-sensitive location on the screen, the mouse press will simply be
ignored and nothing interesting will happen.

(As an interesting exercise, try modifying this program to invoke a
right-button click (BUTTON3_MASK) on a blank area of a Windows screen. 
This should cause a typical Windows popup menu to appear.)

Another delay

The code in Listing 4 includes another two-second delay.  If the mouse
pointer is pointing to the Close button when the mousePress
method is invoked, you should be able to see the Close button in its
depressed state for two seconds. 

Does not terminate Internet Explorer

Simply pressing the left
mouse button on the Close button on Internet Explorer won’t terminate
the program.  A mouse press followed by a mouse release is required to
terminate the program.

Release the left mouse
button

The code in Listing 5 invokes the mouseRelease method
causing the program to behave as if a human user has released
the left mouse button (after first pressing that button).
 

    robot.mouseRelease(InputEvent.BUTTON1_MASK);
  }//end main

}//end class Robot04

Listing 5

Assuming that the mouse pointer is pointing to the Close button on
Internet Explorer, the code in Listing 5 should cause the Close button to
pop back out and should cause the Internet Explorer program to terminate. 
(However, it will probably happen so fast that you probably won’t see the
button pop back out.)

The code in Listing 5 also ends the main method, ends the class
definition, and thus ends the program.

Recap for the program named Robot04

The Java program named Robot04 causes the mouse to move to a location
that is believed to be the location of the Close button for a running
instance of Internet Explorer.

(In the next lesson, I will show you how to determine the location of
components in a Java GUI with more certainty.)

Having moved the mouse pointer to that location, Robot04 clicks the
left mouse button in an attempt to cause the running instance of Internet
Explorer to terminate.  If the location of the mouse pointer was correct,
Internet Explorer will terminate.

Now let’s move along to another program that demonstrates use of the keyboard
by a Robot object.

Description of the program named Robot05

In addition to creating low-level mouse inputs, an object of the Robot
class can create low-level keyboard inputs, just as though a human user
were pressing the keys on the keyboard.

Designed to
work specifically with Windows OS

As written, this program will run correctly
only under the Windows operating system.  This is because it specifically makes use of the Windows program named Notepad.

(However, the program could easily be modified to use a similar editor program under a different operating system.)

Low-level keyboard input

This program illustrates the low-level nature of the behavior of an object of the
Robot class by:

• Starting the Windows Notepad editor program in a separate process.
• Using a Robot object to enter text into the Notepad document.

Running the program

This Java program
does not have a GUI.  To see the intended behavior of this program, simply start the program running
from the command line under the Windows operating system.

Notepad program will start

When this
Java program starts running, it causes the Windows Notepad program to start running
in a separate process with a new empty document.

Enter text into the Notepad document and terminate

Then this program uses a
Robot object to enter the word Hello into the Notepad document. 
After that, the Java program terminates.

When the Java program terminates, the Notepad program is still running, and can be saved to a file, or terminated by the user.

Requires SDK V1.3 or later

As mentioned
earlier, the Robot
class was first released in SDK Version 1.3.  Therefore, as was the case
in the previous program, you will need to be running Version 1.3 or later to
successfully compile and execute this program. 
This program was tested on my machine using SDK 1.4.1 under WinXP

Will discuss in fragments

As usual, I
will discuss this program in fragments.  You can view a complete listing
of Robot05 in Listing 11 near the end of the lesson.

Need some keycode data

The purpose of
this program is to demonstrate the ability of a Java program to enter character data into a non-Java
program.  Therefore, we will need some keycode data.

The code in
Listing 6 creates an array object containing the virtual keycodes for the
word hello.  This data will be used later with the
Robot.keyPress and Robot.keyRelease methods, and the
KeyEvent.VK_SHIFT virtual keycode, to mimic the behavior of a human user entering
the characters Hello into a Notepad document.
 

public class Robot05{
  //Create an array of keycode data
  int keyInput[] = {
      KeyEvent.VK_H,
      KeyEvent.VK_E,
      KeyEvent.VK_L,
      KeyEvent.VK_L,
      KeyEvent.VK_O
  };//end keyInput array

Listing 6

Virtual keycodes are used

In one of the technical documents on its site, Sun has this to say about
specifying keycodes for use with a Robot object:

"Notice that virtual keycodes are used to enter keystrokes; keycodes
are not the same as Java characters. KeyEvent.VK_A corresponds to pressing
the unshifted ‘A’ key on a keyboard. If you specify ‘A’ or ‘a’ instead of
KeyEvent.VK_A, you get unexpected results."

Start Notepad running

The main method begins in Listing 7.  The code in Listing 7
starts the Notepad program running in a separate process.  This should
cause the Notepad program to start running and to become the active window, capable
of accepting input from the keyboard.

  public static void main(String[] args)
                 throws AWTException,IOException{

    Runtime.getRuntime().exec("notepad");

Listing 7

Runtime is not affiliated with the Robot class

Note that the exec method used in Listing 7 has nothing to do with the
Robot class.  The Runtime class and its methods have been
available since JDK1.0.

Here is what Sun has to say about the Runtime class.

"Every Java application has a single instance of class Runtime
that allows the application to interface with the environment in which the
application is running. The current runtime can be obtained from the getRuntime method."

Methods of the Runtime class

The Runtime class provides about twenty instance methods that allow you do
unrelated things such as:

• Determine the amount of free memory in the Java Virtual Machine.
• Run the garbage collector.
• Forcibly terminate the currently running Java virtual machine.
• Run the finalization methods of any objects pending finalization.

The exec method of the Runtime class

In addition, the exec method of the Runtime class makes it
possible to "execute a specified string as a command in a separate process."

The code in Listing 7 executes the string "notepad" as a command in a
separate process.  This, in turn, causes the Windows program named Notepad
to start running and to become the active window.

Get a Robot object

The code in Listing 8 gets and saves a reference to a new Robot
object, which will be used to enter text characters into the Notepad edit
window.
 

    Robot robot = new Robot();

Listing 8

Enter the characters into the Notepad edit window

The code in Listing 9 extracts the keycodes previously stored in the array
and invokes the keyPress method to mimic the action of a human
typist.  This causes the characters represented by those keycodes to be
entered into the Notepad editor.
 

    robot.keyPress(KeyEvent.VK_SHIFT);
    for (int cnt2 = 0;
                 cnt2 < keyInput.length; cnt2++){
      if(cnt2 > 0){
             robot.keyRelease(KeyEvent.VK_SHIFT);
      }//end if
      robot.keyPress(keyInput[cnt2]);
      //Insert a one-half second delay between
      // characters.
      robot.delay(500);
    }//end for loop

  }//main
}//end class Robot05

Listing 9

Note upper and lower case characters

Note the combined use of the keyPress and keyRelease
methods along with KeyEvent.VK_SHIFT virtual keycode to mimic the user holding down the
Shift key for entry of the first character, H.

A one-half second delay

The code in Listing 9 also inserts a one-half second delay following the entry
of each character so that you can see them being individually entered into the
Notepad editor.

End of the program

The code in Listing 9 also signals the end of the main method and the end of
the program.

After all of the characters have been entered into the
Notepad editor, the Java program terminates.

Recap for program Robot05

The program named Robot05 starts a new instance of the program named
Notepad running in a separate process, and then enters the characters Hello
into the Notepad document.  Then the Java program terminates, leaving the
Notepad program running.

I encourage you to copy the code from Listings 10 and 11 into your text editor, compile it, and execute it.  Experiment
with it, making changes, and observing the results of your changes.

Remember, however, that you must be running Java V1.3 or later
to compile and execute these programs, because they make use of Java features that
were first released in V1.3.  (In case you decide to make use of the
mouseWheel method of he Robot class, you must be running SDK V1.4 or
later.)

Summary

In this lesson, I have taught you:

• How to start a non-Java program running in a separate process.
• How to programmatically invoke mouse clicks on non-Java programs with the
  result being the same as that, which would be experienced if a human user were to
  produce those same mouse clicks.
• How to programmatically enter text into a non-Java program that is
  running and ready to accept keyboard input.

What’s Next?

The next lesson in this miniseries will show you how to use a robot to
perform automatic testing on a Java GUI.

The third lesson will show you how to write an animated robot program to
provide a visual demonstration of the use of a Java GUI.

/*File Robot04.java
Copyright 2003 R.G.Baldwin

This program illustrates the low-level nature of
the behavior of an object of the Robot class by
programming a Robot object to click the Close
button of a non-Java program (such as Internet
Explorer) and causing that program to terminate.

This program should work without modification if
you are running WinXP with 1024 by 768 screen
resolution.  If you are running under a different
OS, or a different screen resolution, you may
need to modify the coordinate values in the
program.

To see the desired behavior of this program,
start Internet Explorer, (or some other program)
and maximize it to full-screen mode.  This should
cause the Close button with the X for that
program to appear in the upper right-hand corner
of the screen.

Then run this program.  If all goes well, this
program will use an object of the Robot class
to position the mouse pointer over the Close
button of the other program and then click that
button.  The result will be the standard result
of clicking the Close button on that program.
Frequently, the standard result of clicking the
Close button is to terminate the program.

If you don't get this result, try adjusting the
coordinate values in the program to make certain
that the mouse pointer is automatically
positioned on the Close button before the mouse
click is executed.  Delays are inserted so that
you can view the process in slow motion.

Tested using SDK 1.4.1 under WinXP
************************************************/

import java.awt.*;
import java.awt.event.*;

public class Robot04{
  public static void main(String[] args)
                             throws AWTException{
    Robot robot = new Robot();
    //You may need to modify these coordinates if
    // they fail to place the mouse pointer on
    // the Close button of the non-Java program
    // that you are trying to terminate.
    robot.mouseMove(1005,10);

    //Press and then release the left mouse
    // button.  The delays are provided so that
    // you can view the action.  Watch the upper-
    // right corner of the screen when you run
    // this program.
    robot.delay(2000);
    robot.mousePress(InputEvent.BUTTON1_MASK);
    robot.delay(2000);
    robot.mouseRelease(InputEvent.BUTTON1_MASK);
  }//end main
  //-------------------------------------------//

}//end class Robot04

Listing 10

/*File Robot05.java
Copyright 2003 R.G.Baldwin

Note:  As written, this program will only run
correctly under the Windows operating system,
because it makes use of the Windows program named
Notepad.  However, it could easily be modified to
use a similar program under a different operating
system.

This program illustrates the low-level nature of
the behavior of an object of the Robot class by:

1. Starting the Windows Notepad editor program
in a separate process.
2. Programming a Robot object to enter text into
the Notepad document.

To see the desired behavior of this program,
simply start the program running under the Windws
operating system.

When the program starts running, it causes the
Windows Notepad program to start running with
a new empty document.

Then this program uses a Robot object to enter
the word Hello into the Notepad document, and
terminates.

At this point, the Notepad program is still
running, and can be saved to a file, or
terminated by the user.

Tested using SDK 1.4.1 under WinXP
************************************************/

import java.awt.*;
import java.awt.event.*;
import java.io.IOException;

public class Robot05{
  //Create an array of keycode data
  int keyInput[] = {
      KeyEvent.VK_H,
      KeyEvent.VK_E,
      KeyEvent.VK_L,
      KeyEvent.VK_L,
      KeyEvent.VK_O
  };//end keyInput array

  public static void main(String[] args)
                 throws AWTException,IOException{

    //Start the Windows Notepad program running
    // in a separate process.  It should become
    // the active window, capable of accepting
    // input from the keyboard.
    Runtime.getRuntime().exec("notepad");

    //Get a Robot object that will be used to
    // enter characters into the Notepad document
    Robot robot = new Robot();

    //Enter the keycodes contained in the
    // keyInput array into the Notepad document.
    // Make the first character upper case and
    // the remaining characters lower case.
    robot.keyPress(KeyEvent.VK_SHIFT);
    for (int cnt2 = 0;
                 cnt2 < keyInput.length; cnt2++){
      if(cnt2 > 0){
             robot.keyRelease(KeyEvent.VK_SHIFT);
      }//end if
      robot.keyPress(keyInput[cnt2]);
      //Insert a one-half second delay between
      // characters.
      robot.delay(500);
    }//end for loop
  }//main
}//end class Robot05

Listing 11

About the author

Richard Baldwin
is a college professor (at Austin Community College in Austin, TX) and
private consultant whose primary focus is a combination of Java, C#, and
XML. In addition to the many platform and/or language independent benefits
of Java and C# applications, he believes that a combination of Java, C#,
and XML will become the primary driving force in the delivery of structured
information on the Web.

Richard has participated in numerous consulting projects, and he
frequently provides onsite training at the high-tech companies located
in and around Austin, Texas.  He is the author of Baldwin’s Programming
Tutorials,
which has gained a worldwide following among experienced and aspiring programmers.
He has also published articles in JavaPro magazine.

Richard holds an MSEE degree from Southern Methodist University and
has many years of experience in the application of computer technology
to real-world problems.

Baldwin@DickBaldwin.com

Copyright 2003, Richard G. Baldwin.  Reproduction in whole or in part in any form or medium without express written permission from Richard Baldwin is prohibited.

-end-