Java Programming Notes # 1794
Preface
New features in SDK Version 1.4.0
The recently released JavaTM 2 SDK, Standard Edition Version 1.4 contains a
large number of new features, including the concept
of IO channels. The first lesson in this miniseries, entitled
FileChannel Objects in Java, Background Information, introduced you to the concept of channels
from a read/write IO viewpoint.
Mixed primitive types
The previous lesson, entitled
FileChannel
Objects in Java, Records with Mixed Types, showed you
how to use the FileChannel class along
with the ByteBuffer class to:
- Create records consisting of sequences of data values of mixed primitive
types - Manipulate those records under program control
- Transfer those records between the computer’s memory and a physical disk
file using read/write operations on the channel
Memory-mapped IO
In this lesson, I’m
will teach you how to use the FileChannel class along with the
ByteBuffer class to perform memory-mapped IO for data of type byte.
This is an alternative to the read/write approach discussed in previous
lessons.
This lesson will teach you the basics of memory-mapped IO using data of type
byte.
In the next lesson, I will teach you how to do memory-mapped IO for data
records containing mixed types of data. I will also teach you how to do
memory-mapped IO for different data types using different views of the
data in the buffer.
Viewing tip
You may find it useful to open another copy of this lesson in a separate
browser window. That will make it easier for you to scroll back and
forth among the different listings and figures while you are reading about
them.
Supplementary material
I recommend that you also study the other lessons in my extensive collection
of online Java tutorials. You will find those lessons published at
Gamelan.com.
However, as of the date of this writing, Gamelan doesn’t maintain a consolidated
index of my Java tutorial lessons, and sometimes they are difficult to
locate there. You will find a consolidated index at
www.DickBaldwin.com.
What is a FileChannel?
Sun describes an object of the FileChannel class simply as “A
channel for reading, writing, mapping, and manipulating a file.”
Discussion
and Sample Code
Mapping files into memory
Previous lessons have explained the use of FileChannel objects
for reading and writing files. In this lesson, I will show you how to use FileChannel objects for
mapping files into memory.
Once you map a file into memory, changes that you make to the memory map are
automatically
propagated to the file.
Will discuss sample program in fragments
I will illustrate memory-mapped IO for data of type byte using the sample program named
Channel04.
As usual, I will discuss this program in fragments. You will find a complete listing
of the program in Listing 17 near the end of the lesson.
Description of the program
This program,
which was tested using JDK 1.4.0 under Win2000, illustrates the use of FileChannel objects for
memory-mapped IO.
The program first creates and populates a disk file with data of type
byte, closes all connections between the program and the file, and makes
all references to the data eligible for garbage collection.
Then the program creates a memory map of the existing file,
closes the channel used to create the map, and displays the contents of the map.
Then the program modifies the map, which automatically modifies the contents of the
associated file. (Note that there is no open channel connecting the
file to the program at this point.)
Then
the program reads and displays the modified file.
The above steps illustrate
the basic capability of memory-mapped IO.
Create a
partial read-only map
To illustrate some additional capabilities of
memory-mapped IO, the program then creates a read-only map of
a portion of the modified file and displays it.
The main method
For simplicity, this
program consists of a main method and a static convenience
method used to display the data in a ByteBuffer object.
The first fragment in Listing 1 shows the beginning of the main method.
public static void main(
String[] args){
byte[] array = {65,66,67,68,69,70};
ByteBuffer buf =
ByteBuffer.wrap(array);
Listing 1
|
The wrap method
In an earlier lesson, you learned that one way to create and populate a
ByteBuffer object is to invoke the static wrap method of the
ByteBuffer class to wrap the buffer object around an existing array object
of type byte[]. (I will show you a different way to create a
ByteBuffer object later in this lesson.)
The code in Listing 1 performs such a wrapping operation to create a new
ByteBuffer object, which is populated with six sequential byte values
from 65 through 70 inclusive. This will be the starting point for the
exercises that follow.
Display the data in the buffer
The code in Listing 2 invokes the showBufferData method to display the
contents of the buffer.
showBufferData(buf,"buf-raw data"); Listing 2 |
The showBufferData method was explained in an earlier lesson, and
won’t be discussed further here. You can view the entire method in Listing
17 near the end of the lesson.
The output
The output produced by Listing 2 is shown in Figure 1.
Data for buf-raw data 65 66 67 68 69 70 Figure 1 |
This output simply confirms that the contents of the ByteBuffer object are the same as
the contents of the original array object.
(You also learned in an earlier lesson that changes made to the array
object will be reflected in the buffer and vice versa. In other words, I
can’t tell you how this process is handled in physical memory, but the effect is
as if the array object and the buffer object contain the same data.)
<
Getting a FileChannel object
You can get a FileChannel object that
is connected to an underlying file by invoking the getChannel method
on an existing FileInputStream, FileOutputStream, or RandomAccessFile
object.
A FileChannel object obtained from a FileInputStream object
will be open for reading, but not for writing. A FileChannel
object obtained from a FileOutputStream object will be open for
writing, but not for reading.
A FileChannel object obtained from a RandomAccessFile
object will be open for reading if the RandomAccessFile object was created in the “r”
mode, and will be open for reading and writing if the object was created
in the “rw” mode.
(Note that creating a RandomAccessFile object in rw mode for
a particular file name does not delete an existing file having that name if such
a file already exists.)
Open channel for reading and writing
The code in Listing 3 begins by deleting a file named junk.txt if such
a file already exists.
Then the code in Listing 3 invokes the getChannel method on an
anonymous
RandomAccessFile object’s reference to get a FileChannel
object that can read from and write to a new physical file named junk.txt.
new File("junk.txt").delete();
FileChannel rwCh = (
new RandomAccessFile(
"junk.txt","rw")).
getChannel();
Listing 3
|
The FileChannel object’s reference is stored in the reference variable
named rwCh.
(At this point, the file is empty, with a size of zero.)
Populate the physical disk file
The code in Listing 4 writes the contents of the ByteBuffer object to
the physical file.
System.out.println(
"Bytes written = "
+ rwCh.write(buf));
Listing 4
|
As you learned in a previous lesson, the write method of the
FileChannel class returns the number of bytes written to the file. The
code in Listing 4 displays this value as shown in Figure 2.
Bytes written = 6 Figure 2 |
Disconnect from the file
At this point, for purposes of illustration, I elected to close the
FileChannel object so that it can no longer be used to access the file.
I also elected to
make the original ByteBuffer object and the original array object eligible for garbage collection so that their data is no longer accessible.
Following the execution of the code in Listing 5, the data is available only via the file named
junk.txt.
rwCh.close();
buf = null;
array = null;
Listing 5
|
Get a new FileChannel object
The code in Listing 6 gets a new FileChannel object for reading and
writing the existing file named junk.txt.
rwCh = new RandomAccessFile(
"junk.txt","rw").
getChannel();
Listing 6
|
Map the file into memory
Now we’ve gotten to the interesting part. The code in Listing 7 invokes
the map method of the FileChannel class to map the entire
file named junk.txt into a memory area as a ByteBuffer object.
Then the code in Listing 7 closes the FileChannel object.
long fileSize = rwCh.size();
ByteBuffer mapFile = rwCh.map(
FileChannel.MapMode.READ_WRITE,
0, fileSize);
rwCh.close();
Listing 7
|
(Despite the fact that the channel has been closed, the data in the file
continues to be available via the memory map, as will be demonstrated later.)
A more-detailed discussion
The code in Listing 7 is very important, and I don’t want to pass over it
lightly.
Listing 7 begins by invoking the size method on the FileChannel
object’s reference. The size method returns the current size of the
channel’s file (junk.txt), measured in bytes and returned as type
long.
(At this point, the size of the file is six bytes.)
Invoking the map method
Then the code in Listing 7 uses this size value to invoke the map
method on the FileChannel object’s reference.
The map method deserves some additional discussion.
The MappedByteBuffer class
To begin with, the map method returns a reference to an object of the MappedByteBuffer
class, which I haven’t discussed prior to this point in this series of tutorials. Therefore, I will begin by
explaining the characteristics of a MappedByteBuffer object.
A MappedByteBuffer object
extends ByteBuffer, and is a direct byte buffer whose content is a memory-mapped region of a file.
Created via the map factory method
As you saw in Listing 7, mapped byte buffer objects are created via the map
method of the FileChannel class.
The MappedByteBuffer class
extends the ByteBuffer class, providing operations
that are specific to memory-mapped file regions. Those operations are:
- force – Forces any changes made to this buffer’s content to be
written to the storage device containing the mapped file. - isLoaded – Tells whether or not this buffer’s content is resident
in physical memory. - load – Loads this buffer’s content into physical memory.
Valid until garbage collected
A MappedByteBuffer object and the file mapping that it represents remains valid until
the buffer itself is garbage-collected.
Contents can change or become inaccessible
The contents of a MappedByteBuffer object can change at any time.
This can happen, for example, if the
content of the corresponding region of the mapped
file is changed.
(According to sun, whether or not such changes occur,
and when they occur, is operating-system dependent
and therefore unspecified.)
All or part of a mapped byte buffer may become inaccessible at any time.
This can happen, for
example, if the mapped file is truncated.
An attempt to
access an inaccessible region of a mapped byte buffer will not change the
buffer’s contents and will cause an unspecified exception to
be thrown either at the time of the access or at some later time.
(According to Sun, it is
strongly recommended that appropriate precautions be
taken to avoid the manipulation of a mapped file by this program, or by a
concurrently running program, except to read or write the
file’s content.)
Behavior similar to ByteBuffer object
Otherwise, MappedByteBuffer objects behave no differently than ordinary direct byte
buffers.
Therefore, most of what you have previously learned about manipulating ByteBuffer
objects, (such as getting different views of a ByteBuffer object), also
applies to MappedByteBuffer objects.
The map method parameters
As you saw in Listing 7, the map method of the FileChannel
class requires three parameters:
- mode
- position
- size
Using these parameters, the map method maps a region of the channel’s
file directly into memory.
Mapping modes
A region of a file may be mapped into memory in one of three modes:
- read-only
- read/write
- private
If the file is mapped in read-only mode, any attempt to modify the contents
of the resulting buffer will cause an exception to be thrown. In this
case, the channel must have been opened for reading.
In read/write mode, changes made to the contents of the buffer will be
propagated to the file. (Note, however, that the changes may or may not be
visible to other programs that may have mapped the same file.) In this
case, the channel must have been opened for both reading and writing.
In private mode, changes made to the contents of the buffer will not be
propagated to the file. Similarly, in this case, the channel must have
been opened for both reading and writing.
The position and size parameters of the map method
The position parameter of the map method defines the starting
point in the file that is mapped into the buffer.
The size property defines the number of bytes that will be mapped.
(The combination of the position and size parameters can be used to define
a region of sequential bytes within the file that will be mapped into memory.
This will be illustrated later in the sample program.)
Map the entire file into memory
The invocation of the map method in Listing 7 causes the entire file
named junk.txt to be mapped in read/write mode into the ByteBuffer
object referred to by the reference variable named mapFile.
(Since I had no plans to invoke any of the three methods defined in the
MappedByteBuffer class, it was appropriate for me to save the reference
as type ByteBuffer instead of as type MappedByteBuffer.)
The position, limit, and capacity properties
The MappedByteBuffer object returned by the map method will
have a position of zero. Its limit and capacity will be equal to the
value of the size parameter passed to the map method.
Can close the FileChannel object
Once a mapping is established, it is not dependent on the FileChannel
object that was used to create it. Closing the channel has no effect upon
the validity of the mapping.
Mapping files is expensive
According to Sun:
"For most operating systems, mapping a file into memory is more expensive
than reading or writing a few tens of kilobytes of data via the usual read and
write methods. From the standpoint of performance it is generally only worth
mapping relatively large files into memory."
Display the memory map
The code in Listing 8 displays the contents of the memory map. Note
that since the memory map is simply a ByteBuffer object, the same method
used earlier to display the contents of a ByteBuffer object can be used
to display the contents of the memory map.
showBufferData(
mapFile,"mapFile");
Listing 8
|
The output
The output produced by the code in Listing 8 is shown in Figure 3.
Happily, the data in the memory map matches the original data indicating that no
data corruption or modification has occurred to this point in the program.
Data for mapFile 65 66 67 68 69 70 Figure 3 |
Modify values in the memory map
The code in Listing 9 purposely modifies the contents of the memory map,
which in turn, modifies the contents of the disk file represented by the memory
map (changes to the map propagate to the file).
mapFile.position(2);
mapFile.put((byte)1);
mapFile.put((byte)2);
Listing 9
|
(Note that the channel that was used to create the map was closed before the
modifications were made in Listing 9. As mentioned earlier, once the map
is established, the validity of the map is not dependent on the channel used to
establish the map.)
The code in Listing 9 sets the position property of the map to a value of 2
and then executes two sequential relative put operations on the map. This
causes the values in the third and fourth elements in the map to be changed from
the values of 67 and 68 to the values of 1 and 2.
Display the modified map
The code in Listing 10 displays the contents of the modified memory map.
showBufferData(
mapFile,"mod-mapFile");
Listing 10
|
This produces the output shown in Figure 4
Data for mod-mapFile 65 66 1 2 69 70 Figure 4 |
As you can see, the third and fourth values have been changed relative to
their original value (boldface added for emphasis).
(At the risk of being redundant, I am going to stress that making this change to
the values in the memory map causes the values in the associated disk file named
junk.txt to change accordingly.)
Confirm modifications to the file
Now it’s time to confirm that the modification of the values in the memory
map caused the values stored in the file to be changed accordingly.
The code in Listing 11 gets a FileChannel object that can be used to
read the contents of the file named junk.txt.
(Note that the channel was opened in read-only mode.)
FileChannel newInCh =
new RandomAccessFile(
"junk.txt","r").
getChannel();
Listing 11
|
Create a ByteBuffer object through allocation
The code in Listing 12 invokes the allocate method of the
ByteBuffer class to create a new empty
ByteBuffer object.
ByteBuffer newBuf =
ByteBuffer.allocate(
(int)fileSize);
Listing 12
|
(What is an empty ByteBuffer object? Although I can’t say for
absolute certain, I suspect that each byte in the new ByteBuffer object
will have a value of zero. I base this on the fact that the buffer will
have a backing array of type byte[]. Normally a new array object of
type byte[] will have each of its elements initialized to the default
value of zero (unless they are purposely initialized to some other value).
Thus, an allocated ByteBuffer object wraps a new array object, which has
default initialization of its elements. A reference to the backing array
can be obtained by invoking the array method on the ByteBuffer
object.)
An alternative approach
The allocate method provides an alternative to the wrap method
for creating a new ByteBuffer object. In this case, the size of the
ByteBuffer object was set to match the size of the file, which was
determined earlier in Listing 7.
Populate the ByteBuffer object
The code in Listing 13 uses the read method of the FileChannel
class to read the file contents into the new ByteBuffer object.
System.out.println(
"Bytes read = "
+ newInCh.read(newBuf));
newInCh.close();
Listing 13
|
After reading the data from the file into the ByteBuffer object, the code in Listing 13 closes the
FileChannel object.
Display the data read from the file
The code in Listing 14 invokes the showBufferData method again, this
time to display the data that was read from the disk file into the ByteBuffer
object.
showBufferData(newBuf,"newBuf"); Listing 14 |
The output produced by the code in Listings 13 and 14 is shown in Figure 5
(boldface added for emphasis).
Bytes read = 6 Data for newBuf 65 66 1 2 69 70 Figure 5 |
File data was modified
As you can see, the third and fourth values in the disk file were modified
earlier when changes were made to those two values in the memory map. This
confirms that changes made to the memory map propagate to the disk file
represented by the map.
Mapping a region of a file
You can map an entire file, or a region within a file into a memory map.
The code in Listing 15 gets a new read-only FileChannel object, and
uses it to map a region of the same file named junk.txt into a new
read-only ByteBuffer object.
FileChannel rCh =
new RandomAccessFile(
"junk.txt","r").
getChannel();
ByteBuffer roMapFile = rCh.map(
FileChannel.MapMode.READ_ONLY,
1, fileSize-2);
rCh.close();
Listing 15
|
The definition of the region
The selected region begins at a position value of 1 and includes a number of
bytes that is two less than the number of bytes in the file.
(Thus, the region includes all but the first and last bytes in the file.)
Close the channel
Having mapped the file into memory, the code in Listing 15 closes the
FileChannel object used to perform the mapping operation.
Display partial memory map
The code in Listing 16 displays the contents of the partial memory map.
showBufferData(roMapFile,
"roMapFile");
Listing 16
|
The output produced by the code in Listing 16 is shown in Figure 6.
Data for roMapFile 66 1 2 69 Figure 6 |
As you can see, only four of the six bytes in the file were mapped into the
memory map. (The first and last bytes in the file were excluded.)
A read-only map
If this were a read/write memory map, it would be possible to modify any of
those four values in the map and have the changes propagate to the file.
However, this is a read-only map. Therefore, any attempt to modify any
of the values in the map will cause a ReadOnlyBufferException to be
thrown.
That’s it for now
By now you should understand a quite a lot about the use of FileChannel
objects to create and manipulate memory maps of disk files.
Run the Program
If you haven’t already done so, I encourage you to copy the code from Listing
17 into your text editor, compile it, and execute it. Experiment
with it, making changes, and observing the results of your changes.
Remember, however, that you must be running Java version 1.4.0 or later
to compile and execute this program.
Summary
In this lesson, I have taught you how to use the FileChannel class along with the
ByteBuffer class to perform memory-mapped IO for data of type byte.
This is an alternative to the read/write approach discussed in previous
lessons.
This lesson teaches the basics of memory-mapped IO using data of type
byte.
What’s Next?
In the next lesson, I will teach you how to do memory-mapped IO for data
records containing mixed types of data. I will also teach you how to do
memory-mapped IO for different data types using different views of the data in
a buffer.
Future plans
As time goes on, I plan to publish additional lessons that will help
you learn to use other new IO features including:
- File locks
- Character-set encoders and decoders
- Pattern matching on files using regular expressions
- Socket channels for both clients and servers
- Non-blocking reads
- Non-blocking servers
Complete Program Listing
A complete listing of the program discussed in this lesson is shown in
Listing 17 below.
/* File Channel04.java
Copyright 2002, R.G.Baldwin
Illustrates use of FileChannel objects
for mapped-memory IO.
First creates and populates a disk
file with data of type byte.
Then creates a memory map of the
existing file, and modifies the map,
which modifies the contents of the
file.
Then reads and displays the modified
file.
Then creates a read-only map of part of
the modified file and displays it.
Tested using JDK 1.4.0 under Win2000
The output is:
Data for buf-raw data
65 66 67 68 69 70
Bytes written = 6
Data for mapFile
65 66 67 68 69 70
Data for mod-mapFile
65 66 1 2 69 70
Read/display modified file
Bytes read = 6
Data for newBuf
65 66 1 2 69 70
Data for roMapFile
66 1 2 69
**************************************/
import java.io.*;
import java.nio.channels.*;
import java.nio.*;
class Channel04{
public static void main(
String[] args){
//Create and populate an array obj
byte[] array = {65,66,67,68,69,70};
//Wrap array in a buffer to create
// a ByteBuffer object. Then
// display the contents of the
// buffer.
ByteBuffer buf =
ByteBuffer.wrap(array);
showBufferData(buf,"buf-raw data");
try{
//Delete the file named junk.txt
// if it already exists
new File("junk.txt").delete();
//Get a channel for writing a
// random access file in rw
// mode.
FileChannel rwCh = (
new RandomAccessFile(
"junk.txt","rw")).
getChannel();
//Write buffer data to the file
System.out.println(
"Bytes written = "
+ rwCh.write(buf));
//Close the channel so that it
// can no longer be used to
// access the file.
rwCh.close();
//Make original ByteBuffer object
// and original array object
// eligible for garbage
// collection so that their data
// is no longer accessible. The
// data is now available only
// via the file named junk.txt.
buf = null;
array = null;
//Get a new FileChannel object
// for reading and writing the
// existing file
rwCh = new RandomAccessFile(
"junk.txt","rw").
getChannel();
//Map entire file to memory and
// close the channel
long fileSize = rwCh.size();
ByteBuffer mapFile = rwCh.map(
FileChannel.MapMode.READ_WRITE,
0, fileSize);
rwCh.close();
//Display contents of memory map
showBufferData(
mapFile,"mapFile");
//Change contents of the map and
// hence the file. Note that
// the channel is closed.
mapFile.position(2);
mapFile.put((byte)1);
mapFile.put((byte)2);
//Display new contents of memory
// map
showBufferData(
mapFile,"mod-mapFile");
System.out.println(
"Read/display modified file");
//Get new channel for read only
FileChannel newInCh =
new RandomAccessFile(
"junk.txt","r").
getChannel();
//Allocate (don't wrap) a new
// ByteBuffer
ByteBuffer newBuf =
ByteBuffer.allocate(
(int)fileSize);
//Read file data into the new
// buffer, close the channel, and
// display the data.
System.out.println(
"Bytes read = "
+ newInCh.read(newBuf));
newInCh.close();
showBufferData(newBuf,"newBuf");
//Get new read-only partial
// mapping for file and display
FileChannel rCh =
new RandomAccessFile(
"junk.txt","r").
getChannel();
ByteBuffer roMapFile = rCh.map(
FileChannel.MapMode.READ_ONLY,
1, fileSize-2);
rCh.close();
showBufferData(roMapFile,
"roMapFile");
//Following put throws java.nio.
// ReadOnlyBufferException, so
// it is disabled.
roMapFile.position(2);
//roMapFile.put((byte)88);
}catch(Exception e){
System.out.println(e);}
}// end main
//---------------------------------//
static void showBufferData(
ByteBuffer buf, String name){
//Displays byte buffer contents
//Save position
int pos = buf.position();
//Set position to zero
buf.position(0);
System.out.println(
"Data for " + name);
while(buf.hasRemaining()){
System.out.print(
buf.get() + " ");
}//end while loop
System.out.println();//new line
//Restore position and return
buf.position(pos);
}//end showBufferData
//---------------------------------//
}//end class Channel04 definition
Listing 17
|
Copyright 2002, Richard G. Baldwin. Reproduction in whole or in
part in any form or medium without express written permission from Richard
Baldwin is prohibited.
About the author
Richard Baldwin
is a college professor (at Austin Community College in Austin, TX) and
private consultant whose primary focus is a combination of Java, C#, and
XML. In addition to the many platform and/or language independent benefits
of Java and C# applications, he believes that a combination of Java, C#,
and XML will become the primary driving force in the delivery of structured
information on the Web.
Richard has participated in numerous consulting projects, and he
frequently provides onsite training at the high-tech companies located
in and around Austin, Texas. He is the author of Baldwin’s Programming
Tutorials,
which has gained a worldwide following among experienced and aspiring programmers.
He has also published articles in JavaPro magazine.
Richard holds an MSEE degree from Southern Methodist University and
has many years of experience in the application of computer technology
to real-world problems.
-end-
