August 30, 2014
Hot Topics:
RSS RSS feed Download our iPhone app

Java, Memory-Mapped IO for Data of Mixed Types

  • December 24, 2002
  • By Richard G. Baldwin
  • Send Email »
  • More Articles »

Java Programming Notes # 1796


Preface

New features in SDK Version 1.4.0

The recently released JavaTM 2 SDK, Standard Edition Version 1.4 contains a large number of new features, including the concept of IO channels.  The first lesson in this miniseries, entitled FileChannel Objects in Java, Background Information, introduced you to the concept of channels from a read/write IO viewpoint. 

Memory-mapped IO

The previous lesson, entitled Introduction to Memory-Mapped IO in Java, introduced you to the basics of memory-mapped IO and showed you how to perform memory-mapped IO for data of type byte.

In this lesson, I will show you how to do memory-mapped IO for data records containing mixed types of data.  I will also show you how to manipulate memory maps using view buffers such as FloatBuffer.

Viewing tip

You may find it useful to open another copy of this lesson in a separate browser window.  That will make it easier for you to scroll back and forth among the different listings and figures while you are reading about them.

Supplementary material

I recommend that you also study the other lessons in my extensive collection of online Java tutorials.  You will find those lessons published at Gamelan.com.  However, as of the date of this writing, Gamelan doesn't maintain a consolidated index of my Java tutorial lessons, and sometimes they are difficult to locate there.  You will find a consolidated index at www.DickBaldwin.com.

What is a FileChannel?

Sun describes an object of the FileChannel class simply as "A channel for reading, writing, mapping, and manipulating a file."

Discussion and Sample Code

Once you map a file into memory, changes that you make to the memory map are automatically propagated to the file. 

Methods of the ByteBuffer class apply

Because the memory map is an object instantiated from a subclass of the ByteBuffer class, you can use the methods of the ByteBuffer class to manipulate the data in the memory map, and hence to manipulate the data in the file.

The ByteBuffer class provides numerous methods that allow you to treat the contents of the buffer as different data types.  This includes methods such as:

  • The method named asFloatBuffer, which returns a float view of the data in the ByteBuffer object.  This float view is provided via an object of the FloatBuffer class, and allows you to manipulate the data in the ByteBuffer object as type float.
  • The methods named getFloat and putFloat, which allow you to manipulate the data in the ByteBuffer object as though it were type float rather than type byte.

Will discuss sample program in fragments

I will illustrate memory-mapped IO for data of mixed types using the sample program named Channel05.  As usual, I will discuss this program in fragments.  You will find a complete listing of the program in Listing 11 near the end of the lesson.

Description of the program

This program, which was tested using JDK 1.4.0 under Win2000, illustrates memory-mapped IO for data of mixed types.

The program begins by creating and populating a disk file named junk.txt with data of mixed types.

Then the program creates a memory map of the file and modifies the map.  This, in turn, modifies the contents of the file.

Then the program reads and displays the modified file, demonstrating that it was modified when the contents of the map were modified. 

The code to this point illustrates the use of methods such as putDouble, getDouble, putInt, and getInt to manipulate memory-map data of mixed types.

Following this, the program modifies the map (and hence the disk file) to contain only data of type float.  This portion of the program illustrates the use of methods such as asFloatBuffer to get a float view of the buffer, and to manipulate the data in the map (and the disk file) as type float.

Then the program reads and displays the modified file, the entire contents of which have now been converted to data of type float.

The main method

For simplicity, this program consists of a main method and a couple of static convenience methods for displaying data.

The first fragment in Listing 1 shows the beginning of the main method.

  public static void main(
                        String[] args){
    ByteBuffer buf =
               ByteBuffer.allocate(16);
    buf.putDouble(1.0/3);
    buf.putInt(16000*16000);
    buf.putInt(-32000*32000);

    showMixedData(buf,
        "Display raw buffer contents");

Listing 1

Create, populate, and display

The code in Listing 1 creates, populates, and displays a ByteBuffer object with one double value and two int values.  If you have been studying this series of lessons from the beginning (Understanding the Buffer class in Java), this code should be familiar to you.

(The method named showMixedData is a convenience method of my own design, whose purpose is to display the contents of the ByteBuffer object.  This method, which can be viewed in Listing 11 near the end of the lesson, is very similar to methods that I have explained in previous lessons.  Therefore, I won't discuss it in detail here.)

The output

The output produced by the code in Listing 1 is shown in Figure 1.

Display raw buffer contents
0.3333333333333333
256000000
-1024000000

Figure 1

As you can see, the output consists of one double value and two int values.

The code in Listing 2 creates a file named junk.txt and writes the contents of the ByteBuffer object into the file.

      FileOutputStream fos =
                  new FileOutputStream(
                           "junk.txt");
      FileChannel fChan =
                      fos.getChannel();
      buf.position(0);
      System.out.println(
                    "Bytes written: " +
                     fChan.write(buf));
      System.out.println("File size: "
                + fChan.size() + "\n");

Listing 2

All of this code should also be familiar to you if you have been studying the lessons in this miniseries from the beginning.

The output

The code in Listing 2 produces the output shown in Figure 2.

Bytes written: 16
File size: 16

Figure 2

A file size of sixteen bytes is what we would expect for one double value, (which requires eight bytes), and two int values, (which require four bytes each).

Close streams, etc.

The code in Listing 3 closes streams and channels, and turns objects over to the garbage collector.

      fos.close();
      fChan.close();
      fos = null;
      fChan = null;
      buf = null;

Listing 3

Following the execution of the code in Listing 3, the original data is available to the program only by way of the disk file that now contains it.

Get FileChannel object

The code in Listing 4 gets a read/write FileChannel object, connected to the file named junk.txt, which contain the data.  The FileChannel object will be used to map the contents of the file into a memory map.

      FileChannel rwCh =
                  new RandomAccessFile(
                      "junk.txt","rw").
                          getChannel();

Listing 4

(By now, you are probably getting tired of me telling you that you should already understand the code with no further discussion on my part.  From this point on, if I present code and don't discuss it, I will be assuming that you already understand the code with no further discussion on my part.)

Map the file to memory and display the map

The code in Listing 5:

  • Maps the contents of the file named junk.txt into a read/write memory map.
  • Closes the channel used to perform the mapping operation.
  • Displays the contents of the memory map.
      long fileSize = rwCh.size();
      ByteBuffer mapBuf = rwCh.map(
        FileChannel.MapMode.READ_WRITE,
                          0, fileSize);
      rwCh.close();

      showMixedData(mapBuf,
                       "Map contents");

Listing 5

The output

The output produced by the code in Listing 5 is shown in Figure 3.

Map contents
0.3333333333333333
256000000
-1024000000

Figure 3

Figure 3 shows the contents of the memory map.  Happily, the output values in Figure 3 match the original values originally written to the file, confirming that no data modification or corruption has taken place up to this point in the program.

Modify the map and the file

The code in Listing 6 invokes the relative putInt method on the map object's reference to change the value of the first int value in the map.  (This will also change the value in the first int value stored in the disk file.)

      //Modify one value in the middle
      // of the map buffer
      mapBuf.position(8);
      mapBuf.putInt(127);
      showMixedData(mapBuf,
              "Modified map contents");

Listing 6

The code in Listing 6 causes the int value to be changed from 256000000 to 127.

The output

This is illustrated by Figure 4, which shows the output produced by the code in Listing 6.  Figure 4 confirms that the first int value in the memory map has been modified as described above.

Modified map contents
0.3333333333333333
127
-1024000000

Figure 4

Read and display the modified contents of the disk file

The code in Listing 7 reads the contents of the disk file into a new ByteBuffer object and displays that data.

      //Get new channel for read only
      FileChannel newInCh =
                  new RandomAccessFile(
                       "junk.txt","r").
                          getChannel();

      //Allocate a new ByteBuffer
      ByteBuffer newBuf =
                   ByteBuffer.allocate(
                        (int)fileSize);

      //Read file data into the new
      // buffer, close the channel, and
      // display the data.
      System.out.println(
               "Bytes read = "
               + newInCh.read(newBuf));
      newInCh.close();

      showMixedData(newBuf,
             "Modified file contents");

Listing 7

The output

The output produced by the code in Listing 7 is shown in Figure 5.

Bytes read = 16
Modified file contents
0.3333333333333333
127
-1024000000

Figure 5

Figure 5 confirms that the change made to the first int value in the memory map was automatically propagated to the associated disk file.

Recap

A disk file containing one double value and two int values was mapped to a ByteBuffer object.  The putInt method of the ByteBuffer class was used to modify the first of the two int values.  This change was automatically propagated to the associated disk file.

The disk file was read and displayed, confirming that the change to the memory map was automatically propagated to the disk file.

Memory-mapped IO for single-type data

That completes the illustration of the use of memory-mapped IO for mixed data types.  The remainder of this lesson will concentrate on the use of memory-mapped IO for data of a single type other than type byte.

Change to type float

The code in Listing 8 gets a FloatBuffer view of the ByteBuffer object containing the memory map of the data in the disk file.

      mapBuf.position(0);
      FloatBuffer fBuf =
                mapBuf.asFloatBuffer();

Listing 8

Manipulate the map data as type float

Having gotten a FloatBuffer view of the map, the program can manipulate the contents of the map (and hence the contents of the disk file) as type float.

The code in Listing 9 sets the position property of the FloatBuffer object to zero, and then uses the relative put method of the FloatBuffer class to store four float values in the map.  (This causes the same four float values to be stored in the associated disk file.)

      fBuf.position(0);
      fBuf.put((float)1.0/6);
      fBuf.put((float)2.0/6);
      fBuf.put((float)3.0/6);
      fBuf.put((float)4.0/6);

      showFloatData(fBuf,
              "Modified map contents");

Listing 9

The output

Having stored the four float values in the memory map, the code in Listing 9 invokes the showFloatData method to display the contents of the map as float data.  This produces the four float values shown in Figure 6.

Modified map contents
0.16666667
0.33333334
0.5
0.6666667

Figure 6

Display the file contents

All that now remains is to display the current contents of the file to confirm that the file now contains four float values matching the four float values in the memory map.

This is accomplished by the code in Listing 10.

      //Read and display the modified
      // file data
      //Get new channel for read only
      FileChannel floatInCh =
                  new RandomAccessFile(
                       "junk.txt","r").
                          getChannel();

      //Allocate a new ByteBuffer
      ByteBuffer anotherBuf =
                   ByteBuffer.allocate(
                        (int)fileSize);

      //Read file data into the new
      // buffer, close the channel, and
      // display the data.
      System.out.println(
         "Bytes read = "
         + floatInCh.read(anotherBuf));
      floatInCh.close();

      anotherBuf.position(0);
      FloatBuffer fileBuf =
            anotherBuf.asFloatBuffer();
      showFloatData(fileBuf,
             "Modified file contents");

    }catch(Exception e){
      System.out.println(e);}

Listing 10

The final output

The contents of the disk file, as displayed by the code in Listing 10, are shown in Figure 7.

Bytes read = 16
Modified file contents
0.16666667
0.33333334
0.5
0.6666667

Figure 7

As you can see, the sixteen bytes in the disk file no longer represent one double value followed by two int values.  Now the sixteen bytes represent four float values.

A sequence of bytes

The contents of a disk file are, after all, simply a sequence of bytes.  We can cause those bytes to represent anything that we want them to represent.

The methods of the FileChannel class, and the methods of the Buffer class and its subclasses make it relatively easy for us to cause the sequence of bytes in a disk file to represent any combination of any of the primitive types, (other than boolean).

Those methods also make it possible for us to manipulate primitive data in a memory map and to have all changes made to the memory map automatically propagated to the associated disk file.

That's it for now

By now you should understand a quite a lot about creating and manipulating memory maps of disk files for any combination of primitive types other than boolean.

Run the Program

If you haven't already done so, I encourage you to copy the code from Listing 11 into your text editor, compile it, and execute it.  Experiment with it, making changes, and observing the results of your changes.

Remember, however, that you must be running Java version 1.4.0 or later to compile and execute this program.

Summary

In this lesson, I have taught you how to do memory-mapped IO for data records containing mixed types of data.  I have also taught you how to manipulate memory maps using view buffers such as FloatBuffer.

What's Next?

As time goes on, I plan to publish additional lessons that will help you learn to use other new IO features including:

  • File locks
  • Character-set encoders and decoders
  • Pattern matching on files using regular expressions
  • Socket channels for both clients and servers
  • Non-blocking reads
  • Non-blocking servers

Complete Program Listing

A complete listing of the program discussed in this lesson is shown in Listing 11 below.
 
/* File Channel05.java
Copyright 2002, R.G.Baldwin

Illustrates use of FileChannel objects
for mapped-memory IO for data of mixed
types.

First creates and populates a disk
file with data of mixed types.

Then creates a memory map of the
existing file, and modifies the map,
which modifies the contents of the
file.

Then reads and displays the modified
file.

Then modifies the map to contain only
data of type float.

Then reads and modifies the modified
file.

Tested using JDK 1.4.0 under Win2000

The output is:

Display raw buffer contents
0.3333333333333333
256000000
-1024000000

Bytes written: 16
File size: 16

Map contents
0.3333333333333333
256000000
-1024000000

Modified map contents
0.3333333333333333
127
-1024000000

Bytes read = 16
Modified file contents
0.3333333333333333
127
-1024000000

Modified map contents
0.16666667
0.33333334
0.5
0.6666667

Bytes read = 16
Modified file contents
0.16666667
0.33333334
0.5
0.6666667
**************************************/

import java.io.*;
import java.nio.channels.*;
import java.nio.*;

class Channel05{
  public static void main(
                        String[] args){
    //Create and populate a ByteBuffer
    // object with data of mixed types

    ByteBuffer buf =
               ByteBuffer.allocate(16);
    buf.putDouble(1.0/3);
    buf.putInt(16000*16000);
    buf.putInt(-32000*32000);

    showMixedData(buf,
        "Display raw buffer contents");

    //Write buffer to disk file
    try{
      FileOutputStream fos =
                  new FileOutputStream(
                           "junk.txt");
      FileChannel fChan =
                      fos.getChannel();
      buf.position(0);
      System.out.println(
                    "Bytes written: " +
                     fChan.write(buf));
      System.out.println("File size: "
                + fChan.size() + "\n");

      //Close stream and channel and
      // make objects eligible for
      // garbage collection
      fos.close();
      fChan.close();
      fos = null;
      fChan = null;
      buf = null;

      //Get a new FileChannel object
      // for reading and writing the
      // existing file
      FileChannel rwCh =
                  new RandomAccessFile(
                      "junk.txt","rw").
                          getChannel();

      //Map entire file to memory and
      // close the channel
      long fileSize = rwCh.size();
      ByteBuffer mapBuf = rwCh.map(
        FileChannel.MapMode.READ_WRITE,
                          0, fileSize);
      rwCh.close();

      showMixedData(mapBuf,
                       "Map contents");

      //Modify one value in the middle
      // of the map buffer
      mapBuf.position(8);
      mapBuf.putInt(127);
      showMixedData(mapBuf,
              "Modified map contents");

      //Read and display the contents
      // of the file
      //Get new channel for read only
      FileChannel newInCh =
                  new RandomAccessFile(
                       "junk.txt","r").
                          getChannel();

      //Allocate a new ByteBuffer
      ByteBuffer newBuf =
                   ByteBuffer.allocate(
                        (int)fileSize);

      //Read file data into the new
      // buffer, close the channel, and
      // display the data.
      System.out.println(
               "Bytes read = "
               + newInCh.read(newBuf));
      newInCh.close();

      showMixedData(newBuf,
             "Modified file contents");

      //Now change the type of data in
      // the map and in the file to all
      // float
      mapBuf.position(0);
      FloatBuffer fBuf =
                mapBuf.asFloatBuffer();
      fBuf.position(0);
      fBuf.put((float)1.0/6);
      fBuf.put((float)2.0/6);
      fBuf.put((float)3.0/6);
      fBuf.put((float)4.0/6);
      showFloatData(fBuf,
              "Modified map contents");

      //Read and display the modified
      // file data
      //Get new channel for read only
      FileChannel floatInCh =
                  new RandomAccessFile(
                       "junk.txt","r").
                          getChannel();

      //Allocate a new ByteBuffer
      ByteBuffer anotherBuf =
                   ByteBuffer.allocate(
                        (int)fileSize);

      //Read file data into the new
      // buffer, close the channel, and
      // display the data.
      System.out.println(
         "Bytes read = "
         + floatInCh.read(anotherBuf));
      floatInCh.close();

      anotherBuf.position(0);
      FloatBuffer fileBuf =
            anotherBuf.asFloatBuffer();
      showFloatData(fileBuf,
             "Modified file contents");

    }catch(Exception e){
      System.out.println(e);}

  }// end main
  //---------------------------------//

  static void showMixedData(
        ByteBuffer buf, String label){
    //Displays byte buffer contents

    //Save position
    int pos = buf.position();
    //Set position to zero
    buf.position(0);
    System.out.println(label);
    System.out.println(
                      buf.getDouble());
    System.out.println(buf.getInt());
    System.out.println(buf.getInt());
    System.out.println();//new line
    //Restore position and return
    buf.position(pos);
  }//end showBufferData
  //---------------------------------//

  static void showFloatData(
        FloatBuffer buf, String label){
    //Displays byte buffer contents
    //Save position
    int pos = buf.position();
    //Set position to zero
    buf.position(0);
    System.out.println(label);
    System.out.println(buf.get());
    System.out.println(buf.get());
    System.out.println(buf.get());
    System.out.println(buf.get());
    System.out.println();//new line
    //Restore position and return
    buf.position(pos);
  }//end showBufferData
  //---------------------------------//

}//end class Channel05 definition


Listing 11


Copyright 2002, Richard G. Baldwin.  Reproduction in whole or in part in any form or medium without express written permission from Richard Baldwin is prohibited.

About the author

Richard Baldwin is a college professor (at Austin Community College in Austin, TX) and private consultant whose primary focus is a combination of Java, C#, and XML. In addition to the many platform and/or language independent benefits of Java and C# applications, he believes that a combination of Java, C#, and XML will become the primary driving force in the delivery of structured information on the Web.

Richard has participated in numerous consulting projects, and he frequently provides onsite training at the high-tech companies located in and around Austin, Texas.  He is the author of Baldwin's Programming Tutorials, which has gained a worldwide following among experienced and aspiring programmers. He has also published articles in JavaPro magazine.

Richard holds an MSEE degree from Southern Methodist University and has many years of experience in the application of computer technology to real-world problems.

Baldwin@DickBaldwin.com

-end-
 




Comment and Contribute

 


(Maximum characters: 1200). You have characters left.

 

 


Sitemap | Contact Us

Rocket Fuel