Using Memory-Mapped Files in .NET 4.0

Creating, Expanding and Truncating files

So far, you have learned how to access memory-mapped files that have already existed on disk, or created on the fly for inter-process communications. What if you wanted to create a file from scratch, expand or truncate a file mapped into memory? Luckily, all these three scenarios are straightforward to implement.

First, if you wanted to create a new file to create a memory-mapped view on, you could execute the following code:

  FileStream file = new FileStream(
      @"C:\Temp\MyNewFile.dat", FileMode.CreateNew);
  MemoryMappedFile mmf =
      MemoryMappedFile.CreateFromFile(file, null, 1000);
  MemoryMappedViewAccessor accessor =
      mmf.CreateViewAccessor();

Here, a new file is created by specifying the CreateNew mode in the call to the FileStream's constructor. This will create a new, zero-length file on disk. Such empty files cannot be directly used to create views, and so the CreateFromFile method call must contain a capacity parameter. In the above example, the file will have a capacity of 1,000 bytes, and if nothing else is written to the file, then the file will contain values of zero, i.e. null characters.

Given the above situation of a file with length of 1,000 bytes, how would you continue writing past that limit? If you map a view and use the accessor object's Write method to try to write past the capacity (size) of the file, the operation will silently fail (it is possible that future .NET 4.0 releases will act differently). As such, you cannot simply expand a file by writing past the end of the file, as you could for instance with streams.

How then would you expand a file? The answer lies again in the capacity parameter of the CreateFromFile method call in the MemoryMappedFile class. If you specify a larger capacity than the actual file on disk, then Windows will extend the file to match the capacity given. Naturally, this can only succeed if there is enough free disk space, so capacity increases will not (always) work even if you had enough memory.

The following code listing shows how to expand the previously described 1,000 byte file to 2,000 bytes:

  FileStream file = new FileStream(
      @"C:\Temp\MyNewFile.dat", FileMode.Open);
  MemoryMappedFile mmf =
      MemoryMappedFile.CreateFromFile(file, null, 2000);

The capacity parameter is defined as a C# long, which means that it is a signed 64-bit value (System.Int64). You are then not limited to 2 gigabyte views at a time, but can instead use much larger views. Practically speaking, the only limit is the free virtual address space in your application, around 8 terabytes if you have a 64-bit Windows operating system and compile your .NET application to be a 64-bit application (the x64 platform target mode in Visual Studio). On a regular 32-bit system, the limit is usually less than 2 GB, depending on system setup and available memory. The third common operation, truncation, is done a bit differently than the previous two scenarios: truncating the file must be made at the file level. If you try to specify a capacity parameter value smaller than the actual file on disc, you will get an error stating that the capacity value cannot be smaller than the file size. Thus, you must choose another approach, and one way is the use the FileStream's SetLength method.

To get a size of a file, you could test the Length property of the stream, or use the similarly named property of the System.IO.FileInfo class.

Conclusion

In this article, you learned about memory-mapped files and the managed support classes for them in .NET Framework 4.0. Memory mapping is a useful technique that allows an easy way to read and write to files using simple memory operations. No stream seeking is required, and you do not have to worry about files being larger than can fit into memory: simply map a portion of the file as needed, and you are done.

Memory mapping is also useful in sharing data between an application's threads, but also between processes running on the same system. To share data between processes, all you need to do is give your mapped view object a unique name. If this name matches that in the other process(es), then the data is automatically shared.

With .NET 4.0, you can use managed classes to use memory- mapped files. Reading and writing to a mapped view is done through an accessor object, which can also take a form of a more traditional stream. Getting the accessor object itself is usually a three step process: first open the file using a FileStream, then create a memory mapping object, and finally get the accessor through the mapping object.

The accessor object allows you to easily read and write the most primitive data types, but generic data type support allows you to get to more complex types, including arrays. Strings can be read and written on a byte-per-byte basis; you need to remember to do proper encoding to read and write correctly.

Memory mapping is a valuable technique to access data in files, small or large. With .NET 4.0, managed code developers should learn this new method available to them, and use it whenever needed. It shares as a good alternative to the more traditional methods of accessing files. Happy memory-mapping!

Links

What's New in the .NET Framework 4
The System.IO.MemoryMappedFiles namespace
The MemoryMappedViewAccessor class

About the Author

Jani Järvinen is a software development trainer and consultant in Finland. He is a Microsoft C# MVP and a frequent author and has published three books about software development. He is the group leader of a Finnish software development expert group at ITpro.fi and a board member of the Finnish Visual Studio Team System User Group. His blog can be found at http://www .saunalahti.fi/janij/. You can send him mail by clicking on his name at the top of the article.

Page 4 of 4



This article was originally published on July 14, 2009