Using Memory-Mapped Files in .NET 4.0
Assume you have the need to manipulate multi-gigabyte files and read and write data to them. One option would be to access the file using a sequential stream, which is fine if you need to access the file from the beginning to the end. However, things get more problematic when you need random access. Seeking the stream is naturally a solution, but unfortunately a slow one.
If you have background in Windows API development, then you might be aware of an old technique called memory-mapped files (sometimes abbreviated MMF). The idea of memory-mapped files or file mapping is to load a file into memory so that it appears as a continuous block in your application's address space. Then, reading and writing to the file is simply a matter of accessing the correct memory location. In fact, when the operating system loader fetches your application's EXE or DLL files to execute their code, file mapping is used behind the scenes.
Memory-mapped files and large files are often associated together in the minds of developers, but there's no practical limit to how large or small the files accessed through memory mapping can be. Although using memory mapping for large files make programming easier, you might observe even better performance when using smaller files, as they can fit entirely in the file system cache.
The information and the code listings in this article are based on the .NET 4.0 Beta 1 release, available since May 2009. As is the case with pre-release software, technical details, class names and available methods might change once the final RTM version of .NET becomes available. This is worth keeping in mind while studying or developing against any beta library.
The New Namespace and its Classes
For .NET 4.0 developers, the interesting classes that
work with memory-mapped files live in the new
Presently, this namespace contains four classes and several
enumerations to help you access and secure your file
mappings. The actual implementation is inside the assembly
The most important class for the developer is the MemoryMappedFile class. This class allows you to create a memory-mapped object, from which you can in turn create a view accessor object. You can then use this accessor to manipulate directly the memory block mapped from the file. Manipulation can be done using the convenient Read and Write methods.
Note that since direct pointers are not considered a sound programming practice in the managed world, such an access object is needed to keep things tidy. In traditional Windows API development in native code, you would simply get a pointer to the beginning of your memory block.
That said, the process or acquiring a memory-mapped file and the necessary accessor object, you need to follow three simple steps. First, you need a file stream object that points to (an existing) file on disk. Secondly, you can create the mapping object from this file, and as a final step, you create the accessor object. Here is a code example in C#:
FileStream file = new FileStream( @"C:\Temp\MyFile.dat", FileMode.Open); MemoryMappedFile mmf = MemoryMappedFile.CreateFromFile(file); MemoryMappedViewAccessor accessor = mmf.CreateViewAccessor();
The code first opens a file with the
System.IO.FileStream class, and then passes the
stream object instance to the static
CreateFromFile method of the
MemoryMappedFile class. The third step is to
CreateViewAccessor method of the
In the above code, the
method is called without any parameters. In this case, the
mapping begins from the start of the file (offset zero) and
ends at the last byte of the file. You can however easily
map in any portion of the file. For instance, if your file
would be one gigabyte in size, then you could map, say, a
view at the one megabyte mark with a view size of 10,000
bytes. This could be done with the following call:
MemoryMappedViewAccessor accessor = mmf.CreateViewAccessor(1024 * 1024, 10000);
Later on, you will see more advanced uses for these mapped views. But first, you need to learn about reading from the view.
Page 1 of 4