Windows System Programming Third Edition [Electronic resources] نسخه متنی

Memory-Mapped Files

Dynamic memory in heaps must be physically allocated in a paging file. The OS's memory management controls page movement between physical memory and the paging file and also maps the process's virtual address space to the paging file. When the process terminates, the physical space in the file is deallocated.

Windows' memory-mapped file functionality can also map virtual memory space directly to normal files. This has several advantages.

• There is no need to perform direct file I/O (reads and writes).

• The data structures created in memory will be saved in the file for later use by the same or other programs. Be careful about pointer usage, as Program 5-5 illustrates.

• Convenient and efficient in-memory algorithms (sorts, search trees, string processing, and so on) can process file data even though the file may be much larger than available physical memory. The performance will still be influenced by paging behavior if the file is large.

• File processing performance can be significantly improved in some cases.

• There is no need to manage buffers and the file data they contain. The OS does this hard work and does it efficiently and reliably.

• Multiple processes (Chapter 6) can share memory by mapping their virtual address spaces to the same file or to the paging file (interprocess memory sharing is the principal reason for mapping to the paging file).

• There is no need to consume paging file space.

The OS itself uses memory mapping to implement DLLs and to load and execute executable (.EXE) files. DLLs are described at the end of this chapter.

File Mapping Objects

The first step is to create a file mapping object, which has a handle, on an open file and then map the process's address space to all or part of the file. File mapping objects can be given names so that they are accessible to other processes for shared memory. Also, the mapping object has protection and security attributes and a size.

HANDLE CreateFileMapping (
HANDLE hFile,
LPSECURITY_ATTRIBUTES lpsa,
DWORD dwProtect,
DWORD dwMaximumSizeHigh,
DWORD dwMaximumSizeLow,
LPCTSTR lpMapName)

Return: A file mapping handle, or NULL on failure.

Parameters

hFile is the handle of an open file with protection flags compatible with dwProtect. The value (HANDLE) 0xFFFFFFFF (equivalently, INVALID_HANDLE_VALUE) refers to the paging file, and you can use this value for interprocess memory sharing without creating a separate file.

LPSECURITY_ATTRIBUTES allows the mapping object to be secured.

dwProtect specifies the mapped file access with the following flags. Additional flags are allowed for specialized purposes. For example, the SEC_IMAGE flag specifies an executable image; see the on-line documentation for more information.

• PAGE_READONLY means that the program can only read the pages in the mapped region; it can neither write nor execute them. hFile must have GENERIC_READ access.

• PAGE_READWRITE gives full access to the object if hFile has both GENERIC_READ and GENERIC_WRITE access.

• PAGE_WRITECOPY means that when mapped memory is changed, a private (to the process) copy is written to the paging file and not to the original file. A debugger might use this flag when setting breakpoints in shared code.

dwMaximumSizeHigh and dwMaximumSizeLow specify the size of the mapping object. If it is 0, the current file size is used; be sure to specify a size when using the paging file. If the file is expected to grow, use a size equal to the expected file size, and, if necessary, the file size will be set to that size immediately. Do not map to a file region beyond this specified size; the mapping object cannot grow.

lpMapName names the mapping object, allowing other processes to share the object; the name is case-sensitive. Use NULL if you are not sharing memory.

An error is indicated by a return value of NULL (not INVALID_HANDLE_VALUE).

Obtain a file mapping handle by specifying an existing mapping object name. The name comes from a previous call to CreateFileMapping. Two processes can share memory by sharing a file mapping. The first process creates the named mapping, and subsequent processes open this mapping with the name. The open will fail if the named object does not exist.

HANDLE OpenFileMapping (
DWORD dwDesiredAccess,
BOOL bInheritHandle,
LPCTSTR lpMapName)

Return: A file mapping handle, or NULL on failure.

dwDesiredAccess uses the same set of flags as dwProtect in CreateFileMapping. lpMapName is the name created by a CreateFileMapping call. Handle inheritance (bInheritHandle) is a subject for Chapter 6.

The CloseHandle function, as expected, destroys mapping handles.

Mapping Process Address Space to Mapping Objects

The next step is to allocate virtual memory space and map it to a file through the mapping object. From the programmer's perspective, this allocation is similar to HeapAlloc, although it is much coarser, with larger allocation units. A pointer to the allocated block (or file view) is returned; the difference lies in the fact that the allocated block is mapped to the user-specified file rather than the paging file. The file mapping object plays the same role played by the heap when HeapAlloc is used.

LPVOID MapViewOfFile (
HANDLE hMapObject,
DWORD dwAccess,
DWORD dwOffsetHigh,
DWORD dwOffsetLow,
SIZE_T cbMap)

Return: The starting address of the block (file view), or NULL on failure.

Parameters

hMapObject identifies a file mapping object obtained from either CreateFileMapping or OpenFileMapping.

dwAccess must be compatible with the mapping object's access. The three possible flag values are FILE_MAP_WRITE, FILE_MAP_READ, and FILE_MAP_ALL_ACCESS. (This is the bit-wise "or" of the previous two flags.)

dwOffsetHigh and dwOffsetLow specify the starting location of the mapped file region. The start address must be a multiple of 64K. Use a zero offset to map from the beginning of the file.

cbMap is the size, in bytes, of the mapped region. Zero indicates the entire file at the time of the MapViewOfFile call.

MapViewOfFileEx is similar except that you must specify the starting memory address. This address might, for instance, be the address of an array in the program's data space. Windows fails if the process has already mapped the requested space.

Just as it is necessary to release memory allocated in a heap with HeapFree, it is necessary to release file views.

BOOL UnmapViewOfFile (LPVOID lpBaseAddress)

Figure 5-3 shows the relationship between process address space and a mapped file.

Figure 5-3. Process Address Space Mapped to a File

Chapters 8-10.^[4]

^[4] Statements regarding coherency of mapped views do not apply to networked files. The files must be local.

Figure 5-4. Shared Memory

[View full size image]

UNIX, at the SVR4 and 4.3+BSD releases, supports the mmap function, which is similar to MapViewOfFile. The parameters specify the same information except that there is no mapping object. munmap is the UnmapViewOfFile equivalent. There are no equivalents to the CreateFileMapping and OpenFileMapping functions. Any normal file can be mapped directly. UNIX does not use mapped files to share memory; rather, it has explicit API functions for memory sharing. The UNIX functions are shmget, shmctl, shmat, and shmdt.

File Mapping Limitations

File mapping, as mentioned previously, is a powerful and useful feature. The disparity between Windows' 64-bit file system and 32-bit addressing limits these benefits; Win64 does not have these limitations.

The principal problem is that if the file is large (greater than 23GB in this case), it is not possible to map the entire file into virtual memory space. Furthermore, the entire 3GB will not be available because virtual address space will be allocated for other purposes and available contiguous blocks will be much smaller than the theoretical maximum. Win64 will largely remove this limitation.

When you're dealing with large files that cannot be mapped to one view, create code that carefully maps and unmaps file regions as they are needed. This technique can be as complex as managing memory buffers, although it is not necessary to perform the explicit reads and writes.

File mapping has two other notable limitations.

• A file mapping cannot be expanded. You need to know the maximum size when creating the file mapping, and it may be difficult or impossible to determine this size.

• There is no way to allocate memory within a mapped memory region without creating your own memory management functions. It would be convenient if there were a way to specify a file mapping and a pointer returned by MapViewOfFile and obtain a heap handle.

Summary: File Mapping

Here is the standard sequence required by file mapping.

1.	Open the file. Be certain that it has GENERIC_READ access.
2.	If the file is new, set its length either with CreateFileMapping (step 3 below) or by using SetFilePointer followed by SetEndOfFile.
3.	Map the file with CreateFileMapping or OpenFileMapping.
4.	Create one or more views with MapViewOfFile.
5.	Access the file through memory references. If necessary, change the mapped regions with UnmapViewOfFile and MapViewOfFile.
6.	On completion, perform, in order, UnmapViewOfFile, CloseHandle for the mapping handle, and CloseHandle for the file handle.