Windows System Programming Third Edition [Electronic resources] نسخه متنی

I/O Completion Ports

I/O completion ports, supported only on NT, combine features of both overlapped I/O and independent threads and are most useful in server programs. To see the requirement for this, consider the servers that we built in Chapters 11 and 12, where each client is supported by a distinct worker thread associated with a socket or named pipe instance. This solution works very well when the number of clients is not large.

Consider what would happen, however, if there were 1,000 clients. The current model would then require 1,000 threads, each with a substantial amount of virtual memory space. For example, by default, each thread will consume 1MB of stack space, so 1,000 threads would require 1GB of virtual address space, and thread context switches could increase page fault delays.^{Chapter 9} showed the performance degradation that can result. Therefore, there is a requirement to allow a small pool of worker threads to serve a large number of clients.

^[1] This problem may become less severe in the future with Win64 and larger physical memories.

I/O completion ports provide a solution by allowing you to create a limited number of server threads in a thread pool while having a very large number of named pipe handles (or sockets). Handles are not paired with individual worker server threads; rather, a server thread can process data on any handle that has available data.

An I/O completion port, then, is a set of overlapped handles, and threads wait on the port. When a read or write on one of the handles is complete, one thread is awakened and given the data and the results of the I/O operation. The thread can then process the data and wait on the port again.

The first task is to create an I/O completion port and add overlapped handles to the port.

Managing I/O Completion Ports

A single function, CreateIoCompletionPort, is used both to create the port and to add handles. Since this one function must perform two tasks, the parameter usage is correspondingly complex.

HANDLE CreateIoCompletionPort (
HANDLE FileHandle,
HANDLE ExistingCompletionPort,
DWORD CompletionKey,
DWORD NumberOfConcurrentThreads);

An I/O completion port is a collection of file handles opened in OVERLAPPED mode. FileHandle is an overlapped handle to add to the port. If the value is INVALID_HANDLE_VALUE, a new I/O completion port is created and returned by the function. The next parameter, ExistingCompletionPort, must be NULL in this case.

ExistingCompletionPort is the port created on the first call, and it indicates the port to which the handle in the first parameter is to be added. The function also returns the port handle when the function is successful; NULL indicates failure.

CompletionKey specifies the key that will be included in the completion packet for FileHandle. The key is usually an index to an array of data structures containing an operation type, a handle, and a pointer to the data buffer.

NumberOfConcurrentThreads indicates the maximum number of threads allowed to execute concurrently. Any threads in excess of this number that are waiting on the port will remain blocked even if there is a handle with available data. If this parameter is 0, the number of processors in the system is used as the limit.

An unlimited number of overlapped handles can be associated with an I/O completion port. Call CreateIoCompletionPort initially to create the port and to specify the maximum number of threads. Call the function again for every overlapped handle that is to be associated with the port. Unfortunately, there is no way to remove a handle from a completion port, and this omission limits program flexibility.

The handles associated with a port should not be used with ReadFileEx or WriteFileEx functions. The Microsoft documentation suggests that the files or other objects not be shared using other open handles.

Waiting on an I/O Completion Port

Use ReadFile and WriteFile, along with overlapped structures (no event handle is necessary), to perform I/O on the handles associated with a port. The I/O operation is then queued on the completion port.

A thread waits for a queued overlapped completion not by waiting on an event but by calling GetQueuedCompletionStatus, specifying the completion port. When the calling thread wakes up, the function returns a key that was specified when the handle, whose operation has completed, was initially added to the port, and this key can specify the number of bytes transferred and the identity of the actual handle for the completed operation.

Notice that the thread that initiated the read or write is not necessarily the thread that will receive the completion notification. Any waiting thread can receive completion notification. Therefore, the key must be able to identify the handle of the completed operation.

There is also a time-out associated with the wait.

BOOL GetQueuedCompletionStatus (
HANDLE CompletionPort,
LPDWORD lpNumberOfBytesTransferred,
LPDWORD lpCompletionKey,
LPOVERLAPPED *lpOverlapped,
DWORD dwMilliseconds);

It is sometimes convenient to have an operation not be queued on the I/O completion port. In such a case, a thread can wait on the overlapped event, as shown in Program 14-4 and in an additional example, atouMTCP, on the book''s Web site. In order to specify that an overlapped operation should not be queued on the completion port, you must set the low-order bit in the overlapped structure''s event handle; then you can wait on the event for that specific operation. This is a strange design, but it is documented, although not prominently.

Posting to an I/O Completion Port

A thread can post a completion event, with a key, to a port to satisfy an outstanding call to GetQueuedCompletionStatus. The PostQueuedCompletionStatus function supplies all the required information.

BOOL PostQueuedCompletionStatus (
HANDLE CompletionPort,
DWORD dwNumberOfBytesTransferred,
DWORD dwCompletionKey,
LPOVERLAPPED lpOverlapped);

One technique sometimes used is to provide a bogus key value, such as -1, to wake up waiting threads, even though no operation has completed. Waiting threads should test for bogus key values, and this technique could be used, for example, to signal a thread to shut down.

Alternatives to I/O Completion Ports

Chapter 9 showed how a semaphore can be used to limit the number of ready threads, and this technique is effective in maintaining throughput when many threads compete for limited resources.

We could use the same technique with serverSK (Program 12-2) and serverNP (Program 11-3). All that is required is to wait on the semaphore after the read request completes, perform the request, create the response, and release the semaphore before writing the response. This solution is much simpler than the I/O completion port example in the next section. The only problem is that there may be a large number of threads, each with its own stack space, which will consume virtual memory. The problem can be partly alleviated by carefully measuring the amount of stack space required. Exercise 146 involves experimentation with this alternative solution, and there is an example implementation on the Web site.

There is yet another possibility when creating scalable servers. A limited number of worker threads can take work item packets from a queue (see Chapter 10). The incoming work items can be placed in the queue by one or more boss threads, as shown in Program 10-5.