Java Network Programming (3rd ed) [Electronic resources] نسخه متنی

Chapter 5. Threads

Back in
the good old days of the Net, circa the early 1990s, we
didn't have the Web and HTTP and graphical browsers.
Instead, we had Usenet news and FTP and command-line interfaces, and
we liked it that way! But as good as the good old days were, there
were some problems. For instance, when we were downloading kilobytes
of free software from a popular FTP site over our 2,400 bps modems
using Kermit, we would often encounter error messages like this one:

% ftp eunl.java.sun.com
Connected to eunl.javasoft.com.
220 softwarenl FTP server (wu-2.4.2-academ[BETA- 16]+opie-2.32(1) 981105) 
ready.
Name (eunl.java.sun.com:elharo): anonymous
530-
530-    Server is busy.  Please try again later or try one of our other
530-    ftp servers at ftp.java.sun.com.  Thank you.
530-
530 User anonymous access denied.
Login failed.

In fact, in the days when the Internet had only a few million users
instead of a few hundred million, we were far more likely to come
across an overloaded and congested site than we are today. The
problem was that both the FTP servers bundled with most Unixes and
the third-party FTP servers, such as wu-ftpd,
forked a new process for each connection. 100 simultaneous users
meant 100 additional processes to handle. Since processes are fairly
heavyweight items, too many could rapidly bring a server to its
knees. The problem wasn't that the machines
weren't powerful enough or the network fast enough;
it was that the FTP servers were (and many still are) poorly
implemented. Many more simultaneous users could be served if a new
process wasn't needed for each connection.

Early web servers suffered from this problem as well, although the
problem was masked a little by the transitory nature of HTTP
connections. Since web pages and their embedded images tend to be
small (at least compared to the software archives commonly retrieved
by FTP) and since web browsers "hang
up" the connection after each file is retrieved
instead of staying connected for minutes or hours at a time, web
users don't put nearly as much load on a server as
FTP users do. However, web server performance still degrades as usage
grows. The fundamental problem is that while it's
easy to write code that handles each incoming connection and each new
task as a separate process (at least on Unix), this solution
doesn't scale. By the time a server is attempting to
handle a thousand or more simultaneous connections, performance slows
to a crawl.

There are at least two solutions to this problem. The first is to
reuse processes rather than spawning new ones. When
the server starts up, a fixed number of processes (say, 300) are
spawned to handle requests. Incoming requests are placed in a queue.
Each process removes one request from the queue, services the
request, then returns to the queue to get the next request. There are
still 300 separate processes running, but because all the overhead of
building up and tearing down the processes is avoided, these 300
processes can now do the work of 1,000.
These numbers are rough estimates. Your exact mileage
may vary, especially if your server hasn't yet
reached the volume where scalability issues come into play. Still,
whatever mileage you get out of spawning new processes, you should be
able to do much better by reusing old processes.

The second solution to this problem is to use lightweight threads to
handle connections instead of heavyweight processes. Whereas each
separate process has its own block of memory, threads are easier on
resources because they share memory. Using threads instead of
processes can buy you another factor of three in server performance.
By combining this with a pool of reusable threads (as opposed to a
pool of reusable processes), your server can run nine times faster,
all on the same hardware and network connection! While
it's still the case that most Java virtual machines
keel over somewhere between 700 and 2,000 simultaneous threads, the
impact of running many different threads on the server hardware is
relatively minimal since they all run within one process.
Furthermore, by using a thread pool instead of spawning new threads
for each connection, a server can use fewer than a hundred threads to
handle thousands of connections per minute.

Unfortunately, this increased performance doesn't
come for free. There's a cost in program complexity.
In particular, multithreaded servers (and other multithreaded
programs) require programmers to address concerns that
aren't issues for single-threaded programs,
particularly issues of safety and liveness. Because different threads
share the same memory, it's entirely possible for
one thread to stomp all over the variables and data structures used
by another thread. This is similar to the way one program running on
a non-memory-protected operating system such as Mac OS 9 or Windows
95 can crash the entire system. Consequently, different threads have
to be extremely careful about which resources they use when.
Generally, each thread must agree to use certain resources only when
it's sure those resources can't
change or that it has exclusive access to them. However,
it's also possible for two threads to be too
careful, each waiting for exclusive access to resources it will never
get. This can lead to deadlock, in which two threads are each waiting
for resources the other possesses. Neither thread can proceed without
the resources that the other thread has reserved, but neither is
willing to give up the resources it has already.

There is a third solution to the problem, which in many cases is the most efficient of all, although it's only available in Java 1.4 and later. Selectors enable one thread to query a group of sockets to find out which ones are ready to be read from or written to, and then process the ready sockets sequentially. In this case, the I/O has to be designed around channels and buffers rather than streams. We'll discuss this in Chapter 12, which demonstrates selector-based solutions to the problems solved in this chapter with threads.