Clusters historically have been limited in the throughput they can deliver by the number of shared disk connections possible, because the disks have to be linked physically to each system. Recent advances in serial and fiber channel disk technology have raised disk performance and capacity. But this increased capability has also raised issues of management complexity, because each node introduces another discrete copy of the operating system. With Real Application Clusters, managing the multiple Oracle instances that make up the cluster is essentially the same as managing a single instance, but the multiple machines and the underlying operating systems still make for a greater management burden.
|
Massively Parallel Processing (MPP) systems appeared in the early 1990s to address these limitations. Unlike clusters, the disks in an MPP environment are not shared across multiple nodes, except for high-availability configurations, so the architecture is sometimes called "shared nothing" (shown in Figure 11-4). A special console provides additional operating system-management tools to make management of the many operating system copies appear to be more transparent.
In a typical MPP configuration multiple nodes reside in a single cabinet, but MPP configurations can grow to a series of cabinets with hundreds of nodes. The nodes are linked together via a high-speed switch and may be further linked to other cabinets of nodes via cables that physically connect the switch to the other cabinets. The switch provides each of the nodes with multiple interconnects to adjacent nodes. As nodes are added additional paths are enabled between the nodes, and the total throughput possible by the switch continues to scale with the connections. Individual nodes can be single processor or SMP. Oracle Real Application Clusters can use either configuration, although SMP nodes can be utilized for additional parallelism against data residing on a local node and are becoming the default.
The overall performance is a function of the power of individual nodes, the total number of nodes, the throughput offered by the switch, and the distribution of data. For example, in a business intelligence query, a 64-way SMP platform could outperform an MPP platform with 16 4-way SMP nodes, because these types of queries are usually CPU-bound and MPP interconnects are slower than an SMP memory bus. The tradeoff is that the MPP configuration can be much larger than a single SMP platform (although the cost of the MPP solution can be much higher than that of an SMP system).
The most common implementation of Oracle on MPP utilizes the IBM RS/6000 SP. Prior to including a cluster file system with Oracle Database 10g, Oracle RAC relied on a version of IBM AIX for the SP that included the Virtual Shared Disk (VSD). This software enabled the disk on the SP to appear as a shared disk, even though the disk is directly connected to an individual node. Today, the SP is much less popular, even among IBM customers. Scalability of IBM's 64-bit SMP nodes led to increased deployment of small SMP clusters built from IBM's Regatta class servers (e.g. the p690 and later) and the high-speed switch technology created for the SP as an interconnect.