SAN Extensions
Extending SANs over longer distances is crucial for optimal business continuance and might be required, as industry regulations or government legislation show. To increase the distance between storage networks, the following technologies might be implemented depending on budget allocations and the infrastructure and capacity already in place:Optical networkingFibre Channel over Internet Protocol (FCIP)Internet Fibre Channel Protocol (iFCP)Internet SCSI (iSCSI)
Optical Networking
Optical networks are instrumental in carrying data over long distances. Metropolitan-area networks (MANs) and long-haul networks typically utilize an optical backbone via synchronous optical network/synchronous digital hierarchy (SONET/SDH) or asynchronous transfer mode (ATM). IP over SONET (Packet over SONET [PoS]) has grown in popularity and is increasingly used in applications traditionally reserved for ATM, although it is a long way from replacing ATM entirely.Dense wave division multiplexing (DWDM) and coarse wave division multiplexing (CWDM) are methods of boosting capacity of the physical fiber layer by coalescing (multiplexing) multiple laser wavelengths onto the same fiber. Increasingly, DWDM and CWDM are used to maximize available bandwidth for applications and environments that require increased availability and uptime.DWDM and CWDM both support Enterprise Systems Connection (ESCON), Fibre Connection (FICON), and Fibre Channel (FC), but historically, optical networking has been cost prohibitive for deployment within smaller institutions. Although DWDM and CWDM are not equal in terms of capacity and throughput, the differences in cost (CWDM is significantly cheaper than DWDM) can lead to wider deployment of CWDM as a backbone for extending SANs.Other low-cost alternatives for carrying storage traffic long distances and linking SANs are making inroads with enterprises that seek the full range of data protection available by replicating and sharing data to remote sites.
Fibre Channel over Internet Protocol
FCIP acts as a bridge between two independent SANs by encapsulating FC traffic into IP packets and pushing that data through an IP tunnel. Although there are no physical distance limitations, depending on the capacity of the current IP networking infrastructure, jitter and latency can occur over longer distances. Many organizations already have ample IP networking capacity in place to support some usage models; however, depending on the speed of the network connection, live mirroring, remote clustering, or synchronous (or asynchronous over slower links) replication might not be appropriate. For example, FCIP might be appropriate in the storage service provider space, but overkill on a small campus that can be serviced over native, dark fiber.Figure 2-6 highlights the simplicity and the versatility of FCIP. Joining FC SANs over an IP network using a multiprotocol SAN switch provides long distance replication capabilities while sharing resources over disparate geographies.
Figure 2-6. FCIP Usage

Internet Fibre Channel Protocol
Whereas FCIP is typically hosted over a card on a Fibre Channel switch, iFCP works through a separate gateway device on a network to wrap FC traffic in IP packet format. iFCP gives Fibre Channel devices IP addresses so that the FC devices appear to run native IP. This allows FC devices and FC traffic to be routable (and manageable through network monitoring tools). Like FCIP, there are no physical distance limitations, and iFCP utilizes existing IP and FC resources, specifically, Internet routers and switches and FC HBAs (host bus adapters). iFCP relies on iSNS (Internet Storage Name Server), as does iSCSI, to provide network address and device name resolution. Figure 2-7 shows an iFCP topology with iFCP gateways carrying Fibre Channel traffic on an IP network.
Figure 2-7. iFCP Gateway

Internet SCSI
Internet SCSI (iSCSI) is a protocol designed to encapsulate SCSI commands within IP packets, allowing SCSI traffic to be routed and managed like any other IP packet. Much of iSCSI's adoption thus far can be attributed to its low TCO. The fact that iSCSI leverages existing IP or Gigabit Ethernet and DAS resources makes it an excellent fit for environments that have a high percentage of DAS storage, little investment in Fibre Channel, and less stringent performance requirements for long-distance data replication. iSCSI is often not a good match for high performance and high availability scenarios, such as live mirroring or synchronous replication of databases over long distances; however, iSCSI can be used to mirror or replicate other data types (file and print sharing) to remote locations in a cost-effective fashion. iSCSI can be utilized over existing network interface cards or cards known as TCP Offload Engines (TOE cards), which partially or fully alleviate the burden of sharing client IP traffic over the same pipe as what is used for SCSI data.iSCSI environments can be configured to run over a separate iSCSI router, as illustrated in Figure 2-8, or over an iSCSI blade in a multi-protocol SAN switch.
Figure 2-8. Typical iSCSI Installation
