Now that we've gone over the fine points of planning the SAN, and you have made purchase decisions relevant to the bandwidth and capacity needs of the SAN, it's time to hook up the spigots and get the thing running.
Note
Before you go barreling into these steps, make sure you realize that your Xserve RAIDs need to be "cooked" (powered up for the first 36 hours of its life for a format and verification process) in order to work optimally when added to the SAN. So, if your Xserve RAID is brand new, take it out of its box, hook up both power cords to the Xserve RAID and to the wall, power it up from the button in the rear, and leave it alonefor a day or so. When all the blue lights turn off, hold down the power button for a moment until the unit shuts down. Then you can begin.
First, with all equipment powered down, we will install the necessary internal hardware for our SAN.
1. | Install Fibre Channel host bus adapter (HBA) cards into each Mac (G4, G5, or Xserve) that is being connected to the SAN directly. Installing the Fibre HBA into a G5 PowerMac, in this case, into slot 4 (133 MHz PCI-X)
Installing the Fibre HBA into an Xserve G5
Note Fibre Channel cards are most often installed into PCI-X slot 2 or 3 of G5s (the slots that are closer to the bottom of the computer), reserving the upper PCI-X 133-MHz slot 4 (and its isolated PCI-X bus) for fast cards such as capture cards. However, if a particular machine will not need capture hardware, install the Fibre Channel HBA into slot 4. Alternately, if your G5 has a fast capture card in slot 4, and has a nVidia GeForce 6800 or ATI Radeon 9800 graphics card, your choice of where to put the Fibre card will be limited to PCI slot 3, due to the extra thickness of these graphics cards. There are no limitations to where you install the Fibre Channel controller card on a PowerMac G4 or either G4 or G5 Xserve. Dual-processor G5 PCI slot locations
The next step is necessary if you intend to have the Macs in your SAN access your facility's network or the Internet. |
2. | Install Ethernet network interface cards (NICs) into each Mac (G4 or G5) that needs to connect to the facility's network. Note Wow, it's getting crowded back there. If you have a capture card installed in PCI slot 4 and a Fibre card in slot 2 or 3, your choice is simple: install the NIC in the last remaining slot. However, if you have one of the double-wide graphics cards mentioned in the last note, a Fibre card in PCI slot 3, and a capture card in PCI slot 4, you have run out of slots! Unless you intend to use Airport as your outer network (which has its own security issues), Apple is currently recommending not using a capture card in a G5 that sports a double-wide graphics card that will also be on an Xsan. Alternately, you can now spring for the newer single-slot ATI X800 card that is able to drive 30-inch monitors. In this case, the bandwidth-hungry Fibre card should be reinstalled in PCI slot 4 and the Ethernet NIC in PCI slot 3. Xserve G5s come with two built-in Ethernet ports, so you won't need a NIC for these machines. |
Get your thumb and forefinger readyit's time to make the connections necessary for the Fibre Channel network to function.
We will now hook up the Ethernet network that will handle the metadata. Some installations that are not concerned about connecting the machines on the SAN to the facility's network will create their private network out of the built-in Ethernet ports on the Macs. However, if you need to have your SAN machines also on the facility network, and have installed Ethernet NICs in each G4 or G5 that will be on the SAN, you will then create your metadata network with either the NIC or the built-in port, with the remaining port used to access the outer network. Just make sure to be consistent across all nodes, as to which interface you will use. For Xserve G5s, we will use the en0 (lower) port to configure the metadata network. Later on, you can assign the settings for the facility network to the built-in port on G4s and G5s, and the en1 (upper) port on Xserves.
It's now time to create the directory server that will be responsible for creating and managing the users and groups on the SAN.
If you already have a directory set up at your facility, whether it is being served up by Mac OS X Server's Open Directory or Microsoft Windows' Active Directory, you may want to bind your controller and client nodes on your SAN to that pre-existing directory, using the outer Ethernet network. If so, you can skip this section.
However, on installs of six or fewer client nodes, it is permissible for an Xserve running OS X Server to also act as the controller node for the SAN. For larger installs, you will want OS X Server running on a separate Xserve, and a separate G5 or Xserve acting as the controller node. Think of the difference between a small restaurant and a huge banquet facility: the former can get away with one chef, whereas the latter definitely needs a main chef and a sous-chef.
We will assume you have purchased an Xserve that will serve up directory services for the SAN while simultaneously acting as its primary MDC, and that you have an install of six or fewer client nodes. Therefore, we will proceed as follows:
1. | Configure the Xserve. |
2. | Update the Xserve's OS X Server to the latest version (if needed). |
3. | Create an Open Directory Master on the Xserve. |
4. | Set up the users and groups of our SAN. |
5. | Install Xsan software. |
6. | Configure the Xserve as the primary metadata controller (MDC). |
This section covers the first three of these steps. We will cover the remaining three steps later.
We hope you found the previous sections blissful and mindless, because that state of mind will now end. As we said before, this part will be easier with an Xsan Medallion and/or ACSA-certified integrator at your side or in your head, whichever is more convenient.
Regardless of the size of your SAN, the configuration of OS X Server is the same. Because of space constraints, here is what we will
not cover in this section:
Installing a fresh copy of OS X Server on an Xserve
Updating an Xserve's OS X Server software from 10.X.X to 10.3.X or 10.4.X
Installing, updating, or configuring OS X Server on a G5, rather than an Xserve
Implementing OS X Server on the outer network of the SAN
For more information about these scenarios, you can go to the following online resources:
www.apple.com/server/documentationApple's excellent online documentation page
consultants.apple.comApple's online search tool to find ACTC and ACSA consultants in your area
www.afp548.comAn online forum specifically oriented toward OS X Server implementation
www.macosxhints.comAn online forum focused on OS X in general, but with excellent threads on OS X Server as well
There are also excellent Web resources for Xsan implementation:
www.xsanity.comA Web site specifically dedicated to the foibles of Xsan implementation
www.creativecow.netTheir Apple Xsan forum is a frequent hub of new ideas and solution
Note
The installation process described in the following sections is solely for implementing directory services for use with Xsan. OS X Server has many services to offer your facility, such as Web and mail services, firewall protection, and the like. In the following steps, we will only be implementing directory services for an isolated SAN system.
1. | |
2. | Power up the Xserve. Note Most Xserves are headless, meaning they have no graphics cards installed to drive a monitor. As a result, most are configured and administered remotely. If you already have a PowerMac or PowerBook that has the OS X Server Administration Tools installed on it, you can connect this machine to the Ethernet switch to perform this part of the process, and skip steps 3 and 4. |
3. | Power up another Mac on the network that will serve as a client node. We will use this Mac to configure the Xserve remotely. |
4. | Log in to this Mac on an admin account. |
5. | Install OS X Administration Tools on this client node. More Info The OS X Administration Tools are a free download from www.apple.com/support/downloads. Make sure to download any update to the tools to make sure they are the latest. |
6. | Install Apple Remote Desktop on this client node. More Info Apple Remote Desktop (ARD) is an indispensable part of remote control and configuration of any network. It also costs money. For more information, see www.apple.com/remotedesktop. |
Now that we've installed the OS X Server Administration Tools software on our client, we can connect to and configure our Xserve.
1. | Continuing on the client node, launch the Server Assistant application, located in /Applications/Server/. |
2. | Select "Set up a remote server" and click Continue. You will be presented with a list of servers, and hopefully, through the magic of Bonjour, you will see the Xserve, with the default name of localhost waiting to be configured. |
3. | Double-click in the Password column for the Xserve and type in the first eight characters of the Xserve's serial number, then press Return. Mask box, and finally 10.1.0.1 in the Router box, then click Continue. Note Notice that we have placed the same address in both the IP Address and Router boxes. The metadata network actually has no router, since its information is not being routed to or from a larger network. The reason for the fake router address is for the routing tables of the server, just in case a client machine asks for data not on the metadata network. The server, when queried, will simply drop this request. If you invested in a managed Ethernet switch, the switch itself will have an IP address. In this case, you can assign the router setting for step 8 to the address of your managed switch. |
9. | In the Directory Usage screen, leave the "Set directory usage to" pull-down menu as Standalone Server, then click Continue. Note Most seasoned OS X Server administrators would tell you that you should really create an Open Directory Master once the server is configured and properly running. They would be right, so we'll be promoting this server to an Open Directory Master after we have it up and running. The Assistant then asks if we'd like to employ other services. |
10. | In the Services screen, click the last check boxApple Remote Desktop Clientthen click Continue. We will need ARD on in order to further configure or update the OS of the server using Apple Remote Desktop. |
11. | Select the appropriate time zone from the Time Zone screen, then click Continue. |
12. | In the next screen, Network Time, deselect the "Use a network time server" button, and click Continue. Since we are not connected to the Internet on this metadata network, trying to ping an external time server will just add chatter. |
13. | Finally, review all of your settings and click Apply. After a few minutes of configuration time, you will be presented with a box stating that the server needs to be restarted. |
14. | Click Continue Now to restart the Xserve, or just wait until it restarts itself. |
15. | Quit the Server Assistant. |
Hopefully, you have a frothy beverage to enjoy at this point, because you are 87.3 percent done configuring your Xserve. Mazel Tov!
As you'll recall, we configured our Xserve with a static IP address. In order to talk to it again, we'll have to configure our client node with complementary IP information.
If you have the luxury of having the latest OS X Server install discs, you can skip this step. If not, you'll need to update your server software to the latest version. This is true even if your Xserve is not acting as an MDC. Since our metadata network does not currently connect to the Internet, you will do a little old-fashioned sneaker-net transfer of the update packages to the network.
In some cases, the ARD client we turned on when we configured the Xserve will be an older version. Follow these steps to update the client software on the Xserve to the latest version.
1. | Select the MDC server in the list. |
2. | Choose Manage > Upgrade Client Software. You will be prompted to add this computer to the Master List. |
3. | Enter mdc.local in the DNS name box, and the username and password of the admin account for the server in the appropriate boxes, then click Add. |
4. | In the Upgrade Client Software screen, click Upgrade. It will take a little while to install and start up the new ARD client software on the server. A status window will pop up to inform you of the progress of the upgrade. |
5. | When the upgrade is complete, close the Status window. |
Now that we have the right version of the ARD client software running on the server, it's time to install the packages from the FireWire drive.
1. | Back in the main window of ARD, click your MDC server, then click the Control icon in the toolbar. A login screen appears. |
2. | Enter the admin account username and password, and click the Log In icon. |
3. | Now that you are in, install each package from the FireWire drive. Note If you perform the Xserve G5 firmware update, you will be required to shut down the Xserve and hold down the power button when starting it up again to reprogram the firmware. Read the About document that comes with the firmware update for more information. |
4. | When all your packages have been installed, quit Apple Remote Desktop. |
In the next steps, we will set up the server to serve a directory using Open Directory. This will allow us to create a listing of users and groups that will be universally accessible throughout the network. We'll use the Server Admin application to do this.
1. | From our trusty client node, launch Server Admin, which you'll find in /Applications/Server/. When the program launches, you will be prompted to add a server to administer. |
2. | Enter mdc.local in the Address box, and the username and password of the admin user in the next two boxes, and click Connect. Information about the server loads into the window. |
3. | From the list of services at left, click the Open Directory service. |
4. | Click the Settings tab. |
5. | Change the Role from Standalone Server to Open Directory Master. You'll then be prompted to enter information about the directory. |
6. | The directory admin username and password should be different from your "regular" server admin username and password for security reasons, but it's your choice. The dialog box suggests a username of "diradmin." Fill in the rest as shown in the following image. |
7. |
Now that the directory is created, we'll run Workgroup Manager and populate the directory with some users and groups. The users and groups we'll create will be fake, just for fun, but feel free to substitute your actual users and groups instead. We do this now, because once we set up and configure the client nodes, and bind them to the directory, the users of the SAN will already have been set up in the directory: all your users will have to do is log in to the machine on which they'll be working.
1. | With Server Admin still open, click the handy Workgroup Manager button in the upper-left corner of the interface. Workgroup Manager automatically launches. More Info Since Server Admin and Workgroup Manager are used so often with one another, they have hot link buttons to the other program in the upper-left corner of their interfaces. |
2. | At the Workgroup Manager Connect screen, type mdc.local in the Address box, and the directory administrator username and password in the next two boxes, and click Connect. |
Once we're connected, the main Workgroup Manager appears. From here, we'll first add some groups.
1. | In the upper left of the window, click the Groups tab. |
2. | Click the New Group button in the toolbar. A new group, Untitled 1, with the GID of 1025, appears in the list at the left. |
3. | In the Name box, type Creatives . The short name creatives automatically gets created as you type. |
4. | Click Save. |
5. | Use steps 2 through 4 to create two more groups: Editors , with a GID of 1026, and Graphics , with a GID of 1027. When you're done, your three groups should look like this: |
Next we'll create some users for our directory.
Congratulations are in order again, because we have now finished setting up the first, and most difficult, part of our Directory Server. Now, let's get the SAN actually up and running.
It's time to get the software installed on each machine on the SAN. We'll first get our other client nodes booted and configured on our metadata network, so we can do as much of this as remotely as possible.
Note
Every additional client node must have an admin account, and you should log into this admin account as you configure the machines in the next two subsections.
Now we will install the Xsan software on all of our machines that will be on the SAN, which includes all of our client nodes and our Xserve.
1. | Get back onto our trusty Admin client node, and pop the Xsan Software Install CD into the optical drive. |
2. | When you see the CD mount on the Desktop, double-click the CD icon to open it. |
3. | Find the Install_Xsan.mpkg application and drag it to the Desktop. |
4. | Eject the CD. |
5. | Launch Apple Remote Desktop. |
Now that you've added all the other client nodes onto the metadata network, you'll see their IP addresses in the main Remote Desktop window. Now we will upgrade the ARD client software on all these machines, in one step, like we did with our Xserve earlier.
1. | Select all the client Nodes in the current scanner list. |
2. | Choose Manage > Upgrade Client Software. You will be prompted to add each computer to the Master List. |
3. | Enter the username and password of the admin account for that particular client in the appropriate boxes, then click Add. |
4. | Repeat step 3 for each client node. |
5. | In the Upgrade Client Software screen, click Upgrade. |
6. | When the upgrade is complete, close the Status window. |
From here, you can install the Xsan software on all your machines, and the Xserve, in one step, from one location.
Now that we have all the software installed, it's time to get the client nodes bound to the directory that we set up earlier.
We will configure the Directory Access applicationfound in each client node's /Applications/Utilities folderto look for directory information from our Xserve. This way, end users will be able to log in from a client node and be able to have proper access to different directories on the SAN volume(s). More importantly, each end user and group will have unique user and groups IDs, which will make the MDC very happy.
1. | After our Admin client node has restarted from its Xsan software install, launch Apple Remote Desktop on it. |
2. | Select the first additional client node on the list, and click the Control icon in the toolbar. |
3. | In this client node's Control window, launch the Directory Access application, found in the /Applications/Utilities folder. |
4. | Observe the lock icon in the lower left. If it's locked, click it and authenticate with the admin username and password for that client node. |
5. | In the middle of the window, click the LDAPv3 listing, then click Configure. |
6. | Click the blue disclosure triangle next to Show Options. |
7. | Click New to create a new configuration. |
8. | Type mdc.local in the Server Name or IP Address box. |
9. | De-select the Use For Contacts check box, and click Continue. |
10. | Click OK to leave this sub-window. |
11. | Click OK to complete the binding. |
12. | Quit Directory Access. |
13. | Close the Remote Desktop control window for this node. |
14. | Repeat steps 2 through 13 for each additional client node. |
Now let's remember to bind our Admin client to the directory as well.
1. | Leave Remote Access running, and launch Directory Access on the Admin client. |
2. | For this final client, follow steps 3 through 12 in the preceding exercise. |
Finally, we will shut down all the machines, before we fire up everything in sequence and get ready to configure the SAN.
1. | Back in Remote Desktop, select all machines, including your MDC server, on the Remote Access list (except your Admin client), and choose Manage > Shut Down, then click the Shut Down button. Watch in glee as all the machines shut down. |
2. | Quit Remote Desktop on the Admin client. |
3. | Shut down the Admin client. |
These final steps will allow us to get our storage and our nodes together and create a SAN. Let's go.
It's time to power everything up, in sequence, in order to use Xsan Admin to create and configure our SAN.
The pieces of the puzzle will come together in this next step, where we use the powerful and simple Xsan Admin app to create and mount the SAN.
1. | Once again from the trusty Admin client, launch the Xsan Admin application, located in /Applications/Server. Upon launching, the program will ask for an address to log into. We will give it the address of the machine we want to be the metadata controller, which is our server. |
2. | In the Address box, enter mdc.local , then authenticate with the name and password of the server's admin account, and click Connect. |
3. | Click the Setup tab. Hopefully, you'll see as many entries as you have nodes in your SAN in the list that appears: your MDC server, plus the IP addresses of all of your clients. |
We're now going to validate all our nodes with serial numbers, so make sure to have them handy. Remember, you'll need a unique serial number for each node, including the metadata controller.
This part is really fun: here you get to get to create a huge storage volume by dragging and dropping icons!
1. | In the upper area of the interface, click the LUNs tab. You'll see four (or more or less, depending on the number of Xserve RAIDs you have on hand) LUNs with yellow dots in this list. These LUNs are ready for use, but are not labeled as Apple Cluster File System (ACFS) LUNs for our volume, so we'll do that now. Note For factory-fresh Xserve RAIDs, you'll see two LUNS per Xserve RAID on your Fibre switch. If you see fewer than two, check connections on the switch or on the back of each RAID. |
2. | Double-click the first LUN. A window drops down and asks for a label name. |
3. | Type LUN1 into the box and click OK. |
4. | Repeat steps 2 and 3 for each additional LUN, incrementing the number at the end of the name each time ( LUN2 , LUN3 , LUN4 ). |
5. | When finished, click Save. The labels are applied to the LUNs, which now gleam green. Note Once LUNs are labeled for Xsan's ACFS, they are no longer available for good old HFS+ formatting. In order to get them unlabeled, I'm afraid you'll have to venture into the command-line interface and use the cvlabel command, which will be necessary if you ever intend to use these LUNs for HFS+ again. More info on this in the Xsan Admin Guide. |
6. | In the upper area of the interface, click the Storage tab. Here is our volume playground. |
7. | Click the Volume icon. A window drops down to ask for a volume name. |
8. | Type in the name for your volume, and click OK. An icon for your volume appears in the Storage window. You just entered the name of the disc that the nodes will see mounted on the Desktop. We will leave the other settings (Block Allocation Size and Allocation Strategy) unchanged because they are set up perfectly for general purpose video file usage. You can read more about them in the Xsan Admin Guide. |
9. | Click the Storage Pool icon. A window drops down to ask for information about this storage pool. |
10. | Type POOL1 in the storage pool Name box. |
11. | Click "Any data" for the "Use for" setting. Note We have the option of creating our initial storage pool exclusively for metadata and process journal information only. This is to increase performance on nonmetadata storage pools by isolating the relatively small and articulate metadata to its own pool. One day you might do this. Today, this storage pool will contain both the aforementioned data and regular ordinary data, happily commingling. |
12. | Leave the Stripe Breadth at 256 blocks and click OK. Note Apple says that a Stripe Breadth of 256 blocks yields the best general use performance on fully populated (14-drive) Xserve RAIDs. Also, someone somewhere on our SAN will want read and write permissions, so we'll keep that setting at its default. Finally, the Multipath Method is set to Rotate to switch to a second Fibre cable if the first one fails. A Storage Pool icon appears, indented inside your volume. |
13. | Click the Available LUNs button. A drawer reveals your ready-to-drag, labeled LUNs. |
14. | Grab them and drag them to your pool. Your volume is now built. Look how big that thing is! |
15. | Click the Close Drawer button. You are prompted with a warning about mixing metadata and user data in the same Storage Pool. As mentioned earlier, you should avoid this for best performance, but for our exercise, it'll suffice. |
16. | Click Save. The volume creation process takes just a moment. |
17. | In the upper left of the interface, click the disclosure triangle next to your SAN name. Your volume pops up nested below it. |
18. | |
19. | Click the Start Volume button. Again, another pause as the volume gets ready to be mounted. After a while, you'll see a green light next to the volume name at left. |
Notice now that when clicked on the volume, our button names have changed. Now the interface changes its context to the settings of our volume. This is how we will mount the volume on the nodes.
At this point, you've probably picked this book back up after putting it down hours ago and playing extensively with your volume, establishing crazy speed tests and discovering all kinds of interesting phenomena that happen with file-level locking cluster file systems. So, if you can, put your lab coat away, and let's get down to the big reason why we created a directory in the first place, and that is to create specific areas on the SAN where different users or groups have access and read/write permissions. The exercises here are designed more for stimulating creative solutions for your storage challenges than dictating how they should look, but feel free to steal any part that works for you.
Next, we'll look at Xsan Admin's capability to create root-level (and only root-level) folders on the volume, and then we'll assign specific permission to those folders. If our workflow is simple, and we need general areas where one group can get into and others can't, Xsan Admin will do the trick for us.
Note
We can always set permissions to volume folders deeper than root level by simply creating them in the Finder, getting information about the folder, and setting permission settings there. Even though all the computers on our SAN are bound by a centralized directory, meaning that our directory's users and groups appear in everyone's lists, we'll probably want to make these folders when logged into the server via ARD, or by using OS X Server Workgroup Manager. You can learn more about setting detailed file and folder permissions in the OS X Server Admin Guide.
1. | Back in Xsan Admin, click your volume and then click the Affinities tab. Note You must have the volume mounted on the MDC for the Affinities and Quotas tabs to work in Xsan. |
2. | In the upper-right area, click the plus (+) button to create a root-level folder. A window drops down to ask for a name and affinity for the folder. |
3. | Type Editors Work in the Name box. |
4. | From the Storage Pool Affinity pop-up menu, choose POOL1. |
5. | Click OK. A folder appears nested inside the volume. Note Affinities are an excellent feature of Xsan that we're not able to leverage here, since we have only one measly storage pool. Essentially, we could direct all data placed in this folder to a specific storage pool in our volume, allowing us to save the data to a faster or more secure storage pool. Neat, huh? Read more about it in the Xsan Admin Guide. |
Now let's set specific permissions for this folder.
1. | Click the Editors Work folder. |
2. | In the lower-right area, where the permissions data is, click the plus (+) button. Another drawer appears with a list of all our directory's users. |
3. | Drag the drawer out a little more to see the UIDs for everyone in our crowded list. Note Sorry for the clutter, but this list is a proper server list, which includes process users. These users are not people, but actual system processes that have usernames. |
4. | Click the Groups button of this drawer. Now we see all the groups in our directory, plus all the server groups, with their corresponding GIDs. |
5. | Find the Editors group and drag it to the Group box in the lower portion of the interface window. |
6. | For extra security, change the Other settings to No Access. No one but the Editors group will even be able to get into the folder, let alone read or write to it! |
7. | Using your knowledge of this interface, create another root-level folder called Graphics Work , and assign Read & Write permissions to the Graphics group. Lock this folder to all others, just like you did with the first folder. |
8. | Finally, create a Creative Drop Box folder, and assign Read & Write permissions to the Creatives group, but Write Only access to Others. This way, anyone can drop something in this folder for use by the Creatives. |
9. | Click Save to save all this work. |
10. | Quit Xsan Admin. |
To review: we have the Editors Work folder that will be available only to the Editors group; the Graphics Work folder, available only to the Graphics folks; and the Creative Drop Box, which all Creatives have access to, and anyone else can drop stuff for the Creatives in. Now let's have a look at how our directory works.
What we'll do now is log out of a machine (any client node on the SAN will do), and log in as one of our users, to see the effect of these root-level folders we've created.
1. | Choose Log out (Username) from the Apple menu or press Cmd-Shift-Q on the client node to log out. |
2. | Press Return to confirm. When the login window appears, you will now see an Other icon. |
3. | Click the Other icon. |
4. | Authenticate as Charles (username charles, password charles). |
5. | Open the volume and look at the permissions of the folders. Looks like Charles, who is a graphics guy, can get into the Graphics and Creative folders, but not into the Editors'. |
6. | Log out and authenticate as Albert this time. |
7. | Open the volume and take a look. Albert, being a cutter, can't get into the Graphics folder. This may be a problem, but hopefully you know how to fix it. Let's move on. |
8. | Log out and authenticate as Bonnie. Bonnie, talented cat that she is, has access to all the folders, since she is a member of both the Editors and Graphics groups, and also is a creative. |
9. | Log out and authenticate as the administrator on that machine. |
10. | Open up the volume. |