File System Fundamentals This section looks at some file system basics, including factors to take into account when creating a new file system, creating optimized file systems, and basic file system terminology. In the following sections, the term file system is used to indicate both a type of file system implementation and an actual hierarchical implementation of user or system data. File System Implementation Considerations File systems are one of the most important parts of an operating system. File systems store and manage user data on disk drives and ensure that what is read from storage is identical to what was written. In addition to storing user data in files, file systems also create and manage information about files and about themselves. Besides guaranteeing the integrity of the data, file systems need to be extremely reliable and have excellent performance. Before creating file systems, you need a plan for their layout. The following are some general considerations to be aware of when planning your system: I/O workload should be distributed as evenly as possible across the disk drives. The number of file systems on any one disk should be kept to a minimum. All the Linux file systems are better able to manage fragmentation of a file system in a larger partition/volume than in a small, completely full partition. If a large set of files (in size, number, or both) has characteristics that make the files significantly different from "typical" files, create a separate file system for these files that is tuned to their requirements. The parameters that affect file system performance are set when the file system is initially defined. When a file system is created with the mkfs command, a set of default values is applied to define the file system, unless the defaults are specifically overridden. Although it is possible to use the file system tune command to modify some of the parameters after the file system is defined, not all parameters can be changed after the file system is created. Most of the time, it is simpler to use the mkfs command to define the file system right from the beginning. Advanced planning to create the file system is essential. Later in this chapter, we examine some of the specifics that can be set when a new file system is created. Creating Optimized File Systems mkfs is the front end to each file system's format program and its subprograms. mkfs creates file systems on disk partitions (or volumes, if a volume manager is being used). The mkfs command can usually run without the optional parameters that affect performance. To a large degree, this is because the default parameters are adequate in most cases. mkfs calculates the appropriate parameters to use based on the information it can ascertain about the volume or disk drive, and then it calls the file system mkfs command to actually create the file system. Basic File System Terms Before delving any further into file systems on Linux, it's important to learn some of the common file system terms. These terms are used throughout the remainder of this chapter. A logical block is the smallest unit of storage that can be allocated by the file system. A logical block is measured in bytes, and it may take several blocks to store a single file. A logical volume can be one or more physical disks or some subset of the physical disk space. Block allocation is a method of allocating blocks where the file system allocates one block at a time. In this method, a pointer to every block in a file is maintained and recorded. Ext2 uses block allocation. Extent allocation. Large numbers of contiguous blocks, called extents, are allocated to the file and are tracked as a unit. A pointer needs only to be maintained to the beginning of the extent. Because a single pointer is used to track a large number of blocks, the bookkeeping for large files is much more efficient. Fragmentation is the scattering of files into blocks that are not contiguous and is a problem that all file systems encounter. Fragmentation is caused when files are created and deleted. The fragmentation problem can be solved by having the file system use advanced algorithms to reduce fragmentation. Internal fragmentation occurs when a file does not a fill a block completely. For example, if a file is 10K and a block is 8K, the file system allocates two blocks to hold the file, but 6K is wasted. Notice that as blocks get bigger, so does the waste. External fragmentation occurs when the logical blocks that make up a file are scattered all over the disk. External fragmentation can cause poor performance. An extent is a large number of contiguous blocks. Each extent is described by a triple consisting of (file offset, starting block number, length), where file offset is the offset of the extent's first block from the beginning of the file, starting block number is the first block in the extent, and length is the number of blocks in the extent. Extents are allocated and tracked as a single unit, meaning that a single pointer tracks a group of blocks. For large files, extent allocation is a much more efficient technique than block allocation. Figure 11-1 shows how extents are used. Figure 11-1. An extent is described by its block offset in the file, the location of the first block in the extent, and the length of the extent. File system metadata is the file system's internal data structureseverything concerning a file except the actual data in the file. Metadata includes date and timestamps, ownership information, file access permissions, other security information such as access control lists (if they exist), the file's size, and the storage location or locations on disk. An inode stores all the information about a file except the data itself. An inode is a bookkeeping file for a file (indeed, an inode is a file that consumes blocks, too). An inode contains file permissions, file types, and the number of links to the file. It also contains some direct pointers to file data blocks, pointers to blocks that contain pointers to file data bocks (so-called indirect pointers), and even double- and triple-indirect pointers. Every inode has a unique inode number that distinguishes it from every other inode. A directory is a special kind of file that simply contains pointers to other files. Specifically, the inode for a directory file simply contains the inode numbers of its contents, plus permissions, and so on. A super block contains partition-wide information for the file system such as the size of the partition, size of allocation groups, block size, and so on. Allocation groups divide the space in a partition into chunks and allow the file system resource allocation policies to use methods to achieve high I/O performance. First, the allocation policies cluster disk blocks and disk inodes for related data to achieve locality for the disk. Second, the allocation policies distribute unrelated data throughout the partition in order to accommodate locality. Access control lists (ACLs) allow file owners to specify extended access information about a file, granting additional rights to users/groups other than those owning the file. This form of discretionary access control allows users to manage their own collaborative projects without intervention of system administrators to maintain groups and without granting rights to all users on the system. The Samba environment is a primary user of ACL support. An access control entry (ACE) is an individual entry in an access control list (ACL). The first field of an ACL is the entry type. The second field is a group name, username, numeric UID, or numeric GID, depending on the value of the first field. The third field is the access permission for this ACL. An extended attribute (EA) consists of a name and a value. Applications can attach additional information to a file object in the form of an extended attribute. A file object can have more than one extended attribute. EAs associated with a file object are not part of the file object's data; they are maintained separately and managed by the file system that manages that object. Applications define and associate extended attributes with a file object through file system function calls. File systems in Linux store ACLs by using extended attributes. Figure 11-1 shows how file extents work. If file sample.txt requires 18 blocks, and the file system can allocate one extent of length 8, a second extent of length 5, and a third extent of length 5, the file system would look something like Figure 11-1. The first extent has offset 0 (block A in the file), location 10, and length 8. The second extent has offset 8 (block I), location 20, and length 5. The last extent has offset 13 (block N), location 35, and length 5.

File System Fundamentals This section looks at some file system basics, including factors to take into account when creating a new file system, creating optimized file systems, and basic file system terminology. In the following sections, the term file system is used to indicate both a type of file system implementation and an actual hierarchical implementation of user or system data. File System Implementation Considerations File systems are one of the most important parts of an operating system. File systems store and manage user data on disk drives and ensure that what is read from storage is identical to what was written. In addition to storing user data in files, file systems also create and manage information about files and about themselves. Besides guaranteeing the integrity of the data, file systems need to be extremely reliable and have excellent performance. Before creating file systems, you need a plan for their layout. The following are some general considerations to be aware of when planning your system: I/O workload should be distributed as evenly as possible across the disk drives. The number of file systems on any one disk should be kept to a minimum. All the Linux file systems are better able to manage fragmentation of a file system in a larger partition/volume than in a small, completely full partition. If a large set of files (in size, number, or both) has characteristics that make the files significantly different from "typical" files, create a separate file system for these files that is tuned to their requirements. The parameters that affect file system performance are set when the file system is initially defined. When a file system is created with the mkfs command, a set of default values is applied to define the file system, unless the defaults are specifically overridden. Although it is possible to use the file system tune command to modify some of the parameters after the file system is defined, not all parameters can be changed after the file system is created. Most of the time, it is simpler to use the mkfs command to define the file system right from the beginning. Advanced planning to create the file system is essential. Later in this chapter, we examine some of the specifics that can be set when a new file system is created. Creating Optimized File Systems mkfs is the front end to each file system's format program and its subprograms. mkfs creates file systems on disk partitions (or volumes, if a volume manager is being used). The mkfs command can usually run without the optional parameters that affect performance. To a large degree, this is because the default parameters are adequate in most cases. mkfs calculates the appropriate parameters to use based on the information it can ascertain about the volume or disk drive, and then it calls the file system mkfs command to actually create the file system. Basic File System Terms Before delving any further into file systems on Linux, it's important to learn some of the common file system terms. These terms are used throughout the remainder of this chapter. A logical block is the smallest unit of storage that can be allocated by the file system. A logical block is measured in bytes, and it may take several blocks to store a single file. A logical volume can be one or more physical disks or some subset of the physical disk space. Block allocation is a method of allocating blocks where the file system allocates one block at a time. In this method, a pointer to every block in a file is maintained and recorded. Ext2 uses block allocation. Extent allocation. Large numbers of contiguous blocks, called extents, are allocated to the file and are tracked as a unit. A pointer needs only to be maintained to the beginning of the extent. Because a single pointer is used to track a large number of blocks, the bookkeeping for large files is much more efficient. Fragmentation is the scattering of files into blocks that are not contiguous and is a problem that all file systems encounter. Fragmentation is caused when files are created and deleted. The fragmentation problem can be solved by having the file system use advanced algorithms to reduce fragmentation. Internal fragmentation occurs when a file does not a fill a block completely. For example, if a file is 10K and a block is 8K, the file system allocates two blocks to hold the file, but 6K is wasted. Notice that as blocks get bigger, so does the waste. External fragmentation occurs when the logical blocks that make up a file are scattered all over the disk. External fragmentation can cause poor performance. An extent is a large number of contiguous blocks. Each extent is described by a triple consisting of (file offset, starting block number, length), where file offset is the offset of the extent's first block from the beginning of the file, starting block number is the first block in the extent, and length is the number of blocks in the extent. Extents are allocated and tracked as a single unit, meaning that a single pointer tracks a group of blocks. For large files, extent allocation is a much more efficient technique than block allocation. Figure 11-1 shows how extents are used. Figure 11-1. An extent is described by its block offset in the file, the location of the first block in the extent, and the length of the extent. File system metadata is the file system's internal data structureseverything concerning a file except the actual data in the file. Metadata includes date and timestamps, ownership information, file access permissions, other security information such as access control lists (if they exist), the file's size, and the storage location or locations on disk. An inode stores all the information about a file except the data itself. An inode is a bookkeeping file for a file (indeed, an inode is a file that consumes blocks, too). An inode contains file permissions, file types, and the number of links to the file. It also contains some direct pointers to file data blocks, pointers to blocks that contain pointers to file data bocks (so-called indirect pointers), and even double- and triple-indirect pointers. Every inode has a unique inode number that distinguishes it from every other inode. A directory is a special kind of file that simply contains pointers to other files. Specifically, the inode for a directory file simply contains the inode numbers of its contents, plus permissions, and so on. A super block contains partition-wide information for the file system such as the size of the partition, size of allocation groups, block size, and so on. Allocation groups divide the space in a partition into chunks and allow the file system resource allocation policies to use methods to achieve high I/O performance. First, the allocation policies cluster disk blocks and disk inodes for related data to achieve locality for the disk. Second, the allocation policies distribute unrelated data throughout the partition in order to accommodate locality. Access control lists (ACLs) allow file owners to specify extended access information about a file, granting additional rights to users/groups other than those owning the file. This form of discretionary access control allows users to manage their own collaborative projects without intervention of system administrators to maintain groups and without granting rights to all users on the system. The Samba environment is a primary user of ACL support. An access control entry (ACE) is an individual entry in an access control list (ACL). The first field of an ACL is the entry type. The second field is a group name, username, numeric UID, or numeric GID, depending on the value of the first field. The third field is the access permission for this ACL. An extended attribute (EA) consists of a name and a value. Applications can attach additional information to a file object in the form of an extended attribute. A file object can have more than one extended attribute. EAs associated with a file object are not part of the file object's data; they are maintained separately and managed by the file system that manages that object. Applications define and associate extended attributes with a file object through file system function calls. File systems in Linux store ACLs by using extended attributes. Figure 11-1 shows how file extents work. If file sample.txt requires 18 blocks, and the file system can allocate one extent of length 8, a second extent of length 5, and a third extent of length 5, the file system would look something like Figure 11-1. The first extent has offset 0 (block A in the file), location 10, and length 8. The second extent has offset 8 (block I), location 20, and length 5. The last extent has offset 13 (block N), location 35, and length 5.