Redbooks Paper

Pallavi Galgali Ashish Chaurasia

IBM TotalStorage SAN File System as an Infrastructure for Multimedia Servers

Abstract
Multimedia applications that deal with high-quality video and audio files place special demands on the file system in terms of storage and data retrieval. These applications typically use large files, requiring greater resources from the storage system in terms of capacity and high quality of service for retrieving those files. Storage area networks (SANs) provide an ideal infrastructure to build a multimedia serving environment, but require additional support layers to effectively exploit the SAN with multiple applications in heterogeneous environments. One such layer is the file system. IBM TotalStorage SAN File System simplifies file and data management in SANs by consolidating file systems across servers on different platforms. It is highly scalable for managing large quantities of data and supports optimized access methods for mixed file type environments. SAN File System provides better control of your data, using policy-based automated file placement on tiered storage. This IBM Redpaper discusses: How a SAN satisfies the storage hardware needs of multimedia servers How SAN File System meets the storage, access, and management requirements for media files This paper is intended for storage professionals looking for ways to better implement and manage a multimedia environment. We also provide a brief overview of the architectural design of SAN File System. For more detailed information about SAN File System, see the IBM Redbook IBM TotalStorage: Introducing the SAN File System, SG24-7057.

Copyright IBM Corp. 2005. All rights reserved.

ibm.com/redbooks

Characteristics of multimedia data

There are certain characteristics of media files, compared with other files. Here are the main points of difference: File size: Media files are generally large in size and thus require greater storage space. Retrieval method: The file retrieval pattern of media files is always sequential. Quality of service (QoS) demand: Let us take an example of playing a movie. Consider that we are playing a movie on a normal desktop machine. Now, start different applications in parallel such as e-mail, FTP, and Microsoft PowerPoint. Try to load the system to its maximum limit. At some point, we start seeing the jitter in our movie play. This means that we will not be able to see the movie smoothly, which implies that the media data requests have a specific deadline associated with them. If the system can serve the data within that deadline, we will be able to see the movie properly or smoothly. This is called QoS, or quality of service, demand. The periodic nature of the multimedia data necessitates QoS guarantees in the form of guaranteed throughput and bounded latency. So, our deadline can be expressed in terms of guaranteed throughput and bounded latency. This requires a deterministic nature in the service that is not expected from other types of files. Looking at this special QoS characteristic of media files, we can broadly classify file system data into two broad categories: Real-time (RT) data Non-real-time (NRT) data Files that have a QoS requirement, that is, all media files, belong to the RT data category, and all other data that do not require any deterministic nature from the file system are NRT data.

Why SAN and SAN File System?

Let us now try to understand why we propose the SAN and SAN File System infrastructure with multimedia servers.

Why SAN?
To answer the question of why SAN, consider the scenario in Figure 1.

IBM TotalStorage SAN File System as an Infrastructure for Multimedia Servers

Disk1
Server1 Clients Server2

Disk2

Figure 1 No storage area network

Here, Server1 is a multimedia server or streaming server serving some clients. Server1 can serve a maximum of 10 clients, assuring the required QoS to each one. If more client requests come in, we need to add a new server named Server2. In addition, one needs to copy the data that Disk1 has to Disk2. The job of copying the data and maintaining two consistent copies is very tedious, because the media data is so large in size. A storage area network (SAN) can give us relief from this problem. As shown in Figure 2, Server1 and Server2 can share the data using a SAN. Therefore, using a SAN enables an easy capacity upgrade of your environment.

Disk

S A N

Server1 Clients Server2

Figure 2 Sharing data using a storage area network

Why SAN File System?

The current challenge with SANs is that it is very difficult to use the full potential of a SAN, and as a SAN expands, the management becomes even harder.

IBM TotalStorage SAN File System as an Infrastructure for Multimedia Servers

In a traditional environment, each server has its own file system that must be individually configured and managed. This means that as you scale the number of servers and file systems, you often have to scale the administrative staff. This silo approach to file systems also makes it difficult to set and audit enterprise-wide policies for how corporate information is to be stored and protected. Data sharing between applications that run on different servers is difficult. Traditional SAN infrastructures are just not very flexible, in and of themselves. Some major file and data management issues that exist in todays SANs include: File tasks must be done on each server. Difficult to migrate applications to other servers. Application downtime is required for file system changes. No single view or single access to files or data. No ability to pool files based on quality of service. Therefore, we need a file management system that exploits all the functionality provided by a SAN and also provides an easy interface to manage the data stored in a SAN. This system needs to resolve all concurrency-related issues for a SAN environment and provide a global namespace that will work on multiple platforms. IBM TotalStorage SAN File System is designed to simplify file and data management in your storage area network by consolidating file systems across UNIX, Microsoft Windows, and Linux servers. With SAN File System, files and file systems are viewed and managed as a centralized IT resource with a single point of administrative control. By providing a common SAN-wide file system, SAN File System can support secure, heterogeneous data sharing and centralized policy-based storage management in an open environment. SAN File System is designed to help you to achieve data lifecycle management efficiencies through policy-driven automation and tiered storage management. User-defined policies provide the ability to better match the cost of your storage resources to the value of your data. SAN File System is also designed as a highly scalable solution, supporting both very large files and very large numbers of files without the limitations normally associated with Network File System (NFS) or Common Internet File System (CIFS) implementations. The result is reduced storage management costs, enhanced productivity, and optimized storage resource utilization. SAN File System has been designed to help: Simplify file and database management and improve administrator productivity through policy-based data management Enhance data sharing and collaboration through a single, global namespace across heterogeneous server platforms, with high performance and full locking support Reduce storage needs by pooling available and temporary file space across heterogeneous servers and storage platforms and reducing the need to maintain duplicate data for sharing Improve application and data availability, and lower failover costs by reducing or eliminating application downtime for storage and data management tasks Simplify and lower the cost of data backups through application server free backup and leveraging built-in, file-based FlashCopy functions Provide an on demand storage environment for fast and simple deployment through file level virtualization, policy-based automation, and support of open standards

IBM TotalStorage SAN File System as an Infrastructure for Multimedia Servers

SAN File System architecture

In this section, we discuss the architecture of SAN File System.

Out-of-band virtualization
SAN File System is an out-of-band virtualization implementation, where the data flow is separated from the control flow. This is achieved by separating the data and metadata (data about the data) into different places. Out-of-band virtualization involves moving all mapping and locking tables to a separate server (the metadata server) that contains the metadata of the files. In an out-of-band solution, servers (such as application servers, file servers, and so on) request authorization to data from the metadata server, which grants it, handles locking, and so on. After they are authorized, the servers access the data directly without any metadata server intervention. After a server has obtained access to a file, all I/O will go directly over the SAN to the storage devices. For many operations, the metadata server does not even intervene. Separating the flow of control and data in this manner enables the I/O to use the full bandwidth that a SAN provides, while control can go over a separate network or routes in the SAN that are isolated for this purpose.

How it works
SAN File System is intended to provide a common file system specifically designed for storage networks. A key element of its design is the way that it manages the metadataoverall information about files, such as file location, security permissions, and locks. In many traditional file systems, the metadata resides within individual servers, which can create limitations in sharing and accessing data across servers or across file systems. By managing the metadata on the storage network using a metadata server, instead of individually within all the servers, the design of SAN File System helps move intelligence out from individual servers to the storage network so that it can be available to any server in the network. Figure 3 illustrates this concept.

IP Network for Client/Metadata Cluster Communications

NFS CIFS

External Clients

Metadata Server Cluster

Admin Client

AIX
VFS w/Cache

Solaris
VFS w/Cache

Windows 2000
VFS w/Cache

Linux
VFS w/Cache

Windows Server 2003

IFS w/Cache

Metadata Server

Metadata Store

Storage Network

Metadata Server

Metadata Server

Shared Storage Devices

Multiple Storage pools Data Store

Figure 3 SAN File System architecture

Computers that want to share data and have their storage centrally managed are all connected to the SAN. In SAN File System terms, these are known as clients, because they access SAN File System services; although in the enterprise context, they would most likely

IBM TotalStorage SAN File System as an Infrastructure for Multimedia Servers

be, for example, database servers, application servers, or file servers. The SAN File System client code installed on each attached server captures file requests (such as open, close, create, or lock) and passes this information to the metadata server, which maintains the master catalog of file metadata, notifies the client if a file address changes in flight, handles the file placement policies and information, and handles the locking for file sharing. After SAN File System client software has the metadata, it accesses the file data directly over the SAN. The SAN File System client software enables users and applications to access the global namespace through a virtual file system (VFS) on UNIX and Linux systems and an installable file system (IFS) on Microsoft Windows systems. This layer (VFS/IFS) is built by the operating system (OS) vendors specifically for special-purpose or newer file systems. Metadata servers are clustered for scalability and availability of metadata operations and are often referred to as the metadata server cluster. In a SAN File System server cluster, there is one master metadata server and one or more subordinate metadata servers. Each MDS runs on a separate physical engine in the cluster. Additional metadata servers can be added as required if the workload grows, providing solution scalability. In this manner, SAN File System is designed to provide high-performance access to data and enable sharing across heterogeneous application servers with full-locking and security. SAN File System is designed to allow applications on any server within the SAN to access any file in the network. No changes to the applications typically need to be made. Storage volumes that store SAN File System clients user data (user pools) are separated from storage volumes that store metadata (system pool), as shown at the bottom of Figure 3.

SAN File System: Right fit for multimedia servers

We now describe the different advantages of using SAN File System with multimedia servers.

Better performance with large files

SAN File System provides better performance with large size files. To understand how, let us take an example. Suppose that a multimedia application running in the SAN File System client wants to play a file named movie.avi, which is a large file of 3 GB stored in SAN File System space. The following set of operations are performed: 1. The multimedia application sends the request to open the file so that it can read and play the file. 2. The open request is passed to the SAN File System client driver. 3. The SAN File System client contacts the metadata server to get the required metadata information for the file. 4. In response, the client gets locks and the location information of the data blocks of this file. 5. After this, the SAN File System client can continue reading the data of this file from SAN storage without any need to contact the metadata server. Therefore, reading large files involves less metadata operations. When an activity involves less metadata operations, SAN File System performs better. Because of its out-of-band nature, it removes all bottlenecks and gives uninterrupted access directly to data. Therefore, SAN File System provides better performance with large size files.

Extensive caching
SAN File System does extensive caching of data, as well as metadata at the client side. It separately maintains the metadata cache. The metadata cache is about 32 MB for 32-bit 6
IBM TotalStorage SAN File System as an Infrastructure for Multimedia Servers

machines and 256 MB for 64-bit machines. The data cache is maintained by the normal file system-related operations through the virtual file system (VFS) layer. As the metadata is cached, the client does not need to contact the metadata server for the metadata frequently, so even if the data is not available in the data cache, it can be retrieved very quickly because of the faster Fibre Channel infrastructure support. Therefore, extensive caching of metadata plays an instrumental role in multimedia servers that serve popular files where there are multiple requests for the same file, because the file can be served from the cache itself, which cuts the delay of retrieving the data from storage multiple times.

Prefetching (read-ahead) to meet QoS

As we discussed earlier in this paper, the access pattern for RT data is sequential. And for sequential accesses of data, it is easier to predict the next pattern of blocks to read from disk. So while doing the reads of the current request, a file system can use this predictable pattern, and in background, read the next few blocks of data in parallel and put them in cache. Therefore, if the future data block is read ahead in cache along with current request, it will speed up the operation and help in meeting the deadline. SAN File System implements this prefetching or read-ahead technique for sequential data accesses. Every SAN File System client (Linux, AIX 5L, Microsoft Windows, and Sun Solaris) implements this mechanism to get better performance while doing sequential reads to a large file.

Data placement strategy for better throughput

SAN File System has a data placement strategy that uses a round-robin method for getting better throughput from the storage system. Figure 4 illustrates this method.

SFS Client / Multimedia server

IP Network

SAN
1 2 3

MDS Cluster

Volume1

Volume2

Volume3

Figure 4 Round-robin data placement strategy

Figure 4 shows a part of a very large file represented by the blocks in pink. Each file part is then divided into sections (three in this case), which are striped across the available storage volumes. Therefore, while retrieving data (for example, when playing a media file), these

IBM TotalStorage SAN File System as an Infrastructure for Multimedia Servers

sections can be read concurrently to improve throughput. SAN File System does all this transparently. This placement strategy is called the round-robin method. This method is best suited for when data is going to be retrieved in a parallel fashion.

Policy-based, efficient storage management

SAN File System provides efficient storage management using different policies. Files are located in storage pools. A storage pool consists of one or more storage volumes (LUNs). Storage administrators are able to create rules for SAN File System to determine what storage pool is used to allocate space for a file when the file is created. The storage administrator can use any of the file attributes (such as file name, file type, date created, user ID or group ID, and so on) to create these rules. Through the use of the policy-based rules, SAN File System automates the task of allocating space on the desired storage volumes. You can implement tiered storage and hierarchical storage management (HSM) using this feature. To understand this further, consider the following example, illustrated in Figure 5.

movie1.avi menu.txt
project.log
IP Network

Rules for File Provisioning *.avi, *.mp3 go into Storage Pool1 *.txt, *.doc go into Storage Pool2 *.log goes into storage Pool3
SFS Client

SAN File System

MDS Cluster

Storage Pool 1 Fast movie1.avi

Storage Pool 2 Medium menu.txt

Storage Pool 3 Slow project.log

Figure 5 Policy-based storage management example

Here, we have a tiered storage system with storage pools of different performance characteristics. Storage Pool 1 provides the best throughput. Storage Pool 2 provides medium performance, and Storage Pool 3 is the slowest. We have a policy file that shows the rules we have set. The rules say that all AVI and MP3 files should go in Storage Pool 1, because these are all multimedia files and have deadlines associated with them. Therefore, we need to give the best storage pool to these types of files. Then, all TXT and DOC files go to Storage Pool 2 and all LOG files go to Storage Pool 3. Figure 5 shows the execution of the policy with three example files named movie1.avi, menu.txt, and project.log. SAN File System also provides information lifecycle management capabilities, enabling administrators to specify how files should be automatically moved among storage pools during their lifetime. This feature can potentially lower the overall costs of storage and improve storage space utilization, enabling a balanced use of premium and inexpensive storage matching the objectives of the enterprise. This feature also has the potential to reduce

IBM TotalStorage SAN File System as an Infrastructure for Multimedia Servers

manual storage administration for managing space utilization and reduce the cost of storage management. Therefore, this policy-based storage management can prove helpful in the task of managing media files of large sizes according to their popularity or hit rate. You can write various rules in the policy according to your requirements.

High availability
SAN File System has some self-healing features that make it highly available for multimedia servers: If one metadata server from the metadata cluster fails, SAN File System has a fail-over mechanism with which all the responsibilities of the failed metadata server can be transferred to another, running metadata server. SAN File System has a volume drain feature that provides nondisruptive volume data transfers. This technique is useful if one of the disks fails in the underlying storage and needs to be replaced with a new one. SAN File System also has an automatic call home reporting feature, thus if there is any problem in the system, it automatically reports the problem to an IBM Support Engineer.

Conclusion
In this study of the requirements of multimedia data, we discussed unique characteristics of media data and its special QoS requirements from a file system. We proposed SAN as a hardware solution and SAN File System as a software solution on top of SAN for a typical multimedia data center. In this proposition, we touched on SAN File System architecture, and at the end, we described the features of SAN File System that make it a perfect fit in this media environment. In summary, we emphasized the following features: Better performance with large files Extensive caching Prefetching (read-ahead) to meet QoS Data placement strategy for better throughput Policy-based, efficient storage management High availability Another point that this paper stresses is that, along with meeting the QoS demands of media data, SAN File System provides a complete storage management solution, taking care of the data during its entire life cycle (generally termed information lifecycle management).

Related publications and online resources

The publications listed in this section are considered particularly suitable for a more detailed discussion of the topics covered in this Redpaper. IBM TotalStorage: Introducing the SAN File System, SG24-7057
http://www.redbooks.ibm.com/abstracts/sg247057.html

IBM TotalStorage SAN File System as an Infrastructure for Multimedia Servers

Introduction to Storage Area Networks, SG24-5470

http://www.redbooks.ibm.com/abstracts/sg245470.html

SAN File System product page

http://www.ibm.com/servers/storage/software/virtualization/sfs/index.html

The team that wrote this Redpaper

This Redpaper was produced by a team of specialists from around the world working at the International Technical Support Organization, San Jose Center. Pallavi Galgali is a Software Engineer at the IBM India Software Lab in Pune, India. Pallavi has been involved in development and maintenance projects with products such as SAN File System and Advanced Distributed File System. She has coauthored an IBM Redbook entitled The IBM TotalStorage Solutions Handbook,SG24-5250, and an article on IBM developerWorks titled A comparison of security subsystems on AIX, Linux, and Solaris. She holds a degree in Computer Engineering from Pune Institute of Computer Technology, India. Her areas of expertise include storage networking, file systems, and device drivers. Ashish Chaurasia is a Software Engineer at the IBM India Software Lab in Pune, India. He has been involved in development and maintenance projects with products such as SAN File System, Advanced Distributed File System, and Virtual Private Network. One of his articles has also been published on developerWorks, titled Linux network packet capturing mechanisms. He holds a degree in Computer Engineering from Madhav Institute of Technology and Science, India. His area of expertise include file systems, networking, storage, and device drivers. Thanks to the following people for their contributions to this project: Charlotte Brooks International Technical Support Organization, San Jose Center Leslie Estroff SAN File System Product Manager Steve Gebing SAN File System Marketing Manager Elizabeth Barnes International Technical Support Organization, Austin Center

10

IBM TotalStorage SAN File System as an Infrastructure for Multimedia Servers

Notices
This information was developed for products and services offered in the U.S.A. IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to evaluate and verify the operation of any non-IBM product, program, or service. IBM may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not give you any license to these patents. You can send license inquiries, in writing, to: IBM Director of Licensing, IBM Corporation, North Castle Drive Armonk, NY 10504-1785 U.S.A. The following paragraph does not apply to the United Kingdom or any other country where such provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you. This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice. Any references in this information to non-IBM Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this IBM product and use of those Web sites is at your own risk. IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you. Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. This information contains examples of data and reports used in daily business operations. To illustrate them as completely as possible, the examples include the names of individuals, companies, brands, and products. All of these names are fictitious and any similarity to the names and addresses used by an actual business enterprise is entirely coincidental. COPYRIGHT LICENSE: This information contains sample application programs in source language, which illustrates programming techniques on various operating platforms. You may copy, modify, and distribute these sample programs in any form without payment to IBM, for the purposes of developing, using, marketing or distributing application programs conforming to the application programming interface for the operating platform for which the sample programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore, cannot guarantee or imply reliability, serviceability, or function of these programs. You may copy, modify, and distribute these sample programs in any form without payment to IBM for the purposes of developing, using, marketing, or distributing application programs conforming to IBM's application programming interfaces.

Copyright IBM Corp. 2005. All rights reserved.

11

Send us your comments in one of the following ways: Use the online Contact us review redbook form found at: ibm.com/redbooks Send your comments in an email to: [email protected] Mail your comments to: IBM Corporation, International Technical Support Organization Dept. QXXE Building 80-E2 650 Harry Road San Jose, California 95120-6099 U.S.A.

Trademarks
The following terms are trademarks of the International Business Machines Corporation in the United States, other countries, or both:
AIX 5L AIX developerWorks FlashCopy IBM Lotus Notes Lotus Notes Redbooks (logo) Redbooks TotalStorage

The following terms are trademarks of other companies: Solaris, Sun, and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. Microsoft, Windows, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. UNIX is a registered trademark of The Open Group in the United States and other countries. Linux is a trademark of Linus Torvalds in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others.

12

IBM TotalStorage SAN File System as an Infrastructure for Multimedia Servers