Recently, I was asked to develop a set of evaluation criteria for a NAS installation. The customer wanted a NAS that supported more than 18 TB and scaled to twice that, and they also wanted the NAS to operate with a single file system.
As I generally work with larger requirements in the HPC storage space, I figured I could apply what I know about SAN and shared file systems to NAS and use the same set of requirements questions and evaluation methods. Unfortunately, there isn't 100% overlap between a set of SAN requirements and a set of NAS requirements, and that leads to different evaluation criteria, as I soon found out.
It's All About Requirements
A set of requirements for NAS will have some differences from requirements for a SAN shared file system. Here are some issues that I wanted to understand before developing a request for information (RFI) for NAS vendors:
- What percentage of the total data space was going to be used before the files were removed? This is a big issue, since it can have serious implications for file system fragmentation. In most large SAN shared file system environments I work with, they are using HSM, which reduces the impact of file system fragmentation.
- What types of systems were going to be attached to the NAS system? Different systems have different endians. Changing endians requires significant CPU power or specialized hardware to move bytes from one endian to another. All data is almost always stored in the native ENDIAN of the CPU running the NAS.
- How will the NAS be backed up? Some NAS systems support their own tape system, others support standard NDMP (www.ndmp.org), and others support standard backup software such as Veritas or Legato.
- How much performance was going to be needed and could the base NAS provide the needed performance? Using simple measurements and counts from PCI and PCI-X bus can provide this information.
- What are the reliability requirements for the system and what performance will be needed during a disk failure? I know most of the rebuild issues with RAID devices, but was I was unfamiliar with the tunable settings for NAS devices.
- Did the customer really need an 18 TB file system growing to 35 TB? This seemed like a lot of storage space, given what I knew about most NAS products and the current crop of file systems.
The answers to all of these questions were not as straightforward as you might think, because with NAS you are often combining requirements from many departments or groups to come up with a system.
So after some significant work talking to all of the groups, we came up with a set of requirements that all of the groups could live with. These included requirements for:
- Performance, including performance under load;
- Scalability, to allow a single file system to grow to at least 35 TB;
- Backup of the data on the NAS, including the ability to load the data on a non-NAS platform off site; and
- Uptime, reliability and speed of repair — they needed the MTTR (mean time to repair) to be low for remote sites.
Searching the Web reading vendor marketing claims didn't tell us much, so we created a vendor survey, a list of our questions and a set of criteria by which to evaluate the vendors.
You might think that setting criteria isn't important before you send out the list of questions for the vendors, but this is likely one of the most important decisions you will make during the procurement process. Everyone has their favorite vendors for both good reasons and bad, and you need to put favoritism aside and pick the best vendor for the project and your company. This might or might not be your favorite vendor and might or might not be someone else's favorite vendor, so a list of requirements and how to evaluate the vendors is the key to success and to a harmonious office environment.
Before you write a single word that will be given to vendors, you should create an internal documentation on what is important in the evaluation and why.
Here is an example showing some of what we did for a customer; the critical requirements for the project are listed below. Also, a number of other requirements were given to the vendors in areas such as power and cooling, parts and repair location, and system functionality.
Basic Functionality Requirements
Each of these listed requirements is a basic functional requirement, and any vendor that does not provide this support will not be considered.
File System Size: The vendor must support file system of at least 18 TB today and 35 TB by late 2005.
Comment: This is a yes or no gating requirement; no arguing this one.
Backup: The NAS must be able to be backed up to a tape drive running at full rate with compression, and that data must be able to be restored on another UNIX system.
Comment: Tape wind quality (see Tale of the Tape: Beware of Wind Quality) is important, but equally important is the ability to restore to a different system in an alternate location.
Hot Swap: All of the components on the system must be able to be hot swapped. Any component that cannot be hot swapped must be listed and the vendor must provide detailed reliability statistics, including the Bellcore number (for a good definition of Bellcore failure calculations, see http://www.evaluationengineering.com/archive/articles/0500analyz.htm).
Comment: We wanted the system to be able to be hot swapped for the most common components such as host-to-disk communication hardware. We realized that some parts might not be able to be hot swapped, but wanted to know the failure rate for those parts.
Performance: We required full duplex performance at approximately 35 MB/sec, and read performance could not degrade the write performance requirement. We had an additional performance requirement given the size of the file system that performance could not be degraded by file system fragmentation (for more on this issue, see Storage Focus: File System Fragmentation).
Comment: This customer required a significant amount of performance since they were capturing video streams. Capture of this video data was of the utmost importance, not reading the data, so we wanted to make sure that vendors proposed a system that provided enough bandwidth to capture the streams that were being created. We also wanted to point out to the vendors that if fragmentation was an issue for their file systems — and it is for most file systems — that they should provide enough bandwidth to capture data even if the file system became fragmented. Some vendors might drop out of contention when they realized that their file system could not meet the requirement.
When developing a set of criteria for evaluating vendors, it is important to figure out what is really important to your organization in the usage of a NAS system. The same could be said for any system that you procure, since the process of requirements gathering is pretty much the same for any type of hardware or software.
Once you figure out what is really important, the next step is for everyone to come to an agreement on how to evaluate the vendors' response based on your requirements. Fairness and requirements-based evaluation should be your guiding principle.
You can use a rankings criteria based on colors (red means unacceptable and potentially high risk, yellow means risky and a low rating, blue means the vendor meets the criteria with low risk, and green means the vendor meets the criteria exceptionally well with very low risk) or you might develop a point system (0-10). The method you choose for doing the evaluation is not important; it's about making sure that the evaluation process is objective and provides a solution that meets both your functional and cost requirements, both today and over the product lifecycle. Buying a NAS system is no different than buying anything else; it is all about requirements.
Henry Newman, a regular Enterprise Storage Forum contributor, is an industry consultant with 25 years experience in high-performance computing and storage.
See more articles by Henry Newman.