Performance to/from tape

Updated 9/3/2008 to reflect use of single file-marks on write

The purpose of this document is to provide guidelines to enable users to get the best performance from enstore. The main two factors that affect tape performance are streaming and per file overhead, both of which are affected by file size.

The observed per file overhead of enstore is on the order of  3 seconds to write a file-mark after each file. This corresponds to processing about 30000 files/day. In order to overcome this limitation, your file size should be such that the time it takes to read/write the file on tape is significantly more than this overhead, say a factor of 10. For a 30MB/s 9940B drive, this is about 1GB. For a LTO-3 streaming at 40MB/s it is 1.3 GB, and for an LTO-3 streaming at 80MB/s this is 2.6GB.

There is another very important reason not to write small files. Enstore administrators maintain the integrity of your data by copying a tape to new media when the tape has had too many mounts (nominally 2000), and migrates tapes to denser media to save space and also to move off of media that is becoming obsolete.  A tape with 30,000 files takes more than a day to copy (just writing the file-marks)! It is recommended that no more than 3000 file be written to a single tape. Using the file size recommendations based on overhead considerations limits the number of files/tape to the hundreds.

Tape drives have a rate that they stream at, that is, they continue writing (reading) at this rate without starting and stopping as long as data is written (read) at this rate or faster. A drive will stop streaming to write a file-mark. Keeping a drive streaming provides the best rate possible and does not wear the drive as much as the continual starting and stopping when not streaming.

9940B	30MB/s
LTO-2	35MB/s
LTO-3	40MB/s  and 80MB/s
LTO-4	60MB/s and 120MB/s

Table 1. Drive streaming rates

A drive will not stream for very long when small files are written to it, or if the rate the file is being provided is slower than the drive’s streaming rate(s). Enstore has a large buffer memory on its movers that pretty much assures that reads of large files stream, regardless of how fast they are read out. With writes, the drive will not stream unless the rate provided over the network is sufficient. For 9940B drives, this rate is 30MB/s. LTO-3 (IBM) can stream at 40MB/s or 80MB/s. The usual culprit in not achieving these rates is the disk on the client computer – network rates are usually not an issue. In order to achieve 40 or 80 MB/s off disk, disk striping is usually necessary.

Below is a table of recommended file sizes and disk rates to get the best performance from enstore:

Technology	Recommended File Size	Disk rate to stream writes
9940B	1-2GB	> 30 MB/s
LTO-2	1-2GB	> 35MB/s
LTO-3	2-5GB	> 40-80 MB/s
LTO-4	3-6GB	> 60-120 MB/s

Table 2: Enstore recommended files sizes and rates

Note that these file sizes also keep the drive streaming for a significant time (~1 minute) so that the wear on the drives from stopping/starting is also minimized.