TeraBit Feb 1994

Table of Contents

New Format for NSCEE Newsletter

For the past 3 years SCENE, the NSCEE's newsletter, has been a quarterly publication. Beginning with this issue, January 1994, SCENE will be published monthly. This will enhance our ability to provide our users with timely information about the Center and its activities.

Future issues of SCENE will include topics on

Mass Storage at NSCEE

by Michael Ekedahl, Software Systems Analyst

The NSCEE has achieved another milestone with the procurement of a StoragTek 4410 Automated Cartridge System. This system incorporates a tape robot to automatically mount and unmount nearly 6000 magnetic tape cartridges. The system has an approximate capacity of 2.7 terabytes (2,700,000,000,000 bytes).

StorageTek 4410 Automated Cartridge System

The Automated Cartridge System or Silo is coupled with the existing Cray Research Data Migration software allowing users to access their files 24 hours a day, 365 days a year. The average tape mount time for the silo is 11 seconds.

On the StorageTek system, large amounts of data are kept Nearline and can be accessed transparently by the user. The Cray Data Migration Facility manages data presence and locality with the mass storage complex balancing size of data and frequency of access.

Nearline stores data at a fraction of the cost per megabyte of online storage. Its quick, consistent response time places the 4400 Automated Cartridge System (ACS) much closer to conventional disk system performance.

The system can be upgraded to store over 10 terabytes of data.

netCDF Installation

NetCDF version 2.3.2 has been installed on clark. The following description is taken from the README file that accompanies the distribution. For further information, please see the netcdf (3) and netcdf (3f) man pages.

Unidata netCDF
Version 2.3.2 June 1993

The Unidata network Common Data Form (netCDF) is an interface for scientific data access and a freely-distributed software library that provides an implementation of the interface. The netCDF library also defines a machine-independent format for representing scientific data. Together, the interface, library, and format support the creation, access, and sharing of scientific data. The current netCDF software provides common C and FORTRAN interfaces for applications and data. It has been tested on various common platforms, including several versions of UNIX, VMS, MSDOS, and OS/2.

NetCDF files are self-describing, network-transparent, directly accessible, and extendable. Self-describing means that a netCDF file includes information about the data it contains. Network-transparent means that a netCDF file is represented in a form that can be accessed by computers with different ways of storing integers, characters, and floating-point numbers. Direct-access means that a small subset of a large dataset may be accessed efficiently, without first reading through all the preceding data. Extendable means that data can be appended to a netCDF dataset without copying it or redefining its structure.

NetCDF is useful for supporting access to diverse kinds of scientific data in heterogeneous networking environments and for writing application software that does not depend on application-specific formats. A variety of analysis and display packages have been developed to analyze and display data in netCDF form.

24-Hour Data Migration Access

by Sam West, AIC, Cray Research Inc.

For your convenience, NSCEE's new StorageTek 4410 Automated Cartridge System Nearline subsystem (also known as the Silo, the STK 4410, Mr. Hands) is now online and providing 24-hour access to your migrated data.

Inside, a vision system reads the cartridge labels to select the correct tape. A robotic stet grasps the cartridge and inserts it into the transport, where data is written or read. When the task is completed, the robot returns the cartridge to the appropriate location within the library.

Some mass-storage reminders for clark :

The archive file system, on clark, is accessible from all NSCEE platforms for data archiving needs. Aged files on /archive are automatically migrated to tape on a regular basis.
/u1, /u2
Automigration will soon be enabled on these file systems. More information on automigration will be discussed in upcoming issues of SCENE.
The global setting of the data migration recall mode is 1. This means that programs that access migrated files will block while waiting for the tape mount. Keep this in mind if a file access seems to be taking longer than it should and please wait at least 30 seconds before taking any drastic action.

Internal view of the StorageTek's tape shelves and robotic arm.

For further information about any of the foregoing topics, please see the appropriate man page on clark: archive, data_migration, or dmmode.

4410 Automated Cartridge System Overview

The 4410 Automated Cartridge System consists of four elements: a 4480 Cartridge Subsystem (control units and cartridge drives), one or more Library Storage Modules (LSMs) with attached Library Control Units (LCUs), a Library Management Unit (LMU) and a Host Software Component (HSC). The 4400 ACS is divided into two independent components--the data path and the library control path. The 4480 Cartridge Subsystem represents the data path, and the remaining elements combine to provide the library control path.

The 4480 Cartridge Subsystem utilizes 18-track, 1/2-inch cartridges and is completely compatible with IBM's 3480 manually operated system.

Each 4480 control unit can access up to eight cartridge transports and a cross-coupled control feature allows two control units to access up to 16 cartridge transports. The unit features separate processors for channel and transport operations, a 4-megabyte buffer, improved cartridge recordingcapability (ICRC) digital circuitry plus the flexibility of operation up to 60 feet (18 meters) from the cartridge drive when attached to the 4400 ACS. The 4480 control unit can optionally be configured to support host data channels at speeds up to 4.5 MB/sec.

Each cartridge drive frame contains two or four transports, a fully digitized intelligent servo, a large easy-to-read message display, independent power supplies, automatic tape loading and StorageTek's proprietary thin-film heads. A unique feature, called the Velocity Option, improves the efficiency of transport operations by decreasing the nonproductive rewind and high-speed search functions by approximately 33 percent.

Both the control unit and cartridge drive offer continuously running self-diagnostics and remote diagnostic capability.

The Library Storage Module (LSM) is the key to Nearline storage management. A 12-sided cylindrical structure less than 8 feet (2.4 meters) tall and 11 feet (3.3 meters) in diameter, the LSM maximizes the efficiency of both horizontal and vertical floor space and is designed to house approximately 6,000 cartridges. At 450 megabytes per cartridge, each LSM represents approximately 27 terabytes of storage.

The LSM is managed by an attached Library Control Unit (LCU). When it receives a request from the Library Management Unit, the microprocessor in the LCU commands the robot in the LSM to move to the storage cell or transport where the cartridge is to be moved or located, and to perform the proper operation. In the center of each LSM is a freestanding robotic assembly that retrieves, mounts, and dismounts the cartridges. Up to four cartridge drive frames (with a total of up to 16 transports) can be attached to the outside of the LSM. Up to 16 LSMs can be linked together by means of "pass-thru ports". These allow the exchange of cartridges from one LSM to another when all transports are allocated on a particular LSM. This ensures that any cartridge can be moved to any available transport at any time--and that a limited number of transports can serve a large amount of stored data.

Within a 4410 ACS, cartridges move automatically, quickly and reliably. Inside each LSM, the robot "sees" the correct cartridge (by means of a solid-state camera and two lights mounted on each of the two hands) and "reads" the bar-code volume serial number (VOLSER) on the cartridge's external label. The robot then extracts the cartridge from its cell, Moves it to a transport and mounts it. The robot mounts either 18-track data cartridges or tape-path cleaning cartridges, both stored within the LSM.

A cartridge-access Port allows you to insert or eject up to 21 cartridges at a time per LSM without entering the LSM. Cartridges entered into the LSM are then assigned a cell location. Or thousands of cartridges may be manually placed in an LSM at one time and the Host Software Component will "audit" the contents of the LSM, updating its volume location directory.

The Library Management Unit (LMU) manages up to 16 inter-connected LSMs and can receive, mount, and dismount requests from as many as 16 different hosts. When the LMU receives a request from a host to mount a cartridge, it selects the correct LSM. It sends messages to the LCU to instruct the robot to select the correct cartridge, validate the identification label retrieve the cartridge and mount it in the drive. The LMU then returns status to the Host Software Component.

[an error occurred while processing this directive]