High Performance Computing and Communications: Archived at NLM

By Sally Howe ~
People today might use their smart phones without even thinking about how they work, let alone about the infrastructure that enables us to communicate and compute at a global (actually interplanetary) scale, but today’s reality is built on yesterday’s imagination. Trailblazers had to envision something like today’s information infrastructure and to communicate their foresight in such a way that stakeholders and users would willingly work together over decades to realize it. Much of that envisioning took place in the 1980s and early 1990s and led to the creation of the Federal High Performance Computing and Communications (HPCC) Program.

Begun in 1992, the HPCC Program and its National Coordination Office (NCO) coordinated Federal research and development in high-performance computing, high-capacity and high-speed networking, and information technology by Federal science and technology agencies. HPCC initially included four large agencies—ARPA, DOE, NASA, and NSF—and four smaller ones—EPA, NIST, NLM, and NOAA. There are now 14 agencies in the successor Networking and Information Technology Research and Development (NITRD). NLM participates under the NIH umbrella.

With the formal release of the extensive National Coordination Office for High Performance Computing and Communications Archives, at the National Library of Medicine, we celebrate the 25th anniversary of the September 1992 establishment of the NCO/HPCC. The Federal HPCC Program and the NCO/HPCC and their successors implement the requirements in the High-Performance Computing Act of 1991 and its amendments. The archive is the culmination of work I’ve undertaken for some time with the helpful guidance of HMD archivists. The NCO was located at NLM from September 1992 until 1995 when it moved to NSF space. I worked at the NCO/HPCC from September 1992 until 2007, including as NCO Associate Director from 1998 to 2007.

The HPCC story is like an elephant, with each participant knowing only part of it. The HPCC archival collection focuses on policy and coordination. What follows are some of my views of the animal.

In the 1980s, seemingly insatiable user communities at Federal science and technology agencies and at U.S. colleges and universities had largely the same goals, namely high-capacity and high-speed network access to each other, unique data sets, and ever-faster supercomputers (for example, to do more realistic scientific modeling and simulation). Supercomputer manufacturers were willing to satisfy this need with hardware that was then expensive to buy and maintain. The U.S. Congress convened these and other stakeholders (the library community was an early advocate, the telecommunications industry a late-comer) to develop a shared vision and embody it in the High-Performance Computing Act of 1991 (Public Law 102-194).

So what was the vision? To quote from the HPC Act:

• “Advances in computer science and technology are vital to the Nation’s prosperity, national and economic security, industrial production, engineering, and scientific advancement.”
• “Further research and development, expanded educational programs, improved computer research networks, and more effective technology transfer from government to industry are necessary for the United States to reap fully the benefits of high-performance computing.”
• “A high-capacity and high-speed national research and education computer network would provide researchers and educators with access to computer and information resources and act as a test bed for further research and development of high-capacity and high-speed computer networks.”
• A “multiagency high-performance computing program … would provide American researchers and educators with the computer and information resources they need, and demonstrate how advanced computers, high-capacity and high-speed networks, and electronic data bases can improve the national information infrastructure for use by all Americans.”

The National Research and Education Network (NREN) called for in the HPC Act of 1991 connected geographically distributed research teams, supercomputer centers, and information resources. Image from FY 1992 Blue Book.

The interconnected NSF, NASA, and DOE “backbone” networks and selected regional and campus-area networks. Image from FY 1992 HPCC Blue Book.

Then-Senator (and later Vice President) Albert Gore Jr. (D-TN) initiated and led the Congressional effort that resulted in the passage of the HPC Act, which took four Congresses and six years of painstaking work. A National Software Corporation and artificial intelligence, both of which appeared in early draft legislation, were left on the cutting-room floor. The work was difficult but collegial. One observer at the time opined that no one spoke against the legislation in perhaps thousands of pages of testimony. The last issue holding up passage was leadership for the effort, which was resolved by assigning neither the National Science Foundation nor the Department of Energy as the lead HPCC agency, but rather deciding that “the President [of the United States] shall implement a National High-Performance Computing Program”. The bill passed with strong bipartisan support. I’m told that everyone working in HPCC was ecstatic when that Act was signed into law by President George H.W. Bush. Al Gore was neither present nor acknowledged at the signing ceremony.

The research and development, education, networking, and technology transfer accomplishments of the HPCC Program have profoundly changed—and continue to change—our lives. The list of accomplishments is long, but the most obvious may be the development of technologies found in today’s Internet.

HPCC Grand Challenges described in the FY 1993 HPCC Blue Book

HPCC Grand Challenges described in the FY 1993 HPCC Blue Book

HPCC Grand Challenges described in the FY 1993 HPCC Blue Book

HPCC Grand Challenges described in the FY 1993 HPCC Blue Book

The Act and those accomplishments are the first reason I built this archival collection—what was done was influential and it is worthwhile to document how it happened. It was therefore worth saving and organizing original HPCC materials, and developing a finding aid that makes information about the archival collection publicly available. I appreciate that NLM leadership and people in other Federal organizations, U.S. universities, and U.S. industry shared that view.

Many linear feet of original HPCC materials already resided at NLM, and often only there. From September 1992 to March 1995 NLM Director Donald A.B. Lindberg MD served concurrently as the founding director of the NCO. An NLM librarian built a proper NCO Library and NLM, being a library, had of course saved all of it. When HPCC colleagues at other agencies cleaned out offices, they would usually call me to ask if I wanted their HPCC materials, and I always said yes, so they are part of the archival collection too. (Early plans were that these HPCC materials would become part of Lindberg’s archive, and in any case were always to be housed at NLM.) The archival collection also includes two donations that document other parts of the HPCC elephant. The donation from the National Aeronautics and Space Administration (NASA) documents that large science agency’s research and development in HPCC technologies, including development of supercomputers built from inexpensive commodity-off-the-shelf PCs. The CASC (originally the Coalition for Academic Supercomputing Centers) donation includes its own publications and those from (a) the university research community first in support of what became the HPC Act and then documenting HPCC research results and (b) public policy organizations analyzing HPCC efforts.

With the completion and public announcement of this archival collection, the history of the activities of the HPCC Program and the NCO to realize this vision is now preserved and made available to those interested in its history and others who may be asked to craft future coordinated Federal programs.

The National Coordination Office for High Performance Computing and Communications Archives 1936-2017 document the little-known HPCC Program that quietly helped make today’s information age possible through decades of research and development (R&D) in computing, networking, and information technology, and interoperable Federal research and education networks that are a testbed for networking R&D. This is the first of three posts about the HPCC Program, the NCO/HPCC (especially when it was at NLM), and this archival collection.

The archival collection documents the history of the High-Performance Computing Act of 1991, activities of the eight (now 14) Federal science and technology agencies in the HPCC Program, the National Coordination Office for HPCC that was initially housed at NLM, the successors to the HPCC Program and the NCO/HPCC, the President’s Information Technology Advisory Committee authorized by the Act, and interactions with U.S. universities and industry.

Sally Howe, PhD came from NIST to NLM in 1992 to work at the newly-created National Coordination Office for High Performance Computing and Communications (NCO/HPCC). She served for two and a half years under the NCO’s founding director and now-retired NLM Director Dr. Donald A.B. Lindberg, and for 10 years as NCO Associate Director. Dr. Howe currently serves as an NLM Special Volunteer.