ESS-DIVE Overview: A Scalable, User-Focused Repository for Earth and Environmental Science Data
Authors
Shreyas Cholia1* (scholia@lbl.gov), Deborah A. Agarwal1, Charuleka Varadharajan2, Joan Damerow2, Valerie Hendrix1, Hesham Elbashandy1, Fianna O’Brien1, Emily Robles2, Mario Melara1, Madison Burrus2, Shalki Shrivastava1, Karen Whitenack1, Sarah Poon1, Dylan O’Ryan2, Lauren Core2, Matthew B. Jones3
Institutions
1Scientific Data Division, Lawrence Berkeley National Laboratory, Berkeley, CA; 2Earth and Environmental Sciences Area, Lawrence Berkeley National Laboratory, Berkeley, CA; 3National Center for Ecological Analysis and Synthesis, Santa Barbara, CA
URLs
Abstract
ESS-DIVE’s data repository stores diverse data from DOE-sponsored Earth and environmental research activities. Researchers are focused on providing a scalable, robust repository for long-term curation of ESS data that adheres to Findable, Accessible, Interoperable, and Reusable (FAIR) principles. The current focus of the repository has been across three key areas: (1) data access capabilities; (2) services to support projects providing data to the repository; and (3) standardization of data. The team’s approach is designed around user experience methods and involves significant discussion and involvement of the community in the design and development of the capabilities. The priorities of the repository are continually revised and refined based on input from the community. Researchers are continually improving data access capabilities, including API-driven access along with a new secondary storage layer to support very large hierarchical datasets. To meet the needs of a diverse set of projects, researchers are supporting customized project-centric views of the data that allow for grouped datasets to be displayed together, collaborative management of datasets, and the ability to share datasets within a project. The use of data standards and reporting formats plays a key role in enabling the broadest possible use of data. Researchers have developed nine community reporting formats to enable more structured files with descriptive metadata for diverse data types. In this overview, the team covers key features of ESS-DIVE and how researchers are building a scalable, community-focused data repository.