Using property graphs for rich metadata management in HPC systems

Dong Dai, Robert B. Ross, Philip Carns, Dries Kimpe, Yong Chen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

17 Scopus citations

Abstract

HPC platforms are capable of generating huge amounts of metadata about different entities including jobs, users, and files. Simple metadata, which describe the attributes of these entities (e.g., file size, name, and permissions mode), has been well recorded and used in current systems. However, only a limited amount of rich metadata, which records not only the attributes of entities but also relationships between them, are captured in current HPC systems. Rich metadata may include information from many sources, including users and applications, and must be integrated into a unified framework. Collecting, integrating, processing, and querying such a large volume of metadata pose considerable challenges for HPC systems. In this paper, we propose a rich metadata management approach that unifies metadata into one generic property graph. We argue that this approach supports not only simple metadata operations such as directory traversal and permission validation but also rich metadata operations such as provenance query and security auditing. The property graph approach provides an extensible method to store diverse metadata and presents an opportunity to leverage rapidly evolving graph storage and processing techniques.

Original languageEnglish
Title of host publicationProceedings of PDSW 2014
Subtitle of host publication9th Parallel Data Storage Workshop - Held in Conjunction with SC 2014: The International Conference for High Performance Computing, Networking, Storage and Analysis
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages7-12
Number of pages6
ISBN (Electronic)9781479970254
DOIs
StatePublished - Jan 20 2014
Event9th Parallel Data Storage Workshop, PDSW 2014 - Held in Conjunction with the International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2014 - New Orleans, United States
Duration: Nov 16 2014 → …

Publication series

NameProceedings of PDSW 2014: 9th Parallel Data Storage Workshop - Held in Conjunction with SC 2014: The International Conference for High Performance Computing, Networking, Storage and Analysis

Conference

Conference9th Parallel Data Storage Workshop, PDSW 2014 - Held in Conjunction with the International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2014
CountryUnited States
CityNew Orleans
Period11/16/14 → …

Fingerprint Dive into the research topics of 'Using property graphs for rich metadata management in HPC systems'. Together they form a unique fingerprint.

  • Cite this

    Dai, D., Ross, R. B., Carns, P., Kimpe, D., & Chen, Y. (2014). Using property graphs for rich metadata management in HPC systems. In Proceedings of PDSW 2014: 9th Parallel Data Storage Workshop - Held in Conjunction with SC 2014: The International Conference for High Performance Computing, Networking, Storage and Analysis (pp. 7-12). [7016276] (Proceedings of PDSW 2014: 9th Parallel Data Storage Workshop - Held in Conjunction with SC 2014: The International Conference for High Performance Computing, Networking, Storage and Analysis). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/PDSW.2014.11