论文标题
稀疏数据的高效,可扩展的IO框架:LARCV3
An Efficient, Scalable IO Framework for Sparse Data: larcv3
论文作者
论文摘要
中微子物理学是研究宇宙起源和特性的基本领域之一。许多实验性中微子项目使用复杂的检测器来观察这些颗粒的特性,并转向深度学习和人工智能技术来分析其数据。由此,我们开发了\ texttt {larcv},\ texttt {c ++}和\ texttt {python}基于粒子物理应用程序的稀疏数据的有效io。我们在本文中描述了\ texttt {LARCV}框架和一些基准IO性能测试。 \ texttt {Larcv}旨在在现代HPC系统上大规模地启用破烂和不规则数据的快速效率IO,并且与Python生态系统中最受欢迎的开源数据分析工具兼容。
Neutrino physics is one of the fundamental areas of research into the origins and properties of the Universe. Many experimental neutrino projects use sophisticated detectors to observe properties of these particles, and have turned to deep learning and artificial intelligence techniques to analyze their data. From this, we have developed \texttt{larcv}, a \texttt{C++} and \texttt{Python} based framework for efficient IO of sparse data with particle physics applications in mind. We describe in this paper the \texttt{larcv} framework and some benchmark IO performance tests. \texttt{larcv} is designed to enable fast and efficient IO of ragged and irregular data, at scale on modern HPC systems, and is compatible with the most popular open source data analysis tools in the Python ecosystem.