Observatorio de I+D+i UPM

Memorias de investigación
Ponencias en congresos:
SLoG: A Large-Scale Logging Middleware for HPC and Big Data Convergence
Año:2018
Áreas de investigación
  • Ciencias de la computación y tecnología informática
Datos
Descripción
Cloud developers traditionally rely on purpose-specific services to provide the storage model they need for an application. In contrast, HPC developers have a much more limited choice, typically restricted to a centralized parallel file system for persistent storage. Unfortunately, these systems often offer low performance when subject to highly concurrent, conflicting I/O patterns. This makes difficult the implementation of inherently concurrent data structures such as distributed shared logs. Yet, this data structure is key to applications such as computational steering, data collection from physical sensor grids, or discrete event generators. In this paper we tackle this issue. We present SLoG, shared log middleware providing a shared log abstraction over a parallel file system, designed to circumvent the aforementioned limitations. We evaluate SLoG's design on up to 100,000 cores of the Theta supercomputer: the results show high append velocity at scale while also providing substantial benefits for other persistent backend storage systems.
Internacional
Si
Nombre congreso
38th IEEE International Conference on Distributed Computing Systems (ICDCS)
Tipo de participación
960
Lugar del congreso
Vienna, Austria
Revisores
Si
ISBN o ISSN
978-1-5386-6871-9
DOI
Fecha inicio congreso
02/07/2018
Fecha fin congreso
05/05/2018
Desde la página
1507
Hasta la página
1512
Título de las actas
Proceedings of the 38th IEEE International Conference on Distributed Computing Systems
Esta actividad pertenece a memorias de investigación
Participantes
  • Autor: Pierre Matri (UPM)
  • Autor: Philip Carns (Argonne National Laboratory)
  • Autor: Robert Ross (Argonne National Laboratory)
  • Autor: Alexandru Costan (Inria)
  • Autor: Maria de los Santos Perez Hernandez (UPM)
  • Autor: Gabriel Antoniu (Inria)
Grupos de investigación, Departamentos, Centros e Institutos de I+D+i relacionados
  • Creador: Grupo de Investigación: Ontology Engineering Group
  • Departamento: Arquitectura y Tecnología de Sistemas Informáticos
S2i 2021 Observatorio de investigación @ UPM con la colaboración del Consejo Social UPM
Cofinanciación del MINECO en el marco del Programa INNCIDE 2011 (OTR-2011-0236)
Cofinanciación del MINECO en el marco del Programa INNPACTO (IPT-020000-2010-22)