Observatorio de I+D+i UPM

Memorias de investigación
Research Project:
Large Scale Data Streaming (MS-2007-07)
Year:2007
Research Areas
  • Programming language
Information
Abstract
There are many data streaming applications that will have to cope with massive data streams in which large clusters will be required to be able to cope with the input data stream. We foresee two classes of applications: 1) applications with many continuous queries that can scale by distributing the queries among a pool of sites; 2) Applications with one or a few queries but very massive data that will require parallelizing query operators among a large number of sites to be able to cope with the massive input streaming data. In order to realize a data streaming system able to satisfy the requirements of these two classes of applications for very large clusters (e.g. over 100 sites) it is a real challenge from a scientific point of view. We believe that the advances in System Area Networks (SANs) will enable a new kind of distributed support for data streaming. On one hand, SANs provide high-bandwidth low latency communication that enables end-to-end inter-host memory access in the realm of microseconds. This can be exploited for fast coordination among sites and agile load balancing. On the other hand, these networks are characterized by having Network Interface Cards (NICs) with their own processor with a reasonable capacity and able to access the host memory through DMA. Our previous work enables us to foresee that the envisioned highly scalable data streaming platform can be realized.
International
Si
Project type
Proyectos y convenios en convocatorias públicas competitivas
Company
Microsoft Research Cambridge (PhD Award)
Entity Nationality
REINO UNIDO
Entity size
Gran Empresa (>250)
Granting date
01/10/2006
Participants
  • Director: Ricardo Jimenez Peris (UPM)
  • Participante: Marta Patiño Martinez (UPM)
Research Group, Departaments and Institutes related
  • Creador: Grupo de Investigación: Distributed Systems Labs (LSD) Laboratorio de sistemas distribuidos
  • Departamento: Lenguajes y Sistemas Informáticos e Ingeniería de Software
S2i 2019 Observatorio de investigación @ UPM con la colaboración del Consejo Social UPM
Cofinanciación del MINECO en el marco del Programa INNCIDE 2011 (OTR-2011-0236)
Cofinanciación del MINECO en el marco del Programa INNPACTO (IPT-020000-2010-22)