|
Descripción
|
|
|---|---|
| High dimensional data sets pose important challenges such as the curse of dimensionality and increased computational costs. Dimensionality reduction is therefore a crucial step for most data mining applications. Feature selection techniques allow us to achieve said reduction. However, it is nowadays common to deal with huge data sets, and most existing feature selection algorithms are designed to function in a centralized fashion, which makes them non scalable. Moreover, some of them require the selection process to be validated according to some target, which constrains their applicability to the supervised learning setting. In this paper we propose as novelty a parallel, scalable, exact implementation of an existing centralized, unsupervised feature selection algorithm on Spark, an efficient big data framework for large-scale distributed computation that outperforms MapReduce when applied to multi-pass algorithms. We validate the efficiency of the implementation using 1GB of real Internet traffic captured at a medium-sized ISP. | |
|
Internacional
|
Si |
|
Nombre congreso
|
19th East-European Conference on Advances in Databases and Information Systems (ADBIS). International Workshop on Big Data Applications and Principles (BigDap) |
|
Tipo de participación
|
960 |
|
Lugar del congreso
|
Poitiers, France |
|
Revisores
|
Si |
|
ISBN o ISSN
|
978-3-319-23201-0 |
|
DOI
|
10.1007/978-3-319-23201-0_21 |
|
Fecha inicio congreso
|
08/09/2015 |
|
Fecha fin congreso
|
11/09/2015 |
|
Desde la página
|
186 |
|
Hasta la página
|
196 |
|
Título de las actas
|
New Trends in Databases and Information Systems. Communications in Computer and Information Science Series, Volume 539. (Funded by FP7 ONTIC project, no. 619633) |