|
Descripción
|
|
|---|---|
| High dimensional data sets pose important challenges such as the curse of dimensionality and increased computational costs. Di- mensionality reduction is therefore a crucial step for most data mining applications. Feature selection techniques allow us to achieve said reduction. However, it is nowadays common to deal with huge data sets, and most existing feature selection algorithms are designed to function in a centralized fashion, which makes them non scalable. Moreover, some of them require the selection process to be validated according to some target, which constrains their applicability to the supervised learning setting. In this paper we propose as novelty a parallel, scalable, exact implementation of an existing centralized, unsupervised feature selection algorithm on Spark, an efficient big data framework for large-scale distributed computation that outperforms MapReduce when applied to multi-pass algorithms. We validate the efficiency of the implementation using 1GB of real Internet traffic captured at a medium-sized ISP. | |
|
Internacional
|
Si |
|
DOI
|
10.1007/978-3-319-23201-0 |
|
Edición del Libro
|
|
|
Editorial del Libro
|
Springer |
|
ISBN
|
978-3-319-23200-3 |
|
Serie
|
Communications in Computer and Information Science |
|
Título del Libro
|
New Trends in Databases and Information Systems |
|
Desde página
|
186 |
|
Hasta página
|
196 |