Memorias de investigación
Artículos en revistas:
D'Haro, Luis Fernando, RafaelAutomatic evaluation of end-to-end dialog sysAutomatic evaluation of end-to-end dialog systems withadequacy-fluency metrics
Año:2019
Áreas de investigación
  • Ciencias de la computación y tecnología informática
Datos
Descripción
End-to-end dialog systems are gaining interest due to the recent advances of deep neural networks and the availability of largehuman?human dialog corpora. However, in spite of being of fundamental importance to systematically improve the performanceof this kind of systems, automatic evaluation of the generated dialog utterances is still an unsolved problem. Indeed, most of theproposed objective metrics shown low correlation with human evaluations. In this paper, we evaluate a two-dimensional evalua-tion metric that is designed to operate at sentence level, which considers the syntactic and semantic information carried along theanswers generated by an end-to-end dialog system with respect to a set of references. The proposed metric, when applied to out-puts generated by the systems participating in track 2 of the DSTC-6 challenge, shows a higher correlation with human evalua-tions (up to 12.8% relative improvement at the system level) than the best of the alternative state-of-the-art automatic metricscurrently available
Internacional
Si
JCR del ISI
Si
Título de la revista
Computer Speech And Language
ISSN
0885-2308
Factor de impacto JCR
2,116
Información de impacto
Volumen
55
DOI
10.1016/j.csl.2018.12.004
Número de revista
Desde la página
200
Hasta la página
215
Mes
SIN MES
Ranking
Esta actividad pertenece a memorias de investigación
Participantes
Grupos de investigación, Departamentos, Centros e Institutos de I+D+i relacionados
  • Creador: Grupo de Investigación: Grupo de Tecnología del Habla
  • Departamento: Ingeniería Electrónica