Observatorio de I+D+i UPM

Go Back | Technology Solutions

AUTOMATIC TRANSLATION into Spanish Sign Language (LSE)

More info

[LSE]Commercial_Data_Sheet_UPM_EN(Innovatech) PDF
 
Briefing

Spanish into Spanish Sign Language (LSE) translation system

The system is made up of a speech recognizer (for decoding the spoken utterance into a word sequence), a natural language translator (for converting a word sequence into a sequence of signs belonging to the sign language), and a 3D avatar animation module (for playing back the signs). Based on preliminary experiments in restricted domains, the translation system performance is close to 100%, operating in real-time. The system allows an easy adaptation to a specific application domain. This technology has been develop by the Speech Technology Group at UPM in collaboration with FCNSE (the Spanish Deaf Association). This technology has been included in several industrial prototypes already developed for different scenarios.

 
Technology Solution

Sign Language, translation systemThe system is made up of a speech recognizer (for decoding the spoken utterance into a word sequence), a natural language translator (for converting a word sequence into a sequence of signs belonging to the sign language), and a 3D avatar animation module (for playing back the signs).

The speech recognizer is speaker independent and it is able to recognize continuous speech.

For the natural language translator, three technological approaches have been implemented and evaluated: an example-based strategy, a rule-based translation method and a statistical translator.

Combining these three strategies, it is possible to take advance from their advantages.

 

 

 

Based on preliminary experiments in restricted domains, the translation system performance is close to 100%.

 
Market demands

Deaf people have a lot of communication barriers that generates a significant number of educational, social and cultural problems:

  • Educational needs of the deaf. 47% of the deaf population have no education or are illiterate. Furthermore, only between 1% and 3% of the deaf population has completed college, compared to 21% of the entire Spanish population (INE MECD 1999 and 2000/2001).
  • Social integration needs. Deaf students ends in social guarantee programs 10 times more than the entire student body (INE 1999). Another fact is enlightening that between 45% and 50% of deaf children have psychological disorders compared to the maximum of 25% for the rest of the population (European Society for Mental Health and Deafness 2000). Finally, unemployment is much higher among deaf people, especially among the group of deaf women (INE 2003).

 

These general needs can be specified in the followings point:.

  • Subtitled audiovisual content in sign language. The relevance of this aspect is increasing given the Law about Audiovisual Communication 2010, which imposes very high subtitling needs, requiring automated tools.
  • Translation of web pages into sign language to facilitate deaf people accessing them.
  • Generation of information in sign language to be provided through information points or cash machines.
  • Content translation into sign language for education and training of deaf people.

 

Deafness gives rise to significant communications problems. With this technology it is possible to develop content and services accessible for deaf people.

 
Market potential

Sign Language, Translation systemAccording to the Survey of Disability, Personal Autonomy and Dependency Situations INE (EDAD, 2008), the number of people with disabilities are: 3,847,900 people, more than 8.5% of the population. The number of hearing impaired is 1,064,100, ie 25.20 per thousand population.

From the recognition of LSE (Spanish Sign Language) as an official language in 2007, the number of LSE users is growing significantly.

 

 
Competitive advantages

  • Translation rate is close to 100% for restricted domains.
  • Real-time operation for on-line interaction services.
  • Speaker independence when translation from speech.
  • Easy adaptation to a specific application domain.
  • Flexible to be adapted to other languages or sign languages.
  • Available a sign vocabulary with more than 1000 signs. This vocabulary includes a description of every sign in both SEA (Sistema de Escritura Alfabética) and HamNoSys.
  • These signs are already generated. They can be used in different contexts: it does not require re-design them.
  • There is a user-friendly visual tool for designing new signs in both HamNoSys or SEA.

Sign Language, translation system     Sign Language, translation system

 
References

  • Research group with wide experience in this subject and collaboration with companies.
  • This technology has been developed in collaboration with FCNSE (the Spanish Deaf Association).
  • There are several industrial prototypes already developed for different scenarios: Identity Card renewing, Driving License renewing, Hotel reception and Information Point of the EMT.
 
Development stage

  • Concept
  • R&D
  • Lab-Prototype
  • Industrial Prototype
  • Production
 
Contact

Contact LSE

Rubén San Segundo Hernández

e: lapiz@die.upm.es

José Manuel Pardo Muñoz

e: pardo@die.upm.es

w: http://gth.die.upm.es/

UPM  contact

Innovation, Commercialization and Entrepreneurship Area

Centre of Support for Technological Innovation – UPM

 e: innovacion.tecnologica@upm.es

 
S2i 2019 Observatorio de investigación @ UPM con la colaboración del Consejo Social UPM
Cofinanciación del MINECO en el marco del Programa INNCIDE 2011 (OTR-2011-0236)
Cofinanciación del MINECO en el marco del Programa INNPACTO (IPT-020000-2010-22)