Various Music Notes Rainbow

Breakthrough new technology enables people to hear music through touch

The prototype consists of an audio-tactile algorithm that converts monophonic music into tactile stimuli based on vibrations using “tactile illusions”.

An audio-tactile algorithm developed by scientists from the University of Malaga transmits melodic information through vibration.

A groundbreaking prototype developed by experts from the Department of Electronics at the University of Malaga and members of the Electronics for Instruments and Systems research and development group will allow people with hearing loss to hear music through the sense of touch.

It consists of an audio-tactile algorithm that converts monophonic music into tactile stimuli based on vibrations using “tactile illusions”. According to the researchers, “It’s like ‘hacking’ the nervous system to get a different response to the actual stimulus being sent.”

“The long-term goal is for people who can’t hear to be able to ‘hear’ music,” says researcher Paul Remache, the lead author of this paper, who insists on music’s power to affect mood and its potential as therapy mental disorders and pain management.

The researchers predict that this will result in a portable terminal that can be taken to a concert, as this prototype will be easily transferrable to tech devices such as smartphones.

Researchers at the Faculty of Electronics have developed an audio-tactile algorithm that transmits melodic information through vibration. Source: University of Malaga

music mapping

This young researcher, in collaboration with UMA professors Andrés Trujillo and Fernando Vidal, has developed an algorithm capable of converting musical features and structures from MIDI files – Musical Instrument Digital Interface – into “vibrotactile stimuli”.

Music by touch prototype

The first experiments proved that “tactile illusions” perceived by vibrations evoke in people who hear no other – usually positive – emotional response than that of the original music. Source: University of Malaga

“It’s sort of like mapping music,” explains Remache, adding that it’s possible because this type of file can not only be played and created, but also provides “symbolic representations.”

control vibrations

Current models do not guarantee the correspondence between the emotional response to music and the vibrotactile version of it. In light of this, these UMA engineers propose an arrangement of the “tactile illusions” to enhance and broaden the spectrum of musical features by making vibration more dynamic in terms of movement, change of direction and location.

“It’s a challenging process because the perceivable frequency range of the skin is lower than that of the auditory system, which can lead to the loss of some musical characteristics,” they explain.

Department of Electronics at the University of Málaga

Thanks to a groundbreaking prototype developed by researchers from the Department of Electronics at the University of Malaga, people with hearing loss can hear music through touch. Source: University of Malaga

Different emotional reaction

The results of the first experiments, in which more than fifty volunteers took part, indicate that the arrangement of “tactile illusions” evokes more positive than negative emotions. They are also found to be more enjoyable and stimulating than the audio, and provoke a different emotional response than the original music.

Smart healthcare instrumentation and application

This first prototype was presented at the 11th International Workshop on Haptic & Audio Interaction Design (UK) – the largest international event specializing in these fields of study – after being published in the scientific journal LNCS. The UMA researchers are currently working on a second model and are continuing the experiments.

The research is the result of Paul Remache’s doctoral thesis and is part of the National Plan project “Intelligent Instrumentation and Applications in Healthcare”.

Reference: “Mapping monophonic MIDI tracks to vibrotactile stimuli with tactile illusions” by Byron Remache-Vinueza, Andrés Trujillo-León, Maria-Alena Clim, Fabián Sarmiento-Ortiz, Liliana Topon-Visarrea, Alexander Refsum Jensenius and Fernando Vidal-Verdú, 18 August 2022, Reading music in computer science.
DOI: 10.1007/978-3-031-15019-7_11


#Breakthrough #technology #enables #people #hear #music #touch

Leave a Comment

Your email address will not be published. Required fields are marked *