In our previous articles, we’ve talked about color and psychology, the meaning of colors in different cultures with particular attention to how colors evoke emotions. We know that music holds the same power – we are able to experience a variety of emotions by listening to music, and today we are wondering what is the connection between sounds and colors – can we connect one color with a certain sound and whether these have the same meaning when interpreted by our brains?
To discover more, we are talking to Umut Eldem, a composer, pianist, and art researcher. His main field of research is synesthesia, and he will tell us more about this curious phenomena.
Hi Umut! Your field of interest is synesthesia, could you tell us more about what synesthesia is and why is it so interesting for you?
Synesthesia is a neurological condition in which a person experiences multiple senses from only one sensory input. For example, along with hearing a sound, a person with synesthesia can also feel a certain taste; or while reading they can see that some letters are in different colors. What is particularly interesting for me is when people can see colors and shapes when they hear music, also called chromesthesia (sound-to-color synesthesia). Every synaesthetic person usually has their own colors they associate with different notes, letters, and so on.
Do we know any famous artists who had/have synesthesia?
It is known that certain composers had synesthesia, such as Olivier Messiaen, Franz Liszt, and Duke Ellington, but we didn’t know much about it until recently. Now, we can do MRI scans and see that different parts of the brain are active when people with synesthesia experience an input of sound or other senses, which is different than the reaction of an average brain. Science couldn’t prove it before the technology allowed us, but what is more interesting for me is to understand how these composers correlated notes they wrote and instruments they used to create music with their condition and the colors they saw. This could also help us understand their music better – the ideas they expressed and what we actually identify with.
How can scientific research in this area connect with your artistic research?
There are many scientists researching synesthesia, especially cognitive scientists and neuroscientists, and what they are exploring is a more practical side of synesthesia. They are researching whether is more common to males or females, is it more common with artists, left-handed people, etc. They are trying to answer the question of why is it happening.
As an art researcher, I am looking into the ways that we can have a better understanding of the artistic manifestation of synesthesia, and how to use it to express ourselves as artists in more than one discipline. What I’m exploring is how we can use synesthesia even if we don’t have it, and how an average brain connects colors and sounds. Knowing more about it can help musicians experience musical concepts of the composers they listen to better and bring the process of creating and performing music to the interdisciplinary scene.
What are you working on right now?
My current research is pointed at discovering a general theory of how we connect colors and sounds in order to develop software which can convert music into visual ideas, shapes, and colors in an intuitive way, and to use it as a tool for performing musicians. So, for example, as the violin plays, you can project the video performance which directly responds to the music played. This will, of course, be different from the audio visualizers on media players of computers, where it just mathematically connects frequencies to visuals.
I am also conducting artistic experiments where I can see how visual input affects musicians. I am adding ‘synaesthetic’ colors to their sheet music and finding out if it is easier for them to play like that, or more difficult. The practical result of this research will be to enhance music performance.
Do you use colors in your performances, when you play music?
Before learning about synesthesia, I didn’t. But now, my interest is growing and I am implementing colors in my performances. For example, I did a performance with a narrator, which was reading a piece to the audience, and my soundscape was playing from four speakers set in the room. I used light bulbs which were connected to an app on my phone, so I was able to change the color of the lights based on the tone of the story and also the sound, integrate different colors in different parts of the text.
How did the audience relate to this piece?
The thing is, when you present an audio-visual piece, music, and colors together, the audience will usually find it pleasurable as long as they roughly go along together, so it is difficult to understand these sensations deeply. However, I have received comments from the audience that they did find harmony in the way the lights were responding to the music and the story, and connect to the story on a new level.
Although, what is much more interesting for me, as I said before, is to have a video that responds to live music in an intuitive way to our perception. This might be a new way of experiencing music.
What kind of artistic experiments do you do?
What is interesting for me to work with is improvisation. I like to see how a musician plays a “red triangle” for example. If they see shapes changing on the screen, how will a pianist, a violinist, and a drummer react to these shapes as they change? For me, as an observer, this reaction of music to visual art has an aesthetic value, but for the musicians, it has an insightful significance – by doing this they discover more about themselves and how they correlate visual ideas with the music and which aspect of music they will turn to.
How is synesthetic art different for musicians and visual artists?
Visual art is usually stationary, comparing to music when we talk about paintings for example. There is a static image that we process in our own time and have an individual experience, while music is a more temporal process and we all have the same time to experience it. When a classic artist responds to music visually, they are putting music into one frame. There are synesthetic painters who draw the music they hear, such as contemporary Melissa McCracken. When you are a visual artist with chromesthesia, it is easier to express it because you already see colors and you are creating a visual output. It is also easier for the audience to relate to the synesthesia of a visual artist, but for music, it often requires an explanation or an additional dimension, like a visual element to realize the synesthetic potential of the music.
Visual artists are very inspired by music. Is it a myth that they spontaneously reflect what they hear?
I would say that it depends on the artist, but we all have the ability to connect what we experience between different senses, with or without synaesthesia. We are all reflecting differently, it is all very idiosyncratic. With synaesthetic people, everyone has their own color for the same source, and you will usually get something different. The interesting thing is that even though we always react to music in our own way, there is still a pattern in the big picture. Our experience is unique, but there is a general tendency of people to connect music and art in the same way. For example, we tend to combine warmer colors with fast, happy, exciting music, while we combine colder colors with the opposite- slow and sad. I immediately think of the ‘Blues’ music genre. I believe that the biggest value of my research is understanding this general tendency in combining visuals and music in creating something unique.
What is next that is coming up?
Next week I am delivering a workshop as a part of my project “Drawn to music”, as a part of the project week NextDoors, in Royal Conservatoire of Antwerp, in Belgium. There will be musicians playing in response to visual art, but also the musicians that will draw what they hear. This two-way process will repeat itself and we will see how artists react visually to music and musically to visuals.
If you want to contact Umut and give your input to his research or ask him about it, you can reach him by email. You can also find him on soundcloud.
Umut’s lecture on synesthesia you can find on youtube.
Author: Nina Petrov
