Synaesthesia in Art and Science

Synaesthesia: a condition in which someone experiences things through their senses in an unusual way, for example by experiencing a colour as a sound, or a number as a position in space.

Umut Eldem’s research and works focus on exploring synaesthesia as an artistic expression. Using sounds, colours, and other senses as metaphors for each other his works aim to create synaesthetic worlds and experiences, drawing on the experiences of synaesthetic people. His experiments are being used to explore the aesthetic effects of combining the senses, and to create tools for synaesthetic audio-visualisation and musical performance. Umut Eldem has also given classes and workshops on audiovisual history, technique, and practice.


Synaesthetic Works

Color Jam (2020)

“Color Jam” is an audiovisual jam session, done over Microsoft Teams, during lockdown. Umut triggers the live visuals through musical gestures on his keyboard, while observing how musicians react to color and the relation between color and sound/gesture. The result is a performance where the music reacts to the visuals, which reacts back to the music. The voices have been recorded separately and are mixed together as a multitrack video.

  • Performed by: Marcelo F. Lazcano (Guitar), Ine Vanoeveren (Flute), Thomas Moore (Trombone), Umut Eldem (Live Visuals)


Glimmer (2019)

“Glimmer” is based on the act of breathing, and the intricacies it reveals as one focuses on the different aspects of this natural reflex. The performance focuses on the intuitive presentation of multi-sensory art. While the work is oriented around music, a live audio visualization system, created as a part of the composer’s research, accentuates musical elements (pitch, direction, timbre, form) and transforms them into a visual component.

  • Performed by: Dries Meerts


Hearing Glass (2019)

“Hearing Glass” uses a computer camera to give passers-by a synaesthetic experience. The camera uses a live feed to create a soundscape that comes out of either speakers or headphones, depending on the location and the likelihood of disturbance. A transformed live feed from the camera is shown on the computer screen. The audiovisual transformation from picture to soundscape is based on Eldem’s research on synaesthesia, the experience of multiple senses at once, as the colours and shapes from the pictures are automatically transcribed in accordance with synaesthetic theory.


Prelude (2018)

The text of “Prelude” is based on the ancient creation myths from all throughout the world, a story told by many viewpoints across all continents, with the music, the soundscape, and the colour lights depicting the creation in tandem.

  • Performed by: Atilla Filiz

Color FM (2017)

“Color FM” is part of Eldem’s artworks centered around synaesthesia- the experience of music and colors together. He hid different music boxes, each with their own matching color and original music, in Mirador during the Kolla Festival. The festival-goers could seek them like a treasure hunt, or encounter into them if they went exploring on their own. Each box was placed according to the color and the associated feelings: a dangerous place for red, a place deep in nature for green, etc.



    Presentations

    • The Sound/Colour Relationship in Artistic Practice: Visualising Synaesthetic Principles
      European Platform for Artistic Research in Music (EPARM) Conference 2019

    • Consolidating The Artistic And The Scientific
      Methods in Artistic Research 2019

    • Beyond The Pitch – Applying Synaesthetic Principles to The Artistic Practice
      IASAS Moscow Synaesthesia Conference 2019

    • Between The Senses: Sound, Colour, and Synaesthesia
      Newline 2019

    • Between The Senses: What Our Brains Reveal About Our Connection Between Sound and Colour
      RE:SOUND – Media Art History Conference 2019