Danilo’s Presentation


I’m trying to create a piece of art using primarily the concept of color synesthesia.  I had this idea when I found that when I listen to music or sounds, with proper concentration I can let my senses overlap with one another. Abstract painter and theorist pioneer, Wassily Kandinsky, believed that forms and colors had equal vibrations in the soul. To replicate the synesthetic phenomenon I have been trying to find ways to correlate psycho-physiological reactions such as the rhythm of one’s heart beat (called HEART RATE VARIABILITY or HRV) to a given the audio-visual experience. However, not in a purely scientific manner. So far there are many interesting studies about the more than 60 types of synesthesia. Many of the physio-neurological causes that triggers synesthetic experiences remain unknown. Researchers of the phenomenon are still looking for the cause-and-effect relationship in the diverse forms of synesthesia.


Instead approaching synesthesia by creating audio-visuals where people would see  synesthetic experiences, I’m working in ways where one should create its own synesthetic experience accordingly his own perception. By creating audio-visual stimulus and measuring the alterations in one’s HRV through pulse sensors I hope not only to create a relationship of cause-and-effect but a personal measure of one’s “synesthetic coefficient”

I’m designing my project using Arduino boards and processing as the main platform for building the psycho-physiological visual response. And for the audio-visual I’m using MAX/MSP/JITTER and MIRA(IPad). I also have been studying ways of using “holography”, rear projection in my thesis.

In the website bellow you can see some interesting works that have been created using the concept of synesthesia:


The V-Motion Project:

Stefan Eckert -3D Holographic Show “Space Blues Symphony”

We ran out of time Wednesday before Danilo had a chance to screen Neil Harbisson TED Talk, I listen to color.


Danilo is exploring issue related to Synesthesia, a neurological condition in which stimulation of one sensory or cognitive pathway leads to automatic, involuntary experiences in a second sensory or cognitive pathway.


A couple of weeks ago I caught Colors on Radiolab, which seems interestingly related.



8 Responses to “Danilo’s Presentation”

  1. loudonstearns Says:

    @danilo, so psyched to check out your work! There are two books that you may want to check out in our study of perception. On the visual side the book Visual Perception is awesome.

    On the audio side, check out the work of Albert Bregman’s Auditory Scene Analysis.

    I have never looked at synesthesia, but I have researched visual and audio perception in great depth, those are the best resources I found on the topics.

  2. loudonstearns Says:

    I love the holograms, I wonder if there is an inexpensive way to build a big one. I just feel that making it big makes all the difference here…

    Asher’s horror movie stuff would look awesome as a hologram! Those interesting small frames hovering in the air would be great.

    • isitagiraffe Says:

      or maybe something like this (Asher)

      • loudonstearns Says:

        That is awesome, how did they do it?!?!

      • isitagiraffe Says:

        kinect maybe? one of the comments seems to imply

      • loudonstearns Says:

        The Kinect stuff is awesome, but I am not sure it is necessary for this(maybe it is!).

        This is my guess: When setting up you put a webcam facing the back wall and light the wall as it will be during the instillation, you save this image on the HD. When the instillation is running your software compares the image of the wall to the image it sees in real time; every pixel that is different between the two forms an alpha mask allowing the words to show through on a second projector pointed at the wall from the ceiling and adjusted for angular distortion(keystone).

        That would be my guess, great piece, thank you for this. I love it when the idea is really simple and effective(a shadow of text) but the execution is so hard to imagine.

  3. Augmentationist Interanational Says:

    Really cool reference work Danilo!

  4. isitagiraffe Says:

    “sensors attached to Daito’s fingers not only sending shocks to Masaki’s face but also outputting the results to MaxMSP via Arduino to generate sound . As a result, an interesting situation occurs where touch generated music is directly related to facial expression.”

    some awesome behind the scenes videos here

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: