Featured Exhibitions:
NYCxDESIGN Exhibitions, Time Squares, New York
2019 Westport Maker Fair, New York
ITP Winter Show, Tisch New York Univeristy
IDEATIONS
In this project, we want to offer a different approach of appreciating paintings. A painting is made up of different elements, which combine together to shape a unique personality. Similarly, in an orchestra, the conductor guides different instruments in a band to make harmonious music. By using synesthesia, we want the user to have a new integrated experience of conducting drawings.
The user will pick up the baton to wake different elements of the drawing up and turn it into a musical piece. He then can conduct the music by changing his gesture . There will be two drawings in our project. One is a Chinese traditional drawing while the other is a western landscape drawing. Together they provide a scene of how painting personality differs.
First round of PROTOTYPE & USER TEST
01 FIRST ROUND OF USER TEST
We prototyped with cardboard for the first version of user test. We let tester tried our prototype and noted down their confusion in different stages of user flow.
People are confused when we mention about hanging two painting. They feels they can control both paintings at the same time but they are not allowed to, because our intension is to let user appreciate and conduct each painting separately.
Therefore, instead of having 6 IR sensor on two separate notes, we will only have 3 IR sensors, and users are only able to see one note at a time. The way people choose which painting to interact with is through turning the pages. Also, instead of having a reset button, we want the action itself, turning the page, switch off everything and reset, so user can start interact with the next painting. In order to do that, We place two photoresistors at each side. When one sensor exposed to light and the other is blocked, meaning that the note on the exposed side is selected, the LEDs and sounds reflecting that note will be ready for interaction. When both sensors are exposed, meaning users are Turing pages, all of the LEDs and sounds will be switched off.
DESIGNING THE CIRCUIT
The Arduino will always sent 6 values, the first 3 values represent acceloration on X-axis, Y-axis, and Z-axis, The next three values represent the statue of three scores, 0 means nothing is selected, 1 means the score on the first painting is selected, and 2 means the second.
01 Switch the drawing
Two drawings have their own music. In order to switch the drawing, we place two photo-resistors at each side of the music score. When one sensor exposed to light and the other is blocked, meaning that the note on the exposed side is selected, the LEDs and sounds reflecting that note will be ready for interaction. When both sensors are exposed, meaning users are Turing pages, all of the LEDs and sounds will be switched off.
02 Conduct the music
We use IR-beam sensor to turn a track of music on and off and xyz acceleration sensor to change the synthesizer of the music. The Arduino will always sent 6 values, the first 3 values represent the statue of three scores, 0 means nothing is selected, 1 means the score on the first painting is selected, and 2 means the second. The next three values represent accelation on X-axis, Y-axis, and Z-axis. Max takes in the values and sends them to Ableton.
To improve the handling of our baton, we tried connect with a HDMI cable. The wires inside the HDMI are too thin to break. So we just got some flexible cables and use heat-shrink plastic to protect the pinch points.
The second round of user testing:
AFTER THE WORK
There can be more feedback when user play our project. Like rolling of the painting, light movement and so on. And the gestures of conducting the painting can be more reasonable.
What else can we do to a painting? Step on it? Tore it?
What else could be the place we need music?
Made by Chenyu Sun, Yunze Shi and Nianqi Zhang
