image sources assignment1_sources

photodocumentary : Kinect and Gestures

This slideshow requires JavaScript.

This slideshow requires JavaScript.

My thesis project is a performative, audio-visual installation.
It proposes a new form of communication or language that can be learned through intuitive mappings between the movement of hand gesture shadows and how they behave, in relation to shape, color and sonic output, in a collaborative space of expression.

The shadow-based instrument focuses on how intentionality, learned-actions and decision-making ultimately guide users through the interactive system.

The main goal is for participants to discover and learn the audio-visual language mapped within the system through the joint effort of communicating and recognizing patterns that will emerge as shadow gestures are created over time.

For each person, first read off 6 basic hand gestures : hands together/apart, fingers together/apart, horizontal position, vertical position, clinched fist and a gesture forming a circle, where all fingers are connected. Then I asked is there were certain colors that had any significant meaning or relation to any of the gestures that I listed. Even if they did not have any strong connection of one gesture to color, I asked them to pick and match what seemed most accurate to them. Every participant said they didn’t particularly find one gesture to have a strong connection to a certain color.  Despite this, I did find unseen patterns. The two most consistent ones involved circle-like gesture for reds and pinks and a clinched fist gesture for black or brown.

Although each participant showed this behavior, ultimately color seemed to be arbitrary, especially in the context of shadows since we tend to think of them on a grey-scale rather than in color. This arbitrary relationship between colors and shadow gestures also didn’t seem relevant to my essential design question, specifically regarding color and learned behaviors of shadow gestures involved in the interaction. The movement and intensity of the shadow played a more important role during this experiment. Please watch this documentation video of one user:

Despite the simplicity of this prototype, I received a lot of valuable feedback, mainly regarding the strong interest in opacity of the shadow and it’s movement, shape and speed over time. Most importantly, it made me realize that I have more freedom to do what I want to with color in terms of the interaction and how it can be used in a different perspective. This led me to the decision of assigning a specific color to each hand entering the projected space, mostly because this gives users their own individual identity as they explore the audio-visual instrument as well as a way of automatically understanding what they are controlling and how their individual shadows can be used and also interact with other participants.

This is where I document inspiration + precedents that relate to my thesis project

my tumblr page

systemDiagram