Puppetto

A light weight tangible enhancement of video chat experience

The idea for this project started from hacking everyday objects as Internet of Things (IoT) so that it can enrich conversations between two remote places (e.g., long distance relationship, family who lives outside of the country, etc.). First, we had a brainstorming session in order to figure out how everyday objects can potentially have other channels for additional subtle information. After having this brainstorming session, we decided on a mobile phone, which is now probably one of the most common and the most frequently used everyday objects in our life. Since we also wanted to specifically focus on enhancing the communication experience, we decided to create an attachment to a phone, which can offer a richer mode of communication while having a conversation between remote places. We finalized our idea to make the phone embody physical gestures on the other side as physical movements, like head shaking, leaning and titling. We prototyped the concept as a robotic arm that is attachable to a phone while video chatting, and embodies gestures on the other side. My role was to work on facial gesture recognition through image processing (openFrameworks) and its connection with servo motors (Arduino). In addition, I also made a part of the robotic arm with Rhinoceros and MakerBot.

We finished the first prototype and published the project in the Human Robot Interaction 2015 conference Proceedings. We are also going to present it at the demo session, and we are still continuing on the development.