AR mapping interactive installation

“Boggart” is a mysterious creature in a box, who shows different body parts each time viewers open the lid.

I was responsible for the visual effects on the eyes and programming using Unity 3D.


In the summer of 2017, our class conducted a field trip to Hangzhou’s NetEase AI department. During a workshop, we had the opportunity to use their augmented reality techniques – exploring the latest resources and tools, and improving our skills in return.

Our team used a projection-based AR system, which is under development at NetEase. The projector recognizes different hand gestures and movements, and detects the shapes and depth of simple objects. Given these capabilities, we designed a box-based interactive installation that can be opened. The box has rich interactive effects based on our concept

“One day in the future, researchers find a new life form in the universe. This mysterious life exhibits strong reactions to human contact, showing different body parts each time researchers open the lid … They name it Boggart.”

While in a “standing position,” Boggart shows skin that slowly heaves and falls. When the box is opened, people may discover it staring at their hand through its wired eye. Sometimes, it throws bubbles to communicate with humans – enabling users to whisk and break the bubbles. And sometimes its’ tentacles may even reach outside of the box… Each time the box is opened, it leads to an exciting encounter with the mysterious life form.

Process:  During our research phase, we discovered that the advantages of projection-based AR are primarily from the intriguing combination of virtual images and real objects. Therefore, we focused the controls on opening and closing the box. Every time the viewer opens the box, the “unidentified life” shows different interactive effects, which are intended to surprise and delight.

Our work included 3 main parts: Programming in Unity, Designing interactive effects in C4D, 3D modeling and 3Dprinting.
Since we only had 2 weeks to complete this work, we carefully planned and divided tasks among 5 team members, namely: 2 members focused on learning how to connect the effects and projector in Unity, and the others were responsible for creating the interactive effects and modeling.

Leave a Comment

电子邮件地址不会被公开。 必填项已用*标注