Making 3D Environment Tangible: Bare-Hand Interaction with LightBender Illuminations
Shuqin Zhu
Abstract
What happens when a 3D illumination stops being something you only watch and becomes something you can touch, push, rotate, and reshape? This talk explores how LightBenders, a miniature drone that carries lighting primitive, can transform from flying displays into responsive physical interfaces. We show how users can interact with them using bare hands, feel programmable effects such as stiffness and friction, and move or rotate illuminated objects while the system maintains stable flight. We also introduce a bidirectional editing workflow in which physical changes made to a LightBender are reflected instantly in Blender, connecting digital design with real-world manipulation. Finally, we examine how these interactions can scale from one LightBender to an entire swarm that moves as a coordinated shape and responds collectively to touch, opening the door to playful, tangible, and physics-based 3D environments without headsets or controllers.
Bio
Shuqin Zhu is a Computer Science Ph.D. student at the University of Southern California (USC), advised by Dr. Shahram Ghandeharizadeh. His research focuses on human-computer interaction, physics-based haptic rendering, swarm behavior, and collision avoidance within aerial robotics. Specifically, he designs fundamental control strategies and physical-digital workflows for "Flying Light Specks" (FLSs)—miniature drone swarms equipped with programmable lighting primitives—transforming them into tangible 3D interfaces that users can physically touch and manipulate with bare hands. Through this work, his ultimate vision is to pioneer the next-generation physics-based 3D immersive environments, bridging the gap between digital design and physical reality without the need for headsets or controllers. Currently, he is also working as an intern at TikTok, focusing on Multi-Agent Frameworks.
