What they built

UltroX

# Ultro **Touch nothing. Control everything.** ## The problem Touchscreens assume you can always reach out and tap — not true in an OR, a busy kitchen, a cleanroom, or for anyone with limited hand dexterity. Most gesture demos are flashy toys that spin a 3D object for fun. We wanted gesture control that does real work. ## What it does Ultro is a touchless control hub: a living plasma orb rendered in WebGL, driven entirely by bare-hand gestures over a webcam. Pinch and move to navigate, two-hand pinch to zoom, open palm to select, point to target a control. Behind the orb sit three panels: - **Data** — live system metrics you browse without touching a screen - **Devices** — smart-device toggles fired by gesture, with instant visual + audio confirmation - **Activity** — a log of every action taken Every interaction has a distinct sound, not just a visual pulse — so the interface stays usable when you can't watch the screen closely. A full keyboard fallback covers anyone without a webcam. ## How we built it Three.js renders a 50,000-point additive particle orb with a custom GLSL shader (radial gradient, breathing animation, cursor-driven repel field), composited through multi-pass bloom post-processing. MediaPipe Hands drives real-time landmark tracking with hysteresis-based pinch detection and a small gesture classifier (pinch, palm, fist, point). Web Audio API generates context-aware feedback tied to actions and live data. Next.js powers the app shell. ## Built with Next.js · Three.js · WebGL/GLSL · MediaPipe Hands · Web Audio API · TypeScript