Teaching a room
to see: the ARTrium game.

A VR-ish bomb-defusal game that you play with your whole body. Stand in front of a camera. The room notices you. Dodge the lasers, cut the right wire. Nine-year-olds are extremely hard on software and this entry is mostly about that.

What the game is.

Electronic ARTrium is a touring installation we built in the Georgia Tech IDEA Lab for K–12 STEM outreach. A camera watches the player. A Mediapipe pipeline turns their body into a stream of 3D landmarks. A Unity scene reads those landmarks and places a small humanoid rig in a room full of hazards: lasers to duck, wires to touch, a bomb timer counting down. The goal is not the game — the goal is to teach a middle-schooler that computers can see, and to do it in thirty seconds without a slide deck.

The pipe.

Python owns the camera. Mediapipe gives us 33 pose landmarks in 3D; we smooth them with a small one-euro filter to keep the rig from twitching. Coordinates get serialised to a compact binary frame and sent over a local WebSocket to Unity at whatever frame rate the laptop can sustain — usually 28–32 Hz on the classroom hardware, faster on anything newer. Unity reads the stream, drives the rig, and runs the game logic. Two processes, one socket, no GPU. The whole pipeline runs on a 2019 Dell Precision with integrated graphics because that's what the school has.

Nine-year-olds are the best adversarial test set I have ever had access to.

Surviving children.

The first version of the game assumed the player would stand in front of the camera and behave. Children do not do this. They stand half-out of frame. They stand on each other. They wear hoodies over their hands. They run sideways out of the room and come back. The second version added a "visibility guard" — a small confidence check on the hip and shoulder landmarks — and fell back to a ghost rig if more than 30% of the skeleton went missing. The third version learned to handle two people in frame by picking the one with the larger bounding box. The fourth version, still in progress, wants to gracefully handle wheelchairs.

What it's teaching me.

This project is the reason I'm careful about other people's Wi-Fi now. It's the reason I believe in small, boring bits of state like "last known good pose." It's the reason my PropSage offline demo exists. You cannot predict which failure mode you will see in a gymnasium in south Georgia in October, so the system has to degrade in ways the children never notice. The bomb keeps ticking. That's the whole trick.

Built with the IDEA Lab at Georgia Tech. Source, build instructions, and the outreach-day protocol are on GitHub. Currently touring Atlanta-area middle schools.

← Back to the journal