App Development

The app guides people through a series of expressive hand and face gestures and turns each one into a moment of visual magic. It watches how the player moves and gives feedback in real time, and when they get a move right, the world around them responds with effects built around their body. It brings together computer vision tools that are rarely combined in one app and makes them work as a single real-time system on a phone.

• Hand tracking with MediaPipe

  Google's on-device machine learning reads 21 landmarks per hand from the front camera every frame. We packaged it as our own reusable hand gesture library that recognizes poses and movement.

• Face tracking with ARKit

  We read ARKit's 52 facial blend shapes, plus head position and rotation, to detect expressions such as smiles along with head gestures.

• Combined hand and face gestures

  MediaPipe's 2D hand data and ARKit's 3D face data are brought into a shared screen space, so the app can recognize gestures that need both, such as a hand held near the face.

• People occlusion with Apple Vision

  A custom native iOS plugin feeds Vision's person masks into the render pipeline, so effects pass convincingly behind and around the player.

• Custom URP rendering

  Our own render graph features in Unity's Universal Render Pipeline handle depth-based occlusion, silhouette glows and scene dimming that follow the player's outline.

• Gesture grading

  Each gesture moves through its own state machine and is graded on timing and quality against a rhythm, so the app knows how well a move went, not just whether it happened.

• Movement-driven effects and audio

  Particle systems, VFX Graph effects and layered sound respond directly to where the player's hands are and how they move.

• Guided camera onboarding

  The player is framed and calibrated before the experience begins, and the app waits for the AR camera to settle.

• Built to be testable

  More than 1,700 automated tests cover gesture logic, grading and presentation timing, and builds go out to testers through TestFlight.

We are building a wellness experience that brings Apple Watch biometrics and real-time voice interaction together in one iPhone app. The product is still under wraps, but this is what we engineered for it.

• Custom HealthKit plugin for Unity

  A native iOS plugin streams live heart data from HealthKit into Unity. It ships as a private, reusable Unity package and sets up the Xcode project on every build, including frameworks, entitlements and privacy descriptions.

• Two-way Watch and iPhone link

  A companion watchOS app sends live biometric readings to the iPhone, and the phone sends state back so the watch responds with color and haptics. Several delivery channels keep data flowing when the connection drops or the app is in the background.

• Real-time voice analysis

  On-device pitch and volume detection, fast enough to drive the experience frame by frame with no server round trip.

• Custom iOS audio routing

  A native audio session bridge records from the iPhone mic while playing high-quality audio through AirPods and other Bluetooth headphones, and adapts as they connect or disconnect.

• Procedural music engine

  A runtime synthesizer makes music from what the user does. It needs no audio files and stays harmonious whatever the input.

• Onboarding design and build

  Guided sign-up, sign-in and personalization flows, a modular screen system with animated transitions, a library of reusable components, and layouts that fit every iPhone screen size and safe area.