What Is Real-Time Motion Capture?
What you'll learn: This guide covers how real-time motion capture works across hardware and software — from the live motion capture data pipeline to streaming body and face performance into game engines at sub-40ms latency. You'll understand how to configure live link unreal engine for body and face mocap, what real-time mocap streaming protocols (Live Link, OSC, VMC) are appropriate for different production contexts, how virtual production motion capture integrates with LED volume stages and in-camera VFX workflows, the hardware options across price tiers from iPhone ARKit to optical Vicon/OptiTrack systems, and when a professional animation library delivers better results than running a real-time mocap pipeline for game animation production.
Real-time motion capture is the live streaming of performer movement data directly into a game engine, 3D software, or broadcast pipeline — with zero or near-zero latency between the performer's physical movement and the corresponding character movement in the digital environment. Unlike offline mocap workflows (capture → export → cleanup → import → use), real-time mocap compresses the pipeline: the character moves as the performer moves, in the same moment.
This capability unlocks a range of production workflows that weren't previously accessible: virtual production stages where actors perform against real-time digital backgrounds, VTuber streaming where character and performer are synchronized live, game development previsualization where game characters can be posed and tested with live human input, and interactive experiences where a physical performer controls a digital character in response to real events.
How Real-Time Motion Capture Works
The core technical challenge of real-time mocap is latency — the time between a physical movement and its display in the digital output. Imperceptible latency for live performance is under 40 milliseconds (the threshold at which most people notice audio-visual desync). Most modern real-time mocap systems achieve 15–30ms end-to-end on a well-configured setup.
The data pipeline for real-time mocap:
- Sensor capture: The performer's body is tracked by sensors (inertial IMUs, optical cameras, or depth cameras). Each sensor computes pose data locally at 60–200fps.
- Data aggregation: A central software layer (Rokoko Studio, Axis Studio, MotionBuilder) receives all sensor streams and assembles them into a unified skeleton pose.
- Streaming protocol: The assembled pose is broadcast over a network protocol to the receiving application. Common protocols: Live Link (UE5), OSC (general purpose), VMC Protocol (VTuber ecosystem), BVH stream (DCC tools).
- Retargeting and rendering: The receiving application applies the incoming pose to the target character skeleton (retargeting) and renders the result in real time.
Real-Time Mocap Streaming Into Unreal Engine 5
UE5's Live Link plugin is the standard protocol for real-time data streaming into the engine. Every major mocap hardware manufacturer supports Live Link output.
Live Link Setup
- Enable the Live Link plugin (Edit → Plugins → Live Link)
- Open the Live Link panel (Window → Live Link)
- Add the appropriate source type:
- Face AR Source for iPhone ARKit face capture (via the Live Link Face app)
- Rokoko Studio Source for Rokoko body capture
- Xsens Source for Xsens MVN data
- The subject appears in the Live Link panel with a green connection indicator
Connecting Live Data to a Character
In the character's Animation Blueprint:
1. Add a Live Link Pose node to the AnimGraph
2. Set the Subject Name to match the incoming Live Link subject
3. Set the Retarget Asset to an IK Retarget asset that maps the source skeleton to the character's skeleton
4. Connect the Live Link Pose node to the Output Pose
When the performer moves, the character in the UE5 viewport responds in real time.
Take Recorder: Recording Live Performances
UE5's Take Recorder captures the live data stream as saved animation assets for reuse:
1. Open Window → Take Recorder
2. Add your character actor as a source
3. Press Record — the Live Link stream is captured to Animation Sequences
4. Stop Recording — the Take is saved and can be played back in Sequencer
This is the standard workflow for virtual production: record multiple takes live, review and select the best, then compose into the final cinematic using Sequencer.
Real-Time Mocap Streaming Into Unity
Unity's Animation Rigging package and the Live Link Face plugin provide real-time mocap streaming support.
For body capture: Most hardware vendors (Rokoko, Xsens) provide Unity plugins that stream body animation directly onto a character's Animator. The Rokoko Unity plugin, for example, enables real-time preview of Smartsuit data on a Unity character during capture sessions.
For face capture: Epic's Live Link Face plugin for Unity (separate from the UE5 version) enables iPhone ARKit face streaming to Unity characters. The data drives morph targets directly on the character's SkinnedMeshRenderer.
Recording in Unity: Unity's Animation Recording mode (available in the Animation window) can record incoming transform data to animation clips during live streaming, similar to UE5's Take Recorder.
Hardware Options for Real-Time Mocap
Rokoko Smartsuit Pro II (Body + Hands)
The most popular real-time streaming setup for indie and mid-size productions. The suit streams 27 bones of body data at 100fps to Rokoko Studio, which re-streams to UE5, Unity, Blender, iClone, or VTube Studio simultaneously.
Latency: ~20ms in ideal Wi-Fi conditions. Degraded Wi-Fi or interference adds 10–30ms.
Setup time: 15–30 minutes for calibration on a trained performer.
Xsens MVN Animate (Body)
Professional inertial streaming with more robust magnetic disturbance compensation. Used in AAA game studios for real-time pre-visualization during production. Streams via Axis Studio to UE5 Live Link or MotionBuilder.
Latency: ~15ms.
Setup time: 20–30 minutes.
iPhone + Live Link Face (Face)
Zero hardware cost beyond an iPhone. Streams 52 ARKit blend shape coefficients at 60fps to UE5, Unity, or VTube Studio via the Live Link Face app.
Latency: ~10–15ms on a solid 5GHz Wi-Fi connection.
OptiTrack / Vicon Optical Systems (Body, High-Accuracy)
Camera-based real-time streaming for professional stages. Marker positions stream at 120–360fps via Motive (OptiTrack) or Shogun Live (Vicon) to MotionBuilder and then re-streamed via Live Link to UE5.
Latency: 5–10ms (lowest latency of any system).
Constraint: Fixed stage volume required.
Virtual Production: Real-Time Mocap at Scale
Virtual production is the use of real-time rendering and live mocap in a coordinated production pipeline — replacing traditional green screen with LED volumes, using real-time game engine rendering for in-camera visual effects, and synchronizing physical and digital performers.
LED Volume Production
An LED volume surrounds a physical stage with high-resolution screens displaying the real-time UE5 virtual environment. Cameras capture the physical actors against this digital backdrop — the result is in-camera VFX without post-production compositing.
Real-time mocap in this context drives:
- Digital characters sharing the stage with physical actors
- Physical cameras tracked with 6DOF position/rotation data streamed to UE5 for correct parallax
- Real-time lighting from the UE5 scene affecting the physical set
Previsualization with Live Mocap
Game development studios use real-time mocap for production planning: a designer streams their performance onto a game character and demonstrates how a cutscene should play out — before committing animator time to any specific version. Directors, designers, and animators can evaluate options interactively.
The latency requirement for previsualization is less strict than for live broadcast — 50–80ms is acceptable for in-office previs sessions.
Real-Time vs. Offline Mocap: When to Use Each
| Use Case | Real-Time Mocap | Offline Mocap |
|---|---|---|
| VTuber live streaming | Required | Not applicable |
| Virtual production / LED stage | Required | Not applicable |
| Game previs and blocking | Ideal | Workable but slower |
| Cinematic animation production | Optional (for takes) | Standard (for final output) |
| Gameplay animation production | Optional | Standard |
| Quality ceiling | Limited by hardware latency | No constraint |
Real-time mocap maximizes iteration speed and enables live performance use cases. Offline mocap maximizes output quality because cleanup, retargeting, and polish can be applied before the animation is used.
For most game animation production, the workflow is: real-time for iteration and takes, offline processing for final delivery. Capture live, review in real time, then take the best raw data through the cleanup and retargeting pipeline to produce production-ready assets.
Live Link Unreal Engine: Complete Real-Time Mocap Streaming Configuration
Live link unreal engine is the primary infrastructure for real-time mocap streaming in professional and indie virtual production pipelines. Getting it configured correctly for body and face capture requires understanding each component of the chain — the source plugin, the subject name mapping, the retarget asset, and the Animation Blueprint evaluation order.
Full Configuration Walkthrough
Step 1 — Enable and verify plugins: In UE5, go to Edit → Plugins and enable both "Live Link" and the hardware-specific plugin for your source (Rokoko Studio Link, Xsens MVN Live Link, Face AR Source for iPhone). Restart the editor after enabling new plugins.
Step 2 — Open Live Link panel and add source: Window → Live Link. Click the + Source button and select your hardware source. For Rokoko: select "Rokoko Studio Live Link" and ensure Rokoko Studio is open on the same network with its Live Link output enabled (Rokoko Studio → Settings → Live Streaming → Enable UE Live Link, port 14043). For iPhone face: select "Face AR Source" and enter the iPhone's local IP. The subject appears in the panel with a connection status indicator.
Step 3 — Verify subject name: In the Live Link panel, click on the connected subject to see its name (e.g., "Rokoko Body," "iPhone"). Copy this name exactly — it must match the Subject Name field in your Animation Blueprint's Live Link Pose node.
Step 4 — Create or assign a retarget asset: Real-time mocap streaming requires a Live Link Retarget Asset that maps the source skeleton (e.g., Rokoko's 27-bone skeleton) to your character's skeleton. In UE5, create a LiveLinkRemapAsset (or use IK Retarget for more control). For Rokoko→Mannequin: the Rokoko plugin ships with a pre-built remap asset compatible with the Mannequin skeleton. Reference it in the Live Link Pose node's "Retarget Asset" field.
Step 5 — Animation Blueprint setup: In your character's AnimGraph, place a "Live Link Pose" node. Set Subject Name and Retarget Asset. Connect to Output Pose. In the Event Graph, add "Has Live Link Subject Authority" check to gate evaluation when no stream is active — this prevents the character T-posing when the performer disconnects.
Troubleshooting Common Real-Time Mocap Streaming Issues
Character not moving despite green indicator: Subject name mismatch between Live Link panel and the Live Link Pose node in the AnimGraph. Verify exact string match including capitalization and spaces.
High latency (>50ms visible delay): Wi-Fi interference or bandwidth contention. Move the mocap network to a dedicated 5GHz router with no other devices, or switch to wired ethernet from the capture computer to the switch. Rokoko streams at ~3 Mbps; ensure the network path has no competing high-bandwidth traffic.
Drift accumulating over long sessions: IMU drift in inertial systems (Rokoko, Xsens) is normal over 20–30 minute sessions. In Rokoko Studio, perform a hip calibration reset (the performer stands in T-pose and presses the recalibrate button in Rokoko Studio) without stopping the Live Link stream — the UE5 character briefly snaps back to neutral then continues tracking from the corrected reference.
Face and body out of sync: If body arrives via Rokoko and face via iPhone, both streams are evaluated independently in the Animation Blueprint. Ensure both source frame rates are similar (Rokoko at 100fps, iPhone at 60fps). Add a "Live Link Pose" node for each source, apply them as separate layers (body bones from Rokoko, face morph targets from ARKit) so their evaluation doesn't compete.
When Pre-Built Animation Libraries Replace Real-Time Capture
For game production teams who don't need the live-performance-in-engine capability, pre-built animation libraries often deliver better animation quality at lower total cost than establishing and maintaining a real-time mocap pipeline.
Real-time streaming setups require:
- Hardware investment ($2,500+ for Rokoko; $5,000+ for Xsens)
- Time for setup, calibration, and performer coaching
- Network infrastructure (stable 5GHz Wi-Fi is critical)
- Post-capture cleanup
Professional animation libraries like MoCap Online provide clips that are already cleaned, looped, and engine-ready — the "post-capture cleanup" step that consumes most of the time in a real-time mocap workflow is already done.
FAQ: Real-Time Motion Capture
What is the minimum latency achievable with consumer real-time mocap?
iPhone ARKit face capture via Live Link: approximately 10–15ms. Rokoko Smartsuit body: approximately 15–20ms. This is well below the 40ms perception threshold for most performers in non-broadcast contexts.
Can I use real-time mocap for a live broadcast?
Yes, but network stability becomes critical. Broadcast production typically uses wired ethernet connections from suit to computer and a dedicated switch for the mocap network to eliminate Wi-Fi interference. Dropout during a live broadcast is unrecoverable, so redundancy is standard.
What's the best setup for combining face and body capture in real time?
Rokoko Smartsuit Pro II for body + iPhone Live Link Face for face capture, both streamed into UE5. The body data arrives via the Rokoko plugin as skeletal animation on the body bones; the face data arrives via Live Link Face as blend shape values on the face morph targets. They compose independently on the character in real time.
Does real-time mocap work outdoors?
Inertial systems (Rokoko, Xsens) work outdoors without issue — no camera volume required. iPhone face tracking works outdoors with caution (bright direct sunlight can disrupt the TrueDepth sensor). Optical systems are limited to controlled indoor environments.
What is the difference between live motion capture and offline mocap for game animation production?
Live motion capture delivers the performer's movement to the game engine in real time — the character responds during the capture session, allowing directors to evaluate and iterate immediately. Offline mocap captures data to disk, processes it through cleanup and retargeting, then delivers finished animation assets. For game animation production, the difference is iteration speed versus output quality: live motion capture enables fast creative decisions (you see the shot work or not work instantly), while offline processing enables the cleanup passes (foot planting, noise removal, curve smoothing) that make animation production-ready. Most professional productions use live motion capture for take selection and offline processing for final delivery — the two workflows are complementary, not competing.
How does virtual production motion capture differ from standard game animation capture?
Virtual production motion capture runs inside an active rendering environment rather than a neutral T-pose viewport. The performer wears the mocap hardware on an LED volume stage or in a previs session while the game engine renders the virtual environment in real time around them. Compared to standard game animation capture (performer in suit → data to disk → animator polishes → engine imports), virtual production motion capture requires the entire data pipeline to run simultaneously with the renderer: body pose streams via Live Link to the character rig, camera tracking data streams to the virtual camera, and lighting from the UE5 scene drives the LED panels in sync. The technical requirements are more demanding: network latency budget is stricter (under 20ms to avoid visible desync on the LED wall), frame rate must match the physical camera's shutter, and the mocap system must be time-code synchronized with the camera and lighting systems via SMPTE timecode or genlock.
What real-time mocap streaming protocol should I use — Live Link, OSC, or VMC?
The choice depends on the receiving application. Live link unreal engine is the native protocol for UE5 — use it for any UE5-based workflow (virtual production, game previs, MetaHuman animation). OSC (Open Sound Control) is a general-purpose protocol supported by Blender, TouchDesigner, and many custom applications — use it when UE5 is not the target. VMC Protocol (Virtual Motion Capture) is the VTuber ecosystem standard, supported by VTube Studio, VSeeFace, and VMagicMirror — use it for VTuber avatar streaming. Rokoko Studio supports all three output protocols simultaneously, so you can stream to multiple destinations at once if your production requires it.
Skip the Stream, Start With the Library
Not every production needs a real-time pipeline. For game teams who need production-ready animations now — without building mocap infrastructure — professional animation libraries provide immediate access to cleaned, looped, and engine-formatted clips.
Browse the MoCap Online motion capture animation library for locomotion, combat, sports, and performance packs. Download the free animation pack to test quality without any hardware setup, and check out the animation blog for virtual production and mocap streaming workflow guides.