Matchmoving

Matchmoving (also called camera tracking, 3D tracking, or camera solving) is the process of analyzing 2D footage to reconstruct the 3D camera position, rotation, and lens properties for each frame. The output is a virtual camera that matches the real camera's motion exactly, allowing 3D CGI elements to be rendered and composited into the live-action plate with correct perspective, scale, and motion. Without matchmoving, any 3D element placed in a shot would either drift, slide, or have incorrect parallax relative to the real-world geometry. The process has two main stages. First, 2D feature tracking: the software identifies distinctive points in the footage (corner points, high-contrast edges, texture details) and tracks their pixel positions across frames. A robust solve typically requires 100-500 tracked points per frame, distributed across the image area. Second, 3D camera solving: the software uses the 2D track data, combined with known camera parameters (sensor size, focal length, frame rate), to calculate the 3D position and orientation of the camera for each frame using bundle adjustment algorithms — the same simultaneous refinement method documented by Triggs, McLauchlan and colleagues in the standard photogrammetry literature. The solver also estimates lens distortion (barrel/pincushion) and can undistort the footage before compositing and redistort the CG render to match. The four main matchmoving tools in professional use: PFTrack (the industry standard for film VFX — robust solver with excellent survey data support and object tracking), SynthEyes (fast, affordable, excellent for medium-complexity shots with a good automatic solver), Boujou (one of the oldest automatic trackers, still used in some pipelines but largely superseded by PFTrack), and DaVinci Resolve's built-in Camera Solver (integrated into Fusion, adequate for simple shots but lacks the advanced constraint tools of dedicated matchmove software). For anything with complex camera motion, lens distortion, or survey data requirements, PFTrack or SynthEyes are the professional choice.

Matchmoving fails for predictable reasons. Lack of parallax: if the camera does not translate (only pans and tilts from a fixed point), the solver cannot determine depth — it needs camera movement to triangulate 3D positions. This is why dolly, Steadicam, and handheld shots solve reliably while locked-off tripod shots with only pan/tilt often produce unstable solves. Motion blur: fast camera movement smears the tracked features beyond recognition, causing track dropout. The solution is to increase the search area and lower the tracking threshold, or manually add track points in frames where automatic tracking fails. Reflective and repetitive surfaces cause catastrophic tracking failures. Glass buildings, water surfaces, and polished floors create false features that move independently of the camera. Chain-link fences, brick walls, and tiled floors have repeating patterns that confuse the solver into assigning wrong correspondences. The fix: mask out problematic areas before tracking, or manually place supervised track points on reliable features. Lens distortion must be corrected before tracking — if the footage has significant barrel distortion (common with wide-angle lenses), the tracked points move along curved paths that the solver interprets as camera motion, producing a wobbly or drifting solve. Survey data transforms matchmoving from approximate to precise. If you have a LIDAR scan or measured reference points of the set, you can import them into PFTrack or SynthEyes and constrain the solve to match real-world measurements. This eliminates scale ambiguity (the solver cannot determine absolute scale from 2D footage alone — a 10-meter camera move and a 1-meter move produce identical 2D tracks if the scene is proportionally scaled) and ensures the virtual camera's position matches the real set geometry. For architectural VFX or any shot where a CG building must align with a real structure pixel-perfectly, survey data is not optional — it is required.