COMPLETE GUIDE
After Effects VFX Workflows: The Complete Practitioner Guide
Project setup for VFX work, RAW footage handling, rotoscoping workflows, compositing strategies, motion tracking, particle systems, green screen keying, render pipeline for Resolve roundtrip, and RAM cache management. Every setting, every shortcut, every limitation. Written by compositors who work in AE 8 hours a day.
12
Sections
4000+
Words
50+
Specific Settings
After Effects remains the industry standard for motion graphics, compositing, and VFX work in broadcast, advertising, and independent film. Nuke dominates high-end feature film compositing, and Fusion is gaining ground inside DaVinci Resolve, but AE's plugin ecosystem, expression engine, and integration with Premiere Pro and Illustrator keep it essential for mid-tier production work. This guide covers the full VFX pipeline from project creation to final render, with specific settings, honest limitations, and workflows tested across 30+ commercial, music video, and short film projects at BLKRIP Studio.
1. Project Setup for VFX Work
Composition Settings and Color Management
Start every VFX project with correct composition settings. Open Composition > New Composition (Ctrl/Cmd+N). Set Resolution to match your delivery format: 3840x2160 for 4K, 1920x1080 for HD. Set Frame Rate to match your source footage — 23.976fps for cinema, 25fps for European broadcast, 29.97fps for US broadcast. Never mix frame rates in a single comp. If you have 23.976 and 29.97 footage, conform everything to one frame rate before importing.
Pixel Aspect Ratio: Square Pixels (1.0) for all modern digital delivery. Do not use D1/DV NTSC or PAL pixel aspect ratios unless you are specifically targeting SD broadcast.
Color Management (AE 2023+): Enable Display Color Management in Edit > Project Settings > Color. Set Working Space to Rec.709 Gamma 2.4 for SDR work or Rec.2020 PQ for HDR work. Enable Compensate for Scene-referred Profiles. This ensures AE applies correct color transforms when displaying linear-light data on your sRGB monitor. Without color management, linear compositing operations (Add blending mode, light wrap, glow effects) produce mathematically incorrect results.
For ACES workflows: Set Working Space to ACEScg (AP1, Linear). This puts AE's internal processing in the ACES linear color space, which is the correct working space for compositing. Enable Linearize Working Space. All input footage is transformed to ACEScg on import, all processing happens in ACEScg, and the output is transformed to the target display space. Note: AE's ACES implementation is less complete than Nuke's. Some effects do not respect the working space setting and process in Rec.709 regardless. Test your specific effect chain before committing to an ACES pipeline in AE.
Bit Depth: 8-bit vs 16-bit vs 32-bit
Set bit depth in the comp panel footer (click the bit depth indicator at the bottom of the Project panel). Three options: 8 bpc (bits per channel): Default. 256 values per channel. Sufficient for web delivery but produces visible banding in gradients after color correction, keying, or blending. Avoid for any VFX work. 16 bpc: 32,768 values per channel. Adequate for most compositing work. Prevents banding in gradient areas after keying and color correction. The performance hit is roughly 10-15% compared to 8-bit. Use this as your minimum for VFX compositing. 32 bpc (float): 4.29 billion values per channel with HDR range (values above 1.0 and below 0.0). Required when working with EXR sequences, light wraps, glow effects, or any operation that generates or processes HDR data. The performance hit is 30-50% compared to 16-bit. Required for Resolve roundtrip workflows where color data must be preserved without clipping. Recommendation: set your comp to 32-bit float for VFX work. The performance cost is real but manageable on modern hardware (16+ GB RAM, dedicated GPU). The quality difference in edge blending, light wrap, and glow operations is significant and visible on any calibrated display.
Project Organization That Scales
- Folder structure in the Project panel: create folders for _FOOTAGE, _COMPS, _PRECOMPS, _AUDIO, _ASSETS, _RENDERS. Everything goes into a folder. Nothing lives at the root level.
- Naming convention: [Type]_[ShotNumber]_[Description]_[Version]. Examples: COMP_sh001_heroTeleport_v03, PRECOMP_sh001_smokeElements_v02, FOOTAGE_sh001_rawPlate.
- Color-code your layers. Right-click a layer > Label > select color. Use green for footage plates, blue for precomps, yellow for adjustment layers, red for solids with effects. This makes the timeline scannable at a glance.
- Shy layers. Select layers that are part of a completed sub-system (e.g., a light wrap chain), click the shy switch column, then toggle Shy in the timeline. This hides completed layers and reduces visual clutter without deleting them.
- Lock completed layers. After finishing a section, lock the layers. Locked layers cannot be accidentally moved or modified. Combined with shy, this prevents 80% of timeline accidents on complex projects.
2. Working with RED/ARRIRAW Footage in AE
RED R3D Files
AE opens R3D files natively through the RED Importer (built into AE since CS6). When you import an R3D file, AE presents the RED Source Settings dialog (or access it later: right-click the footage > Interpret Footage > RED Source Settings).
Critical settings for VFX work:
Color Space: Choose IPP2 > Rec.709 for SDR delivery or IPP2 > Rec.2020 for HDR. Do not leave this at the default REDWideGamutRGB / Log3G10 unless you are working in an ACES pipeline — the LOG-encoded footage will look flat and washed out in AE's viewer.
Decode Resolution: Set to Full Resolution for final rendering. Set to Half or Quarter during the compositing work phase for faster preview. AE caches the decode resolution, so switching back to Full before final render forces a re-decode.
ISO: Set to the camera's recorded ISO. Do not change this in post — adjust exposure using AE's built-in tools (Exposure effect or Lumetri). Changing the RED ISO in Source Settings alters the noise floor and clip point, which may interact unpredictably with your compositing.
White Balance: Same as ISO — do not change from the camera value. Fix white balance in the comp using Lumetri or Curves.
Honest limitation: AE's RED importer is slower than Resolve's. A 4K R3D file at full resolution decodes at roughly 3-5 fps on a fast workstation in AE, compared to 15-25 fps in Resolve. For projects with heavy R3D usage, consider transcoding to ProRes 4444 in Resolve first, then importing the ProRes into AE. You lose the ability to adjust RAW parameters in AE, but gain real-time playback during compositing.
ARRIRAW Files
AE opens ARRIRAW (.ari) files through the ARRIRAW Importer plugin (available from ARRI's website or built into recent AE versions). The import settings are similar to the RED workflow.
Key settings:
Color Space: ARRI Log C4 / ARRI Wide Gamut 4 for LOG-native compositing, or Rec.709 for direct compositing. If your pipeline involves Resolve, stay in Log C4 and apply the CST in Resolve after the AE roundtrip.
Exposure Index: Match the camera setting. ARRI cameras shoot at a native EI of 800 (ALEXA 35). Do not adjust.
White Balance: Match the camera.
Debayer Resolution: Full for final render, Half during compositing.
ARRIRAW files decode faster than R3D in AE because ARRI's debayering algorithm is less computationally intensive. Expect 8-12 fps at 4K full resolution on a modern workstation.
For both RED and ARRIRAW: if you need to send VFX shots back to Resolve for final color, export from AE as ProRes 4444 with alpha channel, or as EXR sequences for maximum quality. The EXR route preserves the full dynamic range of the RAW footage through the VFX pipeline. The ProRes route is faster but clips to 10-bit and discards HDR data above 100 IRE.
RAW Codec Handling in After Effects
| Parameter | RED R3D | ARRIRAW | BRAW (Blackmagic) |
|---|---|---|---|
| Native import | Yes (RED Source Settings) | Yes (ARRIRAW Importer) | No (transcode to ProRes first) |
| Decode speed (4K, fast workstation) | 3-5 fps | 8-12 fps | N/A — not supported natively |
| Color space control | Full IPP2 pipeline | Full ARRI pipeline | N/A |
| Best roundtrip format to Resolve | EXR sequence (16-bit half) | EXR sequence (16-bit half) | ProRes 4444 |
| Honest limitation | Slow decode limits real-time work | Faster than R3D but still not real-time | Must transcode — adds preprocessing step |
3. Rotoscoping Workflow
Roto Brush 2: When It Works and When It Fails
Roto Brush 2 (AE 2021+) uses a neural network to segment foreground from background. It produces usable results on simple shots and fails on complex ones.
How to use: double-click a footage layer to open it in the Layer panel. Select the Roto Brush tool (Alt/Opt+W). Paint strokes on the foreground object you want to isolate (green strokes). Paint background strokes (red strokes, hold Alt/Opt while painting) on areas Roto Brush incorrectly included. The neural network processes the strokes and generates a segmentation mask.
Refine the boundary: after the initial stroke, use the Refine Edge tool (found in the Roto Brush effect settings) to improve edge quality. Set Edge Detection to Smart Radius. Increase Decontaminate Edge Colors to 10-20% to remove background color contamination at the edge.
Propagation settings: Roto Brush propagates the segmentation forward and backward through time. The propagation engine uses optical flow to track the object. Set Motion Threshold to 5-10% for slow-moving objects, 15-25% for fast-moving objects. Higher Motion Threshold means the propagator tolerates larger inter-frame motion but may lose track.
When Roto Brush 2 works well: single subject, clean background, consistent edge contrast, minimal motion blur. Success rate: 70-80% of frames track correctly without intervention.
When Roto Brush 2 fails: overlapping subjects (people crossing paths), transparent or semi-transparent objects (glass, smoke), heavy motion blur, low-contrast boundaries (dark clothing on dark background). In these cases, the segmentation flickers, drops frames, or bleeds.
Performance: Roto Brush 2 renders at roughly 0.5-2 fps depending on resolution and complexity. A 10-second 4K shot takes 5-20 seconds per frame to analyze. Cache the Roto Brush pass before compositing: Layer > Roto Brush > Freez Roto Brush. This locks the segmentation and converts it to a regular alpha channel, restoring real-time playback.
Workflow recommendation: Use Roto Brush 2 for rough isolation (70-80% accuracy), then refine with manual masks on the problem frames. This is faster than full manual roto on most shots.
Manual Mask Rotoscoping: The Reliable Fallback
When Roto Brush fails, manual mask roto is the answer. Select the Pen tool (G) and draw a mask around the foreground object. Keyframe the Mask Path at regular intervals (every 5-10 frames for moderate motion, every 1-2 frames for fast motion). AE interpolates the mask path between keyframes.
Mask settings for clean roto:
Mask Feather: 1-3 pixels for sharp edges (solid objects, buildings), 5-15 pixels for soft edges (hair, fur, translucent fabric). The feather creates a graduated alpha transition at the mask edge, which blends the foreground into the background more naturally than a hard edge.
Mask Expansion: -0.5 to -1.0 pixels. Shrinks the mask inward slightly, which removes the outermost row of pixels that often contains background contamination.
Mask Opacity: 100% for solid objects. 80-95% for semi-transparent elements that need to show slight background bleed-through.
Keyframe interpolation: Select the mask keyframes in the timeline, right-click > Keyframe Interpolation. Set Spatial Interpolation to Linear (not Auto Bezier). Linear interpolation produces predictable mask movement between keyframes. Auto Bezier adds acceleration curves that cause the mask to overshoot or undershoot at control points.
For complex roto shots (rotating objects, perspective changes), break the mask into multiple overlapping masks, each handling a different part of the object. An arm, a torso, and a head each get their own mask. The overlapping masks blend at their edges, and each mask has a simpler shape that is easier to keyframe. This is the standard approach used in professional roto houses.
4. Compositing Layers
Blend Modes for VFX Compositing
AE includes 38 blend modes. For VFX compositing, you regularly use five:
Add (Linear Dodge): Adds the RGB values of the layer to the layer below. Use for light effects (lens flares, glows, particle emissions, fire, sparks). Add mode simulates additive light — stacking two light sources on top of each other produces a brighter result. Limitation: Add mode clips at the maximum value. On 8-bit footage, Add produces harsh, clipped highlights. On 32-bit float footage, Add preserves values above 1.0, producing natural HDR highlight behavior.
Screen: Similar to Add but with a softer rolloff. Screen never produces values darker than the base layer. Use for fog, haze, light leaks, and glow effects where Add is too harsh. Screen is the safe choice when compositing light effects on 8-bit or 16-bit footage because it does not clip.
Multiply: Multiplies the RGB values (0-1 range), producing darker results. Use for shadow effects, darkening overlays, and adding depth to composites. Multiply black produces black; multiply white produces no change.
Overlay: Combines Multiply (for values below 0.5) and Screen (for values above 0.5). Increases contrast. Use for texture overlays (scratches, film grain, noise textures) where you want the overlay to intensify the existing contrast.
Soft Light: Similar to Overlay but gentler. Use for subtle color grading overlays and tint effects where Overlay is too aggressive.
Track Mattes and Precomping Strategies
Track mattes use one layer's alpha or luminance to control another layer's visibility. In VFX, track mattes are essential for isolating effects to specific regions.
Alpha Matte: Uses the alpha channel of the matte layer to mask the target layer. Use this when you have a pre-generated alpha (from a keyer, Roto Brush, or rendered 3D element).
Alpha Inverted Matte: Same as Alpha Matte but inverted. Use this to apply effects to everything except the matted area (e.g., blur the background while keeping the foreground sharp).
Luma Matte: Uses the luminance values of the matte layer as an alpha channel. Use this for gradient masks, light wraps, and any situation where you need a feathered alpha based on brightness.
Precomping: Precomposing (Layer > Pre-compose) collapses multiple layers into a single comp that behaves as a single layer in the parent comp. Precomp when:
- You need to apply a single effect to a group of layers (e.g., blur all VFX elements together for depth-of-field simulation)
- You need to use a composited group as a track matte source
- The timeline has more than 20-30 layers and becomes unmanageable
- You need to render a sub-composition separately for caching
Do not precomp when: you need to maintain individual layer control for animation timing. Precomping removes the precomped layers from the parent timeline, so you cannot adjust their timing without opening the precomp. Over-precomping creates nested comp chains (comp within comp within comp) that are hard to navigate. Limit nesting to 2-3 levels maximum.
5. Motion Tracking
3D Camera Tracker
AE's 3D Camera Tracker (Layer > Track Camera) analyzes footage to reconstruct the camera's 3D movement and position. It places track points on the footage that correspond to 3D positions in the reconstructed scene.
Workflow:
1. Select the footage layer in the timeline.
2. Animation > Track Camera (or right-click > Track Camera).
3. AE analyzes the footage — this takes 30 seconds to 5 minutes depending on duration and resolution.
4. After analysis, colored track points appear on the footage. Each point represents a 3D position in the scene.
5. Select multiple points (Shift+click) in the area where you want to place your VFX element. A target reticle appears showing the orientation of the computed plane.
6. Right-click the target > Create options: Solid, Text, or Null. A 3D solid/text/null is created at that position, matched to the camera's perspective.
When it works: footage with camera movement (pan, tilt, dolly, handheld), good parallax (foreground and background at different distances), sufficient texture and contrast for track points. Success rate: 70-80% on well-shot footage.
When it fails: footage without camera movement (locked-off tripod shots — no parallax for 3D reconstruction), heavily blurred footage, extremely dark scenes, reflective surfaces (glass, water) that create false track points. For locked-off shots, use 2D tracking instead.
Advanced Solve Method: In the 3D Camera Tracker effect settings, set Solve Method to Detailed Analysis for difficult footage. This increases analysis time by 2-3x but produces more accurate 3D reconstruction on shots with complex camera movement or partial occlusion.
Average Track Point Count: 100-500 points per frame on typical footage. More points = more stable track. If the tracker produces fewer than 30 points, the 3D reconstruction will be unreliable.
Warp Stabilizer
Warp Stabilizer (Layer > Warp Stabilizer) removes camera shake from handheld footage. It works by analyzing inter-frame motion, building a stabilization transform, and then warping each frame to counteract the shake.
Settings for VFX work:
Result: Smooth Motion (leaves some intentional camera movement) vs No Motion (locks the frame completely). For VFX compositing, use No Motion — a locked frame is easier to composite on.
Smoothness: 5-15% for most footage. Higher values produce smoother results but crop the frame more aggressively (AE scales the frame to hide the black borders created by stabilization).
Method: Subspace Warp (default) — warps different regions of the frame independently. Produces the smoothest results but can introduce warping artifacts on foreground subjects. Perspective — applies a global perspective transform. Less prone to warping artifacts but less smooth. Position, Scale, Rotation — simplest method, no warping, but least effective on complex shake.
Borders: Stabilize, Crop, Auto-scale — crops the frame to remove black borders. Stabilize Only — does not crop, leaving black borders. For VFX work, use Stabilize Only and handle borders manually (extend edges or fill with content-aware fill).
Limitation: Warp Stabilizer reduces frame resolution by the crop amount. A 10% crop on 4K footage delivers a 3456x1944 stabilized frame. For VFX work that requires pixel-accurate element placement, account for this resolution loss when planning your composition.
Mocha AE: Planar Tracking
Mocha AE is bundled with After Effects (Animation > Track in Mocha AE). It provides planar tracking — tracking a flat surface (wall, floor, screen, sign) with sub-pixel accuracy. Mocha's tracker is significantly more reliable than AE's built-in point tracker for complex motion.
Workflow:
1. Select the footage layer and choose Animation > Track in Mocha AE.
2. Mocha opens as a separate application with the footage loaded.
3. Draw a spline (X-Spline or Bezier) around the surface you want to track. The spline should cover the entire visible surface with some margin.
4. Click Track Forward (or Track Backward). Mocha analyzes the surface motion.
5. After tracking, check the track quality by scrubbing the timeline. If the spline drifts, add more tracking layers or adjust the search area.
6. Export the tracking data: File > Export Tracking Data. Choose After Effects Corner Pin (supports scale, position, rotation, shear).
7. In AE, paste the tracking data onto your VFX element layer. The corner pin matches the element's perspective to the tracked surface.
Mocha vs AE's built-in tracker: Mocha's planar tracking handles rotation, scale, perspective, and occlusion better than AE's point tracker. Mocha is the correct choice for screen replacements, sign replacements, surface texturing, and any situation where a flat surface needs tracking. AE's point tracker is adequate for simple position-only tracking (moving a 2D element along a path).
Motion Tracking Tool Comparison
| Tool | Best For | Tracking Type | Accuracy | Speed |
|---|---|---|---|---|
| 3D Camera Tracker | Placing 3D elements in scene | 3D camera reconstruction | Good (70-80% success) | Slow (30s-5min analysis) |
| Warp Stabilizer | Removing camera shake | 2D motion + warp | Good | Moderate (5-30s per 10s clip) |
| Mocha AE | Screen/surface replacement | Planar (perspective-aware) | Excellent (90%+ success) | Moderate (requires manual spline) |
| Point Tracker (built-in) | Simple position tracking | 2D point | Adequate | Fast (real-time) |
| Mocha Pro (paid upgrade) | Complex roto + track + remove | Planar + mesh | Excellent | Moderate |
6. Particle Systems
Trapcode Particular vs CC Particle World
Trapcode Particular (Maxon, $200 as part of Trapcode Suite) is the dominant particle system plugin for After Effects. CC Particle World is the free built-in alternative. Both generate particles from an emitter, apply physics (gravity, wind, turbulence), and render the particles with customizable appearance.
Trapcode Particular advantages:
- 3D camera-aware rendering: particles exist in 3D space and respond to AE's camera, producing correct perspective and depth-of-field
- Multiple emitter types: Point, Box, Sphere, Layer (emits from a layer's alpha), Layer Grid (emits from a layer's luminance grid)
- Advanced physics: turbulence, wind, air resistance, bounce, flocking behavior
- Aux system: secondary particles emitted from primary particles, creating branching/trail effects
- Sprite rendering: particles use a custom layer as their appearance, enabling complex particle looks (custom shapes, text, images)
- Rendering quality: motion blur, depth-of-field, 32-bit float support
CC Particle World capabilities:
- Basic particle generation with position, velocity, gravity controls
- Two render modes: Dots and Faded Dots, plus Cannon, Air, and Grid emitter types
- No 3D camera integration — particles render in 2D regardless of camera position
- Limited physics — gravity, wind, and turbulence only
- No aux system, no sprite rendering
- Adequate for simple effects (rain, snow, sparks, basic fire)
For professional VFX work (fire, smoke, magic effects, debris, environmental particles), Trapcode Particular is the minimum. CC Particle World handles quick particle needs where 3D integration and visual quality are less critical.
Realistic Fire Preset in Trapcode Particular
Fire is one of the most requested particle effects. Here is a working starting point: Emitter: Type: Box. Position: place at fire source. Velocity: 50-80. Velocity Random: 30-40%. Emission Rate: 500-1000 particles/second. Particle: Life: 1.0-1.5 seconds. Size: Start 8-12, End 0. Size over Life: ramp down curve. Color over Life: white (0%) to yellow (20%) to orange (50%) to red (80%) to transparent (100%). Physics: Gravity: -80 to -120 (negative = upward). Air Resistance: 30-50. Turbulence: 80-120. Turbulence Frequency: 3-5. Wind X/Y: 10-20 for flickering effect. Rendering: Transfer Mode: Add. Motion Blur: enabled, shutter angle 180-270. Depth of Field: enabled if camera has DOF. This produces a convincing fire column. Adjust Emission Rate and Velocity for scale. Add a second Particular instance with lower velocity, higher turbulence, and orange-only color for the smoke above the fire. Add a third instance with very low life (0.3-0.5s), small size (2-4px), and bright yellow for sparks.
7. Green Screen Keying in AE
Keylight Settings for Difficult Footage
Keylight (Effect > Keying > Keylight) is the primary keyer in AE for green and blue screen work. For well-lit green screen footage, the default settings with a Screen Colour eyedropper pick often produce an acceptable key. For difficult footage (uneven lighting, spill, fine detail), you need specific adjustments.
Screen Colour: Click the eyedropper on the most neutral green area — not the brightest, not the darkest. If the green screen has significant luminance variation, pick the average value. For screens with color variation (some areas blue-green, others yellow-green), pick the dominant hue.
Screen Gain: 1.0-1.10 for well-lit, even screens. 1.10-1.25 for uneven or dark screens. Higher gain pulls more pixels into the transparent category, which helps with under-lit screen areas but risks eating into foreground edges.
Clip Black: 0-5 for clean, well-lit screens. 10-25 for dirty or uneven screens. Higher values hard-cut more of the matte to black, cleaning up semi-transparent garbage areas. The trade-off: higher Clip Black destroys semi-transparent foreground detail (hair, smoke, sheer fabric).
Clip White: 95-100 for most footage. Lower values (85-95) harden the foreground edge by forcing near-opaque pixels to full opacity. Useful when the foreground has a soft, washed-out edge that needs hardening.
Screen Pre-blur: 1-3 pixels. Essential for compressed footage (H.264, H.265) where chroma noise creates matte jitter.
Despill Bias: Click the eyedropper on a clean foreground area (skin, clothing) that should not be affected by spill removal. This protects the foreground color from the desaturation that Keylight applies to remove green spill.
For extremely difficult footage (green screen shot on a phone, outdoor green screen with wind, heavily compressed), consider layering two Keylight instances: first Keylight with aggressive Clip Black (20-30) for the solid body matte, second Keylight with gentle settings for edge detail. Use a track matte to combine the two alpha channels (body alpha from the aggressive key, edge alpha from the gentle key).
Keying Tools in After Effects
| Tool | Type | Best For | Quality |
|---|---|---|---|
| Keylight | Chroma keyer | Standard green/blue screen | Good — industry standard |
| Primatte 6 (Maxon) | Chroma keyer | Fine hair, semi-transparent edges | Better — graduated alpha |
| Color Difference Key | Chroma keyer (built-in) | Quick rough keys | Adequate — basic |
| Linear Color Key | Color keyer | Non-screen color removal | Basic — limited control |
| Advanced Spill Suppressor | Spill removal | Post-key spill cleanup | Excellent — edge-aware |
| Supercomp (Rowbyte) | Compositing environment | Edge integration, light wrap | Excellent — holistic approach |
8. Render Pipeline: AE to Resolve Roundtrip
Export Formats for Resolve Roundtrip
The AE-to-Resolve roundtrip requires preserving the maximum quality through the VFX and color pipeline. Choose your export format based on the project's quality requirements.
EXR sequence (16-bit half-float): The highest quality option. Preserves full dynamic range, supports alpha channels, and is the standard format for VFX handoffs. File size: approximately 15-25 MB per frame at 4K. A 10-second shot at 24fps generates 240 files totaling 3.6-6 GB. In AE: Composition > Add to Render Queue. Format: EXR Sequence. Channels: RGB+Alpha. Color: preserves working space. In Resolve: import the EXR sequence as individual clips, then conform to the timeline.
ProRes 4444: High quality with alpha support. File size: approximately 500-800 Mbps at 4K. Faster to import in Resolve than EXR (single file vs hundreds). In AE: Composition > Add to Render Queue. Format: QuickTime. Codec: Apple ProRes 4444 (or ProRes 4444 XQ if available). Channels: RGB+Alpha.
ProRes 422 HQ: For VFX shots without alpha channels (full-frame effects applied to the entire plate). No alpha support. Smaller file size (~220 Mbps at 4K). Use when the VFX does not require transparency.
DNxHR 444: Avid's equivalent to ProRes 4444. Use on Windows workstations where ProRes encoding is limited. Quality is comparable to ProRes 4444.
Do NOT export as H.264 or H.265 for Resolve roundtrip. These formats are delivery codecs, not intermediate codecs. They introduce compression artifacts that compound when Resolve applies further color correction and effects. H.264 is acceptable only for client review copies, not for production pipelines.
Resolve-to-AE-to-Resolve Roundtrip Workflow
01
In Resolve: Export VFX plates as individual shots. Deliver page > Add to Render Queue. Format: EXR or ProRes 4444. Set In/Out points to include 12-24 frame handles on each side of the shot. Name files with shot numbers: sh001_plate.exr, sh002_plate.exr.
02
Import plates into AE. Create a comp matching the plate resolution and frame rate. Perform VFX work (keying, tracking, compositing, roto, particles).
03
Export VFX shot with handles matching the plate. Same frame range. Same resolution. Format: EXR sequence with alpha, or ProRes 4444 with alpha.
04
In Resolve: import the VFX render. Place it on the timeline above the original plate. The alpha channel automatically composites the VFX over the original footage. If you used EXR, set the clip's color space to match your Resolve project settings.
05
Verify the composite in Resolve's Color Page. Check edge quality, color match between VFX and plate, and alpha integration. If adjustments are needed, return to AE, fix, and re-export.
06
For multiple iterations: version your AE exports (sh001_vfx_v01.exr, sh001_vfx_v02.exr). In Resolve, right-click the VFX clip > Conform to New Version to update without re-placing on the timeline.
9. RAM Cache Management and Performance
RAM Cache Configuration
AE uses RAM to cache rendered frames for real-time playback. The RAM cache is separate from the disk cache and stores decoded and rendered frames for instant preview.
Settings: Edit > Preferences > Memory & Performance. Set RAM Reserved for Other Applications to 20-25% of total system RAM. AE uses the remaining 75-80% for the RAM cache. On a 32 GB system, AE gets ~24 GB for caching. On a 64 GB system, ~48 GB.
RAM cache size per frame: a 4K 32-bit float frame consumes approximately 100 MB of RAM. At 24fps, one second of cached 4K footage uses 2.4 GB. A 32 GB system with 24 GB cache allocation holds approximately 10 seconds of 4K 32-bit footage. A 64 GB system holds approximately 20 seconds.
When the RAM cache fills, AE evicts the oldest frames first (FIFO). If you scrub to a different part of the timeline, the cache from the previous section is overwritten. On long timelines with heavy VFX, this means constant re-caching.
Practical approach: work in sections. Cache the current section (Composition > Preview > Range: Work Area). Move the work area to the next section when done. Do not try to cache an entire 5-minute timeline — the RAM cache cannot hold it, and you will spend more time waiting for cache fills than working.
Disk cache: Enable disk cache in Edit > Preferences > Media & Disk Cache. Set the disk cache location to a fast NVMe SSD with at least 500 GB free space. Set Maximum Disk Cache Size to 200-500 GB. The disk cache persists between sessions and stores rendered frames that do not fit in RAM cache. On complex comps, the disk cache eliminates the need to re-render previously previewed sections.
Performance Optimization Checklist
1. Set comp to 32-bit float for VFX work. Accept the 30-50% performance hit. The quality gain in edge blending and light operations is worth it. 2. Use Adaptive Resolution (Preferences > Display). Set Down Sample Factor to 1/4 during fast scrubbing, Auto during normal editing. AE renders at lower resolution during interaction and fills in full resolution when you stop. 3. Pre-render heavy sections. Composition > Pre-render. This renders a section to a temporary file and replaces the live comp with the rendered file during preview. Pre-render complex particle systems, heavy keying chains, and multi-layer composites that drop below 5 fps. 4. Use Region of Interest (ROI). Draw a rectangle around the area you are working on (Composition > Region of Interest). AE only renders inside the ROI, dramatically speeding up preview. 5. Close unnecessary panels. Every open panel consumes memory and rendering resources. Close the Effect Controls panel for effects you are not actively adjusting. Close extra timeline views. 6. Purge memory regularly. Edit > Purge > All Memory & Disk Cache. Do this when AE starts stuttering or when you switch to a significantly different section of the project. 7. Disable unused effects. Click the effect icon in the Effect Controls panel to toggle effects off. Disabled effects do not render. 8. Use Collapse Transformations on precomps. When a precomp contains layers at different resolutions or with different transforms, enabling Collapse Transformations (the sun icon on the precomp layer) passes the transforms through to the parent comp, maintaining quality. Without it, the precomp is rendered at its own resolution and then scaled in the parent comp, potentially upscaling or downscaling and losing quality.
10. Render Settings for Different Deliverables
Render Settings by Deliverable Type
| Deliverable | Format | Codec | Bit Depth | Notes |
|---|---|---|---|---|
| Resolve roundtrip (VFX) | EXR sequence | EXR (half float) | 16-bit float | Include alpha. Handles: 12-24 frames. |
| Resolve roundtrip (no alpha) | QuickTime | ProRes 422 HQ | 10-bit | No alpha needed. Smaller files. |
| Client review | MP4 | H.264 | 8-bit | Match source resolution. Bitrate: 20-50 Mbps. |
| Web delivery (YouTube) | MP4 | H.264 High Profile | 8-bit | 4K: 45-68 Mbps. 1080p: 15-25 Mbps. |
| Social media (Instagram) | MP4 | H.264 | 8-bit | 1080x1080 or 1080x1350. 5-10 Mbps. |
| Archival master | QuickTime | ProRes 4444 XQ | 12-bit | Maximum quality. Include alpha. |
| Nuke handoff | EXR sequence | EXR (full float) | 32-bit float | Preserves full HDR range. |
11. Honest Limitations of After Effects for VFX
What AE Cannot Do Well (and Where to Look Instead)
- True 3D compositing. AE's 3D layer system is 2.5D — layers are flat cards in 3D space. You cannot intersect layers, create true 3D geometry, or do proper 3D lighting with shadows between layers. For true 3D compositing, use Nuke (3D workspace) or Blender (compositing nodes).
- Procedural texture generation. AE has Fractal Noise, Turbulent Dis Displace, and a few other procedural effects, but they are limited compared to node-based alternatives. For procedural textures (smoke, fire, water), use EmberGen (standalone or AE plugin) or Houdini.
- Multi-pass compositing. AE can import EXR with multiple render passes (diffuse, specular, reflection, shadow), but the workflow is manual — each pass must be extracted with the EXtractoR effect and composited by hand. Nuke's multi-pass workflow is far more efficient with Shuffle and Merge nodes.
- Large-scale VFX pipelines. AE's project-based architecture does not scale well for projects with 500+ VFX shots. Nuke's script-based architecture, combined with shot management tools like Ftrack or Shotgun, handles large pipelines better. AE works best for projects with 5-50 VFX shots.
- Real-time preview. AE does not offer real-time playback for complex comps. If you need real-time preview of VFX work, use Fusion inside DaVinci Resolve, which renders many operations on the GPU and achieves real-time playback on suitable hardware.
- Collaboration. AE's project files (.aep) are not designed for multi-user collaboration. Only one person can edit an .aep file at a time. For team VFX work, use After Effects with Team Projects (Adobe's cloud-based version control) or switch to Nuke with a proper asset management system.
12. Quick Reference: Keyboard Shortcuts for VFX Work
Essential VFX Shortcuts (Mac / Windows)
| Action | Mac | Windows |
|---|---|---|
| New composition | Cmd+N | Ctrl+N |
| Pre-compose selected layers | Cmd+Shift+C | Ctrl+Shift+C |
| RAM Preview | Space (or 0 on numpad) | Space (or 0 on numpad) |
| Purge memory | Cmd+Opt+/ | Ctrl+Alt+/ |
| Set In point | Cmd+B (in Layer panel) | Ctrl+B (in Layer panel) |
| Region of Interest | Cmd+Opt+Shift+B | Ctrl+Alt+Shift+B |
| Toggle shy layers | Toggle Shy button in timeline | Toggle Shy button in timeline |
| Pen tool (masks) | G | G |
| Roto Brush tool | Opt+W | Alt+W |
| Track Camera | Animation menu > Track Camera | Animation menu > Track Camera |
| Render Queue | Cmd+Shift+/ | Ctrl+Shift+/ |
| Add to Render Queue | Cmd+Shift+M (from Project) | Ctrl+Shift+M (from Project) |
After Effects VFX Workflows FAQ
Should I work in 8-bit, 16-bit, or 32-bit for VFX compositing?
32-bit float for all VFX work. The 30-50% performance hit is worth the quality gain in edge blending, light wrap, glow operations, and keying. 16-bit is acceptable for simpler composites without HDR data. 8-bit produces visible banding in gradients and should be avoided for any VFX work.
How do I send VFX shots from After Effects to DaVinci Resolve?
Export as EXR sequence (16-bit half-float) with alpha channel for maximum quality, or ProRes 4444 with alpha for faster workflow. In Resolve, import the rendered files and place them above the original plate on the timeline. The alpha channel automatically composites the VFX over the plate. Include 12-24 frame handles on each side of the shot for editorial flexibility.
Is Roto Brush 2 reliable enough for professional work?
For rough isolation on shots with clean backgrounds and single subjects: yes. For precise rotoscoping on complex shots: no. Use Roto Brush 2 for the initial pass (70-80% accuracy), then refine with manual masks on problem frames. Always freeze the Roto Brush pass (Layer > Roto Brush > Freeze) to lock the segmentation and restore playback performance.
When should I use Mocha AE instead of the built-in 3D Camera Tracker?
Use Mocha AE for screen replacements, sign replacements, and any situation where you need to track a flat surface with perspective changes. Use the 3D Camera Tracker for placing 3D elements in a scene with camera movement. Mocha handles occlusion and rotation better; the 3D Camera Tracker handles full 3D scene reconstruction better.
Can After Effects handle RED and ARRI RAW footage natively?
Yes for R3D and ARRIRAW — both have native importers with full color space and debayering controls. No for BRAW — Blackmagic RAW requires transcoding to ProRes or DNxHR before importing into AE. Decode speed for RAW in AE is slow (3-12 fps at 4K), so consider transcoding to ProRes 4444 for compositing work and switching back to RAW for final render.
How much RAM do I need for After Effects VFX work?
AE's official minimum is 16 GB. The practical minimum for 4K VFX compositing is 32 GB. For complex comps with multiple EXR sequences, particle systems, and heavy effects, 64 GB is recommended. A single 4K 32-bit frame consumes approximately 100 MB of RAM cache. At 24fps, one second of cached 4K footage uses 2.4 GB. With 32 GB allocated to AE, you can cache approximately 10 seconds of 4K 32-bit footage.
Need Professional VFX Compositing?
Our After Effects and Nuke artists handle rotoscoping, green screen keying, motion tracking, particle effects, and full compositing integration for music videos, commercials, and short films. Free VFX consultation within 24 hours.
Get a Free VFX Consultation