VES Handbook of Visual Effects

Jeffrey A. Okun, VES, is an award-winning Visual Effects Supervisor, who is a Fellow of the VES and the Academy of Motion Pictures, Arts and Sciences as well as the American Society of Cinematographers, and the Television Academy. He has served as the chair and first vice chair of the VES, and the chair of the LA Section. Jeff created the VES awards program and show along with Tim McGovern, VES and Kim Lavery, VES.Okun created visual effects tracking and bidding software in 1992 that is still in wide use within the industry today, as well as the visual effects techniques dubbed the “PeriWinkle Effect” and the “Pencil Effect”.Susan Zwerman, VES, has been a member of the VES since 1998. She is a highly respected Visual Effects Producer who has been producing visual effects for more than 25 years. Zwerman is also a well-known seminar leader and author. As chair of the DGA’s UPM/AD VFX Digital Technology Committee, Susan emphasizes the importance of the visual effects teams to DGA members at visual effects seminars organized under her guidance.In 2013, Susan received the Frank Capra Achievement Award in recognition of career achievement and service to the industry and the Directors Guild of America. She is also a member of the Academy of Motion Picture Arts and Sciences, the Producers Guild of America, Directors Guild of America, and a member and Fellow of the VES.Susan Thurmond O’Neal joined the VES in 1997 and has been deeply involved ever since. She has served on the global Executive Committee in multiple roles, including Treasurer in 2016, second Vice Chair in 2022 and 2023, and first Vice Chair in 2024 and 2025. Over the years, she has chaired the legacy global Education Committee, currently leads the Membership Committee, and in 2019, was honored with the VES Founders Award for meritorious service to the Society.By day, O’Neal is a recruiter for BLT Recruiting, Inc., where she connects top-tier talent with the right opportunities. Her career has spanned roles as Operations Manager at The Mill, Operations Director at Escape Studios LA, and Account Manager at SideFX Software. She got her start in visual effects at Digital Domain in 1993, working in finance and operations before transitioning into production.

• COMING SOON - John KnollCHAPTER 1: INTRODUCTION - Michael Fink, VESWhat is a Visual Effect?What is a Special Effect?Art and TechnologyCHAPTER 2: PRE-PRODUCTION/PREPARATIONOverview - Scott Squires, VESBreaking Down a Script - Budgeting - Scott Squires, VESBallpark BudgetMore Detailed BudgetsBiddingPlate PhotographyTemp ScreeningsReviewing BidsContractsRebidding during ShootingRebidding in PostMonitoring the Budget and ScheduleKeeping the Budget DownExamples of Script and Shot Breakdowns - Susan Zwerman, VES, Jeffrey A. Okun, VESBreakdownsProduction BreakdownsGenerating a VFX Breakdown: The First PassWorking with the Director and Producer - Scott Squires, VESDemo ReelThe MeetingMoving ForwardProduction Departments - Scott Squires, VESProduction DesignCameraWorking with the Cinematographer - Bill Taylor, ASC, VESSpecial Effects - Scott Squires, VESStuntsWardrobeMakeupProductionVisual EffectsEditorialLocationsProduction MeetingDesigning Visual Effects Shots - Scott Squires, VESGuidelines for DirectorsStoryboardsPrevisObjective of the ShotConcept ArtContinuityPhotorealismOriginal ConceptsBudgetReality and MagicCamera AnglesFramingScaleDetailSpeedScaled ImagesDepth of FieldSequence of ShotsCamera MotionLess is MoreAction PacingCG CharactersCreatures and Character DesignPowers of 10 ShotsVisual Effects Techniques - Scott Squires, VESTechnical ConsiderationsAdditional Suggestions for Determining TechniquesDevelopment of Previs Techniques - Mat Beck, Stephanie ArgyHistory and BackgroundNew Potent TechniquesArtificial Intelligence (AI)What is Previs and Other Forms of Visualization? - Hamilton LewisWhat is Previs?Different Types of Visualization and When to Use ThemVisualization: The New EssentialThe Application of Previs: Who Benefits and How? - J. Todd ConstantineThe Benefits of PrevisProject TypesPost Visualization - Scott HankelWhat is Post-Visualization?Why Use Postvis?Who Does Postvis?Cautions and Suggestions for Good Practice - Mat BeckThe Perils of Previs!Passing the Work OnThe Role of the VFX Supervisor in PrevisPrevisualization: Advanced Techniques - Christopher FerriterIntroductionBenefits of VisualizationAdvanced TechniquesCamera Angle Projection - Steven D. KatzIntroduction by Bill Taylor, ASC, VESDrawing What the Lens SeesTechvis - Christopher FerriterWhat is Techvis?How is Techvis used?Advancements in TechvisEmerging Techniques and TrendsEssential Data Captured in TechvisReal Time Engines and Their Previs Applications in VFX - Ryan McCoy IntroductionPre-Production ApplicationsProduction ApplicationsPipeline Advantages Real Time Engines and Their Application in Animation - Ryan McCoy Traditional 3D PipelineReal-Time Pipeline CHAPTER 3: ACQUISITION / SHOOTING Working on Set - Scott Squires, VESCommon Types of Special Effects - Gene RizzardiWhat are Special Effects?A Brief History of Special EffectsThe Special Effects SupervisorWorking with the Visual EffectsVisual Effects in Service to SFXSpecial Effects Design and PlanningStoryboards and PrevisThe Elements: Rain, Wind, and Snow and IceSmoke, Fire, and PyrotechnicsMechanical EffectsFlying Wire Rigs and StuntsSafetyGreenscreen and Bluescreen Photography - Bill Taylor, ASC, VESBest Practices and OtherwiseOverviewFunction of the BackingNegative Scanning and Digital ConversionBacking Uniformity and Screen CorrectionThe Alpha ChannelThe Processed ForegroundThe CompositeRecommended Specifications and PracticesHow to Expose a Greenscreen Shot, and WhySetting Screen BrightnessChoosing the Backing ColorFloor Shots, Virtual SetsForeground LightingControlling Spill LightLighting Virtual SetsTracking MarkersOn-Set PreviewCameras for Bluescreen or Greenscreen PhotographyUnderwater PhotographyOn-Set Data Acquisition - Karen GoulekasCamera ReportTracking MarkersProps for the ActorsCyberscanningDigital PhotosLiDAR/Laser ScanningLens Distortion ChartsHDRI and Chrome BallsLiDAR Scanning and Acquisition - Alan LaskyOn-Set 3D Scanning Systems - Joshua DeHerreraTypes of TechnologyLiDARPhotogrammetryProp ScannersLighting Data - Charles Clavadetscher, Jeffrey A. Okun, VESGathering Lighting DataUsing Conventional Still CamerasReference Shooting ConsiderationsBeware of False Savings!Clean Plates - Scott Squires, VESShooting the Clean PlateLocked-Off CameraMoving CameraOther IssuesPost-ProcessAlternates without Clean PlatesOther Uses for Clean PlatesMonster Sticks - Scott Squires, VESOn-Set Animation Capture: Witness Cam (IBMC) - Joel HynekWireless Non-Video Motion CaptureFactors Affecting Witness CamerasDealing with the Data in Post-ProductionCamera Tracking for Real-Time Visualization - Joe LewisIntroductionCamera Department: Pre-Production and CooperationCamera Tracking Strategies in an LED VolumeEmerging, Innovation and Mature TechnologyTriangulation As a Method of Recording Camera Data - Stuart RobertsonCamera/Subject Positional InformationBasics: The ToolkitBasics: Nodal PointPhotographic Reference - Charles ClavadetscherHow to ProceedShooting Video as a ReferenceRules, Setup, and Testing - Charles ClavadetscherDo a Complete Test Shot!Why Run Through Example or Test Shots?Digital Cinematography - David Stump, ASC, Marty Ollstein, David Reisner, W. Thomas WolfDigital DefinitionsHigh Dynamic Range (HDR)Camera / Lens MetadataCamera TrackingLook ManagementThe Recording SystemVFX Photography - Eric SwensonThe Camera ArrayDesigning an Array ShotTechniciansShoot DaySpecial TechniquesPostThe FutureFilming Live-Action Plates to be Used in VFX - Bill Taylor, ASC, VESCamera Position (Station Point)Angle of ViewLighting ConsiderationsCamera TiltBackground QualityMoving PlatesScouting the Camera PositionsA Case StudyCamera CarsCamera Car Safety IssuesPurpose-Built Crane CarsVibration and Camera StabilizationRoad SpeedPrecautionsPanoramic RigsOn the WaterAir to AirCable SystemsShooting Elements for Compositing - Mark H. WeingartnerWhat Is an Element?Stock FootageTypes of ElementsGeneric versus Shot-Specific ElementsDetermining Element NeedsCheatingBackgroundsBlack BackgroundsLine-UpCamera Format ConsiderationsAssorted Methods for Shooting ElementsHigh-Speed Cinematography and Filming Elements - Jim MatloszCamerasTechniciansDirector of PhotographyLightingMoving the CameraVideo AssistPostSupervising Motion Control - Mark WeingartnerWhat is Motion Control?Performance ChoreographyMultiple-Pass PhotographyScalingImport and Export of Camera Move DataThe DataTypes of Motion Control SystemsMotion Control SoftwareCamera TypesSync and PhaseDealing with ProductionAcquisition of Motion / Still Photographic Textures for Mapping onto CG - Mark WeingartnerPanoramic BackgroundsTiled StillsMotion Tiling and Synchronous PlatesPractical ConsiderationsStills for Textures and LightingStop-Motion - Pete Kozachik, ASCEvolution of Stop-Motion PhotographyThe Time Required to Shoot in Stop-MotionPreparation before ShootingSetting up a Shooting Space for Stop-MotionUse of Motion Control in Stop-MotionUseful CaveatsEvolution of a ShotUse of Stop-Motion in Visual EffectsCHAPTER 4: PERFORMANCE AND MOTION CAPTUREDefining Motion Capture / Performance Capture - Demian Gordon, Nathan Camp, John Root How Motion Capture is Used Across the Industry - Demian Gordon, Nathan Camp, John Root Is Motion Capture Right for a Project? – Demian Gordon, Nathan Camp, John Root, Tracy McSherry The Mocap LookVisual Storytelling and AestheticProduction PlanningPerformanceTechnology and ApproachAligning Needs vs. WantsOptical Motion CaptureMotion Capture for Virtual ProductionMotion Capture for AnimationProduction Planning Scenarios for Motion Capture – Demian Gordon, Nathan Camp, Rhea Aylin Fischer, and John Root Subject Matter of the ShootPerformance and Actor ConsiderationsReal-Time PresentationSubjects Being CapturedVolume Size and BuildSyncing Hardware and SoftwareRigging and Retargeting for Motion CaptureShot ListClient or Vendor DeliveryOptical Motion Capture – Nathan Camp, Demian Gordon , John RootTechnical and ApproachEntry PointBudgetHardwareSoftwareInertial Motion Capture - Rhea Aylin Fischer, Ted O’Brien ,John Root, and Demian Gordon Use CasesTechnology and ApproachEntry PointBudgetHardwareSoftwareMotion Capture for Virtual Production - Nathan Camp, Ted O'BrienUse CasesTechnology and ApproachEntry PointBudgetHardwareSoftwareHand Capture Demian Gordon , Rhea Aylin Fischer, Tracy McSherry. John Root, Use CasesAdvantages and DisadvantagesTechnology and ApproachBudgetFacial Motion Capture – John Root, Demian Gordon, Gareth Edwards, and Oleg AlexanderUse CasesMarkersetLightingFacial HairAdvantages and DisadvantagesEntry PointBudgetFacial Actor SurveyActor Survey - HardwareApplying AIVolumetric Capture / Considerations for Production - Wardrobe, Hair, Makeup, and Blocking - Christina Heller, Skylar Sweetman, Mari YoungCHAPTER 5: VIRTUAL PRODUCTION AND AI TOOLS IN VISUAL EFFECTSVirtual Production - Addison BathWhat is Virtual ProductionHow is Virtual Production Used?Note from the Editors: – Jeffrey A. Okun, VES, Susan Zwerman, VES, Susan Thurmond O’NealCapturing Surfaces with Photogrammetry to Create Assets - Luis CataldiIntroductionCore Principles of PhotogrammetryApplied Principles of Image Capture for PhotogrammetryCapture Methods and WorkflowsFinal ThoughtsPhotogrammetry and LiDAR Sets for Virtual Production - Tripp ToppingOptimizationThe Use of AI in Visual Effects - Lucien HarriotIntroductionOverview of Generative Artificial Intelligence (GenAI)Applications of GenAI in Visual EffectsImpact on Visual Effects ProcessesLimitations, Challenges, and Ethical ConsiderationsFuture Developments in GenAIEmbracing GenAI as a Tool for InnovationSummary of Techniques for NeRFs and 3D Gaussian Splatting - Mike SeymourIntroductionOverview of NeRFs and Gaussian SplattingNeural Radiance Fields (NeRF)3D Gaussian SplattingNeRFs and 3DGS in Combination with Other AILimitations, Challenges, and Ethical ConsiderationsSummary of AI Techniques Used in Visual Effects - Steve WrightA Brief HistoryAI for CompositingImage EnhancementVisual EffectsOther FeaturesAdapting to a World with AICHAPTER 6: POST-PRODUCTION / IMAGE MANIPULATIONResolution and Image Format Considerations - Scott SquiresFormatsTransportResolutionAcademy Color Encoding System (ACES) - Bill Baggelaar, Annie Chang, Steve Tobenkin OverviewUsing ACES in Visual Effects WorkflowsACES Color Space EncodingCompatibilityImage Compression/File Formats for Post-Production - Florian Kainz, Piotr StanczykImage EncodingStill Image CompressionOther Lossless Compression MethodsFile FormatsColor Management - Joseph GoldstoneThe Three GuidelinesDigital Color Image Encodings and Digital CamerasColor Management at the DesktopOpenColorIODigital Intermediate - Tashi TrieuIntroduction to Digital IntermediatesThe ColoristTools of the TradeEmerging TechnologiesGrading with MattesCommunicating Color Through the Visual Effects PipelineVisual Effects Plate GradingThe Evolution of Color GradingShot/Element Pulls and Delivery to VFX - David H. TanakaIntroductionThe Lab“Production” Is to “Lab” asThe MergeThe HandoffVFX EditorialThe SelectThe PullVFX Editorial – Greg Hyman, David H. TanakaEditing within a Shot: The Art of Precompositing (Precomp)How It Came To BeModern Day Tracking and Disseminating of InformationAs the Shot ChangesWrapping It UpEditorial Workflow in Feature Animation - David H. TanakaIntroductionEditorial Crew Staffing and StructureEditorial Involvement with Feature Animation Production StagesCommunication With Artists - Eric DurstStartingWorking with TeamsWorking GloballyReference and PerspectiveShot ProductionCommunicating with Artists in Other DepartmentsCompletionCompositing of Live-Action Elements - Steve WrightOverviewGreen/Blue Screen CapturePre-Processing the Green ScreenGarbage MattesKeying StrategiesMotion Blur and Depth of FieldSpill SuppressionGrain ManagementRotoscopingPost-Composite Processing3D CompositingInspecting the CompRotoscoping and Paint - Marshall Krasser, Sandy HoustonRotoscopingDigital Painting and Plate Reconstruction Deep Compositing - Steve WrightWhat is a Deep Image?Complex Depth InteractionsThe Depth Compositing Edge ProblemThe Re-rendering ProblemDeep Compositing and Live ActionMatte Painting/Creative Environments - Craig Barron, VES, Deak Ferrand, John Knoll, and Robert StrombergMatte Paintings: Art of the Digital RealmWhat Is a Matte Painting?Matte Painting - Pioneers and HistoryVisualizing the Matte Painting Shot in Pre-ProductionOn-Set Supervision for Matte Painting ShotsBasic Skills and Tricks of the TradeMiniatures and Computer-Generated SetsFinding the Best FrameRe-Projected Photo SurveyThe Need for Creative CompositingDigital Environments - Robert StrombergReimagining Artistic Roles in Film: The Rise of the “Environment and Creation Artists”Virtual Production and Game Engine TechnologyFuture of the Environment Creation Artist and AIThe Power of Curiosity in Creativity and InnovationCHAPTER 7 - DIGITAL ELEMENT CREATIONDigital Modeling - Kevin HudsonOverview: The Importance of ModelingTypes of ModelingModel Data TypesDevelopment of ModelsModeling for a Production PipelineEngineering Aspects for PolygonsEngineering Aspects for NURBSRigging and Animation Rigging - Steve PreegRigging: What is It?Animation RiggingDeformation RiggingWhat Makes a Good Rig for Animation – Felix Balbas IntroductionUser and UsesTechnical AspectsTechnology RestrictionsNew TechnologiesTexturing and Surfacing - Ron WoodallThe Importance of Texture PaintingHard Surface ModelsCreature ModelsTypes of Geometry: Their Problems and BenefitsPrepping the Model to Be PaintedTexture CreationVarious Other Map-Driven EffectsTexture Painting in ProductionModel EditingDigital Hair/Fur - Armin Bruderlin, Francois ChardavoineHair Generation ProcessGeneral Issues and SolutionsDigital Feathers - Armin Bruderlin, Francois ChardavoineMorphology of Real FeathersModeling Digital FeathersSimilarities between Hair and FeathersDifferences between Hair and FeathersGeneral Geometry Instancing - Armin Bruderlin, Francois ChardavoineAsset CreationWorld BuildingShot ConsiderationsDynamics and Simulation - Craig Zerouni, Judith CrowHow is a Simulation Created?When is Simulation Appropriate?Tricks and CheatsImportant ConsiderationsPlanning and PreparationSoftware Solutions: A Broad Overview of Current OptionsParticles - Craig ZerouniWhat are Particle Systems?The Next StepThe Birth of ParticlesCreating EffectsRigid Body Dynamics - Craig ZerouniHow Rigid-Body Dynamics are CreatedPotential ProblemsOther IssuesTricks for Getting It DoneDigital Lighting - Andrew WhitehurstLight in Reality and in Computer GraphicsCase Study of Reality Compared with Simple CG SimulationVisual Sophistication through Texture MappingPhysically Derived Shading ModelsBeneath the SurfaceGoals of Lighting in Visual EffectsWork Flow for Successful Creative Digital LightingThe Technologies of Lights in Computer GraphicsDirect Lighting: Source to Surface to CameraReflectionsPhotographed ReflectionsShadowsImage-Based LightingRendering OcclusionAmbient OcclusionReflection OcclusionCreating Light Sources from Environment MapsPhysically Based RenderingPhysically Plausible RenderingVolumetric Lighting EffectsShader Basics - Derek SpearsWhat are Shaders?Shading ModelsBump and DisplacementMap-Based ShadersProcedural ShadersShader DesignAnti-aliasing Considerations2D Compositing - Matt Leonard2D File FormatsImage Quality: Color Bit Depth and ConcatenationLog vs. LinearLow Dynamic Range and High Dynamic Range ImagesMattes and Pre-Multiplied AlphaWorking with Rendered CG ElementsIntegration Techniques2.5D Compositing - Matt LeonardZ-Depth CompositingAdding Depth of FieldAdding Motion BlurRelighting3D Compositing - Matt LeonardWorking with 3D Data in a CompositorPan and TileCamera ProjectionsSet ExtensionsCoverage MappingCubical Camera Setup3D Mattes3D Retouch and CleanupAdjusting Camera MovesParticlesDeep CompositingCrowd Generation and Simulation Techniques - John KnollLive-Action ReplicationSpritesComputer-Generated CrowdsModeling for ReplicationVariationMesh DensityAnimation Cycles for ReplicationMotion CaptureKeyframe AnimationDynamic Motion SynthesisBehaviors and Crowd ControlCG Prosthetic and Actor Enhancements - John KnollOn-Set Tracking and Capture ConsiderationsEye Enhancements3D Techniques2D Techniques2.5D TechniquesSilhouette ChangesRe-Projection3D Products, Systems and Software - Richard KiddDigital Element Creation Process3D Graphics Software3D TrackingSpecial EffectsRenderingTexturingCHAPTER 8 – INTERACTIVE GAMESOverview – David Johnson, VESHow the Gaming Industry and Film/TV Industries are different - David Johnson, VESPublishers and Developers v. Film/Television Studios and VFX Facilities – David Johnson, VESGame and TV/Film Converging with Virtual Production and Cinematics - David Johnson, VESGame Engines and Real-Time Rendering - David Johnson, VESRuntime ComponentUnreal vs. UnityDisciplines and Job Titles - David Johnson, VESGame DesignEngineeringProductionTestArtThe Art DirectorConcept ArtEnvironment ArtistsTexture ArtistCharactersHard ModelingPropsLightingBaked vs. Dynamic Lighting vs. RaytracingShadowsEffectsSystem EffectsEnvironmental FXBreakablesDestructionTech ArtistAnimationUIReal Time Shaders and Materials - David Johnson, VESOptimization and Runtime Budgets - David Johnson, VESPerformance Analysis and Profiling - David Johnson, VESCPU vs. GPU boundTechnical Terminology - David Johnson, VESUser CalibrationLatency“Game Mode” on TelevisionsHDR10PBR – Physically Based RenderingFBXMip MappingFilteringTexelScreen Space Ambient OcclusionLevel of DetailVertex Shaders and Fragment ShadersAR/VR - David Johnson, VESFuture of Gaming - David Johnson, VESOn-Demand Rendering, Cloud Distribution and Ray TracingFast Drives, Nanite and LumenFluid SimulationsAICHAPTER 9 – COMPLETE ANIMATIONWhat Is An Animation Project? - Stephan Vladimir BugajFull Animation vs Visual EffectsDifference Between Visual Effects and Animation - Rob BredowProduction PipelinesProductionAnimation Techniques - Frank GladstoneTraditional AnimationStop-MotionComputer Graphic TechnologyConsiderations for a Full CG-Animated Feature Pipeline - Stephan Vladimir BugajCG Feature Animation PipelineManaging an Animated Film - Don HahnFilm Management and Personal StyleBuilding Brain TrustsBuilding the Core Creative TeamWriting and Visual DevelopmentWorking with a StudioFacilities and EnvironmentManaging the EventThe Production Process: An Animator’s Perspective - Lyndon BarroisWorking on CG-Animated Content in Live-Action FeaturesPlanning the ProcessProductionCharacter and Environment InteractionCHAPTER 10 – VISUAL EFFECTS WORKFLOWSAnalysis of a Production Workflow - Stephan Vladimir BugajFrom Workflow to PipelineService Bureau versus In-House RequirementsDesign of a Production Workflow – Dan RosenFrom Analysis to DesignDeploying a Production Workflow - Stephan Vladimir BugajFrom Design to ImplementationTracking Assets - Stephan Vladimir BugajWhat is Task and Asset Tracking?Commercial Task and Asset Tracking SystemsBuilding Task Asset Tracking SystemsScene Assembly - Stephan Vladimir Bugaj2D Scene Assembly3D Scene Assembly (Compositing)Working Across Multiple Facilities – Scott Squires, VESImagesModelsTexturingAnimationCompositingR&DVersion Control Software - Jase Lindgren, Katie ColeIntroduction to Version ControlVersion Control Uses in Virtual ProductionVersion Control Set-UpCHAPTER 11 – VR / AR (VIRTUAL REALITY / AUGMENTED REALITY)A Note from the Editor Prelude to Virtual Reality & Augmented Reality – Jeffrey A. Okun, VESPre-Production for VR/AR - Chris EdwardsProduction for VR/AR - Grant AndersonPost-Production for VR/AR - Dave FranksEditorial Post-Production for VR/AR - John GrodHow to Direct the Viewer?The Post-ProcessNonlinear Editorial, Timelines and EditsWorld Lock and Forced PerspectiveTypes of Head-Mounted and Handheld Displays - Bernard MendiburuIdentifying the Computer-Mediated RealitiesState of the Art in VR Photonic TechnologiesState of the Art in VR Hardware TechnologiesState of the Art in VR Software TechnologiesUsing an HMD for Visual Effects ProductionChoosing an HMD for Visual Effects ProductionHemispheres and Domes - John GrodGames Mechanics Are What it is All AboutOvercoming Doubt and Preconceived NotionsDome ProjectionsThe Future of DomesThe Evolution of Domes: The MSG Sphere - Phillip Galler Uses of VR - John GrodEnterpriseNarrative StorytellingFuture of VR and AR - Ben Grossmann This is Just the BeginningImmersive Experiences Make Use of Advanced Visual Effects Technology - Philip GallerWhat are Immersive Experiences?Visual Effects’ Impact on Immersive ExperiencesContent Creation for Immersive ExperiencesChallenges for Delivering Content for Immersive ExperiencesWearables and the Future of Immersive ExperiencesAcknowledgmentsAppendix A: Charts and FormulasAppendix B: GlossaryIndex