VES Handbook of Visual Effects

Jeffrey A. Okun, VES, is an award winning Visual Effects Supervisor, having worked on a large number of feature films, commercials and television shows. He started in the inudstry in 1976, and he is currently a member and Fellow of the VES and The Academy of Motion Pictures, Arts and Sciences as well as the American Society of Cinematographers (ASC), the Television Academy and the Editors Guild. He was the chair of the VES for 9 years and 1st vice chair for 16 years, and chair of the L.A. Section for 2 years. He created the VES awards along with Tim McGovern and Kim Lavery, VES.Okun created a visual effects tracking and bidding software in 1992 that is still in wide use within the industry today, as well as the revolutionary visual effects techniques dubbed the "PeriWinkle Effect" (an underwater blue screen technique) and the "Pencil Effect" (accurately predicts the final visual effects count and budget).Susan Zwerman, VES has been a member of the VES since 1998. She is a highly respected Visual Effects Producer who has been producing visual effects for more than 25 years. Zwerman is also a well-known seminar leader and author. As Chair of the DGA’s UPM/AD VFX Digital Technology Committee, Susan emphasizes the importance of the visual effects teams to DGA members at visual effects seminars organized under her guidance.In 2013, Susan received the Frank Capra Achievement Award in recognition of career achievement and service to the industry and the Directors Guild of America. She is also a member of the Academy of Motion Picture Arts and Sciences, the Producers Guild of America and a Fellow of the VES.

• Coming Soon … Chapter 1: IntroductionFirst, Some Ground RulesNow, the IntroductionWhy Use Visual EffectsChapter 2: Pre-Production /PreparationOverviewBreaking Down a Script – Budgeting Ballpark BudgetMore Detailed BudgetsBiddingPlate PhotographyTemp ScreeningsReviewing BidsContractsRebidding during ShootingRebidding in PostMonitoring the Budget and ScheduleKeeping the Budget DownWorking with the Director and ProducerDemo ReelThe MeetingMoving ForwardProduction DepartmentsProduction DesignCameraWorking with the CinematographerSpecial EffectsStuntsWardrobeMakeup ProductionVisual EffectsEditorialLocationsProduction MeetingDesigning Visual Effects ShotsGuidelines for DirectorsStoryboardsPrevisObjective of the ShotConcept ArtContinuityPhotorealismOriginal ConceptsBudgetReality and MagicCamera AnglesFramingScaleDetailSpeedScaled ImagesDepth of FieldSequence of ShotsCamera MotionLess Is MoreAction PacingCG CharactersCreatures and Character DesignPowers of 10 ShotsVisual Effects TechniquesTechnique ConsiderationsAdditional Suggestions for Determining TechniquesDevelopment of Previs TechniquesHistory and BackgroundWhat is Previs and Other Forms of Visualization?What is Previs?Different Types of Visualization and When to Use ThemVisualization: The New EssentialThe Application of Previs: Who Benefits and How?The Benefits of PrevisProject TypesPost-VisualizationWhat is Post-Visualization?Why Use Postvis?Who Does Postvis?Cautions and Suggestions for Good PracticeThe Perils of Previs!Passing the Work OnThe Role of the VFX Supervisor in PrevisPrevis: Advanced TechniquesVisualization UsefulnessVR as a ToolVisualization in EngineRender in EngineVisualization in Real TimeAR as a ToolCamera Angle Projection IntroductionDrawing What the Lens SeesTechvisWhat Is Techvis?Who is Techvis for?Virtual Production What is Virtual Production?How is Virtual Production Used?Chapter 3: Acquisition / ShootingWorking on SetCommon Types of Special EffectsWhat are Special Effects?A Brief History of Special EffectsThe Special Effects CoordinatorWorking with the Visual EffectsVisual Effects in Service to SFXSpecial Effects Design and PlanningStoryboards and PrevisThe Elements: Rain, Wind, and Snow and IceSmoke, Fire, and PyrotechnicsMechanical EffectsFlying Wire Rigs and StuntsSafetyGreenscreen and Bluescreen PhotographBest Practices and OtherwiseOverviewFunction of the BackingNegative Scanning and Digital ConversionBacking Uniformity and Screen CorrectionThe Alpha ChannelThe Processed ForegroundThe CompositeRecommended Specifications and PracticesHow to Expose a Green Screen Shot, and WhySetting Screen BrightnessChoosing the Backing ColorFloor Shots, Virtual SetsForeground LightingControlling Spill LightLighting Virtual SetsTracking MarkersOn-Set PreviewCameras for Blue Screen or Green Screen PhotographyUnderwater PhotographyOn-Set Data AcquisitionCamera ReportTracking MarkersProps for the ActorsCyberscanningDigital PhotosLidar/Laser ScanningLens Distortion ChartsHDRI and Chrome BallsLidar Scanning and AcquisitionOn-Set 3D Scanning SystemsTypes of TechnologyLidarPhotogrammetryProp ScannersLighting DataGathering Lighting DataBeware of False Savings!Using Conventional Still CamerasReference Shooting ConsiderationsClean PlatesShooting the Clean PlateLocked-Off CameraMoving CameraOther IssuesPost-ProcessAlternates without Clean PlatesOther Uses for Clean PlatesMonster SticksOn-Set Animation Capture: Witness Cam (IBMC) Wireless Non-Video Motion CaptureFactors Affecting Witness CamerasDealing with the Data in Post-ProductionCamera Tracking for Real-Time VisualizationCamera Tracking Pre-ProductionThe Camera DepartmentPrior to ShootingCurrent Tracking Systems in UseTriangulation As a Method of Recording Camera DataCamera/Subject Positional InformationBasics: The ToolkitBasics: Nodal PointPhotographic ReferenceHow to ProceedShooting Video as a ReferenceRules, Setup, and Testing Do a Complete Test Shot!Why Run Through Example or Test Shots?Digital CinematographyDigital DefinitionsHigh Dynamic Range (HDR)Lens MetadataLook ManagementThe Recording SystemLens Mapping for VFXVFX PhotographyThe Camera ArrayDesigning an Array ShotTechniciansShoot DaySpecial TechniquesPostThe FutureFilming Live-Action Plates to be Used in VFXCamera Position (Station Point)Angle of ViewLighting ConsiderationsCamera TiltBackground QualityMoving PlatesScouting the Camera PositionsA Case StudyCamera CarsCamera Car Safety IssuesPurpose-Built Crane CarsVibration and Camera StabilizationRoad SpeedPrecautionsPanoramic RigsOn the WaterAir to AirCable SystemsShooting Elements for CompositingWhat Is an Element?Stock FootageTypes of ElementsGeneric versus Shot-Specific ElementsDetermining Element NeedsCheatingBackgroundsBlack BackgroundsLine-UpCamera Format ConsiderationsAssorted Methods for Shooting ElementsHigh-Speed Photography and Filming ElementsCamerasTechniciansDirector of PhotographyLightingApplicationLocking Down the CameraVideo AssistPostSupervising Motion ControlWhat is Motion Control?Performance ChoreographyMultiple-Pass PhotographyScalingImport and Export of Camera Move DataThe DataTypes of Motion Control SystemsMotion Control SoftwareCamera TypesSync and PhaseDealing with ProductionAcquisition of Motion / Still Photographic Textures for Mapping onto CGPanoramic BackgroundsTiled StillsMotion Tiling and Synchronous PlatesPractical ConsiderationsStills for Textures and LightingStop-MotionEvolution of Stop-Motion PhotographyThe Time Required to Shoot in Stop-MotionPreparation before ShootingSetting up a Shooting Space for Stop-MotionUse of Motion Control in Stop-MotionUseful CaveatsEvolution of a ShotUse of Stop-Motion in Visual EffectsChapter 4: Performance and Motion CaptureWhat is Motion Capture?Is Motion Capture Right for a Project?The Mocap LookTechnical SpecificationsEntry PointBudgetWhich Technology is Right for a Project?Gauging a Project’s Needs and ConstraintsPassive Retroreflective OpticalActive OpticalInertialStructured LightDense Stereo ReconstructionBend SensorsPreparing for Motion CaptureActorsMotion Capture SuitsMarker Placement – BodyMarker Placement – FaceRigging for Motion CaptureShot ListTechnology ConsiderationsHardwareThe StrobeMarkersLensesFilterImage SensorOnboard ProcessorInputs/OutputsSetupSoftwareAcquisitionCalibrationPost-ProcessingReconstructionLabelingGap FillingCleaningSolving Motion CaptureFacial Motion CaptureFacial Actor SurveyActor Survey – HardwareReference DataStatistical DataFacial RiggingFacial AcquisitionAudioFacial Motion Capture SolvingReal-Time Motion Capture Real-Time UsesReal-Time LimitationsAlternate TechnologiesMotion Capture ResourcesVirtual ProductionWorld BuildingPrevisualizationOn-Set VisualizationVirtual CinematographyChapter 5: Stereoscopic 3DHow 3D Works Accommodation and ConvergenceInteraxial SeparationToe-in Versus Horizontal Image TranslationParallax or Depth BudgetPositive and Negative ParallaxFloating WindowsFix It in PostStereoscopic DesignThe Emerging Grammar of 3DCreative Use of DepthPrevisualizationAvoiding Painful 3DThe Aesthetic of Scale Cutting for 3DDesigning for Multiple Release FormatsImmersion-Based versus Convergence-Based StereoNative StereoPre-ProductionOn-SetStereography in the Visual Effects ProcessStereography for FinishingHFR as a Solution for Better 3D MoviesVFX Elements and StereoIntroduction – How VFX Elements are UsedNative Stereo ContentMono Capture – Packaged Script and Element DeliveriesMono Capture – Hybrid Approach for Stereo DeliveryMono Capture – Full CG Approach for Stereo DeliveryCreating Depth – Layout and StereographyStereo Camera – Depth WedgesStereo CompositingRequesting a Full CG Stereo Render Mid-ProductionVFX Production Tasks2D to 3D ConversionDepth Creation PreparationVisual Analysis of 2D Depth CuesPre-Production and Previs for ConversionSource and Target PerspectiveShared Shots/Shared WorkflowsMain Stages of 2D-to-3D ConversionMajor 2D-to-3D Conversion WorkflowsSpecial CasesRe-projection Mapping WorkflowPixel Displacement or Pixel ShiftingOther 2D-to-3D Conversion WorkflowsIs "Real" Always Right?2D-to-3D Conversion ManagementStereoscopic Visual EffectsPrepping for the Third DimensionShooting the Third DimensionVisual Effects in the Third DimensionPhotographed ElementsAccuracy and Attention to DetailArtistic Skill LevelData ManagementStereoscopic Digital Intermediate WorkflowStereoscopic 3D Process MilestoneViewing 3D DailiesProjection Screens for Stereoscopic Viewing3D Editorial ProcessesData WorkflowApplying the 3D Grade3D Stereo Deliverables3D Home Video DeliverablesStereoscopic Window The Stereoscopic WindowPlacement of the Window in Relation to the 3D SceneWindow ViolationsWindow Placement LogicHow to Create a Stereoscopic WindowProducing Movies in Three DimensionsDevelopment – Getting the Green LightProduction – What to Look Out ForChapter 6: Post-Production / Image ManipulationResolution and Image Format ConsiderationsFormatsTransportResolutionAcademy Color Encoding System (ACES)ACES ComponentsACES BenefitsACES Color Space EncodingViewing ACESPreparations for Using ACESImage Compression/File Formats for Post ProductionImage EncodingStill Image CompressionOther Lossless Compression MethodsFile FormatsColor Management The Three GuidelinesDigital Color Image Encodings and Digital CamerasColor Management at the DesktopBringing Color Management to Film WorkflowsDigital IntermediateShot Element Pulls and Delivery to VFXIntroductionThe Lab"Production" Is to "Lab" as …The MergeThe HandoffVFX EditorialThe SelectThe PullVFX EditorialEditing within a Shot: The Art of Precompositing (Precomp)How It Came to BeModern Day Tracking and Disseminating of InformationAs the Shot ChangesWrapping It UpEditorial Workflow in Feature AnimationIntroductionEditorial Crew Staffing and StructureEditorial Involvement with Feature Animation Production StagesCommunication with ArtistsStartingWorking with TeamsWorking GloballyReference and PerspectiveShot ProductionCommunicating with Artists in Other DepartmentsCompletionCompositing of Live-Action ElementsModern Digital CompositingScene TrackingRotoscoping and PaintRotoscopingDigital Painting and Plate ReconstructionMatte Paintings/Creative EnvironmentsMatte Paintings: Art of the Digital RealmWhat Is a Matte Painting?Matte Painting Pioneers and HistoryVisualizing the Matte Painting Shot in Pre-ProductionOn-Set Supervision for Matte Painting ShotsBasic Skills and Tricks of the TradeMiniatures and Computer-Generated SetsFinding the Best FrameRe-Projected Photo SurveyThe Need for Creative Compositing3D Matte PaintingChapter 7: Digital Element CreationDigital Modeling Overview: The Importance of ModelingTypes of ModelingModel Data TypesDevelopment of ModelsModeling for a Production PipelineEngineering Aspects for PolygonsEngineering Aspects for NURBSRigging and Animation RiggingRigging: What is It?Animation RiggingDeformation RiggingTexturing And SurfacingThe Importance of Texture PaintingHard Surface ModelsCreature ModelsTypes of Geometry: Their Problems and BenefitsPrepping the Model to Be PaintedTexture CreationVarious Other Map-Driven EffectsTexture Painting in ProductionModel EditingDigital Hair / FurHair Generation ProcessGeneral Issues and SolutionsDigital Feathers Morphology of Real FeathersModeling Digital FeathersSimilarities between Hair and FeathersDifferences between Hair and FeathersGeneral Geometry InstancingAsset CreationWorld BuildingShot ConsiderationsDynamics and SimulationHow is a Simulation Created?When is Simulation Appropriate?Tricks and CheatsImportant ConsiderationsPlanning and PreparationSoftware Solutions: A Broad Overview of Current OptionsParticlesWhat are Particle Systems?The Next StepThe Birth of ParticlesCreating EffectsRigid-Body DynamicsHow Rigid-Body Dynamics are CreatedPotential ProblemsOther IssuesTricks for Getting It DoneDigital LightingLight in Reality and in Computer GraphicsCase Study of Reality Compared with Simple CG SimulationVisual Sophistication through Texture MappingPhysically Derived Shading ModelsBeneath the SurfaceGoals of Lighting in Visual EffectsWork Flow for Successful Creative Digital LightingThe Technologies of Lights in Computer GraphicsDirect Lighting: Source to Surface to CameraReflectionsPhotographed ReflectionsShadowsImage-Based LightingRendering OcclusionAmbient OcclusionReflection OcclusionCreating Light Sources from Environment MapsPhysically Based RenderingPhysically Plausible RenderingVolumetric Lighting EffectsShader BasicsWhat are Shaders?Shading ModelsBump and DisplacementMap-Based ShadersProcedural ShadersShader DesignAnti-aliasing Considerations2D Compositing 2D File FormatsImage Quality: Color Bit Depth and ConcatenationLog vs. LinearLow Dynamic Range and High Dynamic Range ImagesMattes and Pre-Multiplied AlphaWorking with Rendered CG ElementsIntegration Techniques2D Compositing Z-Depth CompositingAdding Depth of FieldAdding Motion BlurRelighting3D Compositing Working with 3D Data in a CompositorPan and TileCamera ProjectionsSet ExtensionsCoverage Mapping3D Mattes3D Retouch and CleanupAdjusting Camera MovesParticlesDeep CompositingCrowd Generation and Simulation TechniquesLive-Action ReplicationSpritesComputer-Generated CrowdsModeling for ReplicationVariationMesh DensityAnimation Cycles for ReplicationMotion CaptureKeyframe AnimationDynamic Motion SynthesisBehaviors and Crowd ControlCG Prosthetics and Actor EnhancementsOn-Set Tracking and Capture ConsiderationsEye Enhancements3D Techniques2D Techniques2D TechniquesSilhouette ChangesRe-Projection3D Products, Systems, and SoftwareDigital Element Creation Process3D Graphics Software3D TrackingSpecial EffectsRenderingTexturingChapter 8: Interactive GamesOverviewHow the Gaming Industry and Film/TV Industries are differentGame Engines and Real-Time Rendering Runtime ComponentDisciplines and Job TitlesGame DesignEngineeringProductionTestArtThe Art DirectorConcept ArtEnvironment ArtistsTexture ArtistCharactersHard ModelingPropsLightingBaked vs. Dynamic LightingShadowsEffectsSystem EffectsEnvironmental FXBreakablesDestructionTech ArtistAnimationUIReal Time Shaders and MaterialsPre-Rendered Cinematics vs. Real Time VisualsOptimization and Runtime BudgetsPerformance Analysis and ProfilingCPU vs. GPU boundTechnical TerminologyUser CalibrationLatency"Game Mode" on TelevisionsHDR10PBR – Physically Based RenderingFBXMip MappingFilteringTexelScreen Space Ambient OcclusionLevel of DetailVertex Shaders and Fragment ShadersAR/VRFuture of GamingOn-Demand Rendering, Cloud Distribution and Ray TracingChapter 9: Complete AnimationWhat Is An Animation Project?Full Animation versus Visual EffectsDifference Between Visual Effects and AnimationProduction PipelinesProductionA Survey and History of Animation TechniquesTraditional AnimationStop-MotionComputer Graphic TechnologyConsiderations for a Full CG-Animated Feature PipelineCG Feature Animation PipelineManaging an Animated FilmFilm Management and Personal StyleBuilding Brain TrustsBuilding the Core Creative TeamWriting and Visual DevelopmentWorking with a StudioFacilities and EnvironmentManaging the EventThe Production Process: An Animator’s PerspectiveWorking on CG-Animated Content in Live-Action FeaturesPlanning the ProcessProductionCharacter and Environment InteractionChapter 10: General Workflow ConsiderationsVirtual Studio TechnologyAnalysis of A Production WorkflowFrom Workflow to PipelineService Bureau versus In-House RequirementsDesign of a Production WorkflowFrom Analysis to DesignDeploying a Production WorkflowFrom Design to ImplementationInfrastructureTracking AssetsWhat is Task and Asset Tracking?Commercial Task and Asset Tracking SystemsBuilding Task and Asset Tracking SystemsScene Assembly3D Scene Assembly2D Scene Assembly (Compositing)Working Across Multiple FacilitiesImagesModelsTexturingAnimationCompositingR&DChapter 11: VR / AR (Virtual / Augmented Reality)A Note from the EditorsPrelude to Virtual Reality & Augmented RealityPre-Production for VR/ARProduction for VR/AR Post-Production for VR/AREditorial Post-Production for VR/AR How to Direct the Viewer?The Post-ProcessNonlinear Editorial, Timelines and EditsWorld Lock and Forced PerspectiveTypes of Head-Mounted and Handheld DisplaysWhat are the HMD and the "VR Presence"?Electronical Designs of HMDs and their Respective PerformancesOptical Designs of HMDs and their Respective UsagesImage Quality Factors in VR HMD DisplaysHemispheres and DomesGame Mechanics Are What it is All AboutOvercoming Doubt and Preconceived NotionsDome ProjectionsThe Future of DomesVR/AR Tracking Displays Uses of VR EnterpriseNarrative StorytellingFuture of VR and AR This is Just the BeginningAcknowledgmentsAppendix A: Charts and FormulasAppendix B: Credits / Titles to Be Submitted in Accordance with VES GuidelinesAppendix C: GlossaryIndex