Research Blog – The Matte
As part of VFX specialization I will be doing rotoscoping for a couple of shots, and so I have decided to try and find the current industry standard for producing high quality mattes.
To integrate digital imagery with real life footage there has to be a way to seamlessly overlay all the elements on each other, this is is done in VFX by producing matte for each frame. A matte is a black and white image that indicates what part of an image is visible and what part isn’t, this allows VFX artists to replace backgrounds or other elements with images they have created. With purely digital sequences the mattes are often automatically made by the software but with real life footage the mattes need to be created by the VFX company. This is done by rotoscoping the silhouette of the desired footage to create a near exact matte of the of the undesired elements of the footage. Rotoscoping basically means creating a matte for each individual frame, this process is aided by the use of green screens as the contrast of colors and light often allow for automated ways of rotoscoping to be used, but often manual rotoscoping is also needed.
This information was extracted from the VES (Visual Effects Society) handbook.
“The best way to begin the task of rotoscoping is to review and study the footage of the subject to be articulated. Things to look for: Can any of the roto be done procedurally (i.e., stabilize the plate and use the inverted data to apply the motion to a single frame)?Will any sections require frame-by-frame hand articulation?Can fine hair detail be extracted using luminance matting?”
“After the plate has been analyzed, select the frame that has the most complex edge to use as the starting point. One big shape is not the ideal method; rather several smaller key shapes are the preferred approach. This allows each shape to be keyframed independently. In most cases, this is needed especially when major motion is involved (i.e., like a running person) and frameby-frame keying is required.”
“Sometimes the most difficult task can come from the need to rotoscope a person who appears to be motionless. Film weave and small movements are always occurring in this situation. Prestabilizing the plate to remove this is helpful, but if that is not a possibility, begin by hand articulating key frames as far apart as possible—then go back and refine as needed. For example, key the spline on 16’s, then refine on 8’s, and if needed, continue on 4’s, then 2’s, and if absolutely necessary, on 1’s. To avoid matte chatter, floating, or popping, the fewer key frames used the smoother and better the mattes will look. Key frames that are divisible by 2 will allow for a more mathematically even keying (i.e., 32, 16, 8, 4, 2). Using odd, or random, key frames makes final refinement more difficult and inconsistent.”
“Some applications provide automated forms of rotoscoping and can be utilized with varying results. But most rotoscoping is still done by hand and is the unsung hero of the visual effects industry. Every day new techniques are being developed and refined as technology advances. Some newer packages utilize 3D layout data to help automate some of the process.”
AVOIDING ROOKIE MISTAKES
In the research I did most of the pipelines begin with the roto artist studying the footage, so they can identify the different methods they can use to rotoscope the footage with the best quality and efficiency.
The most common advice I came upon in my research was to treat the same as animating, starting with the most essential key-frames and gradually breaking down the task with smaller in between frames. It is also recommended to start with the frame with the most detailed shape as that’s the one that will need the most control points.
Making everything one big selection is another mistake of ten made, selecting different body parts and keeping the selections small and simple is key to making the job go smoother.
Okun, J. & Zwerman, S. (2010). The VES handbook of visual effects (pp. 551-553, 569-571). Amsterdam: Focal Press.
The Core Skills of VFX (1st ed.). London. Retrieved from http://creativeskillset.org/assets/0000/0236/The_Core_Skills_of_VFX.pdf
The Horrors of Rotoscoping. (2016). Videomaker.com. Retrieved 7 May 2016, from http://www.videomaker.com/article/f06/17164-the-horrors-of-rotoscoping
What is Rotoscoping? | Rotoscoping Tips. (2014). Digital-Tutors Blog. Retrieved 7 May 2016, from http://blog.digitaltutors.com/understanding-rotoscoping-process-every-vfx-artist-know/