a Matte in Effect*
This article is
the first in a multi-part series on using Paint* and Effect*
by Discreet Logic to composite a CGI character and special effects
into live footage. My 3d software of choice happens to be Lightwave
3d. Even though Lightwave does not benefit from the close integration
with Effect* and Paint* that 3d Studio Max does, they can still
be used together quite well.
There are times
when you have a 'perfect' place to shoot.... but most of the
time, you have to make do with somewhere that is less than perfect.
In this case, I have a test shot for a client where my location
had a small group of trees that the CGI character would need
to be behind. Since I was in a public park, I couldn't cut down
and/or strip the leaves from the trees so I knew I would have
to create a matte around the trees so that they would seem to
be the middle object. The matte will be used on a plane positioned
at the approximate depth of the trees in the scene.
I captured my footage
on my DPS Perception Video Recorder, using an AD-2500 capture
card that DPS was kind enough to loan me for this and other
reviews. For work like this, it is best to use uncompressed
video if you have the capability, or to capture your footage
at the highest data rate possible if compressed video is your
only option. In my case, I captured the video at 6 mb/s ('Digital'
quality setting on the PVR) and then used the Perception Player
utility to trim it down to the exact footage of the shot. Since
the shot also involved the inclusion of some CG pyrotechnic
effects, we used some filter paper attached to a portable reflector
to provide some interactive lighting for it... maybe its not
perfect, but its better than doing it all in post. Since Effect*
does not directly support the DPS Perception, I loaded the video
footage as a Targa sequence using the Perception virtual file
system. (I was operating under WinNT 4.0 sp3. Note that the
Perception virtual file system does not work under Win95/98.)
Start up Effect*
and begin a new project. Set the length of the scene appropriate
to your footage. In my case, my trimmed footage was 400 frames
long (that's 00:00:13.10 in the timecode world). Import your
footage into he project, deinterlacing it as necessary. Save
the project with the footage imported into it. While doing
these procedures, you may want to turn the Display to 1/2 or
even 1/3 resolution to speed up the previews, but all of my
screen shots here are at full resolution for detail purposes.
Next you need to
go through the somewhat arcane procedure of finding the best
frame to draw your matte on. This would probably be best done
outside of Effect*, but you can flip through frames in Effect*
just fine until you find the appropriate frame. You need to
find a frame where the object you are creating a matte for is
relatively stable, and undisturbed by things like motion blur
or field separations. In my case, it was 00:00:06.22. In this
particular frame, the attention of the viewer was focused on
the creature immediately behind the tree, and the camera was
still for a few frames in each direction. I did this intentionally,
knowing I was going to have to pull a matte of the tree later.
The rough matte
go into the Effects menu -> Mask and select 'Freeform'. The
Freeform Mask window will open. Set the matte to 'replace',
and draw a rough matte around the object you are matting. At
this point, it is sufficient to outline the object and its major
features in a rough fashion, and worry about refining them once
you've created a rough matte around the whole object. Clicking
in position and dragging the mouse, you will create a control
point for the matte and pull out spline handles. Don't worry
about precision quite yet, just get the object or objects blocked
out. Clicking on the start point of the matte will make the
matte into a closed loop.
reduce the intensity of the mask. This will make it possible
for you to see areas on both sides of the spline that may need
to be included in the matte.
Refine the edges
you can carefully go along the edges of the matte and refine
its detail. Clicking along the spline path between control points
will create new control points you can use to refine the detailed
areas. Use the rounding capability of the splines to circle
rounded details instead of creating too many extra control points,
as larger numbers of control points will make the matte render
slower (And at this point, you will see the matte rendering.
Areas outside the matte will be darker due to the reduced masking
intensity. Otherwise they would be black) For creating more
drastic changes in the contour of the spline, press control
before clicking on the point handles and you will 'break' the
handles from the control point, allowing you to send the two
control points in different directions. Save your project often
as you work on the matte, and expect this procedure to take
awhile. Don't worry if you need to come back to work on it later,
just note the frame you are working on, and you can re-open
the matte you are working on.
my matte, since it was trees, I had to make holes inside the
matte where you could see through, because the creature
would need to cast a shadow onto the structure behind the trees,
and because there were areas in the tree where you could see
through it. In order to do this, create additional mattes on
top of the first, and set it to Subtract from the previous matte.
Invert is important otherwise you end up with just the area
around the hole as your matte, instead of a hole IN your matte.
You can still use the reduced intensity trick to refine the
edges of these as well, setting the intensity of the 'holes'
to lower just like you did with the original. In some places,
because of the nature and positioning of the leaves, it looked
better if I feathered the edges of the mattes by a few pixels.
your matted object remains still in the scene, then you are
done. If not, then you need to track the motion of the matted
object through the camera's view.
2d tracker is fairly powerful and resilient, and I have found
it to be able to track footage that After Effects cannot. Discreet
simply converted the motion tracker from their higher end tools
like Flint* (now Effect* 5.5 for IRIX) and Flame* into something
useable within Effect*'s environment. They even added a few
things that aren't in the Flame* version (like multithreading,
the new magnification modes for contrast and edge detection,
and the result preview window). We will be using this 2d tracker
to track the movement of the matte object (the tree) through
the scene and use that data to move the matte appropriately.
At this point, the Control Points button should be selected
for your mask. You need to click on the Object button next to
it so you can apply the Tracker to the mask. The Tracker in
Effect* is context-sensitive, meaning that you can apply it
to only a few things in a scene if you have them selected. This
includes the capability to select a few control points of a
mask and track them to the edge they are applied to if you so
Select a tracking point
You need to find
a tracking point, preferably one within the object you are matting.
You need to find two fairly high contrast tracking point that
remains consistent throughout your animation. In my case, the
upper trunk of the larger background tree was the location of
the points I chose. The first point (the red tracker box in
the above picture) is for tracking position data, and the second
(it will have a little '2' next to it, it is white above) is
for tracking Scaling data. I used a longer and narrower reference
point for the second tracker because the changing width of the
tree gave the best reference for scaling data. I extended the
reference box just beyond the edges of the trunk since the tracker
would be using the edges of the trunk for most of its tracking
information. It will also be helpful to temporarily turn off
the mattes for this procedure. To do this, go to the timeline
editor and click the arrow next to 'effects' under your footage.
You should see your mattes listed in the order you created them.
Click the yellow box next to the name of each matte (it looks
kind of like a square LED) and it will go gray, turning off
the matte. Turning them all off will greatly speed up the tracking
process. Depending on how good your tracking point is, you may
need to tweak the Tolerance setting to get a more accurate track
of your object. In my case, I cranked Tolerance down to
50% and achieved a more precise track.
Make sure the tracker
is set to track Position and Scale, set to Relative mode. Select
Tracker one by clicking on it, then track the duration of the
frames for your matte. You should see the matte moving with
the tracker, thought the matte may fall out of scale with the
feature it is tracking. If the tracker loses your object too
much, you will want to go back to the start frame of the track,
reset the tracker, and then lower the Tolerance further. You
may need to change your tracking point, or manually set the
tracking for the points where it loses the reference point.
After you have a good track for the primary matte, you need
to export the track data for Tracker 2 so you can then re-load
this track data for the subsequent mattes.
Importing and exporting tracker data
mattes (the 'holes' in your matte), set up a tracker to
track position, but instead of tracking the matte, simply import
the tracking data from the main matte, and the sub-mattes will
move with the larger matte. Do this for every sub matte and
they will all move together. This works especially well on my
matte since some of the sub-mattes move off screen.
To be fair, I will
tell you you may need to go in at this point and hand-tweak
the position, scale and rotation of your matte or sub mattes
to account for things like lens distortion (or even a light
breeze). But, your tracked mattes should still be reasonable
If you've followed
the instructions, as well as taking some of my suggested actions,
(including tweaking where necessary), you should have a tracked
rotoscoped matte for your use. You can output it and use it
for a variety of things, in my case, I am exporting just the
Alpha channel and using it as an Alpha channel for the main
footage mapped onto a large polygon positioned in from of the
camera in Lightwave. This means I can let my character cast
its shadows onto the objects in the scene without its shadows
falling across the trees it should be 'behind'.
Click the image
to see a small mpeg movie
Carroll is a freelance animator, illustrator and production
designer located in Richmond, VA. He can be reached at [email protected].