(Joint work with Mike Langer, McGill University)
We introduce a new category of image motion called Optical Snow.
Optical snow is the type of motion an observer sees when watching a
snow
fall. Flakes that are closer to the observer appear to move
faster
than flakes which are farther away.
Optical snow also occurs when an observer moves relative to a densely
cluttered
scene.
Note that in both cases, we have dense motion parallax.
Within each image region there can be an arbitrary number of
speeds,
corresponding to objects at different depths.
Despite the apparent complexity of such motion, we still have a rich
perception
of motion and depth structure. Here are two simple image
sequences
showing dense cluttered motion.
| Example 1: Camera moves
horizontally in front of a holly bush. |
Example 2: Camera moves vertically in a field of equal-sized spheres. |
 (click on image to see Mpeg video) |
 (click on image to see Mpeg video) |
Current approaches to motion estimation assume that image motion can be
described
a collection of small image patches, each with a unique image velocity.
The
resulting motion field is referred to as optical flow.
Recently
optical flow models have been extended to allow a small number of flow
measurements
at each point, eg., to allow for transparency effects and multiple
motions
near occlusion boundaries. However, such methods are
inappropriate
for densely cluttered motion, since there can be an arbitrary number of
motions
within any image patch.
We argue that rather than a estimating a unique velocity as optical
flow
methods do, we should fit a linear familty of velocities in
each image
patch. For the case of a camera translating laterally
(ie.,
perpendicular to the viewing direction) this amounts to estimating the
common
direction of image motion. An example of lateral motion is that
seen
be a camera pointing out the side of a moving vehicle. For the
general
case of a forward moving and/or rotating camera, a linear range of
velocities
can be fit within small image patches.
We are currently using frequency-domain methods (ie., spatiotemporal filters) to estimate the direction of motion and the range of speeds. In (Langer and Mann, ICCV'01; Mann and Langer, ICPR'02, Langer and Mann, IJCV'03) we characterize the structure of optical snow in the frequency domain and present algorithms for motion estimation.
NEW: That data and software for these
experiments is available here.