|
A rectangular fish tank, 2'x2'x3', sits on a platform in the center of the room, spotlit from above. Inside swim 6 koi fish, one gold, five white. On the far wall, a video projector displays the changing worldview from the perspective of the gold fish. The computer animation is synchronized to the movements of the fish in the tank using two computer systems working in tandem. The tracking system takes as input a stream of 2-dimensional digital images from three orthogonal cameras: Orange Micro iBot IEEE 1394 units retrofitted with high-quality lenses. The tracking system output is a sequence of estimates of the position and orientation of each fish in the tank, along with a confidence measure for each estimate. The tracking computer is a 2.8 GHz Pentium 4 with 1.5 GB of 533 MHz RAM, running Windows 2000 with Intel's Open Computer Vision libraries, Microsoft DirectX, and custom-written C++ code. Tracking moving targets has a long history in engineering; estimators, observers, and other stochastic filters have been developed to improve accuracy in the presence of noise. Here, tracking is complicated by optical distortions due to the lenses, tank, and water, and by reflections that appear as virtual fish outside the tank. Single-point tracking methods can also fail when fish occlude each other, for example during schooling. The graphics system takes as input the output of the tracking system, the sequence of fish estimates and confidence values. This data is generated at 30 samples per second for each fish. The networking subsystem packets at least three samples at a time and streams raw IEEE floating point values across a local area network using UDP. The graphics system generates, at about 90 frames per second, 2-dimensional images of the tank interior from the perspective of the gold fish. The tank model is based on the current position and orientation of the white fish; the perspective vector is based on the position and orientation of the gold fish. The image is rendered in C++ with OpenGL and GLUT libraries on a Pentium 4 1.6Gz laptop running windows XP with a graphics accelerator card. To maintain a smooth display, the graphics system maintains two fish tank models, one based on the tracking data and one that is virtual, based on simulation. A stochastic proportional controller drives the virtual fish to their real positions. The tracking data model is represented by T_r, a vector of 30 numbers, one 5d vector for each of 5 fish: the first three parameters are non-zero positions, and the last two are orientations about the horizontal and vertical world axes. T_r is updated by the tracking system at 15 Hz. This data is noisy and may be of low confidence. Furthermore, one or more of the fish may get lost, in which case their confidence parameters get set to zero, until they are relocated. The tracking system does some internal checking to insure that individual fish follow semi-continuous trajectories and are tracked consistently, ie, fish don't change their order in T_r. If a problem crops up, the tracking system sets the confidence of the fish involved to zero until it is recovered. The virtual fishtank is represented by T_v, a corresponding vector of 30 numbers. The graphics system renders the live graphics display from T_v, by positioning the camera viewpoint at the head of the gold fish looking forward. Fish in the virtual tank track their counterparts in T_r. We use a Newtonian model to insure the fish swim smoothly and don't collide. When one or more of the fish's confidence in T_r goes to zero (gets lost), the corresponding fish in T_v continue to swim peacefully based on a random process. When a lost fish in T_r is recovered, the corresponding fish in T_v must swim from its current position to the new position, which requires local nonholonomic motion planning. The fish are represented with polyhedral models. Several lighting models: ambient, diffuse, and specular, are combined to simulate the world as perceived by the gold fish.
|