Perception of Non rigid 3D Shapes and Motion

Shape is possibly the most important cue to object identity but many objects deform as they move. Thus, a visual system is required to simultaneously extract the underlying 3D shape, the shape deformations and the global motion of the object. When other cues are not effective the visual system has to rely on motion patterns to achieve this task. Almost all of the studies to date in the human and computer vision have relied on some form of rigidity assumption or on constraints on shape or motion to extract 3D shape. I first show that observers do not need a rigidity assumption to extract 3D shape from motion cues. In addition, observers can not only extract the underlying shape, but also classify deformations using motion cues. I propose a new approach based on scale-space differential analyses of retinal velocity patterns.  This strategy builds on earlier work that linked kinematic differential invariants of the optic flow to specific aspects of 3D shape. I further show that humans are extremely efficient at combining local motion signals with global shape cues to extract object motions for a general deforming object. These analyses provide guidelines on how to construct neurally realistic models for perception of rigid and nonrigid 3D shapes.