Windows of the soul…mirrors of the world?

“Two Columbia University scientists have come up with a computer-based way to extract detailed information from the fleeting images of the world mirrored on the curved surface of the eye.”

Shree Nayar, a professor of computer science and co-director of the Columbia Vision and Graphics Center, took high-resolution photographs of people that include their eyes and, in particular, the transparent part of the eye called the cornea. Then, with a postdoctoral researcher, Ko Nishino, he devised computer algorithms that analyze the images reflected in these natural mirrors, revealing a wealth of information.

The system can automatically recover wide-angle views of what people are looking at, including panoramic details to the left, right and even slightly behind them. It can also calculate where people are gazing – for instance, at a single smiling face in a crowd.