Previous entry: 2005 Broad Street Run: 2,377th Place

Peekaboom: von Ahn and the Future of ESP

by David Sturtz

I attended an interesting lecture today at Drexel given by Luis von Ahn, a Ph.D. student at Carnegie Mellon and inventor of the ESP Game. He spoke about the ESP Game, its potential and uses, and a new game he has designed to aid in developing computer vision.

About a year ago the ESP Game was spreading like wildfire. The game gets users to describe images retrieved from the web by making a game out of the process. Each user is paired with another user and both independently type words describing the same image. When you match your partner you score points (and a potentially valid piece of metadata is discovered). “Taboo words” (i.e. metadata that have been substantiated by several rounds of game play) force the users to come up with new terms.

The data produced by the ESP Game has obvious uses for improving the image search process, but von Ahn mentioned that it also has potential in the areas of accessibility and computer vision. The key finding is the concept of using a game to get work done. An enormous number of hours of human effort have been contributed to the project at no cost other than providing the system. By making the task fun, competitive, and social (in a strange way) people will work for free.

This leads to Peekaboom, von Ahn’s latest game invention, which is sort of like a visual version of Password. Once again players are paired up, but this time only one of the players can see the full image. That player is also given a term (derived from the ESP Game) that she must get the other player to type by progressively revealing portions of the image. There are also pointing and hinting mechanisms to further assist the proccess.

The goal of this new game is to gather data in order to produce better computer vision systems. The data will show (among other things) which areas of the image are important to the identified concept. However, von Ahn’s goal seems to remain finding ways of getting humans to give away their unique computational capacities.

For in-depth info on the ESP Game, see von Ahn, L. & Dabbish, L. (2004). Labeling Images with a Computer Game In the proceedings of ACM CHI 2004. Also, this power-point presentation covering his work on CAPTCHAs, the ESP Game, and into the beginnings of Peekaboom.

Next entry: Yahoo! on Design Patterns