@InProceedings{VonahnLiuBlum2006,
author = {von Ahn, Luis and Liu, Ruoran and Blum, Manuel},
title = {Peekaboom: A Game for Locating Objects in Images},
journal = {Proceedings of the SIGCHI Conference on Human Factors in Computing Systems},
year = {2006},
month = {April},
pages = {55-64}
}
Overview
The article “Peekaboom: A Game for Locating Objects in Images” by Luis von Ahn, Ruoran Liu and Manuel Blum revolves around Peekaboom, a web-based game designed to help computers locate objects in images. It was designed with the objective of collecting training and testing data to create more accurate computer vision algorithms, this online game tries to advance computer science while attempting provide people an enjoyable and satisfying game experience.
Understanding Peekaboom
The game consists of two roles: Peek and Boom. Two randomly chosen players participate by each having of one of these two respective roles. Peek starts off with a blank screen and Boom with a screen with an image. The goal of Boom is to reveal parts of the image to Peek by clicking on an area of the screen while the objective of Peek is to guess the word associated with the image by entering a guess. Peek can enter guesses as to what is the word while Boom can view these guesses and indicate to Peek whether he is “hot” (e.g. the guess is related to the word to be found) or whether he is “cold” (e.g. the guess is not related to the word to be found). When correctly guessed the two participants reverse roles and then proceed with a new image-word pair. If the image-word pair is too difficult players have the option to skip the current image-word pair to go to the next one. Boom has the option of using a “ping” to help Peek. A ping basically makes ripples appear on a specific area of the partially revealed image on Peek’s screen to indicate to Peek that he should focus his guesses on the object in that area of the image (e.g. ripples on the nose to indicate the word relates to the nose specifically not the whole face). In addition, Peek has the option of demanding a hint from Boom. In terms of points, Peek and Boom get points equally regardless of who guesses, hence focusing on collaboration rather than competition.
Type of Data Collected
The images and labels used in this game consist of data collected from a previous experiment with an online game entitled “ESP Game”. This game involved using labels to describe the content of images. That experiment focused on content of image whereas “Peekaboom” is about the location of objects. In addition, Peekaboom collected data on how a word relates to an image, the pixels needed to guess a word, the pixels inside the object, the most salient aspects of the object and the elimination of poor image-word pairs. Therefore, when multiple players go through the same image various elements of data can be combined effectively to give accurate and useful annotations for computer vision.
Preventive Measures
Aware that the data collected can be tainted by participant collusion, the designers of this web-based game incorporated various measures to prevent it from happening. These include a player queue where participants are paired off in matching interval of ten seconds; IP address checks to prevent players attempting to pair with themselves or with people in their geographical proximity; seed images to prevent automated players from corrupting the data collected; and limited freedom to enter guesses to prevent communication between participants.
Evaluating User Statistics / Results
In evaluating the user statistical data the researchers focused on whether Peeakaboom provided users with an enjoyable and satisfying experience and whether the data produced was accurate. In terms of user satisfaction, the researchers indicated that over 90% of the people played more than once and every player in the top scores list played over 800 games or 53hrs in total. In addition, user comments included statements such as “extremely addictive” or “this game is like crack”, suggesting that users enjoyed and were satisfied with their overall experience. In terms of data accuracy, the researchers conducted two experiments with the data collected from 50 image-word pairs. The first compared volunteer-generated bounding boxes for objects within an image to those calculated in Peekaboom while the second experiment consisted of volunteers rating whether the “pings” were inside or outside the object when initiated by the participant (e.g. Boom). In both cases, Peekaboom proved to fairly accurate in term of the collected.
Future Work
Overall, this suggested that the researchers were successful in designing a web-based game that could collect useful and accurate data to advance computer vision while providing the end-user with an entertaining experience. The researchers stress that the concept of problem-solving through online gaming can be generalized to various computing issues and therefore see great potential in future work of human-computer interaction and Artificial Intelligence.