Author | Conference | Journal | Organization | Year | DOI
Look for results that meet for the following criteria:
Academic
Publications
Task search in a human computation market

Task search in a human computation market,10.1145/1837885.1837889,Lydia B. Chilton,John J. Horton,Robert C. Miller,Shiri Azenkot

Edit
Task search in a human computation market   (Citations: 3)
BibTex | RIS | RefWorks Download
In order to understand how a labor market for human computation functions, it is important to know how workers search for tasks. This paper uses two complementary methods to gain insight into how workers search for tasks on Mechanical Turk. First, we perform a high frequency scrape of 36 pages of search results and analyze it by looking at the rate of disappearance of tasks across key ways Mechanical Turk allows workers to sort tasks. Second, we present the results of a survey in which we paid workers for self-reported information about how they search for tasks. Our main findings are that on a large scale, workers sort by which tasks are most recently posted and which have the largest number of tasks available. Furthermore, we find that workers look mostly at the first page of the most recently posted tasks and the first two pages of the tasks with the most available instances but in both categories the position on the result page is unimportant to workers. We observe that at least some employers try to manipulate the position of their task in the search results to exploit the tendency to search for recently posted tasks. On an individual level, we observed workers searching by almost all the possible categories and looking more than 10 pages deep. For a task we posted to Mechanical Turk, we confirmed that a favorable position in the search results do matter: our task with favorable positioning was completed 30 times faster and for less money than when its position was unfavorable.
Published in 2010.
Cumulative Annual
View Publication
The following links allow you to view full publications. These links are maintained by other sources not affiliated with Microsoft Academic Search.
    • ...For example Chilton et al. [4] indicated that most workers use two of the main task sorting mechanisms provided by AMT to find and complete tasks (“recently posted” and “largest number of HITs” orders)...
    • ...(A common reason for this behavior is for the HIT to appear in the first page of the “Most recently posted” list of HITgroups, as many workers pick the tasks to work on from this list [4].) Although it is still unclear what dynamics causes this behavior, the analysis by Barabási indicates that priority-based completion of tasks can lead to such powerlaw distributions [2]...

    Panagiotis G. Ipeirotis. Analyzing the Amazon Mechanical Turk marketplace

    • ...For example, how would the worker experience on Mechanical Turk be different if workers knew requesters’ rejection rates, or the effective wages of HITs? This has been explored in online auctions, especially eBay, but only tentatively in human computation (e.g., [6], which examines task search)...

    M. Six Silberman, et al. Ethics and tactics of professional crowdwork

    • ...Chilton, Horton, Miller, and Azenkot (2010) showed that the criterion most frequently used to find HITs is the “recency” of the HIT (when it was created), and this has led some to periodically add available HITs to the job in order to make it appear as though the HIT is always fresh...
    • ...The offered wage is not often used for finding HITs, and Chilton et al., (2010) found a slight negative relationship at the highest wages between the probability of a HIT being worked on and the wage offered...
    • ...In fact, Chilton et al. (2010) found that the second most frequently used criterion for sorting is the number of HITs offered, since workers look for tasks where the investment in the initial overhead will pay off with lots of work to be done...
    • ...However, recent research on the behavior of workers (Chilton et al., 2010 )d emonstrated that workers had a reservation wage (the least amount of pay for which they would do the task) of only $1.38 per hour, with an average effective hourly wage of $4.80 for workers (Ipeirotis, 2010a)...
    • ...Since the default order in which workers view HITs is by time of creation, with the newest HITs first, a new HIT is seen by quite a few workers right after it has been created (Chilton et al.,2010)...
    • ...Furthermore, since the reservation wage of workers is only $1.38 per hour (Chilton et al., 2010) (with an effective wage of roughly $4.80; Ipeirotis, 2010a), the subjects tend to be less comparable to or expensive than subjects recruited through other means...

    Winter Mason, et al. Conducting behavioral research on Amazon’s Mechanical Turk

Sort by:

Citations (3)

Share this on