amateur economist
Ever since seeing a presentation by Dolores Labs about Amazon's Mechanical Turk, I've been itching for an excuse to play with the system.
I recently saw a thread that highlights the distinction between expected value and utility. Would you take a more likely but lower payoff instead of a less likely but higher payoff? Similarly, the St. Petersburg Paradox takes the problem to its logical extreme. By constructing a game that has a series of increasingly rare payoffs of increasingly larger size, a game with infinite expected value is created.
So I constructed 21 versions of the questions, varying the size of the dollars as well as the rate of payoff for the second outcome.
For one cent apiece, I sent the questions to be answered by one hundred people each, and collated the results. 2100 questions, three hours, and thirty dollars later, I have my results.
Clearly, people (or at least these Turks) begin to cross over at larger values, reaching equilibrium at around $1,000.
While this isn't the most groundbreaking work, it is nice to be able to generate an experiment and gather the results in the course of an evening and then have the results be so pleasing.
The Mechanical Turk is presented as a way to solve problems that are easily explained to people but difficult to implement for computers, frequently described as "artificial artificial intelligence." However, I think some of the most intriguing uses yet will be to explore the edges of our own uniquely human behavior and self-understanding.
Comments
Perhaps you would be willing to test 50% and 5%, so your expected values would be equal. That way, you could look exclusively at people's risk aversion... Good idea, though, and interesting results.
- dsparks
That's an interesting use of the Mechanical Turk. Do you think that the kind of people who would participate in the Turk would be representative of the general population? I don't have an answer, just asking the question.
- JP
Very cool! Somewhat related, I did a decision survey on those "trolley" moral reasoning problems, way back earlier this year ... http://socialscienceplusplus.blogspot.com/2008/01/moral-psychology-on-amazon-mechanical.html
Next up, see if the Kahneman/Tversky prospect theory models fit your data :)
- Brendan O'Connor
If you're interested and have some time, try re-running the experiment with the questions in reverse, so "would you prefer a 5% chance of losing..."
- Rich
Hey, nice work! I also use Mechanical Turk for small and large experiments (some which I have published as academic papers). It's definitely fun to get an experiment going in an evening, but I also find it useful for bouncing ideas off of a group before implementing something or piloting a pre-pilot protocol (to identify the bugs, etc.)
- Brynn Evans