On Labor Day, 1 September 2014, a group of Turkers, researchers, and I, launched the result of the first collective effort to promote ethical practices on the platform: The Guidelines for Ethical Research on Amazon Mechanical Turk. These guidelines are both a means to guide ethical behavior and to back Turker claims when things go wrong.

After almost a year, 231 Turkers and researchers have signed the guidelines. We have seen researchers reference the guidelines on their tasks on AMT and refer to them in discussions at academic conferences. People from 89 different countries in the world have viewed the guidelines with over 20,000 page views. Here’s what we have learned:

Researchers care about fair payment.

Out of all page views of the guidelines 17% were views of the page: Fair Payment. Making it the most popular page after the front page which has 45% of total views. Basics of how to be a good requester took another 12%, and each other page had 1–2%.

Number of page views for top three sections of the guidelines out of a total of 20,000 views.

We can’t say for sure what percentage of these viewers are requesters. But a look into the most common referral paths can give us a clue:

/threads/we-are-dynamo-guidelines-for-academic-requesters-on-mechanical-turk.26933/

/r/mturk/[…]/im_a_researcher_on_mturk_and_i_want_your_opinion

/r/mturk/[…]/helpadvice_new_requester_looking_for_feedback

/r/mturk/[…]/new_requester_question_about_fair_payment

/r/mturk/[…]/want_to_become_a_requester_dont_know_how_much_i

Visitors of the Fair Payment page came from these sources:

Direct: 49.2%

Google: 23.7%

Reddit: 6.38%

MTurkGrind.com: 5.6%

This could mean that it is a challenge for requesters to figure out what a good/fair payment is. They care about doing it right, and they look for online resources for help. Redditor communities and forums such as MTG are one of the main sources for educating requesters.

There has been a steady stream of interest in the guidelines.

Weekly page views since authoring the guidelines.

Pageviews of the guidelines show that there has been continuous interest. We believe that this is due to Turkers’ active conversations with requesters through forums, subReddits, chatrooms, and email threads, as well as web searches by requesters.

The peak in September 2014 is the publishing date of the guidelines, which we announced in a blog post on crowdresearch.org. The period before that is the authoring period that lasted one month.

Here are the most prominent sources of visitors:

Direct: 37.5%

Google: 16.8%

WeAreDynamo.org: 9.4%

MTurkGrind.com: 8.6%

CrowdResearch.org: 7.5%

Reddit: 3.8%

Note that these sources slightly differ from the Fair Payment page’s sources that represents traffic mostly from requesters. This is a mixture of traffic from requesters and Turkers.

Viewers stayed on the document long enough to read it.

The “average time on page” can be a misleading metric. We expect that most viewers will open the link out of curiosity and close it without reading. Still, the average time that viewers spent on the guidelines were long enough for them to read:

Main page (8,600 page views): 02:35

Fair Payment (3,300 page views): 04:09

Basics of how to be a good requester (2,300 page views): 03:12

A majority of viewers were from the US.

People from 89 different countries visited the guidelines, of these 78% were from the US. Following were: Canada, UK, India, and Germany.