Changing Mturk submit button functionality & A new way to prevent duplicate workers on separate HITs

(**Update: I no longer actively maintain this blog. The information below may be out of date, but hopefully is still helpful in getting you closer to solving your problem.**)

Previously, I’ve posted some simple ways to prevent workers from completing your mturk HITs because they already completed an identical one last week or last month, etc. (here, here, and here). Sometimes keeping track of long lists of mturk workers who have completed your previous HITs is difficult. Editing, copying, and pasting a list of 1000 workers that took your survey last time and can’t take it this time takes a while and is prone to mistakes. The best method I can suggest when this is difficult is to start databasing your workers. This is how I manage my Mturk participants, but it requires a decent amount of knowledge of JavaScript, PHP, and MySQL. If that is not your cup of tea, I have been trying to work on an especially simple method using cookies, though it is by definition imperfect.

Cookies are simply small bits of data that a webpage can store on your computer, then later read back. It’s how website “remember” that you are logged in, for instance. Using a small amount of JavaScript, you can set a cookie that is named after your study. The existence of the cookie with that particular name means that the worker completed that study already. When you post a study later and want to exclude people who took your first study, your HIT will have some JavaScript in it that searches for that cookie you created earlier. If it exists, the person took your first study and is excluded, if it doesn’t, they can take your new study. There is no copy and pasting, and no long files full of worker IDs.

However, as I said, it is imperfect. Some users turn off cookies, preventing you from setting the cookie. Some users clear their cookies regularly, deleting the cookie you created. Cookies are computer based, not user based. So a user can go to a different computer and still take your new study. Also, a second person can try taking your study on the same computer and be prevented from doing so because the cookie applies to anyone using that computer, not the specific person. Given the large amount of mturk workers, and the base-rates for the problems I just mentioned, I would guess that this method would prevent about 90% of workers who you don’t want taking your studies from doing so. It may be more or less. The key factors are how many people regularly clear their cookies or complete HITs on multiple computers.

Here is the commented code you can copy and paste into your HIT template window. It first loads jQuery, then adds an event listener to the Submit button, then checks if a cookie exists with the same name as the cookie that gets created when the submit button is clicked. All you need to change is the studyName value to something unique for your study. However, what happens on line 20 is up to you. right now it simply alerts the user that they have taken your study before. Someone more useful would be to prevent any other content from loading and telling them to not accept/return the HIT. Since exactly how it gets implemented can change for HIT to HIT, I didn’t not specify a procedure beyond alerting the user. The cookies are set to last a maximum of 1 year. You can hypothetically make it longer, but likely people have deleted their cookies by then anyway.

Github code here

1: https://code.jquery.com/jquery-1.10.1.min.js 2: 3: //this code assumes jQuery is implemented, though it would not be difficult to do this with straight javascript. .ready() assures the code runs after the submit button is added 4: $(document).ready(function () { 5: var studyName="dm102513";//replace this value with a unique, acceptable cookie name... no spaces and stick with alphanumerics and underscores 6: $("#submitButton").click(function(){ 7: var exdate=new Date(); 8: exdate.setDate(exdate.getDate() + 365); 9: document.cookie=studyName+"=1; expires="+exdate.toUTCString()+"; domain=mturkcontent.com; path=/";//When the user clicks the Submit button, it creates a cookie that you can read later as having already completed this survey 10: }); 11: var i,x,y,ARRcookies=document.cookie.split(";");//variables necessary to read the cookies. Last variables separates all available cookie names into an array we will loop through to find our cookie 12: for (i=0;i