In Mechanical Turk, how do you limit to one HIT pe

Question

I know from communication with Mechanical Turk workers that there is a way to limit the number of HITs a specific worker can complete, but I cant figure out how to do it. Any help would be greatly appreciated!

Answer 1

If you want x unique users, make the number of assignments per HIT equal to x in the "Design" section. Then, when loading your csv file, only put one HIT in the file.

Answer 2

Just to clarify if you want only unique workers to complete a single hit all you have to do is to set the max assignments to however many unique workers you want and the Mechanical Turk by default will assign only unique workers to that hit.

Now if you want unique workers across multiple hits then you have to have to get fancy and use an external question coupled with a script that logs worker id's ext...

Answer 3

I've developed a script that mostly solves this problem. The main idea is to check the worker ID against a database and then hide the HIT if the worker has already completed a related HIT.

So that you don't need to host your own database server, I've made my script available as a (free) service at: http://uniqueturker.myleott.com. Please let me know if you have any trouble using the script, or if you have any questions or suggestions.

I'm also including the script here, in case you wish to use it with your own URL/database. If you go that route, you'll need to set up a web interface to your DB that takes a worker ID and returns "1" if the worker is allowed to work on the HIT and "0" otherwise. Then you'll just replace "YOUR_URL" below to point to that web interface:

<script type="text/javascript">
 (function() {
  var assignmentId = turkGetParam('assignmentId', '');
  if (assignmentId != '' && assignmentId != 'ASSIGNMENT_ID_NOT_AVAILABLE') {
   var workerId = turkGetParam('workerId', '');
   var url = 'http://YOUR_URL/?workerId='+workerId;
   var request = new XMLHttpRequest();
   request.open('GET', url, false);
   request.send();
   if (request.responseText != '1') {
    document.getElementById('mturk_form').style.display = 'none';
    document.getElementsByTagName('body')[0].innerHTML = "You have already completed the maximum number of HITs allowed by this requester. Please click 'Return HIT' to avoid any impact on your approval rating.";
   }
  }
 })();
</script>

Answer 4

Create a hit that really is a single HIT but use javascript to dynamically change the HIT every time it is viewed. Then when posting the HIT set "Number of assignments per HIT" to the number of participants you want. In this way you will only get unique participants.

Depending on the the type of HIT you want to run this is a technique that might work for you. I have used it to randomize the stimuli shown to the participants.

Answer 5

You can also do this with external questions. I run psychology experiments on mechanical turk, so I need unique participants. In addition to requesting that workers only perform one HIT, I use a python script to verify uniqueness. My HITs all run a cgi script to produce the question. The script consults a log file, and if the worker has previously accepted a related job, it politely informs them that because I need unique participants, the HIT won't be available.

I used to do this with qualifications, but found that it really limited participation.

Answer 6

psiTurk (MTurk behavioral research app) automatically prevents workers from repeating HITs using an approach similar to @david-l. Disclosure: I'm one of the developers on the project.