What happened when we gave 13000 work assignments to anonymous internet strangers Taskopus Follow Apr 16, 2019 · 6 min read

Photo by Mario Purisic on Unsplash

We are creating Taskopus - a platform where you can give some strangers the instructions along with some data and expect the work to be done.

The twist is that there is no initial approval of the workers. You can just sign up. Nobody asks for your passport, nationality or address. Not even for payment information. Nobody approves you. You are a perfect stranger.

Another stranger can give you a task and some cryptocurrency for that (Bitcoin Cash (Why?, even if most cryptocurrency freelancers accept Bitcoin)).

As expected, people started to abuse the system from day one.

We expected people to try to create multiple accounts to avoid “per person” limits on tasks. We expected people to be randomly clicking to earn their money as fast as possible.

We even had people come out and tell us how they cheated, though it’s a totally different story.

What we didn’t know was how much should we trust our workers. How many of them would be doing the work well? 90% of workers? 50%? 10%? 0?

We have decided to test that.

The answer was a surprise for us.

What kind of work people are usually paid for?

It’s either the work that is very creative and few people can do it well (think: writers, artists, designers, architects, lawyers) or it’s boring and just needs to be done.

Taskopus is not very suited for the first kind of work, because it requires constant contact between a buyer and a worker. But it’s very well suited for the second type (boring, repetitive, but needs to be done).

The “typos” experiment

So, we have created this spreadsheet:

The “typos” experiment

We call it “the typos experiment”.

So, what exactly is this?

This is a Google Sheet that serves as a basis for this task:

Here is how the task looks to the random worker

Here is a close up of a Google Sheet:

Each row of the Google Sheet will be a separate task for the worker (we call it the “table-based” projects).

As you can see — there are a few columns — “word1a”, “word1b”. One of those is the correct spelling of a random word, one is misspelt. Misspelling can be either at “a” or at “b” spot:

The correct spelling is in blue

That way the worker doesn’t know where the correct word is and has to think. (The worker won’t see the “correct” column, more on that later)

Each typo is a random change of a letter.

We can now create a task template in Taskopus:

The task template

Each row from Google Sheet will be substituted and the resulting task will be given to worker.

We will give up to a maximum of 50 such tasks per worker (so, 250 words). Each task would cost $0.01 in Bitcoin Cash.

Then we will analyse — were the workers clicking randomly or were they actually doing the job. We will do that by comparing the worker’s answer to the “correct” column in the sheet.

In total we have generated 130 rows and will give each to 20 different users.

130 rows * 20 times each row = 2600 total tasks.

2600 tasks with 5 typos each = 13000 typos to check.

At $0.01 per task that comes to $26 for the experiment.

2600 tasks / 50 tasks per person = We can test 52 people or more.

The task limits

Ok, let’s run it.