Token curated registries (TCRs) are an economically incentivized mechanism that promises to address a serious need in decentralized systems: to provide a reliable signal of quality on something you can’t directly observe. The short summary of how they are purported to work goes something like this: people bet on or against quality, value, or truthfulness of an item from some list. The bet is resolved by majority: the side with the highest amount of bets wins. The premise is that the majority will always bet truthfully. Some examples of questions that TCRs are intended to answer: Is John a good roommate? Is Guggenheim a good museum? Is Neptune Diner worth having lunch at?

Here is a curious definition of trust for your consideration: trust is defined as a state where subjective information can be reliably assumed to be objective. Bob tells you that Alice is married. If you trust Bob, you will believe him and will not ask Alice out. If you don’t trust Bob, then you might decide that Bob simply wants to eliminate a competitor.

No wonder that TCRs are a holy grail of decentralized computing. It is all about trustless systems — those where information can be verified directly as objective, without requiring trust. Bitcoin is like this, you don’t need to trust anyone to know that your bitcoin is valid. Why couldn’t you use the same or similar process to verify whether Alice is married?

TCRs have a number of subtle but significant problems. In my view, today’s designs are applied far too broadly. In many domains in which TCRs are being built they can’t possibly work and I personally don’t see a path to fixing them. While I am not saying that a better design is impossible, doing so must start with the understanding of what is wrong with them in the first place. This is what this post is about.

Let’s start from the beginning. TCRs take their roots from thinking on oracles, mechanisms that record publicly observable data onto the blockchain. Publicly observable here means that data that can be cheaply and easily verified by any user of the network. Examples of publicly observable data: air temperature in New York, outcome of the French election, which team won the Super Bowl. Oracles are also token-based and the game is the same: the beliefs of the majority win the race to objectivity. Before understanding why TCRs don’t work, let’s understand why and under what conditions oracles do.

First, let’s examine in details the workings of the oracle. The question is asked, say, “what was the temperature in New York yesterday at noon?” A bounty in the form of some valuable tokens is posted for the answer. The answer is easy to find — just look at or scrape one of hundreds of the websites that report on the weather. Bob (or Bob’s Python script) looks at weather.com, submits the accurate answer “57” to the oracle and posts a bet, in the form of tokens, that his answer will win. If nobody bets against Bob, then Bob collects the bounty and gets his deposit back. The oracle records “57” as the objective value. If somebody agrees with Bob and also reports “57”, then that person will collect half the bounty and Bob will collect the other half (again, deposits returned untouched). More people could vote “57” and this will only stop when the size of the bounty per person starts approaching the transaction fee you have to pay to collect it. So the bounty could be really small, say, 10x the transaction fee.

Why does this work? Let’s see what happens when Alice decides to game the system and provide the wrong answer. She sees that Bob made a deposit of ten dollars in support of the answer “57”. She decides to steal both the bounty and Bob’s ten dollars. In order to do this, she makes a deposit of 20 dollars and submits the answer “-12,000,000”, just to be funny.

Alice will lose, because the moment she bets 20 dollars on the wrong answer, she, effectively, increases the bounty by 20 dollars to those people who report the accurate answer. It would make total sense for someone else (or a group) to come in with another 100$ and support Bob, as that will give them significant profits. If Alice colludes with Vera and Vera bets 100$ on Alice’s “-12,000,000”, this will attract even more players to contribute on the side of Bob.

Oracles work because publicly observable information provides cheap and effective coordination signal to honest players, whereas to rally the majority around the false answer is incredibly hard. Imagine you are Tom, and you see that the New York weather question has two answers “57” and “-12,000,000”, and that, just for the sake of illustration, the value of the bets on each side is the same. Is there a strategy for him to profit from this situation? Yes! He should bet on “57”! Why? Because he knows that “57” will win with overwhelming likelihood. Why? Because this is the answer that’s observably true and so anyone in Tom’s situation will likely bet on “57”. People intuitively know these things. If you want to dig into this even deeper, please see the Appendix.

Let us now notice some assumptions we have made above. The most important assumptions are that: (1) the objective answer exists, (2) it is publicly observable, and (3) it is very cheap to observe it. Let’s imagine that one of these is not true.

Say, the question is: “What is the average value of all stock price ticks for all stocks on NYSE whose ticker symbol begins with ‘S’ in the period between January 21st, 1973 and March 11th, 2003?”. The objective answer can be derived by anyone, and given some terabytes of disk space, and a little python coding, anyone can find this number and report it to the blockchain.

But the cheapness assumption is false. And so this question would certainly not work in an oracle. Why? Let’s say you are Tom. You see a 20000$ bet on the answer 2.35 and a 200$ bet on the answer 7.33. What do you do? Problem is, you know that 200$ is not enough incentive for you to find the answer for yourself, but until you do, you don’t know if 200$ is all you get. In other words, you are not incentivized to even begin to participate in this process. Consequently, Alice, who was the one to bet 20,000$ against Bob’s honest 200$ will likely win this game.

The problem with non-public information is similar to the problem with expensive information. Say, the question is “Does Boris have a Dali print hanging on the wall of his kitchen?”. Problem is, Boris is the only one who knows the correct answer, so Boris is the only one who can provide the coordination signal to Bob and Tom. But does Boris have to tell the truth? No, he can say anything he wants, nobody can verify his answer. The game breaks down, because the truth, not being publicly observable, fails to be the coordination signal that is required for the oracle to work.

Last but not least is the assumption of objectivity. Say, the question is “Is smooth peanut butter better than chunky?”. Say, Bob, a Columbia undergrad, likes smooth peanut butter. Should he bet on “smooth”? There is absolutely no way to expect any concrete outcome and ensure the safety of your bet, because, once again, the coordination signal is missing. (Interestingly, if the question were “Is Bitcoin the best cryptocurrency” huge and totally random wealth redistribution would ensue.)

This takes us back to TCRs, which are just oracles, but in a different context. The purpose of TCRs, unlike oracles, is to collect and curate information that is otherwise hard to ascertain. Whereas oracles are a formal tool to benefit such use-cases as prediction markets and are specifically designed to work on data that is easy to ascertain, TCRs curate information for the benefit of people. They purport to somehow extract the truth of some hard to observe fact from the oracle-like majority betting.

But I have demonstrated above that without the requirements of objectivity, publicity, and cheap observability this type of economic game simply has no reason to converge on a truthful answer. If the TCR is operating in a domain in which these requirements are met, the signal of quality it generates will be stronger (see, for example, Messari), but, and this is where most TCR proposals are made, if these requirements are not met, the TCR will not be useful at all.

It is surprising how often I read about TCRs these days as if they were a done deal and a solved problem. To my knowledge there isn’t a single one that has been shown to generate a good signal and to the best of my ability to comprehend them, I am unable to see why they should.

So there…

Appendix — Coordination according to David Lewis and Thomas Schelling

The best account that I know of for how coordination signals work was given by David Lewis in his “Convention, a Philosophical Study” (See first chapter here), where he summarizes Schelling’s experiments related to coordination without communications. This is so rich in information, that I will put the whole text here for your pleasure.

Schelling has experimented with coordination problems in which the agents cannot communicate. His subjects know only that they share a common understanding of their problem — for instance, they may get instructions describing their problem and stating that everyone one gets the same instructions. It turns out that sophisticated subjects in an experimental setting can often do very well — much better than chance — at solving novel coordination problems without communicating. They try for a coordination equilibrium that is somehow salient: one that stands out from the rest by its uniqueness in some conspicuous respect. It does not have to be uniquely good; indeed, it could be uniquely bad. It merely has to be unique in some way the subjects will notice, expect each other to notice, and so on. If different coordination equilibria are unique in different conspicuous ways, the subjects will need to be alike in the relative importance they attach to different respects of comparison; but often they are enough alike to solve the problem. How can we explain coordination by salience? The subjects might all tend to pick the salient as a last resort, when they have no stronger ground for choice. Or they might expect each other to have that tendency, and act accordingly; or they might expect each other to expect each other to have that tendency and act accordingly, and act accordingly; and so on. Or — more likely — there might be a mixture of these. Their first — and higher — order expectations of a tendency to pick the salient as a last resort would be a system of concordant expectations capable of producing coordination at the salient equilibrium. If their expectations did produce coordination, it would not matter whether anyone really would have picked the salient as a last resort. For each would have had a good reason for his choice, so his choice would not have been a last resort. Thus even in a novel coordination problem — which is an extreme case — the agents can sometimes obtain the concordant expectations they need without communicating. An easier, and more common, case is that of a familiar coordination problem without communication. Here the agents’ source of mutual expectations is precedent: acquaintance with past solved instances of their present coordination problem.

The most important aspect of this is that, according to Schelling, what coordinates people around an answer is not truth per se, but the answer’s salience, which permits coordination without communication. The reason this works for oracles is because in oracles coordinating around the objectively truthful answer doesn’t require communication, but coordinating around the false answer does, and so becomes overwhelmingly hard in a large random crowd of participants.