Every Truth-Seeking Institution We have Is Broken

We need a new institution whose sole incentive is the truth.

The media, ostensibly, seek the truth.

News organizations are supposed to publish only what is true. They are supposed to advance the truth in society.

The media’s incentives have nothing to do with the truth. They are rewarded only for attention. Being truthful is useful to them only if it gets them more attention. Being dishonest is only bad for them if it causes them to lose attention. They have no direct incentive to be honest.

Like almost every agent does, the media often follow their incentive over their mandate. They publish stories that aren’t true, but are emotionally engaging, and then publish weak retractions later. They pass on someone else’s lies — according to unnamed government officials — and, later, wash their hands of responsibility. Or, when caught, they equivocate, and say, “what I really meant was..”

Dan Rather published forged documents, and he’s still a public figure, blathering on about the death of experts. There are plenty of good journalists, though. How many better journalists are out there, that we’ve never heard of, because our brains are still wasting space affording Dan Rather status as a blue-checked public “expert”?

How many honest journalists have low status, and less attention, so that Dan Rather gets to hold on to his public name, and public status, despite helping advance lies? How many other journalists are at the top because of their ability to manipulate an incentive system, rather than their ability to publish truths which embarrass powerful persons? How many journalists followed their mandate over their incentives, and now have less attention for doing so?

Because of the broken incentive system in the news media, the public is robbed three times: We believe things that aren’t true, we unjustly elevate people who told us those false things, and we under-promote people who are more focused on the truth than on advancing their careers.

The academic publishing industry, ostensibly, seeks the truth.

Academic journals are supposed to publish only what is true. They are supposed to advance the truth in society.

The incentives have nothing to do with the truth. Journals are rewarded for selling subscriptions. Academics are rewarded for publishing papers in top journals that sound interesting or engaging. Being truthful is only useful to them if it gets them more citations, more publications, more government funding, and more status in the academic world. Being dishonest is only bad for them if it causes them to lose academic standing or the ability to publish.

Like almost every agent does, academic publishers often follow their incentive over their mandate. They publish papers with claims that aren’t true, but posses statistical validity. They do one ‘experiment’ and then search through hundreds of hypothesis after the fact, to see which hypothesis are statistically congruent with the experiment.

We are seeing a reproducibility crisis in science. Researchers are rewarded much more for new findings than they are for reproducing existing work, much less for failing to reproduce an existing finding. There is little incentive to point out that the emperor is naked, and so few researchers bother to do it.

There are plenty of great researchers out there. How many of them languish unknown in their fields because they have not focused on the game of publishing with citations, and are instead focused on actually advancing the truth?

Because of the broken incentive system in academic publishing, the public is robbed three times: We believe things that aren’t true, we unjustly elevate people who told us those false things, and we under-promote people who are more focused on the truth than on advancing their careers.

The justice system, ostensibly, seeks the truth.

Juries are supposed to determine what the truth is, in complex matters with real human consequences. They are ostensibly aided in this pursuit by prosecutors, defense attorneys, and judges.

No one in this system has an incentive that is aligned with the truth.

Jurors lose money being on juries. They must spend time away from loved ones and family members. They are not rewarded at all for pursuing the truth, aside from the scant possibility for a clean conscience, which might still escape an honest, self aware person who knows they don’t have all the facts.

Prosecutors are rewarded for their convictions. They have very little incentive to bring forth evidence that would exonerate a suspect. This is a legal requirement for them, but prosecutors rarely face prosecution for hiding evidence. This man withheld evidence that would have kept an innocent man from prison. The innocent man was in jail for 24 years. The guilty man got 10 days.

The law required this prosecutor to turn over evidence. His incentives weren’t aligned with his legal duties, and like so many others, he chose to follow his incentives over his duties.

Because of the broken incentives in the justice system, the public is robbed uncountably many times: We believe things that aren’t true. We jail innocent people. We let guilty people go free. We lose our faith in the integrity of the system ostensibly designed to protect us, because it fails to protect us from its own.

Misaligned incentives are extremely dangerous for societies that desire justice. Incentives create power. In a just society, power is mandated by the masses — and the masses coordinate this mandate by some system. Any flaws in that system will give more power to unjust people, and less power to people who prioritize doing the right thing over following their incentives.

How can any of this be fixed?

The media, the academic world, and the courts and all have their own systems for deciding what is and is not true.

The media are supposed to check sources, corroborate facts. There are standards and practices developed over long periods of time. The academic world is subject to peer review, which is supposed to find problems. Experiments are supposed to be widely repeated before their findings are taken as truth. Jurors are supposed to consider only certain facts; jury selection processes and rules of evidence admission adhere to strict standards that are supposed to prevent bias and misrepresentation.

These systems were created over hundreds of years. It would be arrogant in the extreme for us to conclude we could do better by throwing them out and creating a brand new system. I think systems like this — systems which pursue the truth, or claim to — will come and go, with their power rising or falling, in the long term, in accordance to their ability to pursue the truth. I think it is only in the short term that dishonesty pays off; in the long run, society will lose trust in bad-behaving institutions, throwing out the good players with the bad.

Any system that we build, if it makes claims about what is, or is not true, would immediately gain the distrust of anyone who disagrees with the systems’ evaluations. A system that only speaks the truth is worthless if people don't trust it. Is this problem solvable?

Yes, I think it is — but we have to solve the right problem.

Build A New Layer on Top: A Record of Public Statements

I disagree with what you say, but I agree that you are saying it now.

We don’t have to agree on empirical reality. We aren’t going to, and it isn’t desirable to try to force this to happen. We can agree, however, on what people are saying, and especially on what they have said.

Anyone who said, in early 2008, “The economy is on fine footing, no problems here” — I would have a hard time trusting them on the state of the economy. Before I listen to any economic advice from someone, I want to know what they said about the events of 2008 before they happened. If they said everything was fine, I’m much less interested in what they have to say — unless they’ve talked a lot about why they were wrong, and then made some accurate predictions since that time.

The public record doesn’t have to say “This person was wrong” — it just needs to say “This person said exactly these words, at this time”. I can make my own decision about empirical truth, and so can you. We should all be doing this, rather than deferring to experts, as much as we possibly can. When we trust experts, it’s better to do so explicitly, and aware that we are doing so, rather than implicit.

Social truth must be socially constructed. “What was said” cannot be measured and empirically verified, as the laws of physics can. It can only be agreed upon, socially. We need to make sure that social truth can’t be deconstructed and torn down at will.

It will be much, much easier for us to agree on the exact words past politicians have used, than it will be for us to agree on the outcomes of choices or on the results of policies. Agreement on the exact words used can help us come to agreement in the future, about who was wright and who was wrong. It can also help us track agreement on who is weaseling out of questions and who is answering them truthfully.

All decisions about what is true will ultimately have to be made by individuals themselves, but decisions about “what was said” — and what was said about those statements — those we can agree on socially, as the statements happen.

All public statements — by politicians, journalists, and leaders — should be entered into a permanent public record. Claims should individually be recorded at the time they are made.

Anyone should be able to validate a claim, by adding their own statement. A politician says “The situation will improve”, and ten journalists can then sign their own statements saying, “yes, I agree, this politician said this statement.”

I can they go back in the future and decide on my own whether or not the statement maps onto reality. This giant tangled graph of statements bears some resemblance to the ideas behind “blockchain technology” (now a buzzword), including ideas like “tangles” and “hash graphs.”

If I think a politician has dodged a question, I’ll say so in the graph. If i’m on social media, I can use what I think are clear cases of dishonesty behavior as filters to decide who it makes sense for me to interact with. Filter bubbles already exist, and I don’t think we can get rid of them. What we CAN do is signal boost people willing to find lies in what is ostly truth, and find the truth in what is mostly lies.

A record of public statements and our dispositions towards those allows us to do that. It allows persons interested in nuance and subtlety to talk quietly with each other, while loud-mouths can continue shouting their prior beliefs at each other with increasing vehemence.

A public record of statements and our dispositions towards them allows intelligent members of the red tribe to find the best arguments by the blue tribe and consider them seriously, rather that encouraging the loudest members of each tribe to line up and represent the worst of their side has to offer.

Such a public record allows people who can be taken seriously by both tribes to have have this information known, publicly. I can talk to communists, libertarians, statists, and anarchists, and generally understand where they are coming from. I can usually empathize with their dispositions, even if I don’t agree.

I can get usually get a discussion to a point where we are raising key philosophical assumptions behind our worldviews, rather than shouting about what we think the implications of the other person’s beliefs are. I’m proud of this. Yet it doesn’t get me nearly as much of an audience as telling people they’re full of shit. That mismatch in incentives isn’t a good recipe for a healthy public discourse.

A New Incentive

A public record like this creates a new incentive: Anything you said that, years later, turns out to be false, will be permanently recorded. Incentives structures can come and go. I think they’ll grow and die in accordance with their ability to reflect the truth, just like all other organisms. Stating something authoritatively and then being wrong will harm your reputation, even if this happens decades later. The astute, yet arrogant Nassim Taleb would call this incentive ‘skin in the game.’

If all of your public statements are recorded, stating anything with certain, as the absolute truth, becomes risky. Any time Paul Krugman predicts some technology will fail, his twitter feed is filled with images of Fax Machines, because he predicted the internet would have no more impact on society than the fax machine.

That’s all well and good, but ideally we could find columnists who made unlikely predictions that turned out to be correct, and listen to them instead.

Making a statement authoritatively, with uncertainty, and being wrong in public should end your career as a public figure. There are too many good, promising people who have no shot at power, because there are only so many spots at the top.

There is an upper bound on how Famous Names’ there can be. This upper bound is imposed by the fact that we think using pieces of meat, rather than on the abilities of humans themselves. We hate congress, yet most of them end up back in there. There isn’t nearly enough churt at the top.

If you think this would make anyone with power extremely hesitant to say anything with certain, that’s exactly the outcome we all need. I’m certain of this — and if i’m wrong, I should probably not speak again on this issue without you knowing how wrong I was.