The world is pretty awful, in a number of ways. More importantly, the world is, in an important way, undershooting its potential. 

Many of our problems are directly downstream of inadequate institutions, which incentivize those problems, or at least fail to incentivize their solutions. For virtually any part of the world that sucks, you can point to poor institution design that is giving rise to sub-adequate outcomes. [ 1 ]

For many of these broken institutions, there exist proposed alternatives. And for many of those alternatives, we have strong analytic arguments that they would improve or outright solve the relevant problems.

• A Land Value Tax would likely solve many to most of the human problems of inequality (those that are often asserted to be a consequence of “capitalism”, such as many people working bullshit jobs that they hate, for their whole life, and still being too poor to have practical freedom)
• There’s a laundry list of innovations that would solve the incentives problems of academia that lead to some large fraction of published research being wrong.
• Prediction markets could give us demonstrably calibrated estimates of most questions of fact, that take into account all available info.
• Approval voting is one of a number of voting systems that avoid the pitfalls of first-past-the-post elections. But we still use first-past-the-post and so we live with politicians that almost no one likes or supports.

The world is broken. But it largely isn’t for lack of ideas: we (or at least someone) know how to do better. The problem is that we are blocked on implementing those good ideas.

As Yudkosky puts it in Inadequate Equilibria…

Usually when we find trillion-dollar bills lying on the ground in real life, it’s a symptom of (1) a central-command bottleneck that nobody else is allowed to fix, as with the European Central Bank wrecking Europe, or (2) a system with enough moving parts that at least two parts are simultaneously broken, meaning that single actors cannot defy the system. To modify an old aphorism: usually, when things suck, it’s because they suck in a way that’s a Nash equilibrium.

But that doesn’t mean that there’s some fundamental law of nature that those solutions are untenable. If the proposed alternative is any good, it is also a stable equilibrium. If we moved to that world, we would probably stay there.

Or as Robin Hanson says of prediction markets in an interview,

I’d say if you look at the example of cost accounting, you can imagine a world where nobody does cost accounting. You say of your organization, “Let’s do cost accounting here.”

That’s a problem because you’d be heard as saying, “Somebody around here is stealing and we need to find out who.” So that might be discouraged.

In a world where everybody else does cost accounting, you say, “Let’s not do cost accounting here.” That will be heard as saying, “Could we steal and just not talk about it?” which will also seem negative.

Similarly, with prediction markets, you could imagine a world like ours where nobody does them, and then your proposing to do it will send a bad signal. You’re basically saying, “People are bullshitting around here. We need to find out who and get to the truth.”

But in a world where everybody was doing it, it would be similarly hard not to do it. If every project with a deadline had a betting market and you say, “Let’s not have a betting market on our project deadline,” you’d be basically saying, “We’re not going to make the deadline, folks. Can we just set that aside and not even talk about it?”

From a zoomed out view, we can see better incentive schemes that would give us more of what we want, that would allow us to run civilization better. But there’s no straightforward way to transition from our current systems to those better ones. We’re blocked by a combination of collective action problems and powerful stakeholders who benefit from the current equilibrium. So better institutional design remains theoretical.

One step meta

When I look at this problem, it strikes me that this is itself just another institution-design problem. Our norms, incentives, and mechanisms for moving from one equilibrium to another leave a lot to be desired. But, there could exist institutional setups that would solve or preempt the collective action and opposed-stakeholder problems that prevent us from transitioning out of inadequate equilibria. 

This isn’t something that we, as a species, know how to do, in actual practice. But it is something that we could know how to do. And if we did, that would let us build incentive-aligned institutions up to the limit of our theoretical knowledge of which institutions would produce the outcomes that we want. 

And this would, in turn, put the planet on track to adequacy for literally every other problem we face.

I call this the “meta-institutional bottleneck”. I offer it as a contender for the most important problem in the world.

Is this one thing or many?

It may be that this “meta-institutional bottleneck” is really just a name for the set of specific problems of moving from a specific inadequate equilibrium to its corresponding more adequate one. We have many such problems, but they’re different enough that we effectively have to solve each one on its own terms.

My guess is that that’s not true. I posit that there are enough commonalities in the difficulties to solving each inadequate equilibrium, that we can find at least semi-general solutions to those difficulties.

If I imagine someone attempting to intentionally push such equilibrium-switches, over a 60 year career, I imagine that they would learn things that generalize from one problem to the next. Surely there would be plenty of idiosyncratic details to each problem, but I imagine that there would also be common patterns for which one might develop generally applicable approaches.

I have not solved any such problems, at scale, so I’m only speculating. But some possible patterns / approaches that one might discover:

• You might learn that there are a small number of people in a social network / prestige hierarchy who are disproportionately leveraged in switching the behavior of a whole network. If you can convince those people to change their behavior, the rest of the network will follow.
  • Certainly many of these people will be prestigious individuals that are easy to identify. But it could also be the case that there are high-leverage nodes that are not obviously powerful or prestigious, but who nevertheless end up being trend-setters in the network. There may or may not be an arbitrage opportunity in getting the people with the most leverage relative to their power/prestige to switch, and there may or may not be generalized methods for identifying those arbitrage opportunities.
• You might learn that there’s almost always a question of how to compensate stakeholders who are beneficiaries of the current system. You want to run a process that allows the people who benefited from the inadequate institution to save face, or even take the credit for the switch over. Perhaps there are generalized mechanisms that have this property, 
• You might discover the conditions under which dominance insurance contracts can work well, and what their practical limits are.
• You might find that there are repeatable methods for building alternative institutions in parallel to legacy institutions, which gradually take on more and more of the functions of the legacy institution such that there’s an incentive gradient towards a better equilibrium at every step.

Or maybe nothing like any of these. Perhaps when one tries to solve these problems in practice, they learn that this whole analytic frame of incentives and game theory is less productive than some other model.

But whatever form it ends up taking, it seems like there ought to be knowledge to discover here. I guess that the meta-institutional bottleneck is meaningfully a “thing”, and not just a catch-all term for a bunch of mostly unrelated, mostly idiosyncratic problems.

And to the extent that that’s true, there’s room for a community of entrepreneur-researchers iteratively trying to solve this kind of problem, sharing what works and what doesn’t, and collectively iterating to best practices, in the same way that there are currently known best practices for starting a startup.

Only one hump

It seems likely that the meta-institutional bottleneck is a one time hurdle. If we’re able to grow past our current existing institutions, we may not be in the trap again.

For one thing, many of the better institutions that we want to install have built in mechanisms for iterating to even better institutions. If your society is regularly utilizing prediction markets to aggregate information, it becomes much easier to surface, validate, and create common knowledge of superior institution design. A society that is already using prediction and decision markets is much much better positioned to switch to something even better, if something like that is found. Our institutions are our meta-institutions.

But even aside from that, if we do discover general, repeatable, methods for moving from inadequate to adequate equilibria, we will forever be in a better situation for improving our institutions over the future. [ 2 ]

For instance, suppose that Balaji’s “network state” model proves successful, and in 25 years there are a handful of diplomatically recognized sovereign states that were founded in that way. This would probably be a great improvement on the object level, since I expect at least one of those states to incorporate modern institution design, demonstrating their effectiveness (or falsifying their effectiveness!) to the rest of the world. But even if that’s not the case, there would, in that future, be a known playbook by which groups of people could iteratively found sovereign states that conform to whatever ideals a large enough coalition buys into. 

We have a meta-institutional bottleneck in this generation, but if we solve it, in general, we may solve it forever. 

The long game

I guess that 60 to 100 years is enough time for a relatively small group of ~100 competent founder types to radically shift the incentives in society. 

This would be a slog, especially at first. But few individuals plan and build on decades-long time horizons. There’s an advantage to those that are playing a long game. 

And as some of these projects succeed, that would accelerate others. As noted above, there’s a flywheel effect where adequate information propagation and decision institutions disproportionately favor other adequate information propagation and decision institutions.

I could imagine a community of people who share this ideology. They would identify specific inadequate equilibria that could maybe be corrected with some combination of a clever plan and an annoying schlep. Individual founders would form small teams to execute a plan to try to solve that equilibrium. 

Some examples of the sorts of projects that we might push on:

• Normalizing prediction markets
• Attempting to found network states
• Reforming existing government processes.
• Reforming academia 
• Building expertise identification and peer-review processes outside of academia
• Building platforms that recreate the functionality of local governance (eg a community governance platform to evaluate and cast judgement on allegations of sexual harassment)
• Build incentive-compatible outcome-tracking and ranking platforms (for medical professionals, for lawyers, etc.)
• Generally doing the schlep to kickstarter every change that looks like a collective action problem.

Those teams would often fail, and write up post-mortems for the community to engage with. And occasionally they would succeed, to the point of pushing society over a threshold into some new stable equilibrium. In the process, they would find processes and build software tools for solving a specific problem, and then share them with the broader community, to adapt them to other problems.

Over time we’ll get a clearer sense of what common blockers to these institutional shifts are, and then figure out approaches to those blockers. For instance, as Robin Hanson has found, there isn’t actually demand for accurate information from prediction markets, because accurate information interferes with narrative-setting. And in general, most individuals don’t locally prioritize better outcomes over signaling their good qualities, doing the “done thing”, and avoiding blame. 

Noticing these blockers, we treat them as constraints on our exploration process. We need to find institutional reform paths that are themselves compatible with local incentives, just as a technologist steers towards a long term vision by tacking close to what is currently technologically feasible. And it turns out that the local incentives are different than what one might have naively assumed. Metamed’s failure and Robin’s non-traction on prediction markets both represent progress.  [ 3 ]

We would be playing a decades-long game, which means that a crucial consideration would be building and maintaining a lineage that retains essential skill and strategic orientation, over several generations.

Over the course of a century, I think we could plausibly get to a society on planet earth whose core institutions are basically adequate. And from there, the world would be mostly optimized.

But…AI

The problem, of course, is that it doesn’t seem that we have even 30 years until the world is radically transformed and control of the future slips out of human hands. (Because of AI.)

This is what I would be spending my life on, if not for the rapidly approaching singularity.

I would prefer to be pushing on the meta-institutional bottleneck. It seems more rewarding, and more tractable, and a better fit for my skills and disposition. 

But I don’t think I live in that world. 

It’s possible that I should still be doing this. If the AI situation looks intractable enough, maybe the most dignity is in making peace with death, and then backing up and starting to build the kind of sane civilization that would have handled the problem well, if we had been faster. 

---

[ 1 ] – Some of the suckiness of the world is due to currently unavoidable circumstances, diseases like aging and cancer for instance suck. But even those problems are touched by our half-functional institutions. It is not currently the case that a given cancer patient gets the best treatment our planet could in principle afford given our current level of technology. If the norms of medical research were better, or even if the medical literature was better indexed, it would be clearer which treatments worked best. If medical professional’s outcomes were tracked, and scored, it would be possible to buy provably better treatment, which would incentivize diffusion of innovation. And so on.

[ 2 ] – This isn’t literally guaranteed. It could of course be the case that we discover methods that work well in our current world, but that our world changes radically enough that our tools and methods stop being applicable. 

It’s not out of the question that a few network states are successfully founded, before legacy states wise up and coordinate to close that option.

[ 3 ] – Indeed, Robin Hanson’s decades long advocacy of prediction markets and MetaMed’s nascent attempt to reform medicine both seem to me like examples of the kinds of project that we might try to undertake. Neither succeeded, but both gave us more detail on the shape of the problem (noisily, since outcomes are very dependent on luck; a problem that destroyed a first attempt might be defeated by a second attempt).