Successfully propagandized people don’t think they’ve been propagandized; if you would expect to feel the same way in either case, you have to distinguish between the two possibilities using something other than your feelings.

– Duncan Sabien

I wish my dad understood this point.

But it’s pretty emotionally stressful to live in a world where you can’t trust your info streams and you can’t really have a grasp on what’s going on.

Like, if I tell my dad not to trust the New York times, because it will regularly misinform him, and that “science” as in “trust the science” is a fake buzzword, about as likely to be rooted in actual scientific epistemology as not, he has few reactions. But one of them is “What do you want me to do? Become a rationalist?”

And he has a point. He’s just not going to read covid preprints himself, to piece together what’s going on. That would take hours and hours of time that he doesn’t want to spend, it would be hard and annoying and it isn’t like he would have calibrated Bayesian takes at the end.

(To be clear, I didn’t do that with covid either, but I could do it, at least somewhat, if I needed to, and I did do little pieces of it, which puts me on a firmer footing in knowing which epistemic processes to trust.)

Give that he’s not going to do that, and I don’t really think that he should do that, what should he do?

One answer is “just downgrade your confidence in everything. Have a blanket sense of ‘actually, I don’t really know what’s going on.’ ” A fundamental rationalist skill is not making stuff up, and saying “I don’t know.” I did spend a few hours tying to orient on the Ukraine situation, and forcing myself to get all the way to the point of making some quantiative predictions (so that I have the opportunity to be surprised, and notice that I am surprised). But my fundamental stance is “I don’t understand what’s going on, and I know that I don’t understand. (Also here are some specific things that I don’t know.)”

…Ok. Maybe that is feasible. It’s pretty hard to live in a world where you fundamentally don’t know what’s happening, where people assume you have some tribal opinion about stuff and your answer is “I don’t know, I think my views are basically informed by propaganda, and I’m not skilled enough or invested enough to try to do better, so I’m going to not believe or promote my takes.”

But maybe this becomes easier if the goal of your orientation in the world is less to have a take on what’s going on, but is instead to prioritize uncertainties: to figure out which questions seem most relevant for understanding, so that you have _some_ map to orient from, even if it is mostly just a map of your uncertainty.

Extended Paraphrase of Ben and Jessica’s general view [December 2019]

Eli’s Summary of a Conversation with Vassar [October 2020]

Overview of my conversation with Vassar on Wednesday Feb 10: Trauma patterns [February 2021]

Eli’s notes from Spencer’s interview with Vasser [February 2021]

Parsing Ben and Michael’s key points from that giant twitter conversation [May 2021]

When I was practicing touch typing I found that much of the skill was a matter of going as fast as I could, without letting my speed outpace my accuracy. If I could feel that the precision of my finger placements was high, I would put more “oomph” into my typing, pushing harder to go faster. 

But I would often fall into an attractor of “rushing” or “going off the rails”, where I was pushing to go fast in a way that caused my accuracy to fall apart, and I started to “trip over myself”. I made a point to notice this starting to happen and then intentionally slow down (and relax my shoulders) to focus on the precision of my finger placements. The goal was never to rush (because that is counter productive), but to go as fast as possible within that constraint.

I think there might be an analogous thing in my personal productivity. 

When I have a largish amount to get done in a short amount of time, this can be energizing and motivating. My physiological arousal is higher. The my personal tempo faster. There’s a kind of energy or motivation that comes from having things that need to get done, with deadlines, and it boots me up into a higher energy orbital, where my default mental actions are geared towards making progress, instead of random “I don’t feel like it” sort of laziness. There’s a bit of a tailwind behind me.

(Indeed, this kind of pressure is exactly what was missing for most of 2020.)

However, sometimes this pressure gets overwhelming, and my intentionality collapses. It’s too much. Either I don’t have the spaciousness to let my attention fully engage with any given task (which is usually necessary for making progress) because of the competing goal threads, instead only managing a shallow superficial attention, or I’ll get overwhelmed and opt out of all of it by distracting myself.

There’s this important principle that I never want my tailwind to outpace my structure. Having some amount of pressure speeding me along is great, but only if my intentionality is high enough to still absorb everything that’s coming at me, taking in the input of what’s important, orienting to it, and taking action on it.

Too much tail wind and that intentionality collapses.

Which means that I need a control system that keeps those two metrics in sync. I need to notice when my intentionality is starting to collapse, and take actions to slow things down and to shore up my intentionality. 

However, my intentionality can collapse for another reason, other than getting outpaced by motivation-pressure. It also collapses when I’m low on energy and alertness.

My intentionality depends on my personal energy and alertness. When my energy and alertness is depleted, the inner structure of my intentionality tends to collapse. 

(There are some caveats here. For one thing, it is possible to maintain intentionality in a low energy state. Also, I can sometimes depend on external structure as a substitute for intentionality, and external structure depends much less on my personal energy and alertness. But to a first approximation, low energy -> low intentionality.)

As a consequence of this, the control system maintaining my intentionality propagates back to an earlier control system maintaining my energy level. I want to notice when my energy is flagging, when I’m just starting to run on fumes, and take action to shore up my energy, before my intentionality collapses.

Furthermore, because my personal energy and alertness is at the bottom of the stack, a lot of my energy and alertness maintenance is not structured as a control system. I employ strategies to get good sleep, and to exercise every day, independently of my current energy level, because high energy is self sustaining.

Having some practices that are “foundational” rather than implemented as control systems is costlier, because it means that I’ll sometimes engage in them when they are not strongly necessary. But foundational systems are more robust: they have more slack in the system to absorb peterbations.

I’ve written elsewhere about how the biggest factor in my personal productivity is aversions, and skillfully engaging with aversions. It’s maybe not unsurprising that having an aversion to task is relevant to effectively executing on that task. But it is a bit more surprising that having an aversion to some task or consideration, makes it much much less likely that I’ll effectively execute on anything.

The key insight, I think, is engaging deeply in a task entails clearing some mental space.

Aversion to something increases my compulsiveness / distractibility. I’m more likely to take a bathroom break, or to make food for myself, or to rereard old blog posts on my phone (without jotting down my thoughts in the way that makes reading more productive / creative), or to go check twitter and then get stuck in the twitter loop.[1] 

I think this is because I’m feeling some small constant pain, and part of me is compulsively seeking positive stimulation to distract from the pain. Basically holding an aversion makes me more reactive to stray thoughts and affordances of the environment. My immediate actions are driven by a (subtle, but nevertheless dominating) clawing, grasping, drive for positive sensation, instead of flowing from “my deep values”, my sense of what seems cool or alive. 

Most, but not all, forms of creative work, involve making mental space, quieting those distractions so that I can give my full attention to the thing that I’m trying to do. The reason why aversions kill my productivity is that my compulsive stimulation-hunger is too graspy to settle down into any long-threaded thought. That part of me doesn’t want to be still, because it is seeking distraction from the sensation in me.

(The exception is some forms of work that “fit” this compulsiveness, where I can get sucked into compulsively doing some task as a way to distract from the sensation in my body. Sometimes an essay is of the right shape that it can be a hook in just the right way, but most of my work is not like this.)

Generally, when I notice an aversion, I’ll engage with it directly, either by sitting down and meditating, feeling into the sensation in a non semantic way, or by doing focusing / journaling, which is more of a semantic “dialogue”, or something that is a mix of both approaches.

In doing this, I’m first just trying to make space for the sensation, to feel it without distraction, while also being welcoming towards the part of me that is doing the dissociation, and secondly hoping to get more understanding and context, so that I can start planning and taking action regarding the underlying concern of the aversion.

[1] I found myself doing all of these except the last one today, all the while vaguely / liminally aware of the agitation clench in my belly, before I sat down to engage with it directly.

[epistemic status: Brainstorming and first draft thoughts.

Inspired by something that Ruby Bloom wrote and the Paul Christiano episode of the 80,000 hours podcast.]

One claim I sometimes hear about AI alignment [paraphrase]:

“It is really hard to know what sorts of AI alignment work are good, this far out from transformative AI. As we get closer, we’ll have a clearer sense of what AGI / Transformative AI is likely to actually look like, and we’ll have much better traction on what kind of alignment work to do. In fact, it might be the case that MOST of the work of AI alignment is done in the final few years before AGI, when we’ve solved most of the hard capabilities problems already and we can work directly, with good feedback loops, on the sorts of systems that we want to align.”

Usually this is taken to mean that the alignment research that is being done today is primarily to enable or make easier future, more critical, alignment work. But “progress in the field” is only one dimension to consider in boosting the work of alignment researchers in final crunch time.

In this post I want to take the above posit seriously, and consider the implications. If most of the alignment work that will be done is going to be done in the final few years before the deadline, our job in 2021 is mostly to do everything that we can to enable the people working on the problem in the crucial period (which might be us, or our successors, or both) so that they are as well equipped as we can possibly make them.

What are all the ways that we can think of that we can prepare now, for our eventual final exam? What should we be investing in, to improve our efficacy in those final, crucial, years?

The following are some ideas.

Access

For this to matter, our alignment researchers need to be at the cutting edge of AI capabilities, and they need to be positioned such that their work can actually be incorporated into AI systems as they are deployed.

A different kind of work

Most current AI alignment work is pretty abstract and theoretical, for two reasons. 

The first reason is a philosophical / methodological claim: There’s a fundamental “nearest unblocked strategy” / overfitting problem. Patches that correct clear and obvious alignment failures are unlikely to generalize fully, you’ll only have constrained unaligned optimization to channels that you can’t recognize. For this reason, some claim, we need to have an extremely robust, theoretical understanding of intelligence and alignment, ideally at the level of proofs.

The second reason is a practical consideration: we just don’t have powerful AI systems to work with, so there isn’t much in the way of tinkering and getting feedback.

The second objection becomes less relevant in final crunch time: in this scenario, we’ll have powerful systems 1) that will be built along the same lines as the systems that it is crucial to align and 2)  that will have enough intellectual capability to pose at least semi-realistic “creative” alignment failures (ie, current systems are so dumb, and liven in such constrained environments, that it isn’t clear how much we can learn about aligning literal superintelligences from them.)

And even if the first objection ultimately holds, theoretical understanding often (usually?) follows from practical engineering proficiency. It seems like it might be a fruitful path to tinker with semi-powerful systems trying out different alignment approaches empirically, and tinkering to discover new approaches, and then backing up to do robust theory-building given much richer data about what seems to work.

I could imagine sophisticated setups that enable this kind of tinkering and theory building. For instance, I imagine a setup that includes:

• A “sandbox” that afford easy implementation of many different AI architectures and custom combinations of architectures, with a wide variety easy-to-create, easy-to-adjust, training schemes, and a full suite of interpretability tools. We could quickly try out different safety schemes, in different distributions, and observe what kinds of cognition and behavior result.
• A meta AI that observes the sandbox, and all of the experiments therein, to learn general principles of alignment. We could use interpretability tools to use this AI as a “microscope” on the AI alignment problem itself, abstracting out patterns and dynamics that we couldn’t easily have teased out with only our own brains. This meta system might also play some role in designing the experiments to run in the sandbox, to allow it to get the best data to test it’s hypotheses.
• A theorem prover that would formalize the properties and implications of those general alignment principles, to give us crisply specified alignment criteria by which we can evaluate AI designs.

Obviously, working with a full system like this is quite different than abstract, purely theoretical work on decision theory or logical uncertainty. It is closer to the sort of experiments that the OpenAI and Deep Mind safety teams have published, but even that is a pretty far cry from the kind of rapid-feedback tinkering that I’m pointing at here.

Given that the kind of work that leads to research progress might be very different in final crunch time than it is now, it seems worth trying to forecast what shape that work will take and trying to see if there are ways to practice doing that kind of work before final crunch time.

Knowledge

Obviously, when we get to final crunch time, we don’t want to have to spend any time studying fields that we could have studied in the lead-up years. We want to have already learned all the information and ways of thinking that we’ll want to know, then. It seems worth considering what fields we’ll wish we had known when time comes.

The obvious contenders:

• Machine Learning
• Machine Learning interpretability
• All the Math of Intelligence that humanity has yet amassed [Probability theory Causality, etc.]

Some less obvious possibilities:

• Neuroscience?
• Geopolitics, if it turns out that which technical approach is ideal hinges on important facts about the balance of power?
• Computer security?
• Mechanism design in general?

Research methodology / Scientific “rationality”

We want the research teams tackling this problem in final crunch time to have the best scientific methodology and the best cognitive tools / habits for making research progress, that we can manage to provide them.

This maybe includes skills or methods in the domains of:

• Ways to notice as early as possible if you’re following an ultimately-fruitless research path
• Noticing / Resolving /Avoiding blindspots
• Effective research teams
• Original seeing / overcoming theory blindness / hypothesis generation
• ???

Productivity

One obvious thing is to spend time now, investing in habits and strategies for effective productivity. It seems senseless to waste precious hours in our acute crunch time due to procrastination or poor sleep. It is well worth in to solve those problems now. But aside from the general suggestion to get your shit in order and develop good habits now I can think of two more specific things that seem good to do.

Practice no-cost-too-large productive periods

There maybe trades that could make people more productive on the margin, but are too expensive in regular life. For instance, I think that I might conceivably benefit from having a dedicated person who’s job is to always be near me, so that I can duck with them with 0 friction. I’ve experimented a little bit with similar ideas (like having a list of people on call to duck with), but it doesn’t seem worth it for me to pay a whole extra person-salary to have the person be on call, and in the same building, instead of on-call via zoom.

But it is worth it at final crunch time.

It might be worth it to spend some period of time, maybe a week, maybe a month, every year, optimizing unrestrainedly for research productivity, with no heed to cost at all, so that we can practice how to do that. This is possibly a good thing to do anyway, because it might uncover trades that actually, on reflection are worth importing into my regular life.

Optimize rest

One particular subset of personal productivity, that jumps out at me: each person should figure out their actual optimal cadence of rest.

There’s a failure mode that ambitious people commonly fall into, which is working past the point when marginal hours of work are negative. When the whole cosmic endowment is on the line, there will be a natural temptation to push yourself to work as hard as you can, and forgo rest. Obviously, this is a mistake. Rest isn’t just a luxury: it is one of the inputs to productive work.

There is a second level of this error in which one, grudgingly, takes the minimal amount of rest time, and gets back to work. But the amount of rest time required to stay functional is not the optimal amount of rest, the amount the maximizes productive output. Eliezer mused years ago, that he felt kind of guilty about it, but maybe he should actually take two days off between research days, because the quality of his research seemed better on days when he happened to have had two rest days preceding.

In final crunch time, we want everyone to be resting the optimal amount that actually maximizes area under the curve, not the one that maximizes work-hours. We should do binary search now, to figure out what the optimum is.

Also, obviously, we should explore to discover highly effective methods of rest, instead of doing whatever random things seem good (unless, as it turns out, “whatever random thing seems good” is actually the best way to rest).

Picking up new tools

One thing that will be happening in this time, is there will be a flurry of new AI tools that can radically transform thinking and research, perhaps increasingly radical tools coming at a rate of once a month or faster.

Being able to take advantage of those tools and start using them for research immediately, with minimal learning curve, seems extremely high leverage.

If there are things that we can do that increase the ease of picking up new tools and using them to their full potential (instead of, as is common, using only the features afforded by your old tools and only very gradually

Some thoughts (probably bad):

• Could we set up our workflows, somehow, such that it is easy to integrate new tools into them? Like if you already have a flexible, expressive research interface (something like Roam?), and you’re used to regular changes in capability to the backed of the interface?
• Can we just practice? Can we have a competitive game of introducing new tools, and trying to orient to them and figure out how to exploit them creatively as possible?
• Probably it should be some people’s full time job to translate cutting edge developments in AI into useful tools and practical workflows, and then to teach those workflows to the researchers?
• Can we design a meta-tool that helps us figure out how to exploit new tools? Is it possible to train an AI assistant specifically for helping us get the most out of our new AI tools?
• Can we map out the sort of constraints on human thinking and/or the the sorts of tools that will be possible, in advance, so that we can practice with much weaker versions of those tools, and get a sense of how we would use them, so that we’re ready when they arrive?
• Can we try out new tools on psychedelics, to boost neuroplasticity? Is there some other way to temporarily weaken our neural priors? Maybe some kind of training in original seeing?

Staying grounded and stable in spite of the stakes

Obviously, being one of the few hundred people on whom the whole future of the cosmos rests, while the singularity is happening around you, and you are confronted with the stark reality of how doomed we are, is scary and disorienting and destabilizing.

I imagine that that induces all kinds of psychological pressures, that might find release in any of a number of concerning outlets: by deluding one’s self about the situation, by becoming manic and frenetic, by sinking into immovable depression.

We need our people to have the virtue of being able to look the problem in the eye, with all of its terror and disorientation, and stay stable enough to make tough calls, and make them sanely.

We’re called to cultivate a virtue (or maybe a set of virtues) of which I don’t know the true name, but which involve courage and groundless, and determination-without-denial.

I don’t know what is entailed in cultivating that virtue. Perhaps meditation? Maybe testing one’s self at literal risk to one’s life? I would guess that people in other times and places, who needed to face risk to their own lives and that of their families, did have this virtue, or some part of it, and it might be fruitful to investigate those cultures and how that virtue was cultivated.

 

Here’s a sketch. All of the following are generalizations, and some are wrong.

There are rationalists.

The rationalists are unusually intelligent, even I think, for the tech culture that is their sort of backdrop. But they are, by-and-large kind of aspy: on the whole, they are weak on social skills, or their is something broken about their social perceptions (broken in a different way for each one).

Rationalists rely heavily on explicit reasoning, and usually start their journeys pretty disconnected from their bodies.

They are strongly mistake theorists.

They have very very strong STEM-y epidemics. They can follow, and are compelled by arguments. They are masterful at weighing evidence and coming to good conclusions on uncertain questions, where the there is something like a data-set or academic evidence base.

They are honest.

They generally have a good deal of trust and assumption of good faith about other people, or they are cynical of humans and human behavior, using (explicit) models of “signaling” and “evo pysch.”

I think they maybe have a collective blindspot with regards to Power, and are maybe(?) gullible (related to the general assumption about good faith). I suspect that rationalists might find it hard to generate the hypothesis that “this real person right in front of me, right now, is lying to me / trying to manipulate me.”

They are, generally, concerned about ex-risk from advanced AI, and track that as the “most likely thing to kill us all”.

 

There’s also this other cluster of smart people. This includes Leverage-people, and some Thiel people, and some who call themseleves post rationalists.

They are more “humanities” leaning. They probably think that lots of classic philosophy is not only good, but practically useful (where some rationalists would be apt to deride that as the “rambling of dead fools”).

They are more likely to study history or sociology, than math or Machine Learning.

They are keenly aware of the importance of power and power relations, and are better able to take ideology as object, and treat speech as strategic action rather than mere representation of belief.

Their worldview emphasizes “skill”, and extremely skilled people, who shape the world.

They are more likely to think of “beliefs” as having a proper function doing something other than reflecting the true state of the world, for instance, facilitating coordination, or producing an effective psychology. The rationalist would think of instrumentally useful false beliefs as something that is kind of dirty.

They tend to get some factual questions wrong (as near as I can tell): one common one is disregarding IQ, and positing that all mental abilities are a matter of learning.

These people are much more likely to think that institutional decay or civilizational collapse is more pressing than AI.

 

It seems like both these groups have blindspots, but I would really like to have a better sense of the likelyhood of both of these disasters, so it would be good if we could get all the virtues into one place, to look at both of them.

 

 

Epistemic status: wild-eyed inside view impressions, based on narrative and stereotype, mostly devoid of hard facts. I expect hard facts to change my view. [I also note that this story basically accords with my grey-tribe ideology / worldview.]

Over the past couple of weeks, I’ve started following twitter for the first time, and (relatedly) I’ve been reading a lot of Venkatesh Rao’s writing, which is mostly new to me. This combined with the fact that history in 2020 seems to be “moving faster” than it used to, has caused me to start thinking about society, and social class, and some other topics that I’ve not thought much about before.

This shitty post represents an outline of my current, tentative, view of what the heck is going on in the United States these days. As noted in the epistemic status, this synthesis is based on my subjective impressions watching the world unfold, more than rigorous analysis, and accordingly I’m missing a lot of complexity, at best, and totally off base at worst. But nevertheless, this kind of unrigorus making sense of things seems like a good starting point, to guide my empiricism.

Optimism and social stability

There’s a lot of talk about inequality, and the social unrest that income inequality foments. But I think that inequality of income, or of wealth, is a little bit of a red herring. I posit that social unrest mostly stems from pessimism about one’s personal life outcomes.

When folks expect their lot in life to improve, decade by decade, and when they have an expectation that if they work hard, their children can enjoy a better standard of living than they do, they’re pretty content. So long as that is true, they don’t care that much if other people are wealthier than them (especially people who live far away, but I think this is also true of literal neighbors).

Upsetting the social order is risky and dangerous, and is therefore an action of last resort. Most people don’t actually like violence and will avoid it if they can. If there are opportunities for social advancement, they’ll take them.

However, if those opportunities are lacking, for some reason or another, and people feel like they can’t get ahead, they’ll feel frustrated. They have a sense that something unfair is happening, and are likely to adopt the mindset of the pie fallacy: there’s only so much to go around, and one person’s wealth implies another person’s poverty.

In fact, while the pie-fallacy is a fallacy in the general case, the 0-sum mindset is a pretty accurate summary of one’s situation when there are not opportunities to improve your lot in life. Conditioning on that premise, your wellbeing does actually tradeoff against other people’s resources. If there’s some blocker to creating new value, the only thing left is redistribution, either by legal means like taxation, or illegal, like outright revolution.

I think that it is this frustration, of a lack of upward social mobility, much more than inequality specifically, that leads to revolution or attempts at redistribution.

Unfortunately, I think that a huge swaths of Americans actually find themselves in this position, of being unable to improve their lot. The American dream, that you can work hard for most of your life, so that your kids can have a better life than you feels like an empty promise.

In the red-tribe side of things, this looks like the factory worker in the rust belt, whose job was outsourced to a foreign country, leaving him without a clear away to support himself. And as the economic centers of his world dried up, he watched as his community became a shadow of its former vibrant self. For him, the American dream seems to have gone wrong somehow. Somehow he’s worse off than his father was.

On the blue-tribe side of things, this is embodied by the millennial in one of the costal mega-cities, working as part of the gig economy or in dismal retail, who is saddled with enormous college debt,  which somehow didn’t give her much career capital, and who is paying exorbitant rent to for a small room in a shared apartment, or, alternatively, still living with her parents. A child of the nineties, she thought, and her parents thought, that if she worked hard and went to a good college, the world was her oyster. But somehow it doesn’t seem to be working out that way.

I imagine that both of these people feel like they are running in place: they were promised that if they worked hard, they could have a good life. But instead they’re treading water.

Why

Why are we here? It seems like for a number of decades standards of living were rising, and people were doing better than their parents. Why did the American dream stop working?

I think that it is the result of a number of factors.

First, we need to keep in mind that “having a better standard of living than your parents did” was much easier for the Baby Boomers, because that was a lower bar to beat. The Boomer were born right after the Great Depression, and many of them were first or second generation immigrants. If your mom and dad walked off the boat with literally $10 in their pocket, and worked their way up from there, or if they were impoverished farmers whose crops failed in the dust bowl, it is a lot more likely that you’ll end up better off then them.

Second is the great stagnation. The boomers benefited from one of the biggest surges in economic growth in history, driven by the mass deployment of new technology, which created new industries. Riding that wave, life got better for a lot of people. If true growth is much slower now, less benefit will accrue to individuals.

But the biggest thing is that the new economy is exclusionary. Information technology and it’s derivatives, the industries in which almost all the growth we do have is concentrated, is fundamentally about scale, which means that outcomes are compressed into a power-law.

In one of my favorite essays of all time, How to Make Wealth, Paul Graham says, in order to make a lot of wealth, you need to have leverage, in the sense that your decisions scale. And that one of the best ways of getting leverage, is by developing technology.

What is technology? It’s technique. It’s the way we all do things. And when you discover a new way to do things, its value is multiplied by all the people who use it. It is the proverbial fishing rod, rather than the fish. That’s the difference between a startup and a restaurant or a barber shop. You fry eggs or cut hair one customer at a time. Whereas if you solve a technical problem that a lot of people care about, you help everyone who uses your solution. That’s leverage.

Doing work with computers, means that you have 0-marginal cost, which means that if you solve a problem once, you’ve solved it an arbitrarily-large number of times, so you can create a huge amount of value in one go.

But there’s a dark side of leverage, which is that it tends to shift the world towards winner-take all dynamics. A few people who are well suited to this new world (possessing an extreme entrepreneurial mindset, or technical chops, or excellent sales skills, etc.) thrive, producing huge amounts of value. But for the Americans that don’t fit that template, the work-a-day opportunities are becoming less and less attractive, because those roles provide less and less marginal value.

When the marginal costs of goods was non-0, that had a flattening effect on the power law, because even if you couldn’t design a ford car, you provided value to the final product by working on the assembly line. But today, the marginal cost of most everything in the new economy is 0.

Approximately speaking, everyone has been thrust, whether they like it or not, from Taleb’s Mediocristan into Extremistan. More of progress is dominated by a few super-winners, compared to a generation ago, when progress was, though still on a power law, more equitable.

[I feel like this doesn’t quite explain it. Like sure, wages are stagnating. But if costs were falling faster than wages, then in wouldn’t matter: you would be doing better year by year, just by dint of falling cost of living.

So part of the story is that, for some reason, the rents are too damn high, such that people have trouble just getting by.

Why’s that?

Maybe it has something to do with the fact that almost all of the economic opportunities are concentrated in a few mega-cities? You either have to live there, or be resigned to watching the world decay around you?

Does that mean that if zoning were different, so that there were ten times the number of apartments in SF, NYC, Chicago, DC, and Austin, this problem would basically go away, and people could basically rise with the rising tide of the economy? Or if the pandemic causes the world to decentralize, and people can pretty much live and work from anywhere in the world?

I guess I would be surprised if that were the result of either of those changes. Maybe because there are forces that are preying on people and therefore driving net income to some set point above subsistence, but below personal net-growth?

…

It might be that the high cost of living is a consequence of the huge disparities in production power: there are the heroes / robber barons of the new economy, who are mostly living in a few mega cities, and almost all the the wealth flows from there, so you have to be in the mega-cities to have access to any of the opportunities?

Also, don’t forget about the college debt which is part of what makes the cost of living high, and which has been driven up by other factors.]

Reaction

Whatever the cause, I now suspect that there are a lot of people that feel like whatever they do, there’s no way for them to get ahead. They feel, justifiably, disenfranchised.

It’s natural, in a situation where you feel trapped, to try and make sense of what is happening to you. There are a number of narratives that people adopt to orient to the situation that they find themselves in. But I think the two biggest ones are Trumpism, and Wokeism.

The archetypal disenfranchised red-triber is frustrated and confused that somehow things are getting worse for him and his peers. The literal person of Donald Trump exploits those feelings, and provides a narrative that America has been taken advantage of by other countries (for instance, Mexico, feeding us illegal immigrants, and China, who is cheating in trade deals), due to something like weakness on the part of the establishment and the coastal elites (who in any case, seem to alienate him or condescend to him at every turn). Under this narrative, these trends need to be reversed, to make America Great Again ™.

The archetypal millennial blue-triber, in contrast, feels frustrated and confused about how she can’t seem to get ahead. She buys into a narrative that the reason for this is systemic oppression: the problem is racism, the patriarchy, and capitalism, and broadly systems that serve their own interests by exploiting weaker groups. Under this story, what is needed is an overhaul of those oppressive systems.

Note of course that this sort of disenfranchised frustration isn’t the only thing feeding these two ideologies. Lots of people who, personally, have an optimistic trajectory might buy into or act in alignment with either one, for all the usual reasons, from virtue signaling, to tribal mimesis, to thinking seriously about the problems and their origins and coming to the conclusion that the Trumpist or the Wokists are basically correct.

But, I posit that the energy that is driving both movements comes from large swaths of people who feel like, whatever they do, they can’t get ahead. If there were ample individual economic opportunity, people wouldn’t care nearly as much about those problems.

Social media

On top of this, we introduced a new technology to the masses over the past decade and a half. Our social and political lives are organized by twitter and facebook.

This has a divisive, echo-chamber enforcing, effect. The Liberals and the Conservatives have always been at each others throats, railing against the other as an outright evil. But I imagine that there was something very different about the world when everyone watched Walter Cronkite on the news every night. There were strong, emotional disagreements, but everyone was, to a large extent, living in the same world.

No longer. Now, the media landscape is fractured, and virtually every event is spun to support one ideology or another. And now almost everyone has a voice in the conversation, due to social media, where before, there were defacto gate-keeper institutions.

Much more than we used to I think, we live in one of a number of parallel worlds that are layered on top of each other. My guess is that this aggravates tensions, by reinforcing one’s existing narrative. Then, because of the natural memetic incentives, this results in extreme othering and demonization of anyone with a meaningfully different set of priorities.

[It seems like there’s a lot more here, that it might be worth understanding in detail.]

Also, our institutions are falling apart, in the sense of loosing their legitimacy, and in the sense of losing their ability to function in pretty basic ways, for related reasons. Basically, it seems like in this cultural battlefield, every institution has to pick a side. This lack of space for “neutrality” edges out the possibility of non-ideological sense-making. (I’m not satisfied with that explanation. I feel like there is a much more detailed story of the incentives that have torn apart news papers and the universities, for instance.

Covid

On top of all this, we have a pandemic, itself litteraly-insanely politicized, forced a lot of people into enclosed spaces with minimal social contact, for months, and waited we went collectively stir-crazy.

Powder keg

Overall, this reads like a pretty concerning setup. I don’t have a good intuition for how robust the substructure of our world is, but it certainly seems like it is being tested more than any other time in my extremely short lifetime.

Seems maybe bad?

[Epistemic status: a non-rigorous theory, representing my actual belief about how it works.]

Related to: Value Differences As Differently Crystallized Metaphysical Heuristics

In this post I want to outline my understanding of what “values” are, at least for human beings. This idea or something very like it may already have standard terminology in academic philosophy, in which case, I would appreciate being pointed to the relevant references. This may be obvious, but I want to say it to lay the groundwork for a puzzle that I want to talk about in the next post.

Basically, I posit that “values”, in the case of human beings, are crystallization or abstractions of simple response patterns.

[Google doc version for commenting]

Abstracting values from reactions

All animals, have a huge suite of automatic reactions to stimuli, both behavioral and affective, and both learned and hard coded.

When thirsty and near water, a lion will drink. When a rabbit detects a predator, they’ll freeze (and panic?). When in heat (and the opportunity presents itself) a giraffe will  copulate. When his human comes home, a dog will wag its tail and run up in greeting, presumably in a state of excited happiness. [I note that all of my examples are off mammals.]

Some of these behaviors might be pretty complex, but their basic structure is TAP-like: something happens, and there is some response in physiology of the animal. I’m going to call this category of “contextualized behaviors and affects”, “urges”.

Humans understand language, which means that the range of situations that they can respond to is correspondingly vaster than most animals. For instance, a human might be triggered (have a specific kind of fear response) to another human making a speech act.

But that isn’t the main thing that differentiates humans in this context. The big difference between humans and most other animals, is that humans can abstract from a multitude of behaviors, to infer and crystallize the “latent intentionality” among those behaviors.

For instance, an early human can reason,

When I see a tiger, I run, and feel extreme overriding panic. If the tiger catches me, I’ll try to fight it. When a heavy rock falls from a cliff, and I hear it falling, I also have a moment of panic, and duck out of the way.

When I am hungry, I eat. When I am thirsty, I drink.

When other people in my tribe have died, I’ve felt sad, and sometimes angry.

…I guess I don’t want to die.

[edit 2022-06-01: More specifically, what’s going on is that the human simulates a bunch of possible scenarios in which he comes to harm or dies, and has a negatively-valenced (flee, retreat, resist) reaction to each one. He intuits the similarity between those scenarios, to abstract out general concepts of harm or death, and associativity learns a general negatively-valenced reaction to those outcomes. He develops a flee-retreat-resist response to anything that involves his dying. He ends up with a goal of “staying alive”. (By default, all of this happens non-verbally, and without any conscious reflection.)]

From each of these disparate, contextualized, urges-to-action and affective responses (which by the way, I posit are not two distinctly different things, but rather two ends of a spectrum), a person notices the common thread, “what do each of these behaviors seem to be aiming towards?”

And abstracting that goal, from the urges, he/she then “owns” it. He/she thinks of him/herself as an entity wanting, valuing, caring about that thing (rather than a bundle of TAPs, some of which are correlated).

My guess is that this abstraction operation is an application of primate (maybe earlier than primate?) social-modeling software to one’s self. It is too expensive to track all of the individual response behaviors of all of the members of your band, but fortunately, you can compress most of the information by modeling them not as adaption-executors, but as goal directed agents, and keeping track of their goals and their state of knowledge.

When one applies the same trick to one’s own behavior and mental states, one can compress a plethora of detail about a bunch of urges into a compact story about what you want. Wala. You’ve started running an ego, or a self.

This is the origin of “values.” Values are compressions / abstractions / inferences based on / extrapolated from a multitude of low level reactions to different situations.

I think that most animals can’t and don’t do this kind of inference. Chipmunks (I think) don’t have values. They have urges. Humans can, additionally, extrapolate their urges into  values.

I’m pretty sure that something like this process is how people come to their values (in the conventional sense of “the things they prioritize”) in real life.

For instance, I am triggered by claims and structures that I perceive as threats to my autonomy. I flinch away defensively. I think that this has shaped a lot of my personality, and choices, including leading me into prizing rationality.

Furthermore, I posit that something like this process is how people tend to adapt political ideologies. When someone hears about the idea of redistribution, and their visceral sense of that is someone taking things from them, they have a (maybe subtle) aversion / threatened feeling.* This discomfort gives rise to an urge to skepticism of the idea. And if such a person hangs out with a bunch of other people that have similar low-level reactions, eventually, it becomes common knowledge, and this becomes the seed of an ideology, that gets modified and reinforced by all the usual tribal mechanisms.

I think the same basic thing can happen when someone feels (probably less than consciously) threatened by all kinds of ideologies. And this + social mimesis is how people end up with “conservative values” or “liberal values” or “libertarian values” or what have you.

* – I have some model of how this works, the short version being, “stimuli trigger associated (a lot of the action here is in the association function) mental imagery, which gives rise to a valence,  which guides immediate action, modulo further, more consequentialist deliberation. In fact, you can learn to consciously catch glimpses of this happening.

Of course all of this is a simplification. Probably this process occurs hierarchically, where we abstract some goals from TAP-like urges, and then extrapolate more abstract goals from those, and so on until we get to the “top” (if it turns out that there is a “top”, as opposed to a cycle that has some tributaries that flow into it).

For that reason, the abstraction / crystallization / triangulation process is not deterministic. It is probably very path dependent. Two people with the exact same base level pattern of urges, in different contexts will probably grow into people with very different crystallized values.

Values influence behavior

Now a person might abstract out their values from their behavior in a way that is largely non-consequential. They model themselves, and describe themselves, in terms of their values, but that is just talk. The vast majority of their engagement in the world is still composed of the behaviors stemming from their urges in response to specific situations.

But, it also sometimes happens that abstracting out values, and modeling one’s self as an optimizer (or something like an optimizer) for those values, can substantially effect the level of behavior.

For one thing, having a shorthand description of what one cares about means that one one can use that description for deliberation. Now, when considering what to do in a situation, a person might follow a mental process that involves asking how they can achieve some cashed goal, instead of reflexively acting on the basis of the lower level urges that the goal was originally abstracted from.

This means that a person might end up acting in a way that is distinctly in opposition to those low level reactions.

For instance, a person might want status and respect, and they can feel the tug to go drink and socialize “with the guys” of their age group, but they instead stay home and study, because they reason that this will let them get a good job, which will let them get rich, which they equate with having a lot of status.

Or a person might take seriously that they don’t want to die, and sign up for cryonics, even though none of their urges recommended that particular action, and in fact, it flies in the face of their social conformity heuristics.

Furthermore, in this vein a person might notice inconsistencies between their professed values and the way they behave, or between multiple diverse values. And if they are of a logical turn of mind they may attempt to modify their own behavior to be more in line with their values. Thus we end up with moral striving (though moral striving might not be the only version of this dynamic).

Bugs

Just to say this explicitly, humans, uniquely (I think? maybe some other animals also abstract their values), can examine some particular behavior or reaction and consider it to be a bug, a misfiring, where the system is failing to help them achieve their values.

For instance, I’m told that a frog will reflexively flick out it’s tongue to ensnare anything small and black that enters its field of vision. From the perspective of evolution, this is a bug: the behavior is “intended” for catching and eating flies, and eating bits of felt that human researchers throw in the air (or whatever) is not part of the behavior selected for.  [Note: in talking about what evolution “intended”, we’re executing the same mental move of abstracting goals and values from behavior. Evolution is just the fact of what happened to replicate, but we can extrapolate from a bunch of specific contextualized adaptions to reason about what evolution is “trying to do”.]

But, I claim here, that asking “is this behavior a bug, from the frog’s perspective?” is a mis-asked question, because the frog has not abstracted its values from its behaviors, in order to reflect back on its behaviors and judge them.

In the parallel case of a human masturbating, the human can abstract its values from his or her behavior, and could deem masturbation as a 1) bug, a dis-endorsed behavior that arises from a hormonal system that is partially implementing his or her values, but which misfires in this instance, or 2)  as an expression of what he/ she actually values, part of a life worth living.

(Now it might or might not be the case that only one of these options is reflexively stable. If only one of them is, for humans in general, there is still a meaningful sense in which one can be mistaken about which things are Good. That is a person can evaluate something as aligned with their values, but would come to think differently in the limit of reflection.)