[Epistemic status: a non-rigorous theory, representing my actual belief about how it works.]

Related to: Value Differences As Differently Crystallized Metaphysical Heuristics

In this post I want to outline my understanding of what “values” are, at least for human beings. This idea or something very like it may already have standard terminology in academic philosophy, in which case, I would appreciate being pointed to the relevant references. This may be obvious, but I want to say it to lay the groundwork for a puzzle that I want to talk about in the next post.

Basically, I posit that “values”, in the case of human beings, are crystallization or abstractions of simple response patterns.

[Google doc version for commenting]

Abstracting values from reactions

All animals, have a huge suite of automatic reactions to stimuli, both behavioral and affective, and both learned and hard coded.

When thirsty and near water, a lion will drink. When a rabbit detects a predator, they’ll freeze (and panic?). When in heat (and the opportunity presents itself) a giraffe will  copulate. When his human comes home, a dog will wag its tail and run up in greeting, presumably in a state of excited happiness. [I note that all of my examples are off mammals.]

Some of these behaviors might be pretty complex, but their basic structure is TAP-like: something happens, and there is some response in physiology of the animal. I’m going to call this category of “contextualized behaviors and affects”, “urges”.

Humans understand language, which means that the range of situations that they can respond to is correspondingly vaster than most animals. For instance, a human might be triggered (have a specific kind of fear response) to another human making a speech act.

But that isn’t the main thing that differentiates humans in this context. The big difference between humans and most other animals, is that humans can abstract from a multitude of behaviors, to infer and crystallize the “latent intentionality” among those behaviors.

For instance, an early human can reason,

When I see a tiger, I run, and feel extreme overriding panic. If the tiger catches me, I’ll try to fight it. When a heavy rock falls from a cliff, and I hear it falling, I also have a moment of panic, and duck out of the way.

When I am hungry, I eat. When I am thirsty, I drink.

When other people in my tribe have died, I’ve felt sad, and sometimes angry.

…I guess I don’t want to die.

[edit 2022-06-01: More specifically, what’s going on is that the human simulates a bunch of possible scenarios in which he comes to harm or dies, and has a negatively-valenced (flee, retreat, resist) reaction to each one. He intuits the similarity between those scenarios, to abstract out general concepts of harm or death, and associativity learns a general negatively-valenced reaction to those outcomes. He develops a flee-retreat-resist response to anything that involves his dying. He ends up with a goal of “staying alive”. (By default, all of this happens non-verbally, and without any conscious reflection.)]

From each of these disparate, contextualized, urges-to-action and affective responses (which by the way, I posit are not two distinctly different things, but rather two ends of a spectrum), a person notices the common thread, “what do each of these behaviors seem to be aiming towards?”

And abstracting that goal, from the urges, he/she then “owns” it. He/she thinks of him/herself as an entity wanting, valuing, caring about that thing (rather than a bundle of TAPs, some of which are correlated).

My guess is that this abstraction operation is an application of primate (maybe earlier than primate?) social-modeling software to one’s self. It is too expensive to track all of the individual response behaviors of all of the members of your band, but fortunately, you can compress most of the information by modeling them not as adaption-executors, but as goal directed agents, and keeping track of their goals and their state of knowledge.

When one applies the same trick to one’s own behavior and mental states, one can compress a plethora of detail about a bunch of urges into a compact story about what you want. Wala. You’ve started running an ego, or a self.

This is the origin of “values.” Values are compressions / abstractions / inferences based on / extrapolated from a multitude of low level reactions to different situations.

I think that most animals can’t and don’t do this kind of inference. Chipmunks (I think) don’t have values. They have urges. Humans can, additionally, extrapolate their urges into  values.

I’m pretty sure that something like this process is how people come to their values (in the conventional sense of “the things they prioritize”) in real life.

For instance, I am triggered by claims and structures that I perceive as threats to my autonomy. I flinch away defensively. I think that this has shaped a lot of my personality, and choices, including leading me into prizing rationality.

Furthermore, I posit that something like this process is how people tend to adapt political ideologies. When someone hears about the idea of redistribution, and their visceral sense of that is someone taking things from them, they have a (maybe subtle) aversion / threatened feeling.* This discomfort gives rise to an urge to skepticism of the idea. And if such a person hangs out with a bunch of other people that have similar low-level reactions, eventually, it becomes common knowledge, and this becomes the seed of an ideology, that gets modified and reinforced by all the usual tribal mechanisms.

I think the same basic thing can happen when someone feels (probably less than consciously) threatened by all kinds of ideologies. And this + social mimesis is how people end up with “conservative values” or “liberal values” or “libertarian values” or what have you.

* – I have some model of how this works, the short version being, “stimuli trigger associated (a lot of the action here is in the association function) mental imagery, which gives rise to a valence,  which guides immediate action, modulo further, more consequentialist deliberation. In fact, you can learn to consciously catch glimpses of this happening.

Of course all of this is a simplification. Probably this process occurs hierarchically, where we abstract some goals from TAP-like urges, and then extrapolate more abstract goals from those, and so on until we get to the “top” (if it turns out that there is a “top”, as opposed to a cycle that has some tributaries that flow into it).

For that reason, the abstraction / crystallization / triangulation process is not deterministic. It is probably very path dependent. Two people with the exact same base level pattern of urges, in different contexts will probably grow into people with very different crystallized values.

Values influence behavior

Now a person might abstract out their values from their behavior in a way that is largely non-consequential. They model themselves, and describe themselves, in terms of their values, but that is just talk. The vast majority of their engagement in the world is still composed of the behaviors stemming from their urges in response to specific situations.

But, it also sometimes happens that abstracting out values, and modeling one’s self as an optimizer (or something like an optimizer) for those values, can substantially effect the level of behavior.

For one thing, having a shorthand description of what one cares about means that one one can use that description for deliberation. Now, when considering what to do in a situation, a person might follow a mental process that involves asking how they can achieve some cashed goal, instead of reflexively acting on the basis of the lower level urges that the goal was originally abstracted from.

This means that a person might end up acting in a way that is distinctly in opposition to those low level reactions.

For instance, a person might want status and respect, and they can feel the tug to go drink and socialize “with the guys” of their age group, but they instead stay home and study, because they reason that this will let them get a good job, which will let them get rich, which they equate with having a lot of status.

Or a person might take seriously that they don’t want to die, and sign up for cryonics, even though none of their urges recommended that particular action, and in fact, it flies in the face of their social conformity heuristics.

Furthermore, in this vein a person might notice inconsistencies between their professed values and the way they behave, or between multiple diverse values. And if they are of a logical turn of mind they may attempt to modify their own behavior to be more in line with their values. Thus we end up with moral striving (though moral striving might not be the only version of this dynamic).

Bugs

Just to say this explicitly, humans, uniquely (I think? maybe some other animals also abstract their values), can examine some particular behavior or reaction and consider it to be a bug, a misfiring, where the system is failing to help them achieve their values.

For instance, I’m told that a frog will reflexively flick out it’s tongue to ensnare anything small and black that enters its field of vision. From the perspective of evolution, this is a bug: the behavior is “intended” for catching and eating flies, and eating bits of felt that human researchers throw in the air (or whatever) is not part of the behavior selected for.  [Note: in talking about what evolution “intended”, we’re executing the same mental move of abstracting goals and values from behavior. Evolution is just the fact of what happened to replicate, but we can extrapolate from a bunch of specific contextualized adaptions to reason about what evolution is “trying to do”.]

But, I claim here, that asking “is this behavior a bug, from the frog’s perspective?” is a mis-asked question, because the frog has not abstracted its values from its behaviors, in order to reflect back on its behaviors and judge them.

In the parallel case of a human masturbating, the human can abstract its values from his or her behavior, and could deem masturbation as a 1) bug, a dis-endorsed behavior that arises from a hormonal system that is partially implementing his or her values, but which misfires in this instance, or 2)  as an expression of what he/ she actually values, part of a life worth living.

(Now it might or might not be the case that only one of these options is reflexively stable. If only one of them is, for humans in general, there is still a meaningful sense in which one can be mistaken about which things are Good. That is a person can evaluate something as aligned with their values, but would come to think differently in the limit of reflection.)

[This is super rough]

[I’m pretty sure that there is prior work on this question that I don’t know about yet.]

Here’s a conversational pattern that I’ve noticed recently. I observed this specifically in conversations with my parents, but on further reflection, I do this too, and postulate that pretty much everyone does.

Basically, one person will say something, either sharing an opinion, or (maybe this is an atypical rationalist version?) sharing a fact, or something in between.

Then the other person will share an opinion or a fact that is at least somewhat related.

Basically the people go back and forth, sharing what they know about / what they think / what they’re interested in.

For instance, a few weeks ago, someone was telling me about how it wasn’t accidental that Hitler lost the war, which I agreed with, and I shared some anecdotes from my recent reading of the Rise and Fall of the Third Reich, particularly junctures where the NAZIs could have won the war if things had gone slightly differently (like if the Luftwaffe had continued bombing the areas in which the RAF command bunkers, from which they were using RADAR and radio to coordinate their planes, during the battle of Britain).

This may not be a great example, since I might have been making a point about historical contingency, and hindsight bias. I’m very confident that this pattern expressed itself in my most recent conversation with a friend to “catch up” though.

She would say something about something, and I would say something vaguely related.

(Ok. In both these cases, I had phenomenology of “desire to impress” and “pressure to show sophistication.” This is less evident in my conversations with my parents, but it isn’t implausible that there was a less vivid current of the same. That maybe the driving force of this weird thing.)

In any case, this conversational pattern is common, but it is particularly evident when talking with my parents, I’ll often share something, and (misunderstanding what I’m saying) will say something else that is only barely or vaguely related. (I think this might be offered as if it what is being said is in agreement with me.) This particular situation seems to highlight how much conversation is just an opportunity to share things that we want to talk about with someone else, who is likewise taking the opportunity to talk about things that they want to talk about.

What’s going on here? Why do we like doing this?

Some thoughts:

• Maybe we just like the feeling of being listened to, because it validates our opinions and perspectives. This is pretty interesting, given that it’s fake: the other person isn’t actually engaging with our opinions except to riff on them. It’s as if we’re masturbating each-other.
• Maybe this is about intelligence or sophistication signalling? We get to show off our thoughts?
• Maybe it’s a kind of bonding / tribe affirming thing: we pseudo agree with each-other’s propositions (without engaging with them critically)
• We just like talking about what we’re interested in for some reason, and the real question is why we like that.

It just hit me how the fact that this pandemic probably escaped from a lab in China means that this event, with global and personal consequences, is so contingent. There are other Everett branches where it didn’t happen, and I am going about my life as I would have.

I have a weight in my stomach about this. Especially in so far as the quarantine is good for me, and I am learning things or building skills / meta-skills that will have a permanent impact on my life, it is unsettling to think that they depend on fate. Am I going to be consistently less effective in those other worlds, because this “lucky” break didn’t happen? (Interestingly, that is the horrifying thing to me, not the thought of all the luck that I’ve missed out on that went to other worlds.) My growth and power is so fragile.

Of course there have been other events like this: WWII was contingent on Hitler’s birth, and that massively shaped the world I live in now. Most earths don’t look like this one for that reason. There’s still an intact European Jewry, the balance of power looks very different, and science may not have been institutionalized and may not be state-funded. How were Atomic bombs discovered in those world? How did the computer revolution happen?

And of course my personal life is incredibly contingent. If I hadn’t come across Grimoire for the Apprentice Wizard, would I be on a path of ambition? If I hadn’t gone to U Chicago, and met Stefan, would I have have even encountered LessWrong. Reading the sequences, at that time in my life, radically, radically transformed my trajectory.

But this is a huge contingent event occurring after I came into myself as a conscious entity, and after I had “gotten my footing” in the world. This pandemic has changed, and is apt to change a lot for me, but only by luck.

I think that this one stings in particular (at least in part) because, in principle I could have taken time and space to do the things that I’m doing now. But I predict that I wouldn’t have, unless forced. I can feel how that’s a bias. How across the multiverse, I’m less strong than I might otherwise be, because I’m not resisting that temptation. And here, I needed random chance to save me, instead of bootstrapping by my own power.

This is a reminder to me that I want to be an optimizer: I want to be the sort of thing that, whatever sort of universe it finds itself in, convergently steers that world towards the same state. Obviously, the worlds don’t actually converge, because of how much is determined by the randomness, and because I am small. But I want it to be the case that my situation defines the shape of the problems I have to solve, and the tools that I have to work with (and build myself from), but that my situation never sways my basic ability to steer.

I want to be the sort of thing that systematically moves in right direction, no matter where in the multiverse I find myself.

[Probably obvious to many (most?) of my readers, but the obvious often bears saying aloud.]

This morning, as part of my rest day, I did the first exercise of Claudia Altucher’s book, Become an Idea Machine, which is mostly a series of prompts for generating 10 (or more) new ideas, one prompt a day.

The first prompt is to take 10 complaints that you have, and for each one, turn it into a positive expression of gratitude. She gives the example of hating being stuck in traffic every day, and generating from that, “I am grateful that I live in a city that has so much traffic because that means there [are] plenty of opportunities here, and I can meet lots of interesting people.” You’re supposed to do this ten times, on your own problems.

I think that this is a great exercise. I imagine that I tend to be too rigid in my assessment of circumstances, and there are in fact a lot of good things resulting from a “problem”, which I fail to apprehend, because I’m stuck in my existing mindset. Done well, I think this mental motion shouldn’t feel like a self-trick, whereby you convince yourself that “the grapes were probably sour anyway.” Instead it feels like uncovering actual value, henceforth unnoticed, all around you.

As an example, my fourth most pressing problem is the lack of satisfying romantic partnership. And it is natural to focus on the pain or longing of that situation. But also, not having a partner means that I have a lot of solitary time, to think and work and do what I want, which I really value. By default, I think I wouldn’t notice how much I value that alone-time unless I did have a partner, at which point I might start to feel stifled. Hedonically, it seems like a bug if I only pay attention to the aspects of a situation that I dislike.

So I think appreciating what’s good about things that we might reflexively label as bad.

However, I think that it is crucial, if we want to be balanced and sane, that we not lose track of the problem being bad in this process.

It is right and good for the paraplegic to savor the way his disability forces him to slow down and be reflective, and “stop and smell the roses.” But that doesn’t mean that he can’t assess the situation, the pros and cons, and determine that on net, it would be better to be able to walk (and climb and run).

Likewise, it is extremely appropriate to look around at the way the recent pandemic has brought communities together, and the way that it has brought out heroism in many, and feel pleased and proud. But that doesn’t mean that we have to say that the pandemic is good, or that we shouldn’t wish that it hadn’t happened.

The claim “there is often unrecognized good in what seems to be bad” does not entail that “everything that seems bad is actually good”, (or the related “everything happens for a reason”). When we conflate the first with the second(s) them we’re  setting ourselves up for a kind of Stockholm syndrome of the present, a massive status quo bias.

There is more good in the world than we often appreciate by default. And the world is complex, and it is often hard to tell what the ultimate consequence of something will be. But that does not mean that we abdicate our sacred right to weigh up the pros and cons,  make the best assessment we can of whether a thing is good or bad, and then try to steer the world towards the good. When we do otherwise the consequences can be literally catastrophic.

(Of course this is not to say that everything that we reflexively consider bad is in fact bad on net. We might take something that we hate, consider all the benefits that it gives us and all the costs, and conclude that actually, our initial impression was wrong, and we on reflection, prefer the world with the supposed-to-be-bad thing. Just as we might reconsider our reflexive attitude to something that is supposed to be good, and conclude that on-net, we dis prefer it. [For instance, Brienne’s comment in this thread has had me reevaluating my attitude towards romantic love, lately.])