Conceptual precision breaks cooperation, but is necessary for robust cooperation

[Epistemic status: This is really a draft that I should edit into something presentable. This is probably obvious to lots of us, but whatever, I’m rederiving social normality from the ground up. Draft.]

Common, fragile, concepts

There are a number of common, simple, concepts that, when examined closely, appear to break down, or at least be much more complicated than they seemed at first.

For instance, the idea of “I” or who “myself” is. This concept is a standard part of most people’s navigation of the world, but if we turn a philosophical eye to it, we run into all kinds of confusions: am “I” the same person at the person named Eli Tyre who was in high school 10 years ago? What about the person who was resting “20” minutes ago? What about the transporter problem?

This concept is a workhorse of day-to-day living and deciding, but it is shockingly fragile, as evidenced by those edge cases.

Nuance vs. Pragmatism

One might be more or less satisfied with a given level of conceptual clarity around a topic. I might have a pragmatist attitude that ignores or papers over the finicky fragility of concepts, and doesn’t bother much with the nuances of meaning.

Or I might be a stickler for the nuance: really caring, about having clarity around these details, making sure that I understand what I’m talking about.

The same person might have a different attitude in different contexts: I’m a pragmatist when I need to get the milk, and a philosopher when I need to think about cryonics. (But in practice, it also seems like there is a fairly stable trait which represents how much of a stickler someone is.)

Cooperation

Being a stickler for nuance is often detrimental to cooperation. As a case in point, suppose that my neighbor’s cat is sick. The cat really needs to be taken to the vet, but my neighbor is has a crucial business meeting with an important client and if he misses it he’ll be fired. In desperation, my neighbor asks me if I can take his cat to the vet. (He doesn’t know me very well but there’s no one else around and he’s desperate.)

With panic for his beloved pet in his eyes, he asks me, “can I trust you?”

Supposes my response is, “Well, what do you mean by trust? Are you attempting to assess my level of competence? Or are you wanting to know the degree to which our values are aligned? In fact, it’s not even clear if “trust” makes sense outside of a social context which punishes defectors…”

For most normal people, this response sets off all kinds of alarm bells. His was is a simple question, but I seem unwilling to answer. My neighbor now has good reason to think that he can’t trust me: One reason why I would be desiring so much legalistic clarity about what “trust” means, is because I’m intending to hold to the letter of my agreements, but not the spirit, to screw him over while claiming that the precise definition shields me from reproach. Or maybe it means I am something-like-autistic, and I just legitimately don’t understand the concept of trust. In either case, he should be much more reluctant to trust me with his cat.

In this circumstance, it seems like the correct thing to do is put aside nuance, and give the simple answer: “Yes. You can trust me.” The shared social context has a very limited number of buckets (possibly only “yes” and “no”) and in fact the most correct thing to say is “yes” (presuming you in fact will take care of his cat). It is both the case that the available ontology is too simple to support a full answer, and also the case that the response “the available ontology is too simple sot support a full answer” rounds down to “no”, which is not the correct response in this situation.

Being a stickler sabotages cooperation, when that cooperation is shallow.

However, being a stickler is necessary in other  contexts where you are aiming for a more robust cooperation.

For instance, if a partner and I are considering getting married, (or maybe considering breaking up) and she asks me “Are you committed to this relationship?”

In this situation, skipping over the nuance of what is meant by “committed” is probably a mistake. It seems pretty likely that the concepts that she and I reference with that word are not exactly overlapping. And the “edge cases” seem pretty likely to be relevant down the line.

For instance, one of us might be meaning “committed” to be a kind of emotional feeling, and the other might be meaning it to be a measure of resources (of time, attention, life) that you are promising to invest.

Or one of you might feel that “committed” means that you’ll want to spend most of your time together, if circumstances allow. That’s not part of the other’s concept of committed, and in fact, they will feel defensive of their own autonomy when “circumstances allow”, and their partner expects them to spend most of their time together.

Not having clarity about what exactly you’re agreeing to, promising, or signaling to the other, seems like it is undermining the ability for robust cooperation.

Unless you insist on this conceptual nuance, there isn’t actually clarity about the nature of the relationship, and neither party can in full confidence rely on it. (In practice, it maybe more likely that two partners don’t notice this conceptual mismatch, and so do put their weight on the relationship, only to be burned later.)

If I want to have a robust, long standing marriage with my partner, it seems like we really do need to do enough philosophy to be clear about, and have common knowledge about, our shared concepts. [1]

I posit that this is generally true: Insistence on conceptual nuance can undermine cooperation, particularly in “shallow” interactions. But a failure to insist on conceptual nuance can also undermine cooperation, in other contexts.


[1] Although, maybe in some contexts you don’t need to do the philosophy because tradition does this work for you. If culture mandates a very specific set of requirements around marriage, or business dealings, or what have you, you can safely operate on the assumption that your concepts and the other person’s concepts are sufficiently similar for all practical considerations? The cultural transmission is high bandwidth enough that you do both have (practically) the same concepts?

I don’t know.

Addendum: 2019-11-16: I just realized that this dynamic is exactly(?) isomorphic to the valley of bad rationality, but at the interpersonal, instead of the personal level

Consideration Factoring: a relative of Double Crux

[Epistemic status: work in progress, at least insofar as I haven’t really nailed down the type-signature of “factors” in the general case. Nevertheless, I do this or something like this pretty frequently and it works for me. There are probably a bunch of prerequisites, only some of which I’m tracking, though.]

This posts describes a framework I sometimes use when navigating (attempting to get to the truth of and resolve) a disagreement with someone. It is clearly related to the Double Crux framework, but is distinct enough, that I think of it as an alternative to Double Crux. (Though in my personal practice, of course, I sometimes move flexibly between frameworks).

I claim no originality. Just like everything in the space of rationality, many people already do this, or something like this.

Articulating the taste that inclines me to use one method in one conversational circumstance and a different method in a different circumstance is tricky. But a main trigger for using this one is when I am in a conversation with someone, and it seems like they keep “jumping all over the place” or switching between different arguments and considerations. Whenever I try to check if a consideration is a crux (or share an alternative model of that consideration), they bring up a different consideration. The conversation jumps around, and we don’t dig into any one thing for very long. Everything feels kind of slippery somehow.

(I want to emphasize that this pattern does not mean the other person is acting in bad faith. Their belief is probably a compressed gestalt of a bunch of different factors, which are probably not well organized by default. So when you make a counter argument to one point, they refer to their implicit model, and the counterpoint you made seems irrelevant or absurd, and they try to express what that counterpoint is missing.)

When something like that is happening, it’s a trigger to get paper (this process absolutely requires externalized, shared, working memory), and start doing consideration factoring.

Step 1: Factor the Considerations

1a: List factors

The first step is basically to (more-or-less) goal factor. You want to elicit from your partner, all of the considerations that motivate their position, and write those down on a piece of paper.

For me, so far, usually this involves formulating the disagreement as an action or a world state, and then asking what are the important consequences of that action or world-state. If your partner thinks that that it is a good idea to invest 100,000 EA dollars in project X, and you disagree, you might factor all of the good consequences that your partner expects from project X.

However, the type signature of your factors is not always “goods.” I don’t yet have a clean formalism that describes what the correct type signature is, in full generality. But it is something like “reasons why Z is important”, or “ways that Z is important”, where the two of you disagree about the importance of Z.

For instance, I had a disagreement with someone about how important / valuable it is that rationality development happen within CFAR, as opposed to some other context: He thought it was all but crucial, or at least throwing away huge swaths of value, while I thought it didn’t matter much one way or the other. More specifically, he said that he thought that CFAR had a number of valuable resources, that it would be very costly for some outside group to accrue.

So together, we made a list of those resources. We came up with:

  1. Ability to attract talent
  2. The ability to propagate content through the rationality and EA communities.
  3. The Alumni network
  4. Funding
  5. Credibility and good reputation in the rationality community.
  6. Credibility and good reputation in the broader world outside of the rationality community.

My scratch paper:

IMG_3024 2 copy(We agreed that #5 was really only relevant insofar as it contributed to #2, so we lumped them together. The check marks are from later in the conversation, after we resolved some factors.)

Here, we have a disagreement which is something like “how replaceable are the resources that CFAR has accrued”, and we factor into the individual resources, each of which we can engage with separately. (Importantly, when I looked at our list, I thought that for each resource, either 1) it isn’t that important, 2) CFAR doesn’t have much of it, or 3) it would not be very hard for a new group to acquire it from scratch.)

1b: Relevance and completeness checks

Importantly, don’t forget to do relevance and completion checks:

  • If all of these considerations but one were “taken care of” to your satisfaction, would you change your mind about the main disagreement? Or is that last factor doing important work, that you don’t want to loose?
  • If all of these consideration were “taken care of” to your satisfaction, would you change your mind about the main disagreement? Or is something missing?

[Notice that the completeness check and relevance check on each factor, together, is isomorphic to a crux-check on the conjunction of all of the factors.]

Step 2: Investigate each of the factors

Next, discuss each of the factors in turn.

2a: Rank the factors

Do a breadth first analysis of which branches seem most interesting to talk about, where interesting is some combination of “how crux-y that factor is to your view”, “how cruxy that factor is for your partner’s view”, and “how much the two of you disagree about that factor.”

You’ll get to everything eventually, but it makes sense to do the most interesting factors first.

The two of you spend a few minutes superficially discussing each one, and assessing which seems most juicy to continue with first.

2b: Discuss each factor in turn

Usually, I’ll take out a new sheet of paper for each factor.

Here you’ll need to be seriously and continuously applying all of the standard Double Crux / convergence TAPs. In particular, you should be repeatedly...

  • Operationalizing to specific cases
  • Paraphrasing what you understand your partner to have said,
  • Crux checking (for yourself), all of their claims, as they make them.

[I know. I know, I haven’t even written up all of these basics, yet. I’m working on it.]

This is where the work is done, and where most of the skill lies. As a general heuristic, I would not share an alternative model or make a counterargument until we’ve agreed on a specific, visualizable story that describes my partner’s point and I can paraphrase that point to my partner’s satisfaction (pass their ITT).

In general, a huge amount of the heavily lifting is done by being ultra specific. You want to be working with very specific stories with clarity about who is doing what, and what the consequences are.  If my partner says “MIRI needs prestige in order to attract top technical talent”, I’ll attempt to translate that into a specific story…

“Ok, so for instance, there’s a 99.9 percentile programmer, let’s call him Bob, who works at Google, and he comes to an AIRCS workshop, and has a good time, and basically agrees that AI safety is important. But he also doesn’t really want to leave his current job, which is comfortable and prestigious, and so he sort of slides off of the whole x-risk thing. But if MIRI were more prestigious, in the way that say, RAND used to be prestigious (most people who read the New York times know about MIRI, and people are impressed when you say you work at MIRI), Bob is much more likely to actually quit his job and go work at on AI alignment at MIRI?”

…and then check if my partner feels like that story has captured what they were trying to say. (Checking is important! Much of the time, my partner wants to correct my story, in some way. I keep offering modified versions it until I give a version that they certify as capturing their view.)

Very often, telling specific stories clears out misconceptions: either correcting my mistaken understanding of what the other person is saying, or helping me to notice places where some model that I’m proposing doesn’t seem realistic in practice. [One could write several posts on just the skillful use of specificity in converge conversations.]

Similarly, you have to be continually maintaining the attitude of trying to change your own mind, not trying to convince your partner.

Sometimes the factoring is recursive: it makes sense to further subdivide consideration, within each factor. (For instance, in the conversation referenced above about rationality development at CFAR, we took the factor of “CFAR has or could easily get credibility outside of the rationality / EA communities” and asked “what does extra-community credibility buy us?” This produced the factors “access to governments agencies, fortune 500 companies, universities and other places of power” and “leverage for raising the sanity waterline.” Then we might talk about how much each of those sub-factors matter.)

(In my experience) your partner will probably still try and jump around between the factors: you’ll be discussing factor 1, and they’ll bring in a consideration from factor 4. Because of this, one of the things you need to be doing is, gently and firmly, keeping the discussion on one factor at a time. Every time my partner seems to try to jump, I’ll suggest that that seems like it is more relevant to [that other factor], than to this one, and check if they agree. (The checking is really important! It’s pretty likely that I’ve misunderstood what they’re saying.) If they agree, then I’ll say something like “cool, so let’s put that to the side for a moment, and just focus on [the factor we’re talking about], for the moment. We’ll get to [the other factor] in a bit.” I might also make a note of the point they were starting to make on the paper of [the other factor]. Often, they’ll try to jump a few more times, and then get the hang of this.

In general, while you should be leading and facilitating the process, every step should be a consensus between the two of you. You suggest a direction to steer the conversation, and check if that direction seems good to your partner. If they don’t feel interested in moving in that direction, or feel like that is leaving something important out, you should be highly receptive to that.

If at any point your partner feels “caught out”, or annoyed that they’ve trapped themselves, you’ve done something wrong. This procedure and mapping things out on paper should feel something like relieving to them, because we can take things one at a time, and we can trust that everything important will be gotten to.

Sometimes, you will semi-accidentally stumble across a Double Crux for your top level disagreement that cuts across your factors. In this case you could switch to using the Double Crux methodology, or stick with Consideration Factoring. In practice, finding a Double Crux means that it becomes much faster to engage with each new factor, because you’ve already done the core untangling work for each one, before you’ve even started on it.

Conclusion

This is just one framework among a few, but I’ve gotten a lot of mileage from it lately.

Metacognitive space

[Part of my Psychological Principles of Personal Productivity, which I am writing mostly in my Roam, now.]

Metacognitive space is a term of art that refers to a particular first person state / experience. In particular it refers to my propensity to be reflective about my urges and deliberate about the use of my resources.

I think it might literally be having the broader context of my life, including my goals and values, and my personal resource constraints loaded up in peripheral awareness.

Metacognitive space allows me to notice aversions and flinches, and take them as object, so that I can respond to them with Focusing or dialogue, instead of being swept around by them. Similarly, it seems to, in practice, to reduce my propensity to act on immediate urges and temptations.

[Having MCS is the opposite of being [[{Urge-y-ness | reactivity | compulsiveness}]]?]

It allows me to “absorb” and respond to happenings in my environment, including problems and opportunities, taking considered instead of semi-automatic, first response that occurred to me, action. [That sentence there feels a little fake, or maybe about something else, or maybe is just playing into a stereotype?]

When I “run out” of meta cognitive space, I will tend to become ensnared in immediate urges or short term goals. Often this will entail spinning off into distractions, or becoming obsessed with some task (of high or low importance), for up to 10 hours at a time.

Some activities that (I think) contribute to metacogntive awareness:

  • Rest days
  • Having a few free hours between the end of work for the day and going to bed
  • Weekly [[Scheduling]]. (In particular, weekly scheduling clarifies for me the resource constraints on my life.)
  • Daily [[Scheduling]]
  • [[meditation]], including short meditation.
    • Notably, I’m not sure if meditation is much more efficient than just taking the same time to go for a walk. I think it might be or might not be.
  • [[Exercise]]?
  • Waking up early?
  • Starting work as soon as I wake up?
    • [I’m not sure that the thing that this is contributing to is metacogntive space per se.]

[I would like to do a causal analysis on which factors contribute to metacogntive space. Could I identify it in my toggl data with good enough reliability that I can use my toggl data? I guess that’s one of the things I should test? Maybe with a servery asking me to rate my level of metacognitive space for the day every evening?]

Erosion

Usually, I find that I can maintain metacogntive space for about 3 days [test this?] without my upkeep pillars.

Often, this happens with a sense of pressure: I have a number of days of would-be-overwhelm which is translated into pressure for action. This is often good, it adds force and velocity to activity. But it also runs down the resource of my metacognitive space (and probably other resources). If I loose that higher level awareness, that pressure-as-a-forewind, tends to decay into either 1) a harried, scattered, rushed-feeling, 2) a myopic focus on one particular thing that I’m obsessively trying to do (it feels like an itch that I compulsively need to scratch), 3) or flinching way from it all into distraction.

[Metacognitive space is the attribute that makes the difference between absorbing, and then acting gracefully and sensibly to deal with the problems, and harried, flinching, fearful, non-productive overwhelm, in general?]

I make a point, when I am overwhelmed, or would be overwhelmed to make sure to allocate time to maintain my metacognitive space. It is especially important when I feel so busy that I don’t have time for it.

When metacognition is opposed to satisfying your needs, your needs will be opposed to metacognition

One dynamic that I think is in play, is that I have a number of needs, like the need for rest, and maybe the need for sexual release or entertainment/ stimulation. If those needs aren’t being met, there’s a sort of build up of pressure. If choosing consciously and deliberately prohibits those needs getting met, eventually they will sabotage the choosing consciously and deliberately.

From the inside, this feels like “knowing that you ‘shouldn’t’ do something (and sometimes even knowing that you’ll regret it later), but doing it anyway” or “throwing yourself away with abandon”. Often, there’s a sense of doing the dis-endorsed thing quickly, or while carefully not thinking much about it or deliberating about it: you need to do the thing before you convince yourself that you shouldn’t.

[[Research Questions]]

What is the relationship between [[metacognitive space]] and [[Rest]]?

What is the relationship between [[metacognitive space]] and [[Mental Energy]]?

Desires vs. reflexes

[Epistemic status: a quick thought that I had a minute ago.]

There are goals / desires (I want to have sex, I want to stop working, I want to eat ice cream) and there are reflexes (anger, “wasted motions”, complaining about a problem, etc.).

If you try and squash goals / desires, they will often (not always?) resurface around the side, or find some way to get met. (Why not always? What are the difference between those that do and those that don’t?) You need to bargain with them, or design outlet polices for them.

Reflexes on the other hand are strategies / motions that are more or less habitual to you. These you train or untrain.

 

Some musings on human brutality and human evil

[epistemic status: semi-poetic musing]

I’m listening to Dan Carlin’s Hardcore History: Supernova in the East this week. The biggest thing that’s struck me so far is the ubiquity of brutality and atrocity. In this series, Carlin describes the Rape of Nanjing in particular, but he points out that the “police reports” from that atrocity could just as well describe the Roman sack of Cremona, or the Turkish conquest of Byzantium, not to mention the constant brutality of the the Mongol hordes.

I’m left with an awareness that there’s an evil in human nature, an evolutionary darkness, inextricably bound up with us: in the right context, apparently decent, often god-fearing, young men will rape and plunder and murder en mass. There’s violence under the surface.

Luckily, I personally live in a democratic great power that maintains a monopoly on the use of force. At least for me (white and middle class), and at least for now (geopolitics shifts rapidly, and many of the Jews of 1940 Europe, felt that something like the Holocaust could never happen [in their country]), power, in the form of the largest, most technological advanced military ever, and in the form of nuclear weapons, is arrayed to protect me against that violence.

But that protection is bought with blood and brutality. Not just in the sense that America is founded on the destruction of the Native Americans that were here first, and civilization itself was built on the backs of forceful enslavement (though that is very much the case). In the sense that elsewhere in the world, today, that American military might is destroying someone else’s home. I recently learned about the Huế Massacre and other atrocities of the Vietnam war, and I’m sure similar things (perhaps not as bad), happen every year. Humans can’t be trusted not to abuse their power.

It’s almost like a law of nature: if someone has the power to hurt another, that provides opportunity for the darkness in the human soul to flower in violence. It’s like a conservation law of brutality.

No. That’s not right. Brutality is NOT conserved. It can be better or worse. (To say otherwise would be an unacceptable breach of epistemic and ethics). But brutality is inescapable.

So what to do? I the only way I can buy safety for myself and my friends is with violence towards others?

The only solution that I can think of is akin to Paretotopian ideas: could we make it so that there is a monopoly on the use of force, but no human has it?

I’m imagining something like an AGI whose source code was completely transparent: everyone could see and read the its decision theory. And all that it would do is prevent the use of violence, by anyone. Anytime someone attempts to commit violence the nano-machines literally stay their hand. (It might also have to produce immortality pills, and ensure that everyone could access them if they wanted too.) And other than that, it lets humans handle things for themselves. “A limited sovereign on the blockchain.”

I imagine that the great powers would be unwilling to give up their power, unless they felt so under threat (and loss averse), that this seemed like a good compromise. I imagine that “we” would have to bully the world into adopting something like this. The forces of good in human nature would have to have the underhand, for long enough to lock in the status quo, to banish violence forever.

 

 

Two models of anxiety

[This is a confused thought that feels like it is missing something.]

I have two competing models of anxiety.

The first one, is basically the one I outlined here. There’s a part of me that is experiencing a fear or pain, and that part seeks distraction and immediate gratification to compensate for that pain.

But after reading about [[Physiological Arousal]], I have a secondary hypothesis. Instead of postulating a “part” that is motivated to seek distractions, maybe it is just that the fear triggers a fight or flight response, which increases arousal, which causes decreased attentional stability.

These different models suggest different places for intervention: in the one case, I ought to dialogue with the part that is seeking distraction or relief (?), and in the second case, I need to lower my arousal.

Or maybe both of those are mistaken, and I should just intervene on my scattered attention directly, perhaps by holding my attention on some external object for a minute (a kind of micro [[meditation]]).

 

 

Some musings about exercise and time discount rates

[Epistemic status: a half-thought, which I started on earlier today, and which might or might not be a full thought by the time I finish writing this post.]

I’ve long counted exercise as an important component of my overall productivity and functionality. But over the past months my exercise habit has slipped some, without apparent detriment to my focus or productivity. But this week, after coming back from a workshop, my focus and productivity haven’t really booted up.

Here’s a possible story:

Exercise (and maybe mediation) expands the effective time-horizon of my motivation system. By default, I will fall towards attractors of immediate gratification and impulsive action, but after I exercise, I tend to be tracking, and to be motivated by, progress on my longer term goals. [1]

When I am already in the midst of work: my goals are loaded up and the goal threads are primed in short term memory, this sort of short term compulsiveness causes me to fall towards task completion: I feel slightly obsessed about finishing what I’m working on.

But if I’m not already in the stream of work, seeking immediate gratification instead drives me to youtube and web comics and whatever. (Although it is important to note that I did switch my non self tracking web usage to Firefox this week, and I don’t have my usual blockers for youtube and for SMBC set up yet. That might totally account for the effect that I’m describing here.)

In short, when I’m not exercising enough, I have less meta cognitive space for directing my attention and choosing what is best do do. But if I’m in the stream of work already, I need that meta cognitive space less: because I’ll default to doing more of what I’m working on. (Though, I think that I do end up getting obsessed with overall less important things, compared to when I am maintaining metacognitive space). Exercise is most important for booting up and setting myself up to direct my energies.


[1] This might be due to a number of mechanisms:

  • Maybe the physical endorphin effect of exercise has me feeling good, and so my desire for immediate pleasure is sated, freeing up resources for longer term goals.
  • Or maybe exercise involves engaging in intimidate discomfort for the sake of future payoff, and this shifts my “time horizon set point” or something. (Or maybe it’s that exercise is downstream of that change in set point.)
    • If meditation also has this time-horizon shifting effect, that would be evidence for this hypothesis.
    • Also if fasting has this effect.
  • Or maybe, it’s the combination of both of the above: engaging in delayed gratification, with a viscerally experienced payoff, temporarily retrains my motivation system for that kind of thing.)
  • Or something else.

 

Some notes on Von Neumann, as a human being

I recently read Prisoner’s Dilemma, which half an introduction to very elementary game theory, and half a biography of John Von Neumann, and watched this old PBS documentary about the man.

I’m glad I did. Von Neumann has legendary status in my circles, as the smartest person ever to live. [1] Many times I’ve written the words “Von Neumann Level Intelligence” in a AI strategy document, or speculated about how many coordinated Von Neumanns would it take to take over the world. (For reference, I now think that 10 is far too low, mostly because he didn’t seem to have the entrepreneurial or managerial dispositions.)

Learning a little bit more about him was humanizing. Yes, he was the smartest person ever to live, but he was also an actual human being, with actual human traits.

Watching this first clip, I noticed that I was surprised by a number of thing.

  1. That VN had an accent. I had known that he was Hungarian, but somehow it had never quite propagated that he would speak with a Hungarian accent.
  2. That he was middling height (somewhat shorter than the presenter he’s talking too).
  3. The thing he is saying is the sort of thing that I would expect to hear from any scientist in the public eye, “science education is important.” There is something revealing about Von Neumann, despite being the smartest person in the world, saying basically what I would expect Neil DeGrasse Tyson to say in an interview. A lot of the time he was wearing his “scientist / public intellectual” hat, not the “smartest person ever to live” hat.

Some other notes of interest:

He was not a skilled poker player, which punctured my assumption that Von Neumann was omnicompetent. (pg. 5) Nevertheless, poker was among the first inspirations for game theory. (When I told this to Steph, she quipped “Oh. He wasn’t any good at it, so he developed a theory from first principles, describing optimal play?” For all I know, that might be spot on.)

Perhaps relatedly, he claimed he had low sales resistance, and so would have his wife come clothes shopping with him. (pg. 21)


He was sexually crude, and perhaps a bit misogynistic. Eugene Wigner stated that “Johny believed in having sex, in pleasure, but not in emotional attachment. HE was interested in immediate pleasure and little comprehension of emotions in relationships and mostly saw women in terms of their bodies.” The journalist Steve Heimes wrote “upon entering an office where a pretty secretary was working, von Neumann habitually would bend way over, more or less trying to look up her dress.” (pg. 28) Not surprisingly, his relationship with his wife, Klara, was tumultuous, to say the least.

He did however, maintain a strong, life long, relationship with his mother (who died the same year that he did).

Overall, he gives the impression of being a genius, overgrown child.


Unlike many of his colleagues, he seemed not to share the pangs conscience that afflicted many of the bomb creators. Rather than going back to academia following the war, he continued doing work for the government, including the development of the Hydrogen bomb.

Von Neumann advocated preventative war: giving the Soviet union an ultimatum, of joining a world government, backed by the threat of (and probable enaction of) nuclear attack, while the US still had a nuclear monopoly. He famously said of the matter, “If you say why not bomb them tomorrow, I say why not today? If you say today at 5 o’clock, I say why not 1 o’clock.”

This attitude was certainly influenced by his work on game theory, but it should also be noted that Von Neumann hated communism.

Richard Feynman reports that Von Neumann, in their walks through the Los Alamos desert, convinced him to adopt and attitude of “social irresponsibility”, that one “didn’t have to be responsible for the world he was in.”


Prisoner’s dilemma says that he and his collaborators “pursued patents less aggressively than the could have”. Edward Teller commented, “probably the IBM company owes half its money to John Von Neumann.” (pg. 76)

So he was not very entrepreneurial, which is a bit of a shame, because if he had the disposition he probably could have made sooooo much money / really taken substantial steps towards taking over the world. (He certainly had the energy to be an entrepreneur: he only slept for a few hours a night, and was working for basically all his working hours.


He famously always wore a grey oxford 3 piece suit, including when playing tennis with Stanislaw Ulam, or when riding a donkey down the grand canyon. But, I am not clear why. Was that more comfortable? Did he think it made him look good? Did he just not want to have to ever think about clothing, and so preferred to be over-hot in the middle of the Los Alamos desert, rather than need to think about if today was “shirt sleeves weather”?


Von Neumann himself once commented on the strange fact of so many Hungarian geniuses growing up in such a small area, in his generation:

Stanislaw Ulam recalled that when Von Neumann was asked about this “statistically unlikely” Hungarian phenomenon, Von Neumann “would say that it was a coincidence of some cultural factors which he could not make precise: an external pressure on the whole society of this part of Central Europe, a subconscious feeling of extreme insecurity in individual, and the necessity of producing the unusual or facing extinction.” (pg. 66)


One thing that surprised me most was that it seems that, despite being possibly the smartest person in modernity, he would have benefited from attending a CFAR workshop.

For one thing, at the end of his life, he was terrified of dying. But throughout the course of his life he made many reckless choices with his health.

He ate gluttonously and became fatter and fatter over the course of his life. (One friend remarked that he “could count anything but calories.”)

Furthermore, he seemed to regularly risk his life when driving.

Von Neuman was an aggressive and apparently reckless driver. He supposedly totaled his car every year or so. An intersection in Princeton was nicknamed “Von Neumann corner” for all the auto accidents he had there. records of accidents and speeding arrests are preserved in his papers. [The book goes on to list a number of such accidents.] (pg. 25)

(Amusingly, Von Neumann’s reckless driving seems due, not to drinking and driving, but to singing and driving. “He would sway back and forth, turning the steering wheel in time with the music.”)

I think I would call this a bug.

On another thread, one of his friends (the documentary didn’t identify which) expressed that he was over-impressed by powerful people, and didn’t make effective tradeoffs.

I wish he’d been more economical with his time in that respect. For example, if people called him to Washington or elsewhere, he would very readily go and so on, instead of having these people come to him. It was much more important, I think, he should have saved his time and effort.

He felt, when the government called, [that] one had to go, it was a patriotic duty, and as I said before he was a very devoted citizen of the country. And I think one of the things that particularly pleased him was any recognition that came sort-of from the government. In fact, in that sense I felt that he was sometimes somewhat peculiar that he would be impressed by government officials or generals and so on. If a big uniform appeared that made more of an impression than it should have. It was odd.

But it shows that he was a person of many different and sometimes self contradictory facets, I think.

Stanislaw Ulam speculated, “I think he had a hidden admiration for people and organizations that could be tough and ruthless.” (pg. 179)

From these statements, it seems like Von Neumann leapt at chances to seem useful or important to the government, somewhat unreflectively.

These anecdotes suggest that Von Neumann would have gotten value out of Goal Factoring, or Units of Exchange, or IDC (possibly there was something deeper going on, regarding a blindspots around death, or status, but I think the point still stands, and he would have benefited from IDC).

Despite being the discoverer/ inventor of VNM Utility theory, and founding the field of Game Theory (concerned with rational choice), it seems to me that Von Neumann did far less to import the insights of the math into his actual life than say, Critch.

(I wonder aloud if this is because Von Neumann was born and came of age before the development of cognitive science. I speculate that the importance of actually applying theories of rationality in practice, only becomes obvious after Tversky and Kahneman demonstrate that humans are not rational by default. (In evidence against this view: Eliezer seems to have been very concerned with thinking clearly, and being sane, before encountering Heuristics and Biases in his (I belive) mid 20s. He was exposed to Evo Psych though.))


Also, he converted to Catholicism at the end of his life, based on Pascal’s Wager. He commented “So long as there is the possibility of eternal damnation for nonbelievers it is more logical to be a believer at the end”, and “There probably has to be a God. Many things are easier to explain if there is than if there isn’t.”

(According to wikipedia, this deathbed conversion did not give him much comfort.)

This suggests that he would have gotten value out of reading the sequences, in addition to attending a CFAR workshop.

 

Basic Double Crux pattern example

Followup to: The Basic Double Crux Pattern

Intro

This is a continuation of my previous post, in which I outlined the what I currently see as the core pattern of double crux finding. This post will most be an example of that pattern in action.

While this example will be more realistic than the intentionally silly example in the last post, it will still be simplified, for the sake of brevity and clarity. There’s a tradeoff between realism and clarity of demonstration. The following conversation is mostly-fictional (though inspired by Double Cruxes that I have actually observed). It is intended to be realistic enough that this could have been a real conversation, but is also specifically demonstrating the basic Double Crux pattern, (as opposed to other moves or techniques that are often relevant.)

Review

As a review, the key conversational moves of Double Crux are…

  1. Finding a single crux:
    1. Checking for understanding of P1’s point.
    2. Checking if that point is a crux for P1.
  2. Finding a double crux:
    1. Checking P2’s belief about P1’s crux.
    2. Checking if P2’s crux is also a crux for P1.

As an excise, I encourage you to go through the example below below and track what is currently happening in the context of this frame: Which parts of the conversation correspond to which conversational move?

Note that, in general, participants will be intent on saying all kinds of things that aren’t cruxes, and part of the facilitator’s role is to keep the conversation on track, while also engaging with their concerns.

Example: Carol and Daniel on gun control legislation

[Important note: This example is entirely fictional. It doesn’t represent the views, of anyone, including me. The arguments that these two character’s make are constructions of arguments that I have heard from CFAR participants, and things that I heard in podcasts years ago, and things I made up. I have no idea if these are in fact the most important considerations about gun control, or if the facts that these characters reference are actually true, only that they are plausible positions that a person could hold.]

Two people, Carol and Daniel, are having a discussion about politics. In particular, they’re discussing new gun laws that would heavily restrict citizens ability to buy and sell firearms. Carol is in favor of the laws, but Daniel is opposed.

Facilitator: Daniel, do you want to tell us why you think this law is a bad idea?

Daniel: Look every time there’s a mass shooting, everyone gets up in arms about it, and politicians scramble over each other to show how tough they are, and how bad an event the shooting was, and then they pass a bunch of laws that don’t actually help, and add a bunch of red tape for people who want to buy firearms for legitimate uses.

Facilitator: Ok. There were a lot of things in there, but one part of is you said that these laws don’t work?

Daniel: That’s right.

Facilitator: you mean that they don’t reduce incidence of mass shooting?

Daniel: Yeah, but mass shootings are almost a rounding error.

Facilitator: [suggesting an operationalization] So what do you mean when you say they don’t help? That they don’t reduce the murder rate?

Daniel: Yeah. That’s right.

Facilitator: Ok. If you found out that this law in particular did reduce the US murder rate, would you be in favor of it?

Daniel: Well, it would have to have a significant impact. If the effect is so small as to be basically unnoticeable, then it probably isn’t worth the constraint on millions of people across the US.

Facilitator: Ok. But if you knew that this law would “meaningfully” reduce the murder rate, would you change your mind about this law? (Where meaningfully means something like “more than 5%”?)

Daniel: Yeah, that sounds right.

Facilitator: Great. Ok. So it sounds like whether this law would impact the murder rate is a crux for you.

Carol, do you think that this law would reduce the murder rate?

Carol: Yeah.

Facilitator: By more than 5%?

Carol: I’m not sure by how much specifically, but something like that.

Facilitator: Ok. If you found it that this law would actually have no effect (or close to no effect) on the murder rate would you change your mind?

Carol: I think it would reduce the murder rate.

Facilitator: Cool. You currently think that this law would reduce the murder rate. I want to talk about why you think that in a moment. But first, I want to check if it’s a crux. Suppose you found really strong evidence that this this law actually wouldn’t reduce the murder rate at all (I realize that you currently think that it will), would you still support this law?

Carol: Yeah. I guess there wouldn’t be much of a point to it in that case.

Facilitator: Cool. So it sounds like this is a double Crux. You, Daniel, would change your mind about this law if you found out that it would improve the murder rate, substantially, and you, Carol, would change your mind, if you found out it wouldn’t improve the murder rate. High-five!

The participants high-five, and write down their cruxes.

[insert picture]

Facilitator: Alright. So it sounds like we should talk about this double crux, why do each of you think that this would improve the murder rate?

Carol: Well, I’m aware of other countries that implemented similar reforms, and it did in fact improve incidents of violent crime in those countries.

Facilitator: Ok. So the reason why you think that this law would reduce the murder rate is that it worked in other countries?

Carol: That’s right.

Facilitator: Ok. Suppose you found out that actually laws like this did not help in those other countries? Would that change your mind about how likely this reform is to work in the US?

Carol: I’m pretty sure that it did work. In Sweden, for instance.

Facilitator: Ok. Suppose you found out that the evidence in those countries, like Sweden, didn’t hold up, or it turns out you misread the studies or something.

Carol: If all those studies turned out to be bunk…Yeah. I guess that would cause me to change my mind.

Facilitator: Cool. So it sounds like that’s a crux for you.

Daniel, presumably, you think that laws like this don’t lower the murder rate in other countries. Is that right?

Daniel: Well, I don’t know about those studies in particular. But I’m generally skeptical here, because this topic is politicized. I think you can’t rely on most of the papers in this area.

Facilitator: Ok. Is that a crux for you?

I don’t think so. There are a lot of gun law interventions that are effective in other countries, but don’t work in the US, because there are just so many more guns in the US, than in other countries in the world. Like, because there are so many guns already in circulation, making it harder to buy a gun today, makes almost no difference for how easy it is to get your hands on a gun. In other countries where there are fewer weapons lying around, things like gun buybacks, or restrictions on gun purchases, have more of an effect, because the size of the pool of the existing guns is larger, and so those interventions have a much smaller proportional impact.

Like, you have to account for the black market. There’s a big black market in weapons in the US, much bigger than in most European countries, I think.

Facilitator: [Summarizing] Ok. It sounds like you’re saying that you don’t think making harder to buy guns has an impact on the murder rate because, if you want to kill someone in the US, you can get your hand on a gun one way or another.

Daniel: Yeah.

Facilitator: Ok. Is that a crux for you? Suppose that you found out that, actually reforms like this one do make it much harder for would-be criminals to get a gun. Would that change your mind about whether this reform would shift the murder rate?

Daniel: Let’s see…I might want to think about it more, but yeah, I think if I knew that it actually made it harder for those people to get guns, that would change my mind, but it would have to be a lot harder. Like, it would have to be hard enough that those people end up just not having guns, not like they would have to wait an extra 2 weeks.

Facilitator: Ok. Cool. If laws like this made it so that fewer criminals had guns, you would expect those laws to improve the murder rate?

Daniel: yeah.

Facilitator: Carol, do you expect that this new gun law would cause fewer criminals to own guns?

Carol: Yeah. But it’s not just the guns. The other thing is the ammunition. This law would not just make it harder to buy guns, but would make it harder to buy bullets. It doesn’t matter how criminals got their guns (they might have inherited them from their grandmother, for all I know), if they can’t buy bullets for those guns.

Facilitator: OH-k. That’s pretty interesting. You’re saying that the way that you limit gun violence is by restricting the flow of ammunition.

Carol: Exactly.

Daniel: Huh. Ok. I hadn’t thought about that. That does seem pretty relevant.

Facilitator (to Daniel): Is that a crux for you? If this law makes it harder for criminals to get bullets, would you expect it to impact the murder rate?

Daniel: Yeah. I hadn’t thought about that. Yeah, if would-be murderers have trouble getting ammunition, then I would expect that to have an impact on the murder rate. But I don’t know how they get bullets. It might be mostly via black market channels that aren’t much effected by this kind of regulation. Also, it maybe that it’s similar to guns, in that there are already so many bullets in circulation, that it

Facilitator (to Daniel): Cool. Let’s hold off on that question until we check if it is also a crux for Carol.

Facilitator (to Carol): Is that a crux? Suppose that you found out that criminals are able to get their hands on ammunition via “unofficial”, illegal methods. Would that change your mind about how likely this policy is to reduce the murder rate in the US?

Carol: Let me think.

. . .

I’m still kind of suspicious. I would want to see some hard data of states or countries that have the a similar gun culture to the US, and also have this sorts of restrictions-

Daniel [Interrupting]: Yeah, I would obviously want that too.

Carol: …but in the absence of evidence like that, then it makes sense that if criminals  get guns and ammo via illegal channels, then a reform that makes it harder to buy weapons wouldn’t have any impact on the murder rate.

Facilitator: Ok. Awesome. So it sounds like we have a Double Crux, “would this regulation limit the availability of ammunition for would-be killers.” Separately, you Carol, have a single crux about “Would this make it harder for criminals to buy guns?” We could talk about that, but Daniel doesn’t think it’s as relevant.

Carol: No. I’m excited to just talk about the ammo for now.

Facilitator: Cool. That also makes sense to me.

And maybe underneath “Would this regulation limit the availably of ammunition to would-be killers”, there’s another question of “How do criminals mostly get their ammunition?” If they get their ammo from sources that are relevantly affected by these sorts of restrictions, then we expect it to restrict the flow of ammo to criminals. But if they have a stockpile, or get it from the black market (which get’s it from other countries, or something), then we wouldn’t expect the availability of ammo for criminals to be impacted much.

Does that sound right?

[Nods]

Facilitator: Ok then. Cool. I think you guys should high five.

Carol and Daniel high five, and the write the new cruxes on their piece of paper.

[image]

Facilitator: Also, it sounds like there was maybe another Double Crux there? That is, if there was evidence from a country or state that tried a policy like this one, and also had a similar number of guns in circulation?

Does that sound right?

Daniel: Yeah.

Carol: Yep.

Facilitator: Ok. That just seems good to note, even  if it isn’t actionable right now. If either of you encounter evidence of that sort, it seems like it would be relevant to your beliefs.

But it seems like the most useful thing to do right now is to see if we can figure out how criminals typically get ammunition.

Daniel and Carol researched this question, found a bunch of conflicting evidence, but settled on a shared epistemic state about how most criminals get their ammo. One or both of them updated, and they lived with a more detailed model of this small part world ever after.

Discussion

Hopefully this example gives a flavor of what a Double Crux conversation is (often) like.

Again, I want to point out that this example uses the person of the facilitator for convenience. While I do usually recommend getting a third person to be a facilitator, it is also totally possible for one or both of the conversational participants to execute these moves and steer the conversation.

This conversation is specifically intended to demonstrate the basic Double Crux pattern (although the normal order was disrupted a bit towards the end of the conversation), but nevertheless, we see some examples of other principles or techniques of productive disagreement resolution.

Distillation: As is common, Daniel said a lot of things in his opening statement, not all of which are relevant, or cruxes. One of the things that the facilitator needs to do is draw out the key point or points from all of those words, and then check if those are crucial. Much of the work of Double Crux is drawing out the key points each person is making, so you can check if they are cruxes.

Operationalizing: This particular example was pretty low on operationalization, but nevertheless, early on, the conversation settled on the operationalization of “reducing the murder rate”.

Persistent crux checking: Every time either Carol, or Daniel made a point, the facilitator would ask them if it was a crux.

Signposting: The facilitator frequently states what just happened (“It sounds like that’s a crux for you.”). This is often surprisingly helpful to do in a conversation. The facilitator

Distinguishing between whether a proposition is a crux, and what the truth value of a crux is: One thing that you’ll notice is that several times, the facilitator asks Carol if some proposition “A” is a crux for her, and she responds by saying, in effect, “I think A is true”. This is a fairly common thing that happens, even for quite smart people. It is important to make mental space between the questions “is X a crux for Y?”, and “is X true?”. It is often very tempting to jump in and refute a claim, before checking if it is a crux. But it is important to keep those questions separate.

False starts: There were several places where the facilitator checked for a Double Crux, and “missed”. This is natural and normal.

This conversation was weird on in that most of the considerations named were cruxes for someone. People are often inclined to share consideration, that upon reflection, they agree don’t matter at all.

Conversational avenues avoided: This is maybe the key value add of Double Crux. For every false start, there was a long, and ultimately irrelevant conversation that could have been had.

It might have been tempting for Daniel to argue about whether gun control regulation actually worked in other countries. (It is just sooo juicy to tell people exactly how they are wrong, when they’re wrong.) But that would have been a pointless exercise, (for Daniel’s belief improvement, at least), because Daniel thinks  that the data from other countries is irrelevant. At every point they stay on a track that both parties consider promising for updating.

(Of course, sometimes you do have information that is relevant to your partner’s crux, but not to your own, and it is good and proper to share that info with them.)

Inside view / outside view separation: Towards the end, we saw Carol distinguishing between outside view reasoning based on empirical data, and inside view reasoning based on thinking through probable consequences. She clearly separated these two, for herself. I think this is sometimes a useful move, so that one can feel psychologically permitted to do blue-sky reasoning, without forgetting that you would defer to the empirical data over and above your theorizing.

Conclusion

I also again want to clarify my claims:

  • I am not claiming that this pattern solves all disagreements.
  • I am not claiming that this is the only thing a person should do in conversation.
  • I am not claiming that one should always stick religiously to this pattern, even when Double Cruxing. (Though it is good to try sticking religiously to this pattern as a training exercise. This is one of the exercises that I have people do at Double Crux trainings.)

I am saying…

  1. This is my current best attempt to delineate how finding double cruxes usually works, and
  2. I have found double cruxes to be extremely useful for productively resolving disagreements.

 

Hope this helps. More to come, soon probably.

 

[As always, I’m generally willing to facilitate conversations and disagreements (Double Crux or not) for rationalists and EAs. Email me at eli [at] rationality [dot] org if that’s something you’re interested in.]

The Basic Double Crux Pattern

[This is a draft, to be posted on LessWrong soon.]

I’ve spent a lot of time developing tools and frameworks for bridging “intractable” disagreements. I’m also the person affiliated with CFAR who has taught Double Crux the most, and done the most work on it.

People often express to me something to the effect, “The important thing about Double Crux is all the low level habits of mind: being curious, being open to changing your mind, paraphrasing to check that you’ve understood, operationalizing, etc. The ‘Double Crux’ framework, itself is not very important.”

I half agree with that sentiment. I do think that those low level cognitive and conversational patterns are the most important thing, and at Double Crux trainings that I have run, most of the time is spent focusing on specific exercises to instill those low level TAPs.

However, I don’t think that the only value of the Double Crux schema is in training those low level habits. Double cruxes are extremely powerful machines that allow one to identify, if not the most efficient conversational path, a very high efficiency conversational path. Effectively navigating down a chain of Double Cruxes is like magic. So I’m sad when people write it off as useless.

In this post, I’m going to try and outline the basic Double Crux pattern, the series of 4 moves that makes Double Crux work, and give a (simple, silly) example of that pattern in action.

These four moves are not (always) sufficient for making a Double Crux conversation work, that does depend on a number of other mental habits and TAPs, but this pattern is, according to me, at the core of the Double Crux formalism.

The pattern:

The core Double Crux pattern is as follows. For simplicity, I have described this in the form of a 3-person Double Crux conversation, with two participants and a facilitator. Of course, one can execute these same moves in a 2 person conversation, as one of the participants. But that additional complexity is hard to manage for beginners.

The pattern has two parts (finding a crux, and finding a double crux), and each part is composed of 2 main facilitation moves.

Those four moves are…

  1. Clarifying that you understood the first person’s point.
  2. Checking if that point is a crux
  3. Checking the second person’s belief about the truth value of the first person’s crux.
  4. Checking the if the first person’s crux is also a crux for the second person.

In practice

The conversational flow of these moves looks something like this:

Finding a crux of participant 1:

P1: I think [x] because of [y]

Facilitator: (paraphrasing, and checking for understanding) It sounds like you think [x] because of [y]?

P1: Yep!

Facilitator: (checking for cruxyness) If you didn’t think [y], would you change your mind about [x]?

P1: Yes.

Facilitator: (signposting) It sounds like [y] is a crux for [x] for you.

Checking if it is also a crux for participant 2

Facilitator: Do you think [y]?

P2: No.

Facilitator: (checking for a Double Crux) if you did think [y] would that change your mind about [x]?

P2: Yes.

Facilitator: It sounds like [y] is a Double Crux

[Recurse, running the same pattern on [Y] ]

Obviously, in actual conversation, there is a lot more complexity, and a lot of other things that are going on.

For one thing, I’ve only outlined the best case pattern, where the participants give exactly the most convenient answer for moving the conversation forward (yes, yes, no, yes). In actual practice, it is quite likely that one of those answers will be reversed, and you’ll have to compensate.

For another thing, this formalism is rarely so simple. You might have to do a lot of conversational work to clarify the claims enough that you can ask if B is a crux for A (for instance when B is nonsensical to one of the participants). Getting through each of these steps might take fifteen minutes, in which case rather than four basic moves, this pattern describes four phases of conversation. (I claim that one of the core skills of a savvy facilitator is tracking which stage the conversation is at, which goals have you successfully hit, and which is the current proximal subgoal.)

There is also a judgment call about which person to treat as “participant 1” (the person who generates the point that is tested for cruxyness). As a first order heuristic, the person who is closer to making a positive claim over and above the default, should usually be the “p1”. But this is only one heuristic.

Example:

This is an intentionally silly, over-the-top-example, for demonstrating the the pattern without any unnecessary complexity. I’ll publish a somewhat more realistic example in the next few days.

Two people, Alex and Barbra, disagree about tea: Alex thinks that tea is great, and drinks it all the time, and thinks that more people should drink tea, and Barbra thinks that tea is bad, and no one should drink tea.

Facilitator: So, Barbra, why do you think tea is bad?

Barbra: Well it’s really quite simple. You see, tea causes cancer.

Facilitator: Let me check if I’ve got that: you think that tea causes cancer?

Barbra: That’s right.

Facilitator: Wow. Ok. Well if you found out that tea actually didn’t cause cancer, would you be fine with people drinking tea?

Barbra: Yeah. Really the main thing that I’m concerned with is the cancer-causing. If tea didn’t cause cancer, then it seems like tea would be fine.

Facilitator: Cool. Well it sounds like this is a crux for you Barb. Alex, do you currently think that tea causes cancer?

Alex: No. That sounds like crazy-talk to me.

Facilitator: Ok. But aside from how realistic it seems right now, if you found out that tea actually does cause cancer, would you change your mind about people drinking tea?

Alex: Well, to be honest, I’ve always been opposed to cancer, so yeah, if I found out that tea causes cancer, then I would think that people shouldn’t drink tea.

Facilitator: Well, it sounds like we have a double crux!

In a real conversation, it often doesn’t goes this smoothly. But this is the rhythm of Double Crux, at least as I apply it.

That’s the basic Double Crux pattern. As noted there are a number of other methods and sub-skills that are (often) necessary to make a Double Crux conversation work, but this is my current best attempt at a minimum compression of the basic engine of finding double cruxes.

I made up a more realistic example here, and I’m might make more or better examples.