Basic Double Crux pattern example

Followup to: The Basic Double Crux Pattern


This is a continuation of my previous post, in which I outlined the what I currently see as the core pattern of double crux finding. This post will most be an example of that pattern in action.

While this example will be more realistic than the intentionally silly example in the last post, it will still be simplified, for the sake of brevity and clarity. There’s a tradeoff between realism and clarity of demonstration. The following conversation is mostly-fictional (though inspired by Double Cruxes that I have actually observed). It is intended to be realistic enough that this could have been a real conversation, but is also specifically demonstrating the basic Double Crux pattern, (as opposed to other moves or techniques that are often relevant.)


As a review, the key conversational moves of Double Crux are…

  1. Finding a single crux:
    1. Checking for understanding of P1’s point.
    2. Checking if that point is a crux for P1.
  2. Finding a double crux:
    1. Checking P2’s belief about P1’s crux.
    2. Checking if P2’s crux is also a crux for P1.

As an excise, I encourage you to go through the example below below and track what is currently happening in the context of this frame: Which parts of the conversation correspond to which conversational move?

Note that, in general, participants will be intent on saying all kinds of things that aren’t cruxes, and part of the facilitator’s role is to keep the conversation on track, while also engaging with their concerns.

Example: Carol and Daniel on gun control legislation

[Important note: This example is entirely fictional. It doesn’t represent the views, of anyone, including me. The arguments that these two character’s make are constructions of arguments that I have heard from CFAR participants, and things that I heard in podcasts years ago, and things I made up. I have no idea if these are in fact the most important considerations about gun control, or if the facts that these characters reference are actually true, only that they are plausible positions that a person could hold.]

Two people, Carol and Daniel, are having a discussion about politics. In particular, they’re discussing new gun laws that would heavily restrict citizens ability to buy and sell firearms. Carol is in favor of the laws, but Daniel is opposed.

Facilitator: Daniel, do you want to tell us why you think this law is a bad idea?

Daniel: Look every time there’s a mass shooting, everyone gets up in arms about it, and politicians scramble over each other to show how tough they are, and how bad an event the shooting was, and then they pass a bunch of laws that don’t actually help, and add a bunch of red tape for people who want to buy firearms for legitimate uses.

Facilitator: Ok. There were a lot of things in there, but one part of is you said that these laws don’t work?

Daniel: That’s right.

Facilitator: you mean that they don’t reduce incidence of mass shooting?

Daniel: Yeah, but mass shootings are almost a rounding error.

Facilitator: [suggesting an operationalization] So what do you mean when you say they don’t help? That they don’t reduce the murder rate?

Daniel: Yeah. That’s right.

Facilitator: Ok. If you found out that this law in particular did reduce the US murder rate, would you be in favor of it?

Daniel: Well, it would have to have a significant impact. If the effect is so small as to be basically unnoticeable, then it probably isn’t worth the constraint on millions of people across the US.

Facilitator: Ok. But if you knew that this law would “meaningfully” reduce the murder rate, would you change your mind about this law? (Where meaningfully means something like “more than 5%”?)

Daniel: Yeah, that sounds right.

Facilitator: Great. Ok. So it sounds like whether this law would impact the murder rate is a crux for you.

Carol, do you think that this law would reduce the murder rate?

Carol: Yeah.

Facilitator: By more than 5%?

Carol: I’m not sure by how much specifically, but something like that.

Facilitator: Ok. If you found it that this law would actually have no effect (or close to no effect) on the murder rate would you change your mind?

Carol: I think it would reduce the murder rate.

Facilitator: Cool. You currently think that this law would reduce the murder rate. I want to talk about why you think that in a moment. But first, I want to check if it’s a crux. Suppose you found really strong evidence that this this law actually wouldn’t reduce the murder rate at all (I realize that you currently think that it will), would you still support this law?

Carol: Yeah. I guess there wouldn’t be much of a point to it in that case.

Facilitator: Cool. So it sounds like this is a double Crux. You, Daniel, would change your mind about this law if you found out that it would improve the murder rate, substantially, and you, Carol, would change your mind, if you found out it wouldn’t improve the murder rate. High-five!

The participants high-five, and write down their cruxes.

[insert picture]

Facilitator: Alright. So it sounds like we should talk about this double crux, why do each of you think that this would improve the murder rate?

Carol: Well, I’m aware of other countries that implemented similar reforms, and it did in fact improve incidents of violent crime in those countries.

Facilitator: Ok. So the reason why you think that this law would reduce the murder rate is that it worked in other countries?

Carol: That’s right.

Facilitator: Ok. Suppose you found out that actually laws like this did not help in those other countries? Would that change your mind about how likely this reform is to work in the US?

Carol: I’m pretty sure that it did work. In Sweden, for instance.

Facilitator: Ok. Suppose you found out that the evidence in those countries, like Sweden, didn’t hold up, or it turns out you misread the studies or something.

Carol: If all those studies turned out to be bunk…Yeah. I guess that would cause me to change my mind.

Facilitator: Cool. So it sounds like that’s a crux for you.

Daniel, presumably, you think that laws like this don’t lower the murder rate in other countries. Is that right?

Daniel: Well, I don’t know about those studies in particular. But I’m generally skeptical here, because this topic is politicized. I think you can’t rely on most of the papers in this area.

Facilitator: Ok. Is that a crux for you?

I don’t think so. There are a lot of gun law interventions that are effective in other countries, but don’t work in the US, because there are just so many more guns in the US, than in other countries in the world. Like, because there are so many guns already in circulation, making it harder to buy a gun today, makes almost no difference for how easy it is to get your hands on a gun. In other countries where there are fewer weapons lying around, things like gun buybacks, or restrictions on gun purchases, have more of an effect, because the size of the pool of the existing guns is larger, and so those interventions have a much smaller proportional impact.

Like, you have to account for the black market. There’s a big black market in weapons in the US, much bigger than in most European countries, I think.

Facilitator: [Summarizing] Ok. It sounds like you’re saying that you don’t think making harder to buy guns has an impact on the murder rate because, if you want to kill someone in the US, you can get your hand on a gun one way or another.

Daniel: Yeah.

Facilitator: Ok. Is that a crux for you? Suppose that you found out that, actually reforms like this one do make it much harder for would-be criminals to get a gun. Would that change your mind about whether this reform would shift the murder rate?

Daniel: Let’s see…I might want to think about it more, but yeah, I think if I knew that it actually made it harder for those people to get guns, that would change my mind, but it would have to be a lot harder. Like, it would have to be hard enough that those people end up just not having guns, not like they would have to wait an extra 2 weeks.

Facilitator: Ok. Cool. If laws like this made it so that fewer criminals had guns, you would expect those laws to improve the murder rate?

Daniel: yeah.

Facilitator: Carol, do you expect that this new gun law would cause fewer criminals to own guns?

Carol: Yeah. But it’s not just the guns. The other thing is the ammunition. This law would not just make it harder to buy guns, but would make it harder to buy bullets. It doesn’t matter how criminals got their guns (they might have inherited them from their grandmother, for all I know), if they can’t buy bullets for those guns.

Facilitator: OH-k. That’s pretty interesting. You’re saying that the way that you limit gun violence is by restricting the flow of ammunition.

Carol: Exactly.

Daniel: Huh. Ok. I hadn’t thought about that. That does seem pretty relevant.

Facilitator (to Daniel): Is that a crux for you? If this law makes it harder for criminals to get bullets, would you expect it to impact the murder rate?

Daniel: Yeah. I hadn’t thought about that. Yeah, if would-be murderers have trouble getting ammunition, then I would expect that to have an impact on the murder rate. But I don’t know how they get bullets. It might be mostly via black market channels that aren’t much effected by this kind of regulation. Also, it maybe that it’s similar to guns, in that there are already so many bullets in circulation, that it

Facilitator (to Daniel): Cool. Let’s hold off on that question until we check if it is also a crux for Carol.

Facilitator (to Carol): Is that a crux? Suppose that you found out that criminals are able to get their hands on ammunition via “unofficial”, illegal methods. Would that change your mind about how likely this policy is to reduce the murder rate in the US?

Carol: Let me think.

. . .

I’m still kind of suspicious. I would want to see some hard data of states or countries that have the a similar gun culture to the US, and also have this sorts of restrictions-

Daniel [Interrupting]: Yeah, I would obviously want that too.

Carol: …but in the absence of evidence like that, then it makes sense that if criminals  get guns and ammo via illegal channels, then a reform that makes it harder to buy weapons wouldn’t have any impact on the murder rate.

Facilitator: Ok. Awesome. So it sounds like we have a Double Crux, “would this regulation limit the availability of ammunition for would-be killers.” Separately, you Carol, have a single crux about “Would this make it harder for criminals to buy guns?” We could talk about that, but Daniel doesn’t think it’s as relevant.

Carol: No. I’m excited to just talk about the ammo for now.

Facilitator: Cool. That also makes sense to me.

And maybe underneath “Would this regulation limit the availably of ammunition to would-be killers”, there’s another question of “How do criminals mostly get their ammunition?” If they get their ammo from sources that are relevantly affected by these sorts of restrictions, then we expect it to restrict the flow of ammo to criminals. But if they have a stockpile, or get it from the black market (which get’s it from other countries, or something), then we wouldn’t expect the availability of ammo for criminals to be impacted much.

Does that sound right?


Facilitator: Ok then. Cool. I think you guys should high five.

Carol and Daniel high five, and the write the new cruxes on their piece of paper.


Facilitator: Also, it sounds like there was maybe another Double Crux there? That is, if there was evidence from a country or state that tried a policy like this one, and also had a similar number of guns in circulation?

Does that sound right?

Daniel: Yeah.

Carol: Yep.

Facilitator: Ok. That just seems good to note, even  if it isn’t actionable right now. If either of you encounter evidence of that sort, it seems like it would be relevant to your beliefs.

But it seems like the most useful thing to do right now is to see if we can figure out how criminals typically get ammunition.

Daniel and Carol researched this question, found a bunch of conflicting evidence, but settled on a shared epistemic state about how most criminals get their ammo. One or both of them updated, and they lived with a more detailed model of this small part world ever after.


Hopefully this example gives a flavor of what a Double Crux conversation is (often) like.

Again, I want to point out that this example uses the person of the facilitator for convenience. While I do usually recommend getting a third person to be a facilitator, it is also totally possible for one or both of the conversational participants to execute these moves and steer the conversation.

This conversation is specifically intended to demonstrate the basic Double Crux pattern (although the normal order was disrupted a bit towards the end of the conversation), but nevertheless, we see some examples of other principles or techniques of productive disagreement resolution.

Distillation: As is common, Daniel said a lot of things in his opening statement, not all of which are relevant, or cruxes. One of the things that the facilitator needs to do is draw out the key point or points from all of those words, and then check if those are crucial. Much of the work of Double Crux is drawing out the key points each person is making, so you can check if they are cruxes.

Operationalizing: This particular example was pretty low on operationalization, but nevertheless, early on, the conversation settled on the operationalization of “reducing the murder rate”.

Persistent crux checking: Every time either Carol, or Daniel made a point, the facilitator would ask them if it was a crux.

Signposting: The facilitator frequently states what just happened (“It sounds like that’s a crux for you.”). This is often surprisingly helpful to do in a conversation. The facilitator

Distinguishing between whether a proposition is a crux, and what the truth value of a crux is: One thing that you’ll notice is that several times, the facilitator asks Carol if some proposition “A” is a crux for her, and she responds by saying, in effect, “I think A is true”. This is a fairly common thing that happens, even for quite smart people. It is important to make mental space between the questions “is X a crux for Y?”, and “is X true?”. It is often very tempting to jump in and refute a claim, before checking if it is a crux. But it is important to keep those questions separate.

False starts: There were several places where the facilitator checked for a Double Crux, and “missed”. This is natural and normal.

This conversation was weird on in that most of the considerations named were cruxes for someone. People are often inclined to share consideration, that upon reflection, they agree don’t matter at all.

Conversational avenues avoided: This is maybe the key value add of Double Crux. For every false start, there was a long, and ultimately irrelevant conversation that could have been had.

It might have been tempting for Daniel to argue about whether gun control regulation actually worked in other countries. (It is just sooo juicy to tell people exactly how they are wrong, when they’re wrong.) But that would have been a pointless exercise, (for Daniel’s belief improvement, at least), because Daniel thinks  that the data from other countries is irrelevant. At every point they stay on a track that both parties consider promising for updating.

(Of course, sometimes you do have information that is relevant to your partner’s crux, but not to your own, and it is good and proper to share that info with them.)

Inside view / outside view separation: Towards the end, we saw Carol distinguishing between outside view reasoning based on empirical data, and inside view reasoning based on thinking through probable consequences. She clearly separated these two, for herself. I think this is sometimes a useful move, so that one can feel psychologically permitted to do blue-sky reasoning, without forgetting that you would defer to the empirical data over and above your theorizing.


I also again want to clarify my claims:

  • I am not claiming that this pattern solves all disagreements.
  • I am not claiming that this is the only thing a person should do in conversation.
  • I am not claiming that one should always stick religiously to this pattern, even when Double Cruxing. (Though it is good to try sticking religiously to this pattern as a training exercise. This is one of the exercises that I have people do at Double Crux trainings.)

I am saying…

  1. This is my current best attempt to delineate how finding double cruxes usually works, and
  2. I have found double cruxes to be extremely useful for productively resolving disagreements.


Hope this helps. More to come, soon probably.


[As always, I’m generally willing to facilitate conversations and disagreements (Double Crux or not) for rationalists and EAs. Email me at eli [at] rationality [dot] org if that’s something you’re interested in.]

The Basic Double Crux Pattern

[This is a draft, to be posted on LessWrong soon.]

I’ve spent a lot of time developing tools and frameworks for bridging “intractable” disagreements. I’m also the person affiliated with CFAR who has taught Double Crux the most, and done the most work on it.

People often express to me something to the effect, “The important thing about Double Crux is all the low level habits of mind: being curious, being open to changing your mind, paraphrasing to check that you’ve understood, operationalizing, etc. The ‘Double Crux’ framework, itself is not very important.”

I half agree with that sentiment. I do think that those low level cognitive and conversational patterns are the most important thing, and at Double Crux trainings that I have run, most of the time is spent focusing on specific exercises to instill those low level TAPs.

However, I don’t think that the only value of the Double Crux schema is in training those low level habits. Double cruxes are extremely powerful machines that allow one to identify, if not the most efficient conversational path, a very high efficiency conversational path. Effectively navigating down a chain of Double Cruxes is like magic. So I’m sad when people write it off as useless.

In this post, I’m going to try and outline the basic Double Crux pattern, the series of 4 moves that makes Double Crux work, and give a (simple, silly) example of that pattern in action.

These four moves are not (always) sufficient for making a Double Crux conversation work, that does depend on a number of other mental habits and TAPs, but this pattern is, according to me, at the core of the Double Crux formalism.

The pattern:

The core Double Crux pattern is as follows. For simplicity, I have described this in the form of a 3-person Double Crux conversation, with two participants and a facilitator. Of course, one can execute these same moves in a 2 person conversation, as one of the participants. But that additional complexity is hard to manage for beginners.

The pattern has two parts (finding a crux, and finding a double crux), and each part is composed of 2 main facilitation moves.

Those four moves are…

  1. Clarifying that you understood the first person’s point.
  2. Checking if that point is a crux
  3. Checking the second person’s belief about the truth value of the first person’s crux.
  4. Checking the if the first person’s crux is also a crux for the second person.

In practice

The conversational flow of these moves looks something like this:

Finding a crux of participant 1:

P1: I think [x] because of [y]

Facilitator: (paraphrasing, and checking for understanding) It sounds like you think [x] because of [y]?

P1: Yep!

Facilitator: (checking for cruxyness) If you didn’t think [y], would you change your mind about [x]?

P1: Yes.

Facilitator: (signposting) It sounds like [y] is a crux for [x] for you.

Checking if it is also a crux for participant 2

Facilitator: Do you think [y]?

P2: No.

Facilitator: (checking for a Double Crux) if you did think [y] would that change your mind about [x]?

P2: Yes.

Facilitator: It sounds like [y] is a Double Crux

[Recurse, running the same pattern on [Y] ]

Obviously, in actual conversation, there is a lot more complexity, and a lot of other things that are going on.

For one thing, I’ve only outlined the best case pattern, where the participants give exactly the most convenient answer for moving the conversation forward (yes, yes, no, yes). In actual practice, it is quite likely that one of those answers will be reversed, and you’ll have to compensate.

For another thing, this formalism is rarely so simple. You might have to do a lot of conversational work to clarify the claims enough that you can ask if B is a crux for A (for instance when B is nonsensical to one of the participants). Getting through each of these steps might take fifteen minutes, in which case rather than four basic moves, this pattern describes four phases of conversation. (I claim that one of the core skills of a savvy facilitator is tracking which stage the conversation is at, which goals have you successfully hit, and which is the current proximal subgoal.)

There is also a judgment call about which person to treat as “participant 1” (the person who generates the point that is tested for cruxyness). As a first order heuristic, the person who is closer to making a positive claim over and above the default, should usually be the “p1”. But this is only one heuristic.


This is an intentionally silly, over-the-top-example, for demonstrating the the pattern without any unnecessary complexity. I’ll publish a somewhat more realistic example in the next few days.

Two people, Alex and Barbra, disagree about tea: Alex thinks that tea is great, and drinks it all the time, and thinks that more people should drink tea, and Barbra thinks that tea is bad, and no one should drink tea.

Facilitator: So, Barbra, why do you think tea is bad?

Barbra: Well it’s really quite simple. You see, tea causes cancer.

Facilitator: Let me check if I’ve got that: you think that tea causes cancer?

Barbra: That’s right.

Facilitator: Wow. Ok. Well if you found out that tea actually didn’t cause cancer, would you be fine with people drinking tea?

Barbra: Yeah. Really the main thing that I’m concerned with is the cancer-causing. If tea didn’t cause cancer, then it seems like tea would be fine.

Facilitator: Cool. Well it sounds like this is a crux for you Barb. Alex, do you currently think that tea causes cancer?

Alex: No. That sounds like crazy-talk to me.

Facilitator: Ok. But aside from how realistic it seems right now, if you found out that tea actually does cause cancer, would you change your mind about people drinking tea?

Alex: Well, to be honest, I’ve always been opposed to cancer, so yeah, if I found out that tea causes cancer, then I would think that people shouldn’t drink tea.

Facilitator: Well, it sounds like we have a double crux!

In a real conversation, it often doesn’t goes this smoothly. But this is the rhythm of Double Crux, at least as I apply it.

That’s the basic Double Crux pattern. As noted there are a number of other methods and sub-skills that are (often) necessary to make a Double Crux conversation work, but this is my current best attempt at a minimum compression of the basic engine of finding double cruxes.

I made up a more realistic example here, and I’m might make more or better examples.


Some things I think about Double Crux and related topics

I’ve spent a lot of my discretionary time working on the broad problem of developing tools for bridging deep disagreements. I’m also probably the person who has spent the most time explicitly thinking about and working with CFAR’s Double Crux framework. It seems good for at least some of my high level thoughts to be written up some place, even if I’m not going to go into detail, defend, or substantiate, most of them.

The following are my own beliefs and do not necessarily represent CFAR, or anyone else. I, of course, reserve the right to change my mind.

Here are some things I currently belive:


  1. Double Crux is one (highly important) tool/ framework among many. I want to distinguish between the the overall art of untangling and resolving deep disagreements and the Double Crux tool in particular. The Double Crux framework is maybe the most important tool (that I know of) for that goal, but it is only one tool/framework in an ensemble.
    1. Some other tools/ frameworks, that are not strictly part of Double Crux (but which are sometimes crucial to bridging disagreements) include NVC, methods for managing people’s intentions and goals, various forms of co-articulation (helping to draw out an inchoate model from one’s conversational partner), etc.
    2. In some contexts other tools are substitutes for Double Crux (ie another framework is more useful) and in some cases other tools are helpful or necessary compliments (ie they solve problems or smooth the process within the Double Crux frame).
    3. In particular, my personal conversational facilitation repertoire is about 60%  Double Crux-related techniques, and 40% other frameworks that are not strictly within the frame of Double Crux.
  2. Just to say it clearly: I don’t think Double Crux is the only way to resolve disagreements, or the best way in all contexts. (Though I think it may be the best way, that I know of, in a plurality of common contexts?)
  3. The ideal use case for Double Crux is when…
    1. There are two people…
    2. …who have a real, action-relevant, decision…
    3. …that they need to make together (they can’t just do their own different things)…
    4. …in which both people have strong, visceral intuitions.
  4. Double Cruxes are almost always conversations between two people’s system 1’s.
  5. You can Double Crux between two people’s unendorsed intuitions. (For instance, Alice and Bob are discussing a question about open borders. They both agree that neither of them are economists, and that neither of them trust their intuitions here, and that if they had to actually make this decision, it would would be crucial to spend a lot of time doing research and examining the evidence and consulting experts. But nevertheless Alices current intuition leans in favor of Open Borders , and Bob’s current intuition leans agains Open Borders. This is a great starting point for a Double Crux.)
  6. Double cruxes (as in a crux that is shared by both parties in a disagreement) are common, and useful. Most disagreements have implicit Double Cruxes, though identifying them can sometimes be tricky.
  7. Conjunctive cruxes (I would change my mind about X, if I changed my mind about Y and about Z, but not if I only changed my mind about Y or Z) are common.
  8. Folks sometimes object that Double Crux won’t work, because their belief depends on a large number of considerations, each one of which has only a small impact on their overall belief, and so no one consideration is a crux. In practice, I find that there are double cruxes to be found even in cases where people expect their beliefs have this structure.
    1. Theoretically, it makes sense that we would find double cruxes here: if a person has a strong disagreement (including a disagreement of intuition) with someone else, we should expect that there are a small number of considerations doing most of the work of causing one person to think one thing and the other to think something else. It is improbable that each person’s beliefs depend on 50 factors, and for Alice, most of those 50 factors point in one direction, and for Bob, most of those 50 factors point in the other direction, unless the details of those factors are not independent. If considerations are correlated, you can abstract out the fact or belief that that generates the differing predictions in all of those separate considerations. That “generating belief” is a crux.
    2. That said, there is a different conversational approach that I sometimes use, which involves delineating all of the key considerations (then doing Goal-factoring style relevance and completeness checks), and then dealing with each consideration one at time (often via a fractal tree structure: listing the key considerations of each of the higher level considerations).
      1. This approach absolutely requires paper, and skillful (firm, gentle) facilitation, because people will almost universally try and hop around between considerations, and they need to be viscerally assured that their other concerns are recorded and will be dealt with in due course, in order to engage deeply with any given consideration.
  9. About 60% of the power of Double Crux comes from being specific.
    1. I quite like Liron’s recent sequence on being specific. It re-reminded me of some basic things that have been helpful in several recent conversations. In particular, I like the move of having a conversational paint a specific, best case scenario, as a starting point for discussion.
      1. (However, I’m concerned about Less Wrong readers trying this with a spirit of trying to “catch out” one’s conversational partner in inconsistency, instead of trying to understand what their partner wants to say, and thereby shooting themselves in the foot. I think the attitude of looking to “catch out” is usually counterproductive to both understanding and to persuasion. People rarely change their mind when they feel like you have trapped them in some inconsistency, but they often do change their mind if they feel like you’ve actually heard and understood their belief / what they are trying to say / what they are trying to defend, and then provide relevant evidence and argument. In general (but not universally) it is more productive to adopt a collaborative attitude of trying to sincerely trying to help a person articulate, clarify, and substantiate the point your partner is trying to make, even if you suspect that their point is ultimately wrong and confused.)
  10. Many (~50%) disagreements evaporate upon operationalization, but this happens less frequently than people think: and if you seem to agree about all of the facts, and agree about all specific operationalizations, but nevertheless seem to have differing attitudes about a question, that should be a flag. [I have a post that I’ll publish soon about this problem.]
  11. You should be using paper when Double Cruxing. Keep track of the chain of Double Cruxes, and keep them in view.
  12. People talk past eachother all the time, and often don’t notice it. Frequently paraphrasing your current understand of what your conversational partner is saying, helps with this. [There is a lot more to say about this problem, and details about how to solve it effectively].
  13. I don’t endorse the Double Crux “algorithm” described in the canonical post. That is, I don’t think that the best way to steer a Double Crux conversation, is to hew to those 5 steps in that order. Actually finding double cruxes is, in practice, much more complicated, and there are a large number of heuristics and TAPs that make the process work. I regard that algorithm as an early (and self conscious) attempt to delineate moves that would help move a conversation towards double cruxes.
  14. This is my current best attempt at distilling the core moves that makes Double Crux work, though this leaves out a lot.
  15. In practice, I think that double cruxes most frequently emerge not from people independently generating their own list cruxes (though this is useful). Rather double cruxes usually emerge from the move of “checking if the point that your partner made is a crux for you.”
  16. I strongly endorse facilitation of basically all tricky conversations, Double Crux oriented or not. It is much easier to have a third party track the meta and help steer, instead of the participants, who’s working memory is (and should be) full of the object level.
  17. So called, “Triple Crux” is not a feasible operation. If you have more than two stakeholders, have two of them Double Crux, and then have one of those two Double Crux with the third person. Things get exponentially trickier as you add more people. I don’t think that Double Crux is a feasible method for coordinating more than ~ 6 people.
  18. Double Crux is much easier when both parties are interested in truth-seeking and in changing their mind, and are assuming good faith about the other. But, these are not strict prerequisites, and unilateral Double Crux is totally a thing.
  19. People being defensive, emotional, or ego-filled does not preclude a productive Double Crux. Some particular auxilary skills are required for navigating those situations, however.
    1. This is a good start for the relevant skills.
  20. If a person wants to get better at Double Crux skills, I recommend they cross-train with IDC. Any move that works in IDC you should try in Double Crux. Any move that works in Double Crux you should try in IDC. This will seem silly sometimes, but I am pretty serious about it, even in the silly-seeming cases. I’ve learned a lot this way.
  21. I don’t think Double Crux necessarily runs into a problem of “black box beliefs” wherein one can no longer make progress because one or both parties comes down to a fundamental disagreement about System 1 heuristics/ models that they learned from some training data, but into which they can’t introspect. Almost always, there are ways to draw out those models.
    1. The simplest way to do this (which is not the only or best way, depending on the circumstances, involves generating many examples and testing the “black box” against them. Vary the hypothetical situations to triangulate to the exact circumstances in which the “black box” outputs which suggestions.
    2. I am not making the universal claim that one never runs into black box beliefs, that can’t be dealt with.
  22. Disagreements rarely come down to “fundamental value disagreements”. If you think that you have gotten to a disagreement about fundamental values, I suspect there was another conversational tact that would have been more productive.
  23. Also, you can totally Double Crux about values. In practice, you can often treat values like beliefs: often there is some evidence that a person could observe, at least in principle, that would convince them to hold or not hold some “fundamental” value.
    1. I am not making the claim that there are no such thing as fundamental values, or that all values are Double Crux-able.
  24. A semi-esoteric point: cruxes are  (or can be) contiguous with operationalizations. For instance, if I’m having a disagreement about whether advertising produces value on net, I might operationalize to “beer commercials, in particular, produce value on net”, which (if I think that operationalization actually captures the original question) is isomorphic to “The value of beer commercials is a crux for the value of advertising.  I would change my mind about advertising in general, if I changed my mind about beer commercials.” (In this is an evidential crux, as opposed to the more common causal crux. (More on this distinction in future posts.))
  25. People’s beliefs are strongly informed by their incentives. This makes me somewhat less optimistic about tools in this space than I would otherwise be, but I still think there’s hope.
  26. There are a number of gaps in the repertoire of conversational tools that I’m currently aware of. One of the most important holes is the lack of method for dealing with psychological blindspots. These days, I often run out of ability to make a conversation go well when we bump into a blindspot in one person or the other (sometimes, there seem to be psychological blindspots on both sides). Tools wanted, in this domain.

(The Double Crux class)

  1. Knowing how to identify Double Cruxes can be kind of tricky, and I don’t think that most participants learn the knack from the 55 to 70 minute Double Crux class at a CFAR workshop.
  2. Currently, I think I can teach the basic knack (not including all the other heuristics and skills) to a person in about 3 hours, but I’m still playing around with the how to do this most efficiently. (The “Basic Double Crux pattern” post is the distillation of my current approach.)
    1. This is one development avenue that would particularly benefit from from parallel search: If you feel like you “get” Double Crux, and can identify Double Cruxes fairly reliably and quickly, it might be helpful if you explicated your process.
  3. That said, there are a lot of relevant compliments and sub-skills to Double Crux, and to bridging disagreements more generally.
  4. The most important function of the Double Crux class at CFAR workshops is teaching and propagating the concept of a “crux”, and to a lesser extent, the concept of a “double crux”. These are very useful shorthands for one’s personal thinking and for discourse, which are great to have in the collective lexicon.

(Some other things)

  1. Personally, I am mostly focused on developing deep methods (perhaps for training high-expertise specialists) that increase the range of problems (in this case, disagreements) that the x-risk ecosystem can solve at all. I care more about this goal this than developing shallow tools that are useful “out of the box” for smart non-specialists, or in trying to change the conversational norms of various relevant communities (though both of those are secondary goals.)
  2. I am highly skeptical of teaching many-to-most of the important skills for bridging deep disagreement, via anything other than ~one-on-one, in-person interaction.
  3. As such, I am publishing my drafts of Double Crux stuff here, over the next few weeks. But making writeups for the public internet, particularly of content that I don’t yet know how to teach via writing, is not a priority for me.

I have a standing offer to facilitate conversations and disagreements (Double Crux or not) for rationalists and EAs. Email me at eli [at] rationality [dot] org if that’s something you’re interested in.

On Double Crux tests and tournaments

Most of the tests I’ve heard people pitch for DC don’t seem very valuable to me, and I want to at least gesture at why.

Other folks seem to be thinking of Double Crux as a complete method, to be directly compared with other methods: “which one works better”. I think of Double Crux as one (very important) pattern in an ensemble for the overall goal of bridging disagreements. “Testing Double Crux”, as I often hear people talk about it, sounds to me a little like “testing bank shots” in basketball: it is clearly useful sometimes, it isn’t always the right thing to go for, and it depends heavily on personal skill.

I think that example overstates it somewhat: Double Crux is more of a broad framework for disagreement bridging than bankshots are for basketball. And that’s not to say that you can’t test bank shots: it’s plausible that there are superstitions about it, and it isn’t as effective as many practitioner’s belive. But the value of information seems lower to me (at least at this stage, where approximately no one has put in more than 20 hours in explicitly training disagreement bridging, compared to basketball, which has hundreds of highly skilled experts.)

I would be more excited in organizing a “disagreement resolution tournament”, where experts who have developed their art and trained to excellence, compete, rather than (for instance) a setup where we give 20 undergrads a 30 minute long double crux lecture with 30 minutes of practice, and compare them to a control group.

(That second things isn’t useless, but I care a lot less about developing shallow tools that are helpful for ~0-skilled folks, out of the box, than I do about deep experts who increase the range of problems (in this case, disagreements) that humanity  / the x-risk ecosystem can solve at all.)

The logistics of such a tournament seem hard to make work, because there’s not an obvious way to standardized disagreements to resolve, and in practice there are very few highly skilled experts of differing schools. So the value of information in 2019 still seems low. But it seems more promising than most of the tests I hear proposed.

Capability testing as a pseudo fire alarm

[epistemic status: a thought I had]

It seems like it would be useful to have very fine-grained measures of how smart / capable a general reasoner is, because this would allow an AGI project to car-fully avoid creating a system smart enough to pose an existential risk.

I’m imagining slowly feeding a system more training data (or, alternatively, iteratively training a system with slightly more compute), and regularly checking its capability. When the system reaches “chimpanzee level” (whatever that means), you stop training it (or giving it more compute resources).

This might even be a kind of fire-alarm. If you have a known predetermined battery of tests, then when some lab develops a system that scores “at the chimp level” at that battery, that might be a signal to everyone, that it’s time to pool our resources and figure out safety. (Of course, this event might alternatively precipitate a race, as everyone tries to get to human-level first.)

Probably the best way to do this would be for both training data, and compute / architecture. Start with a given architecture, then train it, slowly increasing the amount or quality of the training data, with regular tests (done on “spurs”, the agent should never have eposodic memory of the . When increasing training data plateaus, iteratively improve the architecture in some way, either by giving the systems it more compute resources, or maybe making small adjustment. Again train the new version of the system, with regular tests. If you ever start to get very steep improvement, slow down and run tests more frequently.

Naively, it seems like a setup like this would prevent an AI team from overshooting and making a system that is much more capable that they think (which gives rise to all kinds of problems, like treacherous turns), regardless of how close “chimp” is to “human” on some absolute intelligence scale.

Some considerations:

  • This of course, depends on having metrics that work. Of course, it seems very likely that early general AIs will be minds with a very unusual “shape”, such that it has sophisticated reasoning abilities along dimensions that are not natural categories to humans, while being weak on many of the dimensions we’re testing for.
    • Although, this seems a bit surprising, since presumably we would be training it to hit those metrics? Somehow it would get better, faster, at a kind of reasoning we were not intending to train, than at the kind of reasoning that we were optimizing for?
    • Also, can we just design metrics that get around this problem, by measuring exactly what we care about, i.e. danger? Something like a metric for consequentialist reasoning?
  • This idea depends on the system not deceiving you, or intentionally underperforming on your metrics.
    • Naively, this doesn’t seem like that much of an issue, if you are doing tests regularly enough. I’m presuming (perhaps wrongly), that a chimpanzee is not smart enough to infer that it is an agent in a simulation that is overseen by human agents, and that it should try to decide those humans. That is, we stop our progression before the agent is at the point where deception is a consideration.
      • Which means we really need to be confident about how smart something needs to be before it tries to deceive us?
        • Really? Its first attempts at tricking humans will be bad. Just as the attempts of animals to trick us will be pretty bad.
  • At least one “danger point”, is when the system is capable enough to realize the instrumental value of self improving by seizing more resources.
    • How smart is this?
      • My guess, is really smart. Animals come pre-loaded with all kinds of instincts that cause them to seek out food, water, etc. These AI systems would not have an instinct to seek more training data / computation. Most humans don’t reason their way into finding ways to improve their own reasoning. If there was a chimp, even loose in the internet (whatever that means), would it figure out to make itself smarter?
      • If the agent has experienced (and has memories of) rounds of getting smarter, as the humans give it more resources, and can identify that these improvements allow it to get more of what it wants, it might instrumentally reason that it should figure out how to get more compute / training data. But it seems easy to have a setup such that no system has episodic memories previous improvement rounds.
        • [Note: This makes a lot less sense for an agent of the active inference paradigm]
          • Could I salvage it somehow? Maybe by making some kind of principled distinction between learning in the sense of “getting better at reasoning” (procedural), and learning in the sense of “acquiring information about the environment” (episodic).



IQ and impulsiveness

[Real short post. Random. Complete speculation.]

Childhood lead exposure reduces one’s IQ, and also causes one to be more impulsive and aggressive.

I always assumed that the impulsiveness was due, basically, to your executive function machinery working less well. So you have less self control.

But maybe the reason for the IQ-impulsiveness connection, is that if you have a lower IQ, all of your subagents/ subprocesses are less smart. Because they’re worse at planning and modeling the world, the only way they know how to get their needs met are very direct, very simple, action-plans/ strategies. It’s not so much that you’re better at controlling your anger, as the part of you that would be angry is less so, because it has other ways of getting its needs met.