This is quoted from an explanation by Lahwran (blog), part of a larger post on LessWrong, and sourced from an original claim and example by Andrew Critch. To my knowledge, Critch has never posted it online. I found myself wanting to reference this divorced from the remainder of the post, so I have reproduced it here. None of these words are my own. (If in the distant future this is preserved while the originals are not, then my apologies, I feel the same way about several ancient Greek philosophers, but at least I’ve cleared up that I haven’t edited it.)
A RECURSION EXAMPLE
I found the claim that humans regularly social-model 5+ levels deep hard to believe at first, but Critch had an example to back it up, which I attempt to recreate here.
Fair warning, it’s a somewhat complicated example to follow, unless you imagine yourself actually there. I only share it for the purpose of arguing that this sort of thing actually can happen; if you can’t follow it, then it’s possible the point stands without it. I had to invent notation in order to make sure I got the example right, and I’m still not sure I did.
(I’m sorry this is sort of contrived. Making these examples fully natural is really really hard.)
- You’re back in your teens, and friends with Kris and Gary. You hang out frequently and have a lot of goofy inside jokes and banter.
- Tonight, Gary’s mom has invited you and Kris over for dinner.
- You get to Gary’s house several hours early, but he’s still working on homework. You go upstairs and borrow his bed for a nap.
- Later, you’re awoken by the activity as Kris arrives, and Gary’s mom shouts a greeting from the other room: “Hey, Kris! Your hair smells bad.”. Kris responds with “Yours as well.” This goes back and forth, with Gary, Kris, and Gary’s mom fluidly exchanging insults as they chat. You’re surprised – you didn’t know Kris knew Gary’s mom.
- Later, you go downstairs to say hi. Gary’s mom says “welcome to the land of the living!” and invites you all to sit and eat.
- Partway through eating, Kris says “Gary, you look like a slob.”
- You feel embarrassed in front of Gary’s mom, and say “Kris, don’t be an ass.”
- You knew they had been bantering happily earlier. If you hadn’t had an audience, you’d have just chuckled and joined in. What happened here?
If you’d like, pause for a moment and see if you can figure it out.
You, Gary, and Kris all feel comfortable bantering around each other. Clearly, Gary and Kris feel comfortable around Gary’s mom, as well. But the reason you were uncomfortable is that you know Gary’s mom thought you were asleep when Kris got there, and you hadn’t known they were cool before, so as far as Gary’s mom knows, you think she thinks kris is just being an ass. So you respond to that.
Let me try saying that again. Here’s some notation for describing it:
X => Y: X correctly believes Y
X ~> Y: X incorrectly believes Y
X ?? Y: X does not know Y
X=Y=Z=...: X and Y and Z and … are comfortable bantering
And here’s an explanation in that notation:
Kris=You=Gary: Kris, You, and Gary are comfortable bantering.
Gary=Kris=Gary's mom: Gary, Kris, and Gary’s mom are comfortable bantering.
You => [gary=Gary's mom=kris]: You know they’re comfortable bantering.
Gary's mom ~> [You ?? [gary=Gary's mom=kris]]: Gary’s mom doesn’t know you know.
You => [Gary's mom ~> [You ?? [gary=Gary's mom=kris]]]: You know Gary’s mom doesn’t know you know they’re comfortable bantering.
And to you in the moment, this crazy recursion just feels like a bit of anxiety, fuzzyness, and an urge to call Kris out so Gary’s mom doesn’t think you’re ok with Kris being rude.
Now, this is a somewhat unusual example. It has to be set up just right in order to get such a deep recursion. The main character’s reaction is sort of unhealthy/fake – better would have been to clarify that you overheard them bantering earlier. As far as I can tell, the primary case where things get this hairy is when there’s uncertainty. But it does actually get this deep – this is a situation pretty similar to ones I’ve found myself in before.
There’s a key thing here: when things like this happen, you react nearly immediately. You don’t need to sit and ponder, you just immediately feel embarrassed for Kris, and react right away. Even though in order to figure out explicitly what you were worried about, you would have had to think about it four levels deep.
If you ask people about this, and it takes deep recursion to figure out what’s going on, I expect you will generally get confused non-answers, such as “I just had a feeling”. I also expect that when people give confused non-answers, it is almost always because of weird recursion things happening.
In Critch’s original lightning talk, he gave this as an argument that the human social skills module is the one that just automatically gets this right. I agree with that, but I want to add: I think that that module is the same one that evaluates people for trust and tracks their needs and generally deals with imagining other people.