Harry: 0:00

Two different people seeing the same facts, if what they’re seeing is ambiguous, are likely to come to different conclusions. And that starts to create an opportunity for us to organize at a higher level around recognizing that we’re seeing something that’s ambiguous and talking about its ambiguity, and then expressing how we’ve come to know something in a way that allows us to have a more fruitful conversation. Hey, Jorge, it’s great to see you as well.

Jorge: 0:48

You know, folks listening on the audio feed are not going to be able to see this, but you’re wearing a very bright red shirt today.

Harry: 0:55

I am. I’m so excited about it. I created a Traction Heroes merch shirt, and I’m wearing it out in the world.

Jorge: 1:06

That is amazing. I, uh… You say you created it, so I’m assuming it’s a sample of one. I kinda want one.

Harry: 1:14

I, I— you just tell me what size and what color, and I’ll make it happen.

Jorge: 1:18

Well, I think it’s only one color, but, uh, uh, but, uh, I’ll tell you what the size is offline because this will not make for good entertainment for folks.

Harry: 1:27

Okay. I brought a, I brought a great reading today. I’ve been rereading a book that really inspired me a year or so ago. And, as is typical, I’ll share the title and author after the reading. But, crazy as it is, a friend of mine called me and said that they were really struggling with some of the issues related to safety and artificial intelligence, and they were asking if I would consider looking into the issues of misalignment and some of the challenges related to ontology, the applications of ontologies and epistemology to securing AIs to prevent them from kind of going rogue and, you know, causing serious problems. And what I reflected on, weirdly, is this book that I read that has nothing to do with artificial intelligence at all. It has to do with the human mind. And so I’m just about done rereading the book. And as I was kind of moving through the book, I’m always paying attention to whether or not there might be something interesting to talk about with you to get your thoughts on and to share with folks out there in the world. So this is a reading from that book, which is really deeply interesting and inspiring and also may, it turns out, apply to this new world of artificial intelligence that we are thinking about. You ready?

Jorge: 3:03

And it’s— Yeah, please.

Harry: 3:05

Them. It’s a powerful word. And the research in both psychology and neuroscience suggests that because our identities have so much more to do with group loyalty, the very word itself, identity, is best thought of as that which identifies us as, well, but more importantly, not them. It’s a basic human drive like hunger or sleep. We’re built by primate genes that construct primate brains that carry within them innate mental states that can be triggered by sensory input. Among them are empathy, sympathy, jealousy, shame, and embarrassment. These mental states, which happen to us, which we feel without asking to feel them, clue us into our nature. As social primates, we can’t help but care a great deal about what others think about us. Humans aren’t just social animals, we are ultra-social animals. We’re the kind of primate that survives by forming and maintaining groups. Much of our innate psychology is about grouping up and then nurturing the group, working to curate cohesion. If the group survives, we survive. So a lot of our drives, our motivations like shame, embarrassment, ostracism, and so on, have more to do with keeping the group strong than keeping any one member, including ourselves, healthy. In other words, we’re willing to sacrifice ourselves and others for the group if it comes to that. There are lots of terms for this in modern psychology, political science, sociology, and so on. The author prefers tribal psychology, but it’s also called extreme partisanship, cultural cognition, etc. Whatever the label, the latest evidence coming out of social science is clear. Humans value being good members of their groups much more than they value being right. So much so that as long as the group satisfies those needs, we will choose to be wrong if it keeps us in good standing with our peers. When social psychologist Brooke Harrington was asked about her thoughts on this, she summed it up by saying, and I’m paraphrasing, “Social death is more frightening than physical death.” This is why we feel deeply threatened when a new idea challenges the ones that have become part of our identity. For some ideas, the ones that identify us as members of a group, we don’t reason as individuals, we reason as members of a tribe. We wanna seem trustworthy, and reputation management as a trustworthy individual often supersedes most other concerns, even our own mortality. This is not entirely irrational. A human alone in this world faces a lot of difficulty, but being alone in the world before modern times was almost always certain death. So we carry with us an innate drive to form groups, join groups, remain in those groups, and oppose other groups. But once you can identify them, you start favoring us. So much so that given a choice between an outcome that favors both groups a lot or one that favors both much less, but still favors yours more than theirs, that’s the one you’ll pick. If you add any conflict over resources of any kind, humans will instinctively enter us versus them thinking, even if that’s not the overall most beneficial strategy, and this is where tribal psychology gets really weird. In times of great conflict where groups are in close contact with each other or communicating with each other a lot, individuals will work hard, extra hard to identify themselves to each other as us and not them. In such an environment, anything can become a signal of loyalty, and how you signal will paint you as a good, loyal member or a traitor. What you wear, the music you like, what car you drive, all of it. If an attitude or a belief or a stated opinion on an issue that was previously neutral becomes an identifier, it becomes a badge of loyalty or a symbol of shame, a signal to others that you are or are not trustworthy.

Jorge: 8:20

Okay, so this sounds very topical, w- first of all. Second of all, I’m very curious to hear about the connection with AI safety. And third of all, I feel a little sad because I just read a book that is very relevant to this subject, and I was hoping to bring it to one of our episodes. And, uh, I think I’m just gonna talk about it here because, um, very, very relevant to what you just read. So I’m very curious, where is this from?

Harry: 8:48

Okay. Uh, well, first, I struggled to read a little bit of this today. It wasn’t as smooth as I would’ve otherwise liked. But this is a book called How Minds Change: The Surprising Science of Belief, Opinion, and Persuasion by David McRaney.

Jorge: 9:05

That sounds really topical, uh, in general. Uh, w- you know, changing minds, um, and being swept along by tribal thinking is obviously something that, uh, one need only fire up a news app and, uh, and be exposed to it, right? So, so very topical. Um, again, I’m super curious about the connection with AI because you brought this up, uh, in the context of AI safety. What’s the connection there?

Harry: 9:35

Yeah. So this is really, really interesting. And I’m only at the, I would say, the very beginning of what I would characterize as deep research into what the issues are here. But the first thing that I looked into when I started diving into issues related to misalignment and AI training and sort of good agents going bad was— first, this was prompted by a series of events, data breaches and whatnot, including the one most recently with Hugging Face you may have read about, and, where a swarm of agents went and did some no good, and they knew better, we would like to think. And as I started looking into some of the issues, it became apparent that two things were probably at the heart of this. One of them is that agents don’t have—they don’t have a connection to the real world. That is to say, they are, as best I can tell, completely unlikely to experience, and I’m putting the word experience in double quotes, in any way that we understand the material negative consequences of their actions because they don’t interact with what would cause those negative experiences as a function of how they relate to what’s going on outside of the digital domain. That was the first thing. And the second thing that jumped out to me was this is fundamentally an epistemological problem. That is to say how, how— epistemology is the study of how we understand truth. And as human beings, it’s a very complex subject, right? There’s a whole discipline in philosophy related to how we understand what is or isn’t true. And it occurred to me that the intersection between AI training and confidence and truth, and then consequences, need more explicit linkages in order for AI, AIs, and specifically agents, not to be able to operate completely insulated from what one might otherwise characterize as reality.

Jorge: 12:30

One of my pet peeves is that the phrase artificial intelligence anthropomorphizes a set of technologies that are not human and unlike humans in many ways.

Harry: 12:45

Mm-hm.

Jorge: 12:45

You mentioned this in the reading, or I got this from the reading, that there are evolutionary reasons why we lapse to benefiting our tribe. AI. Agents, which I’m even struggling to use this terminology because I do think it’s inappropriate to talk about them in this way. But AI-powered systems with some degree of autonomy and agency do not have the same evolutionary background that we do, right? And all they know is what they’ve been trained on, which in the case of LLMs is our language output. On the one hand. The oth- on the other hand, what I hear you saying here, which is also kind of an evolutionary fact, is that these systems lack our ability to experience reality in the same way that we do, including the effects of their actions. So again, I think that one of the fundamental mistakes here is anthropomorphizing these systems. And, um, I think we’ve already gone down that path, so we can’t quite squeeze that toothpaste back into the bottle, right? What did you… I’m just curious about this. I, yeah, having said that, and, you know, I realize that that was a bit abstract. I can see how if the metaphor being used to describe the actions of rogue systems is that they swarmed in some way, I can see why one would gravitate to looking at the phrase you used was tribal psychology, right? It’s like, um, the agent somehow is drawn by the actions of their fellow agents, and suddenly, you didn’t use this phrase, but like something like a mob mentality ensues.

Harry: 15:14

Mm-hm.

Jorge: 15:16

That’s the sense I got from the reading. Have you found anything in this book that you think is actually going to help you with the project of helping draw guidelines for AI safety?

Harry: 15:33

So I think the answer to that is yes, in that because this book is not written for that, this is really a book that’s written to help mortals understand how our opinions and values and beliefs develop and how they can change. And it places enough of the science in context to make that clear that the book itself provides a kind of overarching framework for thinking about this AI-related subject. Obviously the connection is loose enough given that agents are not, you know, back to your comment about anthropomorphizing the AI. You know, agents and artificial intelligence, thinking about them in humanistic terms may in fact dramatically limit how we ought to be thinking about them as entities that have capability and to a certain extent, agency in the world. I’m trying to choose my word, words carefully not to overload all the terms, and it’s hard to do. But given this overarching framework and some of the examples in the book, the way the book is constructed is chapter by chapter, it looks at different aspects of how we come to believe and unbelieve things. And so one of the places, for example, that there’s a linkage between sort of the humanity of our engagement in the world and how agents might come to know or come to have an operational, quote-unquote, quote, belief system is through social proof, right? The concept of, you know, your people you know or authorities or folks you respect or like can tell you something and that will have a certain amount of weight, right? That may inform your belief system. So for example, if my very dear friend Mark goes out and buys a particular car, that may influence my belief that that car is better than something somebody who I don’t know tells me is a good car. So that’s a form of social proof. And so by analogy, I jump over to the AI/agent side and I’m thinking, okay, well, if an agent is out there wondering if you will, wondering whether to value something to a greater or lesser extent or to have a greater or lesser amount of confidence in something, having other agents then provide that information may increase that initial agent’s confidence level, right? So there is a form of social, potential social proof in how agents interact. And then that can be, it can create an either virtuous or vicious cycle of truthiness, if you will, or belief. And so it occurred to me that this book does a pretty good job, better than anything else I’ve read. And I really got it the second time I read it much more so than the first time, of helping us understand what a belief actually is in concrete enough terms that you can look at the world in a new way. And as a leader trying to get traction, for example, there you can apply these ideas to yourself, but it’s also very helpful when you’re working with other people and trying to get other folks to see things in a more nuanced way, that recognizing that a belief is a construct that is informed by your priors and assumptions, and that the greater something is ambiguous, the more likely it will fall to one side or the other based on your priors and your assumptions, and that different people are likely to fall to different sides of that bell-shaped curve. So two different people seeing the same facts, if what they’re seeing is ambiguous, are likely to come to different conclusions. And that starts to create an opportunity for us to organize at a higher level around recognizing that we’re seeing something that’s ambiguous and talking about its ambiguity, and then expressing how we’ve come to know something in a way that allows us to have a more fruitful conversation, rather than just coming to an unconscious or subconscious belief about something and then doubling down on it because that’s what we, quote-unquote, “know to be true.”

Jorge: 20:58

Having worked with these systems in various capacities, I’ve come to see what you’re describing here as an architecture problem,

Harry: 21:10

Yes.

Jorge: 21:11

perhaps for self-serving reasons, in that There is the training corpus, and there are, there might be, again, I’m gonna anthropomorphize, but like what you could just characterize as like beliefs or values might be inherent in the training corpus. And there are things that you can do add to the context that the system is operating with, steer those values in particular directions. So they, you know, the commercial models have guardrails, so that they don’t engage in behavior that we deem inappropriate or illegal, right? That’s an architectural decision. And the other architectural piece here, which is implied by what you’re saying, is that there need to be feedback mechanisms where you take some kind of action or say something and your milieu responds. You know, the world pushes back in some way or responds in some way, and then you internalize that feedback and adjust your behavior. Have you seen the book, The Righteous Mind, I think it’s called, by Jonathan Haidt?

Harry: 22:51

I have. I’ve read it. It’s a very, very good book.

Jorge: 22:55

Yeah, I was thinking of that book as you were describing this because it has this model in there about how we make decisions. And I remember, like, the image that he uses is that of a rider on an elephant. And he says that the elephant is kind of our intuitions, and the rider is like the strategic reasoning mind. Like the rider thinks that they’re driving the elephant, but really the elephant’s gonna go where the elephant’s gonna go, right? And again, I think this is why I’m so kind of adamant about not anthropomorphizing these systems because I do not think that they have the same kinds of intuitions that people do. They don’t have the same kind of I was gonna use the word innate. That might not be the right word, but value systems that we do. They might, you know, they’re a big part cultural. So again, I think that these are architectural problems, and I, um, when I describe myself as architect, as someone who architects intelligence, this is the kind of thing that I’m talking about. So this is very near and dear to my interests. I’m wondering how it might be of interest to someone looking to gain traction, and I’m wondering if we could connect those dots. Like, what’s the snow chains on this?

Harry: 24:16

Yeah, and I was trying to get to some of that a little earlier, and I apologize it wasn’t on the nose enough, as they say in the movie industry. But I think where this book succeeds in a profound way is helping us understand, even if we can’t apply it to ourselves in real time because our own beliefs are developed outside of our awareness, once you understand how some of these mechanisms work, for example, disambiguation is often not as straightforward as it seems because two different people can disambiguate in radically different ways looking at identically, looking at identical things. But as a leader or a manager or somebody in a position of either structural or positional or de facto power, when you are engaged in a debate, when you are working with a team, when you are trying to get people to coalesce around an idea, it’s rarely the simple situations that require deep leadership to show up. And therefore, having better tools and understanding better constructs for how to bring people together is key. And if you take away nothing more from this book than understanding that situations which are ambiguous, that is to say they’re not just vague, but they’re ambiguous, they’re likely to fall to one side or the other, are likely to be seen in two different ways, and those people that see them in two different ways are gonna believe both are true, or each one rather is true. And so if you can step back from that, it allows you not to fall into the trap of either thinking that you’re right, thinking that someone’s wrong, and also that helping people see how something can be ambiguous and that if they had a set of priors and assumptions that would lead to one conclusion, that might be very different from the set of priors and assumptions that would lead to another conclusion. And there you have a conversation.

Jorge: 26:57

I love what you’re saying because I’ve read a lot of advice that says that the way to design effective agentic systems is to adopt good management practices.

Harry: 27:08

Hmm, uh-hm.

Jorge: 27:09

And in this case, you’ve looked at good management practices for the sake of designing agentic systems, and it feels like we’ve circled all the way back to— and we can learn from that how to, you know, better manage humans too. You know? And we should not anthropomorphize AI, but we should definitely anthropomorphize our coworkers.

Harry: 27:34

Yeah. I definitely recommend this book. This is one of those few books that… So I’ve listened to it now twice. I’m reading it with my eyes electronically, and I’m gonna buy a physical copy and cut the spine and turn it into a workbook and really take thoughtful notes on it.

Jorge: 27:51

Well, I promised at the top of the episode here that I was going to recommend the book that I read recently.

Harry: 27:58

Mm-hmm.

Jorge: 27:59

I think might help you also in this path. I don’t know if it’s as relevant now that I’ve heard the full argument, but it is a really good book. It’s called The One and the Ninety-nine by Luke Burgis. It came out recently. It’s a fairly recent book. Burgis wrote a previous book about René Girard, the whole, what’s that called? The, you know, this whole mimetic thing where how kind of group wants emerge.

Harry: 28:38

Hmm. Mm-hmm.

Jorge: 28:39

And this latest book, The One and the Ninety-nine, is about how to preserve your individuality and agency despite the tribal psychology that you’re describing. The, you know, despite the fact that we live in these worlds, we live in these contexts where we are very much drawn by group psychology and kind of sacrifice our, you know, our genuine selves. So that seems like a good chaser for that particular, that particular cocktail.

Harry: 29:19

I love it. I’m definitely gonna go pick it up. It sounds, um connected enough that I wouldn’t be surprised if I learned something, um, that’s quite additive to this.

Jorge: 29:33

Awesome. Well, thank you, Harry, for bringing that. It’s, um, it sounds like it’s something that I should definitely read because it’s very much in my line of interest.

Harry: 29:44

All right. I think you’ll enjoy it. Thank you.