I recently read the book “How Will You Measure Your Life?” by Harvard Business Professor Clayton Christensen, which early on emphasizes how you need theory to guide behavior. Theory in psychology has become a mild obsession of mine recently*, and so I listened eagerly as he described how absolutely vital theory was to guiding businesses—and our own lives. But pretty soon I realized that what Christensen meant by theory was radically different from what I have been thinking about.
My demarcation problem: was I really reading Theory?
Source: From rawpixel on Pexels
When I talk about theory, I tend to invoke certain scientific ideals. A good theory should be consistent with what people have observed, typically meaning the results of experiments. A good theory should be specific: you should know exactly what it’s talking about and how it relates to specific situations (typically experiments). A good theory should make accurate predictions about what will happen in those situations. A good theory should try to explain the key features of a system in a relatively simple way that can be understood by anyone who puts in the time to learn it. A good theory should apply to as many situations as possible. I could go on… (I kind of did!).
When Christensen talks about theory, he tends to start with a story. For example, he explained his theory of disruptive technology to the chairman of Intel by telling the story of how steel minimills became successful by starting to attack the “low end” of the market, focusing on creating the cheap products effectively. Ultimately they were able to work up the market and become dominant players in creating all steel products. The chairman of Intel had asked him to talk about how to apply his theory to the particular business case of Intel, but Christensen refused to do that. He just told his story of how businesses investing in minimills had overtaken much larger competitor businesses, and then he allowed the Intel chairman to make his own inferences about how it applied to his business case.
Christensen boasts that after hearing his talk, the Intel chairman then laid out the key elements of a hugely successful and lasting business strategy—implying that the talk had been a successful case of business consulting. As importantly, it was a vindication of his theory’s importance. But Christensen hadn’t said anything specific about how the theory worked, what it predicted, or what Intel should do. His theory were missing almost all of the things I have been thinking that a good theory needs. What had he really contributed?
Christensen had told a story, and in telling the story he had made specific conceptual distinctions—that is, he divided up all the known facts about the world into a limited number of important things. The story was implicitly an instruction about what to pay attention to. These things were what you needed to think about when you were in this kind of situation. For example, the market should be thought of as having sections ranging from low to high, and these were important. There wasn’t a discussion of how the needs or desires of consumers were changing. This wasn’t a thing in his story, and so it wasn’t something you needed to worry about.
Theory as a story or explanation can tell us what to focus and act on.
Source: From Skitterphoto on Pexels.
The story also had a specific description of how one thing causes another. Because certain companies started at the low end of the market, they were then able to move up the market and be more successful. This was the key causal relationship to worry about. There might have been dozens of other potential things to consider—how many other competitors were there? How much had the government invested in basic science in the area leading up to the new technology? Who had the bigger social network? But Christensen’s story just focused on one. This implicitly told his listener which factors would lead to change, and so what should happen next.
Overall, this feels more like theory as a parable than as science. After he’s explained his theory, we can’t tell for sure when it should and shouldn’t apply. There’s just an expectation that it might fit certain kinds of situations. His theory doesn’t make specific predictions, but just broad, qualitative ones: starting with cheap investments down market will lead to future success. We don’t know if his theory explains all the relevant previous observations, or even what all the relevant observations are, because we don’t really know when it should and shouldn’t apply. But it’s safe to assume that there are probably important cases when starting with cheap investments down-market didn’t lead to big business success. These negative cases don’t really seem to be relevant. It also doesn’t really seem like this theory is meant to be as widely applicable as possible. There isn’t a sense that if this doesn’t apply to all fields with where technology might be relevant, the theory will fail or be refuted.
Stories teach us how to think about the world, but don’t all meet the standards of science.
Source: From mentatdgt on Pexels.
But the thing is, it does seem like it helped the chair of Intel. And I can understand the value of this kind of theory. When I was in middle school, I learned a bit about Freud’s theories of the unconscious, and about the idea of ego defense mechanisms (which was an extension his thinking by his daughter, Anna Freud). It made me think about my classmates having secret anxieties or insecurity that they didn’t express—and may not have even been consciously aware of—but that nonetheless drove their behavior. It made me consider the possibility that when I felt like I was being targeted or put down, it was about the other person employing a strategy to maintain their own self-esteem. Maybe I actually hadn’t done anything socially bad or stupid. These were all useful things for me to think when I was navigating middle school, as they helped take some of the sting out of changes in my social standing.
After having received formal training in psychology as a research discipline, I no longer give much weight to what Freud said as a scientific theory. (Indeed, Freudian thinking was a key target of Karl Popper’s when he was developing his account of what divided science from non-science; Popper thought Freud’s work was a prime example of non-science.) But I still sometimes use the tools his “theory” gave me: a set of ideas about what things other people are made of and how these things cause behaviors. Freud, like Christensen, was looking very closely at specific cases. For Freud, these were the cases of people who came to him seeking psychiatric treatment; for Christensen, businesses succeeding or failing. They both would then try to explain what had happened in these cases by telling specific stories about them.
Ultimately, these stories aren’t really in the realm of being right or wrong. As long as they’re not directly contradicting the observed facts of the case, they’re all “right.” They’re proposed explanations, but other people in the same situations might have told different stories to explain what happened. People are well-equipped to understand these stories and to extract and use their “morals”—the messages they have about how to live. They ultimately live or die by how many people find them to be helpful, with helpful being a broad category that includes making people feel good about how they have acted, helping people feel more confident in a decision they have to make, or justifying themselves to others.
Stories are the original theory. But we don’t really think about stories in terms of being true or false, more or less accurate. They’re just ways of dividing up and connecting the things we see or hear about, and they’re sometimes useful and sometimes not. In any given situation, you can take one or two (or many) off the shelf and see if they give you any insight. Nothing is definitive. This paradigm is where the most influential psychological theory of all time began, and it’s how a lot of popular business and self-help books still work. What I’m struggling to learn is whether—and how—we can turn psychological theory into something more than that**.
* Another recent obsession has been figuring out how to make sure I live a meaningful life, now that I have finished my PhD and will potentially have more than one career option open to me.
** This is not to say that others are not already working in that direction, or that important theory hasn’t been developed in psychology. Cognitive and mathematical psychology include several instances of testable, and specific theory. The areas in which I work, social and personality psychology, has not historically emphasized this much. The broader point of this article is reasoning through what theory might mean.