C. Thi Nguyen on Why Measuring Everything Ruins Everything

Metadata

Author: Yascha Mounk
Full Title: C. Thi Nguyen on Why Measuring Everything Ruins Everything
Category: #articles

Highlights

What you do, he argues, is go through a process of performing a rough and inept version of the activity, guided by a simple description. That helps you see more of what it is about. You then revise your sense of what is valuable and how to guide yourself, do it again, and arrive at something more refined. This happens in climbing, in philosophy, in cooking. Over time, you come to see the subtle value. But you need a blunt, simple version at first.
- Tags: #favorite
A lot of this started because, in two different places, I saw the same distinction. When I was in graduate school, my advisor, Barbara Herman, a Kantian ethicist, once said in a graduate seminar, after I said something, “You’re failing to distinguish between a goal and a purpose.” I replied that those were just the same word. She said, “No. When you have your friends over for a night of cards, the goal is to win, and the purpose is to have fun.” You know, deeply, that unless you are a very strange person, if you lose but had a great time in that context, it was a good evening. It was not a wasted evening.
That distinction is extremely important, and games illuminate something fascinating about us. Our purpose is often to have fun, or to relax, but the only way to get there is by hyper-focusing on a goal. I climb to clear my head, and I cannot clear my head by trying to clear my head. I clear my head by climbing. The same is true of fishing. One reason I fish is that staring intensely at the surface of the water, searching for rising trout, creates a meditative stance that I cannot reach directly.
We need very clear goals to get us into mental states we cannot otherwise approach. What games are, in a sense, are pre-packaged mental states. They say, “Do this,” and suddenly you are hyper-focused on balance. “Do this,” and suddenly you are focused on complex logical interplay, or geometric patterns in chess. They are pre-packaged.
In the process, though, you often encounter people who say that for the first two years they were happy, and then after five or ten years they start saying, “I’ve lost the joy in it.” One way of putting it is that it is no longer a hobby, it is a job. But I think the reason is that this distinction has been lost. Earlier, winning was the goal, but the purpose was to have fun. Now the purpose is to make an income.
The second point, which is the really interesting one in the context of jobs, is that we often modulate the goal itself. When we are playing games, because we are not locked into a particular one, we can change the goal as we go. This happens a lot in fishing. Sometimes I fish for easy fish. Sometimes I fish to catch a lot of fish. Sometimes I go fishing to try to catch one big fish. I can modulate the goal depending on my mood, the environment, or what feels fun that day. That is possible because the goal is not linked to something downstream.
Once it becomes a job, you are locked in. You cannot modify the goal or tailor it to yourself. There is a deep sense in which what makes games distinctive lies in the broader ecosystem of play. This is a nineteenth-century insight about aesthetics. Kant argues that aesthetics is the realm where we are free to think about things differently because we are not locked into a practical purpose. Play is the space where we can reconceive what we are doing.
There is a related distinction from R.G. Collingwood, the philosopher of art, between art and craft. Craft is when we know exactly what we are trying to do in advance. We know the specifications and the ingredients, and we are locked into a procedure. Art is when you discover what you are trying to do as you are doing it, and discover what you are thinking as you go. That is exactly what you are pointing to. When you can no longer play with the goal, you can no longer reconfigure the activity.
- Tags: #favorite
What she is saying is more interesting. Life is going to consist of playing games, not just fun games on a game night with friends, but games like wanting to achieve at some endeavor. I want to become a professional philosopher, and that will involve publishing in certain places and jumping through a certain number of hoops. Even if it is inevitable that you will have to play certain kinds of games in life, you need to make sure that you do not get so caught up in a particular game and its rules that you end up doing an activity you do not actually want to be doing. So the right move is not to stop playing games. It is to keep reflecting on whether you are still playing the game that you actually want to be playing.
A croissant in Brooklyn is different because a large portion of the market understands what makes that croissant valuable. When I point to other cases where prices miss, they tend to involve small subcommunities that care deeply about something idiosyncratic, where the value is not particularly public or accessible. I wonder whether, in the croissant case, what is really going on is that, through historical and cultural happenstance, that value is broadly appreciated and therefore priced appropriately, whereas in other cases it is not.
One of my persistent worries about transparency is that it is extremely hard, from the outside, to tell the difference between bias and subtle expertise. That problem shows up very clearly in aesthetic domains. One of the ways I started thinking about this years ago was through quantified scoring systems and wine culture, and more generally through art. It is very hard to tell the difference between a group of posers and a group of people who are genuinely responding to something you do not yet see.
For a long time, I thought hard bop jazz, Coltrane and that whole tradition, was just random fast noise, and that people who claimed to love it were hipster poseurs. Then I got it, and I realized I had been wrong the entire time. That experience stuck with me.
So yes, I think there is a fascinating structural similarity here. Large-scale, blunt systems like markets and transparency metrics can break open closed spaces and disrupt entrenched bias, but they do so in a way that simultaneously misses an enormous amount of subtlety. Both effects come together in the same move.
We need some metrics in order to be able to have productive processes, to be relatively efficient, to make sure that we actually are achieving something. When you’re trying to bake bread for your neighborhood, whether at the end of this process you have two loaves of bread and most people starve or a hundred loaves of bread and most people can have breakfast makes an obvious difference. On the other hand, there are all of these different areas in which the metrics we use end up either misdirecting our productive processes or robbing us of the joy that activity should actually entail. In the most straightforward form, in some forms of central planning, you might end up with 100 loaves of bread, but they’re actually inedible. That’s not well captured in the metrics. As long as the 100 loaves of bread come out of the oven, it doesn’t matter that nobody will actually be able to eat them.
There are two fantasies that people have when they approach metrics. One fantasy is that the metric captures everything that’s important and we just need to optimize for it. The other fantasy is one that I had lived in in the past, which is that these things are just wholly terrible. They’re evil. They miss everything that’s important. We should just get rid of them and enter some kind of non-metricized utopia.
Again, I think the obvious example is letter grades. Everyone agrees what an A means, a B means, a C means, approximately. So we can all collect information into the same bucket and it aggregates instantly. Porter’s insight here is that the very thing that makes metrics socially powerful is the design process that removes context and nuance—the thing that makes it travel. He calls this the portability theory.
He has this beautiful moment, I think you in particular would like this, where he says something like, “information is a specific thing, a way, a kind of human understanding that’s been prepared to travel to distant strangers and be understood in different contexts”. I find that so profound, and I think that stabs right at the heart of this inescapable tension. The reason metrics have social power is that they’re de-nuanced and that’s not something that we can hope to get around. Instead, we should expect that there’s this constant trade-off and the things that are most legible at scale will be the least nuanced and we’re gonna have to use those to interact with each other. But also, we have to be constantly aware that they’ve achieved that kind of social centrality precisely because they’re de-nuancing. Then it becomes hard because that’s exactly the terms that everyone can understand instantly. Again, as Porter points out, that was the design goal. We achieved the design goal.
Nguyen: That’s a superb question. So it seems really doubtful that any metric is good or bad across the board. It’s highly contextual. Here’s one of my favorite examples. I think a lot of us know that BMI, Body Mass Index, is a terrible way to manage your health if people just try to make their BMI go down. But that’s not what BMI was built for. BMI was built as a large-scale public health measure that was like a kind of litmus test. If across an entire nation, BMI suddenly jumps five points suddenly, we know that something has happened. Mounk: Or if it goes down a lot, which is not the problem today in the United States. If BMI suddenly craters, well, perhaps we’ve just discovered GLP-1, but most likely it’s because there’s a famine going on. So it’s really useful at that level. Nguyen
A standard example for me is what we actually care about in a lot of cases is intellectual and emotional maturity, but that doesn’t have legal objectivity. Eighteen years of age does have legal objectivity, so even if that’s imprecise, we use that.
I recently had Atul Gawande on the podcast, and he was talking about a very similar topic, which is that we are great at extending the lifespan by a few months at the end of people’s lives. But, first, most healthcare spending is on those few months of life. Second, the quality of life that people have in that time tends to be terrible. They are constantly in the hospital, and they are in significant pain. You are drawing out the process of dying, often much longer than patients would want it to be. That is downstream from the fact that the metric of “is this patient alive?” or “is this patient dead?” is pretty easy to measure, despite some hard cases. The metric of “was it a good death or a bad death?” is much harder. On the whole, did life go better for living those extra eight weeks in pain, half-conscious in the hospital, or not? That is incredibly difficult to assess.
Part of the reason I wrote this book was that I ran into a puzzle that I did not know if anyone else was interested in, but I could not stop thinking about. I had written a bunch of stuff about games, and I had written that stuff because I had gotten frustrated with attempts to praise games as a kind of cinema, where all people talked about were the cutscenes, the dialogue, and the graphics. They did not talk about freedom or the quality of action. I really wanted to talk about how it felt, how rich the decisions were, and how interesting the decisions were. One of the most interesting things I found was this moment from the great German board game designer Reiner Knizia, where he says, “The most important part of my game designer toolbox is the scoring system, because it sets the player’s desires.” I was a game player, and I thought that was exactly right. I was also a philosopher, and I thought, my God, that is so true and so deep, and so weird to put that so starkly.
What a game designer is doing is describing an alternate self with alternate desires and alternate abilities.
Rock climbing tuned me into something else, and it did so because of the scoring system. It told me to climb harder climbs inside a ranking system. Every rock climb has a community-established difficulty rating, and the internal scoring system is to climb harder routes. Interestingly, to climb harder routes I had to learn how to control and perceive my body. Over the course of that, I learned that movement was beautiful, and that subtle movement was incredibly beautiful, often in a way very similar to the beauty I found in philosophy, this kind of fine-grained subtlety, but also really different. I got that because of the rock-climbing scoring system. After about five years, that scoring system stopped being useful to me, partially because I am just a mediocre climber with poor athletic ability. When I plateaued, the scoring system was good as long as I could keep improving. But at another stage of my life, as a parent and academic who is not that athletic, it started to destroy the joy of climbing to stay within the same scoring system. So I created an alternate goal set for myself. I started aiming to climb moderate routes as elegantly as I could, and I found again what I loved.
Mounk: One of the things that really struck me is a student who, I think, watched one of your videos. I do not think it was a student of yours. She realized, in a very typical story, that she was a high-achieving student who worked really hard and was successful at gaming all of the metrics that life threw at her, and that she was unhappy because that was not what she actually wanted to do. I think she wrote on the background of her phone something like, “Is this the game you want to be playing?” It is a kind of self-admonition, a reminder to always ask that question. It is not a simplistic version of this idea, which is very popular, that says, “Stop playing the game. Reject all the metrics. Just go hang out with your friends on a beach.” That is not the question she is asking.