The Human Curriculum No. 3: We Are Not the Center of Intelligence

When I was 8 years old, I stood in the middle of the street I grew up on after dinner one night, with a cheap, plastic telescope in my hand. I had it pointed straight up at the sky, as if that little contraption would give me eyes to see into the worlds beyond ours. I don’t know what came over me, but I was determined to see something different and I thought that telescope was the way.

I remember feeling the vastness of our universe and the magnitude of all its stars. For a brief moment, I sensed how small our existence, my existence, in this particular corner of this particular planet was. And in that moment, I didn’t feel afraid. I felt wonder, awe, and an inexplicable sense of gratitude.

When William Shatner ascended into space many years later, he described something similar. At first, looking into the blackness of the world beyond Earth led to immense grief:

It was among the strongest feelings of grief I have ever encountered. The contrast between the vicious coldness of space and the warm nurturing of Earth below filled me with overwhelming sadness…My trip to space was supposed to be a celebration; instead, it felt like a funeral.

But then Shatner absorbed the Overview effect and found that same sense of awe of our tiny existence as humans on this planet.

In this insignificance we share, we have one gift that other species perhaps do not: we are aware—not only of our insignificance, but the grandeur around us that makes us insignificant.

The third installment of The Human Curriculum covers John Markoff’s Machines of Loving Grace — a history of artificial intelligence reaching back to the 1950s. Reading it produced something close to what Shatner felt. How little we seem to understand what makes us intelligent in the first place, and how much that drives our never-ending quest to build intelligent machines.

No. 3: Machines of Loving Grace

John Markoff / 2015 / The Past

The book traces the arc of AI from the first chess playing programs and early expert systems through the AI winters, the rise of robotics, Sebastian Thrun’s autonomous vehicle Stanley winning the 2005 DARPA Grand Challenge and driving 132 miles through the Mojave Desert without a single human command, and eventually the neural network renaissance led by Geoffrey Hinton, whose deep learning breakthroughs at the University of Toronto set the stage for everything happening in AI today.

The history of artificial intelligence is full of moments that felt like arrival.

The clearest example of this moment for me was AlphaGo. In March 2016, DeepMind’s system beat Lee Sedol, one of the strongest Go players in the world. In the second game, on the 37th move, AlphaGo placed a stone on the fifth row near the edge of the board. Human Go tradition strongly disfavored moves like that at that stage of the game and so the commentators initially assumed it was a mistake. But it wasn’t — it was a move no human player would have made and it won.

When the match was over, AlphaGo had won 4-1 and the response from the AI community shifted almost immediately. While acknowledging the feat, experts noted that AlphaGo had required vastly more training games than any human would need to learn and that real intelligence would require embodiment and the ability to navigate the messy real world. A board game, however, complex, was still a board game.

We moved the goalposts just as we always do.

This wasn’t the only example. It happened with Deep Blue in 1997 when we said chess was just calculation. It happened with self-driving cars when we promised full autonomy and discovered how much of human driving was soft reasoning we couldn't quite formalize. Every generation has had its eureka moment, the breakthrough that felt like the finish line was finally in sight. And every time, as we got closer, we realized something fundamental was missing.

I can’t help but wonder if LLMs are another one of those moments we’re living through right now. On one hand, it feels like the most credible attempt yet. We have the right confluence of pieces for the first time across compute and scale, algorithmic and architecture breakthroughs, enough human knowledge on the Internet to make language the substrate of intelligence. This generation of building intelligence in machines will go further than any previous one.

Autotelic Intelligence

But the pattern of this history makes me deeply suspicious. Each time we got close, we discovered that we had been describing intelligence too narrowly. We were measuring what was legible and calling that the whole thing.

A recurring theme of The Human Curriculum is my attempt to define what makes us human. Venkatesh Rao offers an insightful way to think about what’s missing in our definition of human intelligence in his essay Intelligence Reconsidered. He draws a distinction between functional intelligence — thinking as a tool to achieve some end, and autotelic intelligence — thinking for the pleasure of thinking itself.

Functional intelligence is about trying to win, it is fundamentally a conceptualization of the brain as a finite-game machine. Autotelic intelligence on the other hand, is about continuing the game; the brain conceptualized as an infinite-game machine.

The human brain, I am convinced, is fundamentally an autotelic intelligence: it thinks because it likes to think, not because it must to survive. While it has functional capabilities, it has been defined by a dominant autotelic side for all of recorded history and possibly deep into hominid prehistory.

LLMs are remarkably good at the functional layer, but they’re not even trying to do the autotelic part and I’m not sure they can. The autotelic layer is the thinking that happens when nothing is at stake. The moment you optimize a thought for an outcome, you’ve made it functional. You cannot engineer your way to a mind that thinks for the pleasure of thinking and it is not covered by any benchmark today.

When we write the next chapter in the book about the LLM era, there will be many superhuman feats. The world will change in ways we can’t yet imagine. But the same thread that stitches together every previous chapter will run through this one too. We will have come a long way. We will feel so close. And the thing we couldn’t quite capture, the thing that always seems to be just out of reach, will still be the part of us that thinks for free.

The Sun, The Earth, The Universe

Perhaps the biggest tragedy in our quest to build machines of loving grace is a lesson we should have learned centuries ago. When Copernicus realized the Earth was not the center of the universe, he challenged a mental model that placed humans at the center of things.

We keep thinking we’ve found the answer. We keep placing ourselves at the center of the universe. But the truth has always been in front of us — we aren’t. We placed ourselves at the top of the chain of intelligence. We aren’t quite there either. The history of artificial intelligence is, in this sense, less a story about machines and more a story about how slowly we update our sense of where we sit in the order of things.

We are smarter than we have ever been as a species. And we are still much dumber than we’ll ever admit. Every time we build a machine that can do something we thought only we could do, we are given another chance to be honest about what we are. We keep flinching from it. We move the goalposts. We redefine what counts as real intelligence. We protect the idea of our own centrality because we can’t quite imagine ourselves without it.

When I went for my morning walk the other day, I found myself staring up again at the sky. This time, the sun was shining through the tree branches, the birds were singing somewhere overhead. And for another moment, I felt the same insignificance I felt as an eight year old in the middle of the street.

Me a little older, the sounds around me a little quieter, but that familiar feeling. I thought about my little corner of the universe, it being such a tiny drop of importance in the vast sea of insignificance. And the fact that I was even thinking that, for no reason at all, made me smile.