Fighting the Collapse is a Form of Human Intelligence

Two and a half years ago, I experienced collapse.

I grew up as a curious kid. I used to stare at the stars, marvel at the world around me, and dream about what my life could become.

But somewhere along the way, my focus quietly shifted.

I followed a high-achieving path: excelling in academics, getting into a good school, landing prestigious roles at competitive tech companies. Focus itself wasn’t the problem. In many ways, it helped me move forward. Collapse, for me, was a shift in what I optimized for.

Without realizing it, I stopped optimizing for curiosity and growth, and began optimizing for perception. I measured myself through other people’s expectations.

When the path stopped being linear and I hit real challenges in my career, I assumed something was wrong with me. I listened closely to feedback about my weaknesses. I felt the fear of not being enough. And slowly, I lost touch with myself.

I stopped feeling curious about the world. I stopped asking simple questions because I worried people might think I was dumb. I stopped believing in my own ability to create and shape things.

I collapsed because my focus narrowed too far around external validation — around fitting in, being liked, and conforming to a mold I thought I was supposed to fit.

Eventually, I noticed what was happening. And I learned how to fight back.

With LLMs, I rediscovered a sense of self-belief and wonder by building again. I asked questions without fear of judgment, I let myself be curious about what I could become again, rather than who I was supposed to be.

It took me a while to realize this wasn’t just a personal story. The more time I spent thinking about AI, the more familiar their behavior began to feel. Models collapse, just as humans do. But humans can notice collapse as it’s happening and learn how to fight it, whereas LLMs don’t even have awareness that it’s occurring.

That distinction is changing how I view intelligence itself.

What Human Collapse Feels Like

I don’t have a perfect definition of human collapse because I experienced it before I understood it conceptually. But I recognize it instantly when others describe it.

Ed Catmull captures it beautifully in Creativity Inc :

We begin life, as children, being open to the ideas of others because we need to be open to learn. Most of what children encounter, after all, are things they’ve never seen before. The child has no choice but to embrace the new. If this openness is so wonderful, however, why do we lose it as we grow up? Where, along the way, do we turn from the wide-eyed child into the adult who fears surprises and has all the answers and seeks to control all outcomes?

Collapse isn’t inherently bad. If you spent your entire life exploring possibility, you’d never act. Collapse focuses us. It gives us direction.

But when collapse goes too far, we begin optimizing our entire lives to fit within a narrow distribution. We stop learning — not because we can’t, but because we quietly decide we already know who we are.

Cyan Banister describes this perfectly on Jackson Dahl’s Dialectic:

We all have this mental model of what we were supposed to be when we were grown up. If you look back, everyone would always ask you: what are you gonna do when you grow up? You might have a fantasy. I had fantasies, and I remember those fantasies of what I was gonna be, how I saw myself in the future.

Adults around us in society and culture stamp it out of us. They tell us that it’s wrong to be foolish, it’s wrong to play. At some point, it’s very discouraged. We create these masks.

This too is human collapse: losing the curiosity we enter the world with.

What Model Collapse Looks Like

Over the last few months, I’ve listened closely to some of the most prominent AI researchers discuss a growing gap between LLM-based intelligence and human intelligence:

• Ilya Sutskever – We’re moving from the age of scaling to the age of research

• Andrej Karpathy - AGI is still a decade away

• Richard Sutton - Father of RL thinks LLMs are a dead end

• Demis Hassabis - The future of intelligence

Across each conversation, they mention the same limitation again and again: model collapse.

Today’s LLMs collapse too quickly within distribution. Pre-training compresses them toward the center of observed data. Post-training and reinforcement learning (RL) reward conformity, safety, and predictability over variance.

Richard Sutton describes this as a fundamental failure of gradient descent to generalize well. It solves the problem it’s trained on, but struggles to adapt when conditions change.

Gradient descent will not make you generalize well. It will make you solve the problem. It will not make you, if you get new data, generalize in a good way.

Generalization means to train on one thing that’ll affect what you do on other things. We know deep learning is really bad at this. For example, we know that if you train on some new thing, it will often catastrophically interfere with all the old things that you knew. This is exactly bad generalization.

Generalization, as I said, is some kind of influence of training on one state on other states. The fact that you generalize is not necessarily good or bad. You can generalize poorly, you can generalize well. Generalization always will happen, but we need algorithms that will cause the generalization to be good rather than bad.

This matters because not collapsing is a form of good generalization. Knowing when to narrow — and when not to — is the difference between brittle intelligence and adaptive intelligence.

Andrej Karpathy makes this concrete:

That’s because all of the samples you get from models are silently collapsed. Silently—it is not obvious if you look at any individual example of it—they occupy a very tiny manifold of the possible space of thoughts about content. The LLMs, when they come off, they’re what we call “collapsed.” They have a collapsed data distribution. One easy way to see it is to go to ChatGPT and ask it, “Tell me a joke.” It only has like three jokes. It’s not giving you the whole breadth of possible jokes. It knows like three jokes. They’re silently collapsed.

You’re not getting the richness and the diversity and the entropy from these models as you would get from humans. Humans are a lot noisier, but at least they’re not biased, in a statistical sense. They’re not silently collapsed. They maintain a huge amount of entropy. So how do you get synthetic data generation to work despite the collapse and while maintaining the entropy? That’s a research problem.

Models collapse to fit distribution in much the same way humans do over time.

Why Our Incentive Structures Reinforce Collapse

We reinforce collapse in AI because it’s a good thing for the capital flows of the world. We build evals, guardrails, and applications that demand determinism. We reward models for staying inside the distribution.

We do this because the value reallocation game is focused on automation. These models give us incredible, unseen leverage to automate narrow forms of intelligence.

Capital allocators prefer predictability. You need collapsed generators, in models and in humans alike.

So we actively incentivize collapse.

Why Fighting Collapse Matters

Even as researchers worry about the gap between human intelligence and LLM-based intelligence, two truths can coexist — a paradox:

(1) LLMs are here to stay. We are on the scaling path and it will continue to work until it doesn’t. We haven’t seen how transformative this technology will be, because the models will get better and so will our ability to collapse them in more and more workflows. Coding agents may eventually look like training wheels for powerful AI on this path. And whether this path leads to a form of AGI or not is almost beside the point.

(2) Human-like intelligence requires knowing when and how to fight collapse. This form of awareness is something we have yet to see in machines.

You can argue whether that matters. But I’d counter it’s the most important gap to understand. Even LLMs hallucinating, something we frame as flaws, hint at something deeper we don’t yet grasp.

Learning how to fight collapse is what made me feel human again. I had intuition. I had the ability to sense where my life was headed and course-correct before the evidence was overwhelming.

Most of us are already doing this. We anticipate mistakes. We adapt ahead of feedback. We change not only in response to data, but in response to meaning.

Maybe LLMs are the wrong architectural paradigm. Maybe we need a new research direction — something like online or continual learning, where systems can develop awareness of when to collapse and when not to. Or agents interacting with each other in complex environments until awareness emerges. Or maybe we have to look inwards first: better understand how our own consciousness evolved biologically.

I don’t really know.

What I do know is that this gap — the ability to notice collapse and fight it — is something that defines us.

No matter where you are in your life, I’ll leave you with this invitation: notice where you might be acting on old fears or assumptions about yourself. Ask yourself whether your view of who you are has quietly collapsed. And ask whether you know how to fight it.

I suspect you do.

Thanks to Linus Lee, Rohan Chitalia, Chris Rappoli, and Dilip Rajan for reading drafts of this post and offering thoughtful feedback.