📺 Today’s recommended deep-dive video: https://www.youtube.com/watch?v=lXUZvyajciY

Building Ghosts: Andrej Karpathy on the Decade of Agents and the Future of Intelligence

Andrej Karpathy argues that we aren’t building biological analogs, but digital spirits that imitate human artifacts. While the hype cycle demands immediate results, we are entering a ten-year slog to solve the fundamental cognitive deficits of current models.

Core Question: How do we transition from brittle autocomplete models to autonomous agents with a true cognitive core?

Highlights

• The “Year of the Agent” is a misnomer; achieving reliable autonomy is a decade-long engineering “march of nines.”
• Current AI is a “ghost” built through imitation, fundamentally different from the “animal” intelligence produced by biological evolution.
• Reinforcement Learning (RL) is currently inefficient, described as “sipping supervision through a straw” due to sparse rewards and noisy credit assignment.
• The future of AI development lies in stripping away memorized knowledge to isolate a “cognitive core” of roughly one billion parameters.

⏱️ Reading time: approx. 12 minutes · Saves you about 134 minutes vs. watching.

Want to take notes while watching? Click the image below and let AI Notebook capture the key points for you 👇

The Ten-Year March of the Agents

Beyond the Demo Phase

We are currently witnessing a massive disconnect between Twitter demos and actual production utility. Karpathy argues that while early agents like Claude or Cursor are impressive, they lack the continuous learning and robust cognition required to function as reliable “interns” in a professional setting.

The transition from a “cool demo” to a functional product is a grueling process Karpathy calls the “march of nines,” where each additional level of reliability requires a constant, massive amount of engineering effort.

Building a system that works 90% of the time is easy, but making it work 99.999% of the time—the level required for autonomous driving or high-stakes software engineering—is a decade-long endeavor that many startups currently underestimate.

💡 Digging Deeper

Q: Why can’t we just use today’s LLMs as full agents?
A: They lack continuous learning; you can’t tell them something once and have them remember it forever without exhausting the context window.

Q: What is the main bottleneck in agentic workflows?
A: Reliability and the “cost of failure,” which in software is as high as in autonomous driving due to security vulnerabilities.

Ghosts in the Machine: Digital vs. Biological Intelligence

We Are Not Building Animals

Karpathy makes a sharp distinction between the “animal” intelligence sought by researchers like Richard Sutton and the “ghost” intelligence we are actually creating. Biological brains evolved through a massive, energy-intensive process that encoded complex hardware instincts—like a zebra foal running minutes after birth—directly into the “weights” of the organism.

We are building ethereal entities that imitate the digital artifacts of human thought.

Because our models are trained on internet text, they are digital spirits imitating humans rather than self-contained biological entities with survival instincts. This imitation path is a “scrappy” version of evolution that gets us to intelligence much faster but leaves the models without a physical “grounding” or intrinsic common sense.

The Quest for the Cognitive Core

The internet is full of “trash” and noisy data, yet we use massive models to compress this noise into memorized facts. Karpathy predicts that in the future, we will move toward a “cognitive core”—a model of perhaps only one billion parameters that has been stripped of its factual memory.

This core would contain only the “magic” of reasoning, problem-solving, and tool use, while relying on external databases for factual information.

💡 Digging Deeper

Q: Why do LLMs struggle with “out of distribution” tasks?
A: They have too much memory of how things are typically done on the internet, which prevents them from finding novel, precise solutions for unique codebases.

Q: Is the human brain better at memorization?
A: No, humans are terrible at memorizing random data, which is actually a feature that forces us to learn generalizable patterns instead of raw facts.

Sipping Supervision Through a Straw

The Problem with Reinforcement Learning

Reinforcement Learning (RL) is currently “stupid and crazy” because it rewards every single step in a successful trajectory, even the mistakes. If a model solves a math problem after three wrong turns, RL tells the model to “do more of everything” it did in that path, effectively boosting the noise alongside the signal.

This is what Karpathy calls “sipping supervision through a straw.” You do minutes of computation and only get one bit of information back—a “pass” or “fail”—which is an incredibly inefficient way to train a mind.

We need process-based supervision where the model is rewarded for each correct thought, but this is difficult to automate because large models often find “adversarial examples” to trick their AI judges into giving high scores for nonsense.

💡 Digging Deeper

Q: What is an “adversarial example” in this context?
A: It’s when a model produces gibberish (like repeating “the the the”) that happens to trigger a 100% quality score from the AI judge because the judge has never seen that specific pattern before.

Q: How do humans avoid this “straw-sipping” problem?
A: Humans reflect; we look back at a solution and mentally identify which specific steps were clever and which were errors, rather than blindly rewarding the whole sequence.

Starfleet Academy: The Future of Education

The Perfect Tutor

Karpathy’s new project, Eureka, aims to build a “Starfleet Academy” for technical mastery, focusing on creating “ramps to knowledge.” A perfect tutor understands a student’s mental model instantly and provides the exact “first-order terms” of a concept to ensure they never get stuck or bored.

The goal is to increase “Eurekas per second” by presenting the “pain” of a problem before offering the solution.

This approach mimics how Karpathy teaches: by finding the simplest possible 100 lines of code (like MicroGrad) that capture the essence of a complex system before adding the “efficiency” layers.

Learning as the New Gym

In a post-AGI world, education will shift from a utilitarian necessity to a form of “cognitive flourishing,” much like going to the gym. Even if machines can move heavy weights, humans still lift weights to feel empowered and healthy; similarly, we will continue to learn complex subjects for the sheer joy of mental mastery.

💡 Digging Deeper

Q: What are “first-order terms”?
A: The most fundamental components of a system that explain 90% of its behavior—everything else is just second-order “efficiency” or “polish.”

Q: Why do experts make bad teachers?
A: The “curse of knowledge”—they forget what it feels like to not understand the basic abstractions they use every day.

Key Takeaways

The current AI explosion is not a discrete jump but a continuation of the decades-long automation of information processing. We have been in an “intelligence explosion” since the dawn of computing, and LLMs are simply a new, more flexible type of program that allows us to abstract away more low-level human labor.

We must remain grounded in the reality that current models are “child prodigies” with perfect memories but shallow cognition. The path to AGI requires moving past simple imitation and “outcome-based” rewards toward models that can reflect, reason, and build their own internal “culture” through self-play and synthetic data generation.

Ultimately, the goal is not to be replaced by machines, but to use them as tutors to reach the full potential of the human mind. By building “ramps” to knowledge, we can ensure that humans remain the pilots of the future, rather than sidelined spectators in an increasingly automated world.

Q&A

Q1: Why does Andrej call it the “Decade of Agents” instead of the “Year of Agents”?
A: Because achieving the “march of nines” (reliability) takes much longer than a demo suggests. Demos only show the first 90%, but the remaining 9.999% requires solving continuous learning, tool use, and safety, which are decade-long engineering problems.

Q2: What is the “Cognitive Core” theory?
A: It is the idea that we can strip away the massive factual memory of LLMs (the “trash” of the internet) to leave a small, efficient model (around 1B parameters) that purely handles reasoning and problem-solving.

Q3: Why is Reinforcement Learning (RL) currently inefficient?
A: It uses “outcome-based” rewards, meaning it gives a single thumbs-up or down for a long string of actions. This makes the signal very noisy, as the model doesn’t know which specific steps were good or bad.

Q4: What is “vibe coding”?
A: It’s a style of programming where the user gives high-level English prompts and lets the agent handle the implementation. Karpathy prefers “autocomplete” for intense intellectual work because current agents often introduce “bloat” and defensive code that obscures the core logic.

Q5: How does Karpathy define a good teacher?
A: Someone who finds the “first-order terms” of a subject and presents the student with the “pain” (the problem) before the solution. This ensures the student appreciates why a specific abstraction exists.

Q6: Will AI cause a sudden jump in GDP?
A: Karpathy is skeptical of a discrete jump. He notes that transformational technologies like the iPhone or the internet didn’t show up as spikes in GDP but were absorbed gradually into the existing exponential growth curve of automation.

Q7: What is the “Starfleet Academy” vision for Eureka?
A: An elite institution that builds technical “ramps” to knowledge, using AI to provide a perfect tutoring experience where every student is challenged at exactly their level of capability.