📺 Today’s recommended deep-dive video: https://www.youtube.com/watch?v=LCEmiRjPEtQ

Software 3.0: Andrej Karpathy on the Shift to Natural Language Programming

Software is undergoing its most fundamental shift in 70 years, transitioning from rigid, hand-written instructions to fluid, natural language prompts. Andrej Karpathy explores how large language models (LLMs) are becoming a new kind of operating system, effectively turning every English speaker into a developer.

Core Question: How should we design software and workflows when the primary “computer” is now a fallible, human-like simulator?

Highlights

• The evolution of programming from Software 1.0 (code) to 2.0 (weights) to 3.0 (natural language prompts).
• Why LLMs are essentially a new operating system currently stuck in a “1960s-style” time-sharing era.
• The “Autonomy Slider” concept: balancing manual human verification with agentic machine generation.
• The rise of “Vibe Coding” and the urgent need to build “agent-legible” digital infrastructure.

⏱️ Reading time: approx. 7 minutes · Saves you about 32 minutes vs. watching.

Want to take notes while watching? Click the image below and let AI Notebook capture the key points for you 👇

The Three Eras of Software

From Binary Instructions to English Prompts

We are witnessing a rapid architectural transition in how humans command machines. For decades, Software 1.0 defined the industry, consisting of explicit code written in languages like C++ or Python to provide deterministic instructions. It was precise, but brittle, requiring humans to map out every logical branch by hand.

Software 2.0 shifted the burden of “writing” to optimization algorithms. Instead of writing the code, we curated datasets and used neural networks to learn weights—essentially parameters that act as a new kind of software. Karpathy notes that during his time at Tesla, Software 2.0 “ate” the stack, as neural networks replaced thousands of lines of C++ code for tasks like image stitching and path planning.

Now, we have entered the era of Software 3.0.

In this paradigm, the neural network itself has become programmable through natural language. Your English prompts are the new source code, and the LLM is the compiler. This is unprecedented because it democratizes software creation; for the first time in history, the barrier to entry is not a syntax manual, but the ability to communicate a vision clearly in one’s native tongue.

💡 Digging Deeper

Q: Is Software 1.0 becoming obsolete?
A: No, but its footprint is shrinking. It is best used for explicit, deterministic logic, while 2.0 and 3.0 handle high-dimensional patterns and reasoning.

Q: How does Software 3.0 differ from 2.0 if both use neural networks?
A: Software 2.0 involves “frozen” weights specialized for one task (like image recognition). Software 3.0 uses general-purpose models that are “programmed” on the fly via the context window.

Q: What is the “GitHub” of Software 2.0?
A: Karpathy identifies Hugging Face as the primary repository for the weights and models that define the 2.0 era.

The LLM as a New Operating System

Time-Sharing and the 1960s Parallel

If the LLM is the new computer, we are currently living in its 1960s era. Much like the early mainframes, LLM compute is incredibly expensive and centralized in massive cloud “fabs” owned by a few labs. We interact with these “computers” through time-sharing—where millions of users occupy small slices of a batch—waiting for the “intelligence utility” to beam down results.

The architecture of an LLM mirrors a traditional OS more closely than people realize. The LLM itself acts as the CPU, while the context window serves as the volatile RAM (working memory). It has the ability to read and write to files, use tools (peripherals), and orchestrate complex problem-solving sequences.

However, we are still waiting for the “Personal Computing” revolution of AI.

Currently, our interaction is largely limited to a “terminal” interface—the chat box. We lack a universal Graphical User Interface (GUI) for AI that would allow for more intuitive auditing and interaction across different tasks. While some are experimenting with local LLMs on Mac Minis, the sheer cost of compute keeps the most powerful “operating systems” locked in the cloud for now.

💡 Digging Deeper

Q: Why does Karpathy compare current AI to the 1960s?
A: Because compute is centralized and expensive, requiring users to “dial in” to a shared mainframe rather than running high-end intelligence locally.

Q: What is the main limitation of the context window as RAM?
A: It suffers from “anterograde amnesia.” Once the session ends or the window fills up, the “spirit” loses its memory and cannot natively learn from the interaction without external fine-tuning.

Q: How is technology diffusion “flipped” for LLMs?
A: Usually, the military gets tech first, then consumers. With LLMs, consumers used it to “boil eggs” and write emails before governments or corporations could even figure out their security protocols.

The Psychology of “People Spirits”

Managing the Autistic Savant

To program an LLM effectively, you must understand its unique “stochastic psychology.” These models are simulations of human collective intelligence, and as such, they display human-like brilliance alongside baffling cognitive deficits. Karpathy likens them to “People Spirits” or an autistic savant, similar to the character in the film Rainman.

They possess encyclopedic knowledge but struggle with basic self-reflection. An LLM can recall obscure cryptographic hashes but might insist that “9.11 is greater than 9.9.” This “jagged intelligence” means they are superhuman in breadth but prone to hallucinations and logical trips that no human would ever make.

Verification is the ultimate bottleneck in Software 3.0.

Because these systems are fallible, the human role shifts from generation to verification. The goal of any good AI app should be to make this verification loop as fast as possible. If an AI generates 10,000 lines of code at once, it becomes a liability rather than an asset, because the human “verifier” cannot possibly audit that much information quickly enough to ensure safety and correctness.

💡 Digging Deeper

Q: What movies does Karpathy suggest for understanding LLM memory?
A: Memento and 50 First Dates, because the protagonists’ “weights” are fixed while their “context window” resets constantly.

Q: What is the “strawberry” problem?
A: A famous example of an LLM logic failure where it fails to correctly count the number of ‘r’s in the word “strawberry” due to tokenization issues.

Q: Why is “keeping the AI on a leash” important?
A: To prevent “agentic spin,” where an over-reactive AI takes too many autonomous steps without human check-ins, leading to buggy or insecure outcomes.

Partial Autonomy and the Future of Work

The Autonomy Slider and Vibe Coding

The future of software is not a binary choice between “human-made” and “fully autonomous.” Instead, it exists on an “Autonomy Slider.” Tools like Cursor (for coding) or Perplexity (for search) succeed because they allow the user to adjust how much control they cede to the AI. You might start with simple autocomplete (low autonomy) and move to full repository-wide agents (high autonomy) only when the task is well-defined.

This shift has birthed “Vibe Coding”—a meme-turned-reality where people build entire applications by simply describing the “vibe” or functional requirements to an AI. Karpathy himself “vibe coded” an iOS app and a restaurant menu generator despite not knowing the underlying languages fluently. It represents a “gateway drug” to software development, making the creative process more about intent than syntax.

However, for agents to truly work, we must build “agent-legible” infrastructure.

Currently, AI agents have to “click” buttons on websites designed for human eyes, which is inefficient. We need a new standard: llms.txt files, markdown-based documentation, and APIs that replace “click here” instructions with direct machine-readable commands. We must meet the LLMs halfway by making our digital world easy for them to ingest and manipulate.

💡 Digging Deeper

Q: What is “Vibe Coding”?
A: A style of development where the programmer focuses on high-level prompting and “vibes” rather than writing every line of syntax, often enabling non-experts to build functional apps.

Q: Why is the “GUI” still important for AI?
A: Because reading text is effortful, but looking at a visual diff (red/green) is a “highway to the brain,” allowing humans to verify AI work much faster.

Q: What is llms.txt?
A: A proposed standard, similar to robots.txt, that provides a concise, markdown summary of a website’s content specifically for LLMs to read.

Key Takeaways

The transition to Software 3.0 represents a massive opportunity to rewrite the digital world. By treating LLMs as a new kind of operating system, we can build applications that are partially autonomous, significantly increasing human productivity. However, this requires a paradigm shift: we must move away from building purely for human consumers and start building for “people spirits”—AI agents that can reason, act, and assist, provided we keep them on a sufficiently short leash.

As we move the “Autonomy Slider” from left to right over the next decade, the role of the developer will evolve. It will become less about mastering the minutiae of syntax and more about mastering the art of the verification loop. The goal is to create “Iron Man suits”—software that augments human capability through a seamless blend of manual control and agentic assistance.

Q&A

Q1: Why does Andrej Karpathy think now is a unique time to enter the industry?
A: Software is changing fundamentally for the first time in 70 years. The rise of Software 3.0 means there is a massive amount of legacy code to be rewritten and new types of “vibe-coded” applications to be invented.

Q2: What is the “Autonomy Slider”?
A: It is a design philosophy for AI tools where the user can choose the level of agentic help, ranging from simple suggestions (like a spell-checker) to full task delegation (like an agent writing an entire feature).

Q3: How does the “1960s computing” analogy apply to modern AI?
A: Just as early computers were centralized mainframes that users accessed via terminals, current state-of-the-art LLMs are centralized in massive data centers and accessed via “time-sharing” API calls.

Q4: What is the main danger of “Agentic Spin”?
A: If an AI is given too much autonomy without human verification, it can generate vast amounts of work—such as 10,000 lines of code—that might contain subtle bugs or security flaws, overwhelming the human’s ability to audit it.

Q5: What are “People Spirits”?
A: It is Karpathy’s term for LLMs, highlighting that they are stochastic simulations of human behavior and intelligence, complete with human-like strengths and cognitive blind spots.

Q6: Why should documentation be provided in Markdown?
A: Markdown is highly legible for LLMs compared to raw HTML or complex PDFs. Providing “agent-friendly” docs like llms.txt helps AI tools understand and use software libraries more accurately.

Q7: Will “Vibe Coding” replace traditional software engineering?
A: While it lowers the barrier for building apps, the “hard” parts of software—like devops, security, payments, and authentication—still require significant technical oversight and traditional integration work.