DeepMind's (Code) World Models

When Letting LLMs Write the Simulator Matters More Than Letting Them Play

Yann LeCun has argued for years that next-token prediction is insufficient for human-level intelligence and that AI needs world models that learn how the world works. I agree, even if I see the solution not in the model per se, but in the agentic framework around it. It’s one of the reasons why, since 2023, I have sensors as part of my agent definition because they are required to perceive the world.

We all have a naive model of the world. If we turn a glass of water around, we expect the water to splash on the floor. World models in agentic systems are internal representations that let a system anticipate what happens next and plan accordingly

If there is one thing I have learned, given my own work on memory architecture (i, ii, iii), and also object permanence, is that providing the right frame of reference to understand everything that’s dumped into the context window increases accuracy manifold.

Recently, LeCun’s Joint-Embedding Predictive Architecture (JEPA) pushed this concept even further. Of course, originally, the term world-model is more often applicable to VLAMs, but I am certain that we can extend the concept to more general agent architectures. Especially, since I see them as a milestone on the road to robotics. In this post, I argue that a world model doesn’t need to predict everything. Not every pixel. Not every grain of rice.

The model should learn to abstract a representation of the world, based on logic and simple rules, that captures what’s predictable and discards everything isn’t.

And in a way, that is exactly what the Architect from the movie The Matrix: Reloaded also said.

The first Matrix I designed was quite naturally perfect, it was a work of art – flawless, sublime. A triumph equalled only by its monumental failure. The inevitability of its doom is apparent to me now as a consequence of the imperfection inherent in every human being.

What is predictable can be controlled.

Kevin Kelly, in his seminal work Out of Control, calls this concept the emergence of control, where it arises from successfully harnessing what is predictable rather than comprehensively modeling everything.

I think that’s the right framing,

But I’d like to take it one step further:

A world model perceives the world through sensors. In autonomous driving, we talk about cameras and LIDAR. But I think this concept can be extended to purely software agents as well.

I have implemented sensors in my agent architectures for a long time. In my case, it is not a car that needs to react to sensing an object in the street, but an unusual volume pattern, or an arriving email from a colleague.

The sensor picks up this new information and provides it to the world model.

The world model is a predictable abstraction of the world. The world model then provides the information to the planner, who is tasked with finding the right action to take. Given this action, the model forecasts the impact on the world of the action, and then hands it over to the agent to execute the action.

Here, the term agency is really important.

It defines the difference between automation and agent.

As now, after the action has been implemented, the world has changed, and the state of the world has moved one frame further. A stock has been bought, sold, or not. A response email has been written.

Each action moves the agent temporally and sequentially through context. The state of the world changes.

Each state transition describes “what comes next, given what just happened”.

This temporal and sequential framing allows us to validate what we are seing make sense as the sensory observation is scoped to whatever sequence of events the agent pays attention to. I will touch on these components again when I discuss hierarchies.

Models work the best when they rely on very basic logic. The sun is up during the day. If I can see the sun (a sensor), then it must be daytime.

A code world model is an extension to the standard world model, as it describes the world through such codified logic. Conceptually, Code World Models have evolved out of a traditional reinforcement learning school of thought that operates within a refined environment. The benefit of using a code world model is that we can just execute the code and observe its effects instead of building a holistic model of the environment.

Since the boundaries of the world model are assumed finite, it is possible to define structured, rule-based regularities that make the future predictable enough to control. In its most basic form, we define

• States: “A turn moves through draw, discard, or knock phases, and the legal actions are different in each one.”

• Invariants: “A player can never launch more ships than the planet currently holds, and the code enforces this on every call rather than trusting the caller.”

• Conditions: “If the attacking force exceeds the garrison, the planet changes ownership.”

Each state is temporal. And, I think that is really important to understand. While a rule describes how each moment transitions into the next one, the code world model enables long-running, executable accumulation of exactly these kinds of one-sentence rules, applied one step at a time.

Just like a game loop.

And I think that’s where the concept gets really interesting.

Monte Carlos Tree Search

We begin this section with a discussion of the constraints of simulated environments. In simulated environments, there are generally two very different ways to get a language model to behave intelligently within it. The popular approach is to treat the model as the policy itself: You feed it the rules, the current state, and the model predicts the next state. You see this commonly in reinforcement learning algorithms like PPO. Here, the model becomes an “intuitive player” that pattern-matches its way through Chess openings or Poker ranges. Since this approach is traditional RL and therefore quite well known, I won’t go into too much detail.

I think its more interesting, instead of having the model play the game a billion times, creating a model that writes the game. That means, given a natural-language rulebook and a handful of example trajectories, have it produce a working simulator. This simulator can then be used to predict state transitions, enumerate legal moves, and expected score outcomes.

We then hand that simulator to Information Set Monte Carlo Tree Search (ISMCTS) and let that do the playing.

Monte Carlo Tree Search is an algorithm that builds a search tree incrementally over four repeated phases:

Selection traverses the tree from the root using a policy that balances exploiting high-value nodes against exploring under-visited ones, until it reaches a leaf;

Expansion adds one or more new child nodes to that leaf;

Rollout (simulation) plays out a sequence of actions, often randomly or via a fast policy, from the new node to estimate a value Q; and

Backpropagation propagates that value back up through every node visited in the selection path, updating their visit counts and value estimates.

Repeating these four steps many times concentrates simulation effort on promising branches, so the visit counts at the root eventually approximate the optimal action distribution without needing to exhaustively search the full tree.

Generator simulation and ISMCTS together form the concept of the Code World Model, or CWM. First introduced in this recent paper from Google DeepMind, “Code World Models for General Game Playing,” the team pushes this idea further by introducing imperfect information, stochasticity, and games the model has never seen before.

This paper fits perfectly to Yann LeCun’s long-running argument that world models, not next-token prediction, are the road to machine intelligence. Given my own learings on self-correcting systems, I can see their point.

While both concepts, code-as-world-model and learned-latent-world-model, are technically different, they try to answer the same underlying question.

“If I have the current state and want to do an action, what will the next state be?”

Data to Code Translation

The DeepMind paper defines the job of the LLM as translation. You give it a textual description of a game’s rules and add a small number of recorded trajectories from random play. Trajectories here refer to recorded sequences of (state, action, next-state) tuples, i.e., actual play-throughs of the game, logged turn by turn.

The Code World Model is now tasked to produce code that implements the game engine and a standard interface to it.

The interface is:

1. a state transition function,

2. a legal-move enumerator,

3. an observation function,

4. a reward function, and a termination check.

This interface is similar to OpenSpiel’s API, the open-source game research framework. Therefore, the resulting code can immediately be dropped into existing, battle-tested planning algorithms.

Why does this matter?

Well, in my opinion, there are three advantages this buys you, which are exactly the three things that make “LLM as policy” approaches fragile.

Verifiability. A CWM is, by construction, a formal specification of the game’s rules. If the planner wants to know which moves are legal at the current state, it doesn’t ask the LLM and hope; it calls a function and gets a deterministic answer. The notorious failure mode of LLM-as-policy agents, and something I also encountered in my Game Theory work, is that these models propose illegal moves.

Strategic depth. Once you have a working simulator, you can throw arbitrary amounts of search at it. MCTS with a thousand simulations per move doesn’t care whether the underlying model “feels” confident; it just brute-force explores the game tree. This is the classic System 1 / System 2 split that Daniel Kahneman made famous. The LLM supplies a fast, intuitive understanding of the what of the game, while the search algorithm supplies the deliberate, grounded lookahead execution that intuition alone can’t match.

Generalization. For all forms of robotics training, a system do to one thing and do this one thing extremely well has been possible for a long time. The road to AGI might be paved with CWM’s ability to generalize better to unknown game situations. An LLM asked to translate rules into code is doing something it has seen a copious amount of: turning a specification into a program. The meta-task, i.e., data-to-code translation, is one that the model has practiced at scale, even if the specific game is novel. That’s why the paper’s out-of-distribution games (invented specifically so they can’t be in any LLM’s training data) are such an important part of the evaluation.

Inference-as-code

Most prior work on LLM-synthesized world models, WorldCoder or GIF-MCTS, assumed fully observed, deterministic environments. For generalization, especially if we can bring this concept into capital markets, that is a problem and also a huge simplification. Chess and Go are fully observed: every piece is visible to both players at all times. Poker is not. Bridge is not. Most real strategic situations, negotiations, markets, multi-agent coordination are not.

The DeepMind paper’s most substantial contribution, in my opinion, is the contribution of “inference as code.”

My argument is that if you’re playing Poker you only know your own cards and the public information. How do you predict what your opponent might be holding?

Card counting is one option. Another is having Information Set MCTS handle this by sampling states to ground truth, testing consistency with sensor observations, running searches across the distribution, and aggregating statistics.

The problem is that ISMCTS needs a way to generate those plausible ground-truth samples.

That’s where inference-as-code comes in.

In the DeepMind paper, the team tasks the LLM to synthesize a second piece of code, an inference function that takes an observation and action history and samples a plausible, complete history of the game, including the parts the player doesn’t see: the chance player’s card deals, the opponent’s hidden actions, everything.

Because the CWM’s transition function is deterministic, sampling a plausible history is equivalent to sampling a plausible hidden state, running the sampled history through the CWM, and getting back a hidden state that’s consistent with everything you’ve actually observed.

For every point in a trajectory where you, the player, took an action, the world model performs a unit test asserting that replaying the sampled history reproduces your actual observation and your actual action at that point. If the inference function passes all these tests, the sampled histories are guaranteed to belong to the support of the true posterior, not necessarily the exact right distribution, but at least plausible, valid states the game could actually be in.

In other words, the result is a set of hidden states that are actually possible given everything you've observed, as opposed to states that are ruled out by what you've seen.

Open vs. Closed

One of the paper’s more practically important contributions is a distinction it calls “open deck” versus “closed deck” synthesis.

Open deck synthesis assumes that during the training/synthesis phase, you have access to the hidden information in your offline trajectories. You’re building the model in a “design room” where you can see everyone’s cards, even though the deployed agent never will. This is a reasonable assumption in several real scenarios: cooperative training environments, game-design phases where developers have full state access, or expert demonstrations that come with hindsight annotations. It’s also the assumption made by the most directly comparable prior work, “POMDP Coder.”

Closed deck synthesis drops this assumption entirely. The CWM only sees what the player saw: your own observations and your own actions, never the hidden state, not even after the fact. The paper presents this as the realistic scenario for an agent thrown into a genuinely novel online game, and as far as the authors are aware, no prior CWM work has addressed it.

The technical solution here is elegant: a kind of code-based autoencoder.

The LLM still produces a CWM and a hidden-history inference function, but the unit tests are restricted to only what’s verifiable without hidden information.

Essentially, “does the round-trip from observation through inferred hidden history back to observation reproduce what I actually saw?”

The inference function acts as an encoder, the CWM acts as a decoder, and the combination of game rules plus the required API serves as the regularizer that keeps this from collapsing into a trivial, useless latent space (the kind of degenerate solution that plain autoencoders are notorious for if you don’t constrain them).

Performance does degrade in the closed-deck setting, sometimes substantially. Bargaining’s online inference accuracy drops from 100% (open deck) to 76% (closed deck). Gin Rummy, already the hard case, becomes essentially unusable: inference accuracy collapses to roughly 9-54% depending on whether you’re looking at train, test, or online numbers, none of which is good. But here’s the genuinely surprising finding: despite this synthesis-quality degradation, game-play performance against Gemini 2.5 Pro doesn’t degrade nearly as much. The CWM-ISMCTS agent built on a noticeably worse closed-deck model still beats or matches the LLM-as-policy baseline in most games. The paper even notes a counterintuitive improvement in one game (Hand of War) under closed-deck conditions, hypothesizing that the freedom to synthesize a simpler state space — one not constrained to match some externally-imposed “true” hidden state — might sometimes help rather than hurt.

The practical lesson: synthesis-accuracy metrics and downstream-performance metrics are not the same thing, and a CWM that’s “wrong” in calibrated, support-preserving ways can still be useful for planning, because what the planner actually needs is a valid state to reason from, not necessarily the state.

Tree Search vs. Conversation

Neither a CWM nor an inference function will come out right on the first try. For all forms of machine learning we need a way to provide corrective feedback.

During training, the refinement loop generates unit tests from the offline trajectories: for each recorded transition, a test checks whether the CWM’s prediction matches what actually happened, across states, observations, rewards, and move legality. The CWM is refined until either every test passes or the compute budget runs out.

The CWM operates two mechanisms. Conversation and Tree Search.

“Conversation” is a sequential agentic loop where each failed test’s stack trace gets appended to the conversation history, and the model is asked to fix it.

“Tree search” is more elaborate. Here, multiple candidate CWMs are maintained simultaneously in a refinement tree, and Thompson sampling decides which candidate to refine next, weighted toward candidates with either high test-pass rates or few refinement attempts so far, essentially borrowing the bandit-style exploration/exploitation logic that underlies MCTS itself, but applied to code candidates rather than game moves.

The key practical advantage of tree search is the ability to backtrack: if a line of refinement is going nowhere, the search can abandon it and try a different branch, rather than being stuck digging deeper into a conversational dead end.

For the perfect-information games in the paper, both approaches converge to perfect transition accuracy quickly, but tree search is also resilient under imperfect-information conditions.

Synthesizing the Value Function

After the CWM itself and the inference functions, a Code World Model requires a third artifact: heuristic value functions (HVF). The HVF is used to initialize the value of newly-expanded leaf nodes in MCTS and ISMCTS rather than relying purely on random rollouts. Unlike the CWM and inference functions, value functions aren’t refined against ground truth as there is no “correct” heuristic to test against.

Instead, several candidates are generated in one shot, and the best one is selected through a local tournament: agents using different value functions (including a “no value function, just rollouts” baseline) play against each other on the synthesized CWM, and the winner is kept.

Where Code World Models and JEPA meet

It’s worth being precise about what’s actually shared between the DeepMind CWM line of work and the world-model research associated with Yann LeCun, because the overlap is conceptual rather than architectural.

LeCun’s “Path Towards Autonomous Machine Intelligence” position paper and succeeding work are about learned, latent-space world models. JEPA’s central bet is that generative models, which try to predict every pixel of the future, are solving the wrong problem; the world is full of irrelevant, unpredictable detail, and a model that tries to reconstruct all of it will waste capacity and fail. Instead, JEPA-style architectures learn an abstract representation of “what matters” and make predictions in that abstract space, closer to how, in LeCun’s framing, a child develops an intuitive model of gravity and object permanence without ever needing to predict the exact trajectory of every falling object down to the pixel.

The stable-worldmodel library operationalizes this research. Their implementation is effectively “a learned simulator that predicts how an environment evolves in response to actions, enabling agents to plan by imagining future outcomes,” providing fundamentals like: standardized dataset collection from simulated environments, training pipelines for world models, and solvers (including CEM, the Cross-Entropy Method, a classic sampling-based planning algorithm) that use the learned world model for planning. The workflow mimics model-based RL: collect rollouts, train a model that predicts state transitions, then plan against that model rather than against the real environment.

So where’s the connection to Code World Models?

Not in the representation, CWMs are symbolic (literal Python code with explicit state dictionaries and deterministic functions), while JEPA-style world models are learned and latent (neural networks operating on abstract embeddings, typically for continuous, high-dimensional domains like video and robotics).

But the function is identical: both are “a thing you can run forward to imagine what happens next, so that a planner doesn’t need to act blind.”

This actually clarifies when each approach makes sense. Code World Models shine when the domain has crisp, describable rules. I.e., board games, card games, turn-based strategy, anything where “the rules” is a coherent concept that fits in a prompt and where exact, verifiable transitions matter (you really don’t want your Poker agent confused about whether a flush beats a straight).

Learned latent world models shine when the domain is continuous, high-dimensional, and resistant to crisp specification, robotic manipulation, video, and physical dynamics, where “writing down the rules” isn’t really a coherent idea, because the rules are physics, at a level of detail no one’s going to type into a prompt.

In both approaches, the model that “understands” the domain is explicitly decoupled from the system that acts in it.

In CWM, the LLM’s job ends at producing a simulator; MCTS or ISMCTS does the planning.

In JEPA-based systems, the learned world model’s job is to predict consequences in representation space; the CEM solver or other planning algorithm does the planning.

Both architectures are, in a sense, rejections of the “one model does everything end-to-end” approach, whether that’s an LLM directly emitting moves, or a generative video model directly emitting pixels of an imagined future. The bet, in both cases, is that separating “knowing how the world works” from “deciding what to do about it” produces systems that are more verifiable, more sample-efficient, and more capable of the kind of deep, deliberate lookahead that neither pure pattern-matching nor pure generation reliably delivers on its own.

What This Should Mean in Practice

We are building agents here. So if you’re also building one to run a game, then if this game is operating an environment with quirks, edge cases, or undocumented behaviors that diverge from any official rulebook, then the CWM approach offers a concrete, falsifiable workflow.

You need to:

1. get the LLM to write the simulator,

2. generate unit tests from real trajectories,

3. refine until those tests pass, and

4. only then let a classical planner loose on the result.

The paper’s results, CWM-based agents matching or beating a frontier “thinking” LLM used as a direct policy in nine out of ten games, including ones invented specifically to be unfamiliar, suggest this isn’t a niche technique.

source: figure 4

It’s a genuinely different division of labor between language models and search, one where the LLM’s most valuable contribution might not be the move it suggests, but the simulator it writes.

Unresolved cases are instructive, too. Gin Rummy’s procedural, multi-stage scoring logic remains the hard frontier, a reminder that “translate the rules into code” is itself a task with a difficulty gradient, and that gradient tracks procedural depth more than it tracks how complex a game seems to a human player. For anyone applying this approach to their own domain, that’s probably the single most useful diagnostic question to ask up front: how many sequential, conditional phases does a single turn actually involve? The more of those there are, the more refinement budget, and the more carefully-chosen unit tests — you should expect to need before the world model the LLM writes actually matches the world it’s describing.

Thank you for reading this far.

Kindly share this post if you think it provides value to you. Thank you!