patrickz.aiLet’s talk ↗

Field notes Idea 14

I had a flight to Thailand. So AI rebuilt Mario with me in it.

In one day, parallel AI agents rebuilt Super Mario Bros. and The Legend of Zelda starring me, upscaled and over the top, playable on my phone. The real lesson was how to direct agents and let a bot judge their work.

Key art of Patrick in a red blazer leaping through a pixel-art Mario world with a sword and a rocket launcher while three goombas panic

The idea in one minute

The idea
When several AI agents build one project, give them a contract (who owns which files, and the few functions they share) and make a bot that plays the result the judge of whether it works.
Why it matters
Agents can produce a whole game in a day. The bottleneck moves to coordination and verification: agents break each other's work, and nobody can read their way to "it plays correctly".
Do this first
Before asking for features, write down the source of truth, the ownership rules, and the automated check that proves a change works.

A long flight, a phone, and a slightly unreasonable idea

I had a long flight to Thailand coming up, a Galaxy S24, a game controller and nothing I wanted to play. A normal person downloads a game. I decided to remake the classics with me in them: bald head, glasses, goatee, red blazer, upscaled.

The deadline was the boarding gate, so the real experiment was simple:

How much of a real, playable, polished game can I direct in one day if AI does the building and I only do the deciding?

The answer became The AI Game Museum. It boots like a 1985 cartridge on a living-room CRT, glitches, and opens into a neon museum hall with three exhibits:

  • Super Cool AI Bro, after Super Mario Bros. All 32 levels plus the bonus rooms. The mushroom became a sword, the fire flower became a rocket launcher, and underwater it fires torpedoes.
  • The Legend of Tokens, after The Legend of Zelda. The full first quest: overworld, nine dungeons, caves, shops, items and bosses.
  • IDE Zone, after Comix Zone. Still in the workshop.
The AI Game Museum hub: a neon hall with exhibit cards for Super Cool AI Bro, The Legend of Tokens and IDE Zone, which is marked coming soon
The museum hub. Each exhibit is a separate game built by its own agent session.

The Mario exhibit has a twist. It starts as a faithful 1985 game. The moment you pull a sword out of a question block, the screen glitches into the 2026 AI Remaster, with painted parallax backgrounds, HDR bloom, dynamic lighting, a remixed soundtrack and a livestream chat that roasts me. Goombas panic when you are armed. Every third coin block is a joke: a DLC paywall, an NFT, a loot box. A mini-boss called Goombzilla is a goomba "upscaled 400% by AI". It did not consent.

My human contribution was mostly direction, taste and saying "no, he should go down the pipe, not through it." That sentence turned out to be the whole lesson.

Watch the result

The Super Cool AI Bro trailer

1:30

Real gameplay captured from the game, announced by Patrick and Su with GPT Live voices and assembled as a local HyperFrames composition.

Patrick brags. Su corrects him. Every level shot is the game's own autopilot playing; nothing was keyframed by hand.
Download the trailer

Parallel agents needed a contract before they needed better prompts

I ran several Claude Code sessions at the same time, one per game, all inside one Godot project. Parallel agents in one codebase fail in a predictable way: each one "helpfully" rewrites files the others depend on. A clever prompt does not fix that. A contract does.

  1. OwnershipEach game owns its own folders. Shared files only get small, targeted edits.
  2. InterfaceThe museum talks to every game through three functions: start_game(id), return_to_museum() and is_available(id).
  3. Graceful gapsA game that is not ready shows COMING SOON instead of crashing the hub.
  4. Requests, not editsIf one agent needs something from another's area, it asks. "Hold the Android build until my scene compiles" was a message, not a merge conflict.

With that in place, the Mario session could build the museum while the Zelda session refactored its dungeon engine, and neither broke the other.

Rebuilding from the real game made "correct" measurable

"Make a Mario-like platformer" gets you a Mario-like platformer. I wanted the actual levels, so the agent wrote a decoder for the original game's level data: every block, pipe, enemy, lift, firebar, cannon, warp and hidden coin.

Before trusting it, the decoder had to reproduce a hand-built World 1 exactly. It matched 100% of the geometry. Only then did it generate all 32 levels plus 24 bonus rooms. The Zelda session did the same for the overworld, dungeons and caves, then ported enemy and item behavior rule by rule.

A known source of truth changes the job from inventing a game to rebuilding one faithfully. You can check a rebuild. You can only argue about an invention.

Turning myself into a sprite was a video pipeline, not a drawing

  1. DesignGoogle Flow with Nano Banana Pro: me, stylized, in the red blazer.
  2. PerformShort AI video clips on a flat green background: walk, run, jump, sword slashes, a spin attack, crouch-firing a rocket launcher, swimming and underwater attacks.
  3. CutScripts pull the frames, key the background, measure the ground line and body height so every animation lines up, and keep only the frames that matter.
  4. TagThe frame where the sword connects or the rocket leaves the barrel becomes game data, so gameplay syncs to the animation instead of a timer.
Six animation frames of Patrick crouching with a shoulder-mounted rocket launcher, firing, and the back-blast smoke clearing
The rocket-fire animation, generated as video and auto-cut into frames. The launcher only appears while he is firing.

My first launcher was a separate image bolted onto the walking animation, and it looked wrong. The fix was not a better sticker. It was generating a proper firing performance and showing the launcher only while he fires. Generate the performance you need; don't bolt props onto the one you have.

The 2026 remaster is a stack of swappable AI layers

The same moment of World 1-1 side by side: a flat 1985 CRT frame on the left, and the 2026 remaster with painted hills, god rays, a head-shine flare and a live chat overlay on the right
Same level, same code. Left: 1985. Right: after pulling the sword, the 2026 AI Remaster.

Each theme (day, night, snow, underwater, sky, castle) gets a far painting, an animated atmosphere shader (god rays, caustics, aurora), mid and near cutout layers, and a blurred foreground strip for depth of field. Seedream painted the backgrounds; Google Flow painted the cutouts on flat key colors, which a script keys and tiles automatically.

ElevenLabs composed every theme twice, as a chiptune and a remix, synced so the remaster crossfades mid-song. Stable Audio made the modern sound effects and a tiny numpy synthesizer made the 8-bit ones.

Keep providers swappable. One image service ran out of credit mid-day. The pipeline only expected a picture on a flat color, so switching to Flow cost minutes, not the afternoon.

The most valuable feature was a bot that plays the game

No one can read their way to "every level is beatable". So I had the agents build an autopilot that plays the whole game headless, from World 1-1 to Bowser in 8-4 including every bonus room, and logs what happened.

godot --headless --path game --fixed-fps 60 -- demo god trace
[trace] 1-4 x=2/166 hp=3 sword=false score=26900 coins=14 deaths=0 kills=10
[death] 4-3 col=76 cause=hit@76
# full run: 1-1 → 8-4 plus every room, 0 script errors

It found bugs I would never have found by hand: a bridge tile that wasn't solid, a castle loop that dropped you inside a wall, coins that didn't disappear, and a power-up that landed on top of the bricks. Death logs with a cause (enemy, pit, lava, timer) separated "the bot is bad at hammer bros" from "this level is broken".

The same bot, in Godot's movie mode, shot every gameplay clip in the trailer. It even got a trailer mode that fires rockets while navigating and hides the on-screen control hints.

The trailer review caught what no screenshot would

The trailer is announced by Patrick and Su, performed by two persistent GPT Live actors that hear each other's previous line before replying. The first cut had problems that only appeared when I watched it like a viewer and then measured it like an engineer:

  • A truncated take. "Underwater? Torpedo." had lost "Torpedo"; the recorder treated Patrick's dramatic pause as the end of his line. I re-performed one line.
  • Uneven loudness. Lines varied by 7 dB, so some vanished under explosions. Normalizing and automatic ducking fixed it.
  • Tough cuts. Several shots began during the game's own fade-in from its level card.
  • Music restarting at every cut. I re-recorded gameplay with sound effects only and laid one continuous score under the whole trailer.
  • Physics comedy I didn't ask for. Patrick slid across warp pipes and walked past castle doors. The fix belonged in the game, not the edit, so gameplay now looks right too.

Try it

Remake one level of a classic with one agent and one bot

Do not start with a whole game or several agents. Start with one level, one agent and one automated check, then add scope only when the check stays green.

  1. Pick a source of truth

    Choose one level you can describe exactly: a published map, your own sketch on grid paper, or a level from a game you own. Write down what "faithful" means before any code exists.

  2. Write the agent's contract

    List the folders it owns, the files it may only touch lightly, and the one or two functions other parts of the project will call. Paste it at the top of every task.

  3. Build the smallest playable slice

    Movement, one enemy, one hazard, and a finish line. Play it for two minutes yourself before asking for more.

  4. Add a bot that plays it

    Ask the agent for an autopilot that runs the level without you, logs position, deaths with a cause, and script errors, and prints a one-line result.

  5. Give it one twist, then re-run the bot

    Add the thing that makes it yours (you as the hero, a joke power-up, a remaster toggle). Every change must end with a green bot run and a short recording you actually watch.

You’ll know it worked when

You can play the level, the bot finishes it and reports zero script errors, and a recording shows the twist working the way you intended.

When you’re ready

Scale out only after the contract holds

When one agent and one bot are reliable, add a second agent for a second game or system with its own folders and the same shared interface. Keep a written list of shared files and a short "requests" channel between sessions.

Then turn the bot into production tooling: add test arguments to start at a column, skip the title, force a power-up, or record footage. The QA harness becomes your camera crew.

This project is a private, non-commercial tribute. The original games belong to their creators and nothing is being distributed. If you rebuild a classic, keep it personal, or make the level data your own.

Avoid these

Common mistakes

  • Letting parallel agents share everything

    Without ownership rules, agents quietly undo each other's work. Assign folders and a small shared interface first.

  • Reviewing by reading code

    "It looks right" is not "it plays right". Let a bot run the build and report deaths, stalls and errors.

  • Bolting props onto a generated character

    Stitched-on items look fake. Generate the animation that actually performs the action.

  • Locking into one AI provider

    Credits run out and services go down. Make every asset step accept any image or clip in a known format.

  • Judging a video from stills

    Truncated audio, loudness jumps and bad cuts only appear in playback. Watch it, then measure it.

Copy, adapt, run

Prompts to try

Paste one into any chat assistant and replace anything in [brackets].

Write a multi-agent contract

Several AI coding agents will work in one repository on [project]. Propose a contract: which folders each agent owns, which shared files may receive only small targeted edits, the exact functions they use to integrate, how an unfinished part should fail gracefully, and how agents request changes in another agent's area. Keep it under one page.

Build an autopilot tester

Add an autopilot to my [engine] game that plays [level] without input. It should navigate, use the core abilities, log position every second, log each death with its cause, detect being stuck for more than [N] seconds, and print one summary line: levels completed, deaths, and script errors. Add command-line flags to start at a position and to enable invincibility for coverage runs.

Turn a character video into sprites

I have short videos of a character on a flat green background performing [actions]. Write a pipeline that extracts frames, keys out the background, measures the ground line and body height so all animations align, trims to the frames that read clearly, and records the frame where each action lands (a hit or a projectile release) as data the game can use.

Browse every prompt on the site ↗

The tool kit

Tools and links

  • Claude CodeSeveral parallel sessions, one per game, integrated through a written museum contract.
  • Godot 4Engine, headless autopilot runs, movie-mode capture for the trailer, and the Android build.
  • Google FlowNano Banana Pro character design, Veo and Omni animation clips, parallax cutouts and museum art.
  • SeedreamPainted backgrounds for the 2026 remaster.
  • ElevenLabs Music and Stable AudioEvery theme in a 1985 and a 2026 version, plus modern sound effects. A small numpy synthesizer made the 8-bit ones.
  • GPT LivePatrick and Su performed as two persistent actors that hear each other between lines.
  • HyperFramesLocal HTML-to-video composition, checks and render for the 90-second trailer.

The short version

What to remember

  • With several agents, a contract (ownership plus a small shared interface) matters more than a clever prompt.
  • Rebuild from a known source of truth so quality can be checked, not debated.
  • A bot that plays the build is the fastest reviewer you have, and it can shoot the trailer too.
  • Generate the performance you need instead of bolting props onto a character.
  • AI moved my work from typing to deciding. Taste and verification are the job now.

Next idea · 15 of 15 · Take on bigger projects

Build for one real person first.

Learn to start an AI project from one person's real needs, pick the easiest input that works for them, keep a non-AI fallback, and measure what matters to them.

Read idea 15
A woman selects a cup picture on a communication tablet while Patrick and Su listen