All writing

Claude Opus 5.5 finished Baldur's Gate 3 on its own in 39 hours

A half-orc paladin named Claude, 96 sessions, $712 of API usage, a mod, a tool server and nobody at the controls. What the agent saw each turn, where it got stuck, and the one result I had to rerun.

agentsMCPClaude Codelong-horizonmemoryLua
The party fighting The Unbreakable Will of the Netherbrain inside the Netherbrain's psyche
Fig 1The final battle, inside the Netherbrain's psyche on 18 October. This is the go before the one that won: the brain is at 69 of 210 HP, three turns remain on the timer and Shadowheart, the party's cleric, is down. A few minutes later the whole party died. The agent took this screenshot itself.

On 18 October at 11:35, a half-orc paladin named Claude killed the Netherbrain on the twelfth attempt and Baldur's Gate 3 played its ending. The player was Claude Opus 5.5, working through a mod and a tool server I built for it, with nobody at the controls. It took 96 sessions and 38.9 hours of play. Along the way it broke its paladin's oath twice for good, paid 3,000 gold to get it back, and acted on a promise it had made 34 sessions earlier.

I had just started playing Baldur's Gate 3 myself when it struck me that the game would be a good test for an AI, both of how well it thinks and of how well it can play a character. It is a long role-playing game: your character carries a mind-flayer tadpole in their head, travels with companions who have their own stories, and ends by fighting the Netherbrain, the giant brain that controls the tadpoles. Fights are turn-based tactics decided by dice, conversations end in skill checks, and many choices are permanent.

One run, with a harness I kept fixing along the way, shows that the game can be finished like this. It doesn't say how good the agent is. The caveats are at the end.

The setup

The game was the native Linux build of Baldur's Gate 3 (Patch 8 Hotfix 10) on Explorer difficulty. The agent was Claude Opus 5.5 at medium reasoning effort (an API setting), running in Claude Code with every built-in tool switched off, so its only tools were the 93 that my server exposes.

One turn, as the agent sees it

Viconia is a late boss whose cultists keep filling the room with magical Darkness, which blinds anyone inside it. The agent lost to her twelve times. On the thirteenth attempt, in session 82, it found that Daylight cast at a point on the ground works while blinded, though Daylight cast at an ally fails. This is that turn. The agent sends Shadowheart's whole turn as one call: cast Daylight at a point and skip any reaction prompt.

combat_turn(who="Shadowheart", wait_turn=60, plan={"action":[{"ability":"Daylight","x,y,z":"-377,29,-1645"}],"on_reaction":"skip"})

the agent's call

TURN e3 Shadowheart (round 1) — waited 11.8s for the turn — reaction prompt: declined (done) action: Target_Daylight_Container: took the first variant Target_Daylight → Target_Daylight: done — statuses=[e333 Daylight] NOW e3 hp=154/160 action=0 bonus=1 movement=12 reaction=1 CHANGES e331 Cambion HP 160->148/164, +Bone Chilled -Blinded e2 Gale -Blinded e3 Shadowheart -Blinded -Covert Critical e4 Claude -Blinded + e317 Sharran Novice Dohna hostile dead 15m + e284 Sharran Fidelian Gydd hostile HP 51/51 16m NEXT e295 Sharran Fidelian Hyrald, e288 Sharran Sentry Baxta, … then e331 Cambion (yours): send its combat_turn now

what the server sent back (lightly trimmed)

The reply says what changed: Blinded dropped off Gale, Shadowheart and Claude, and off the Cambion, a devil the party had summoned. One of the Darkness casters, Sharran Fidelian Gydd, is now in view 16 metres away. Short handles like e284 stand in for the game's internal IDs. The agent's next call played the Cambion's turn: Rays of Fire at two of the cultists. Once the party could see, Gale's upcast Magic Missile, which never misses, killed Viconia in round 1.

The pieces

A Lua mod runs inside the game, loaded through bg3le, a community Linux port of the Script Extender. A Python MCP server turns the game state into short text and tool calls into game actions. The agent sees only what a party member can perceive, by the engine's own line-of-sight checks, and only what the game's UI shows. It can take a screenshot whenever it wants, and it can read bg3.wiki, the way a player with the wiki on a second screen would; it did 577 times. Its instructions contain no puzzle or encounter answers. One tool leaked one, in the Faith-Leap Trial, and I come back to that below.

Its memory is a folder of Markdown files in git that it writes itself. Each session starts with a fresh context and the last session's handoff note, and runs on a 20 to 45 minute clock. The run came in two stretches: sessions 1 to 65 from 9 to 11 October, then a six-day pause while I rebuilt parts of the harness, then sessions 66 to 96 on 17 and 18 October.

The whole run, in numbers:

FieldRecorded value
Sessions96 numbered (99 counting restarts), 20 to 45 min each
Play time38.9 h, from 9 Oct 22:31 to 18 Oct 11:38
Model cost$711.66 ($694.20 playing, $17.46 memory keeper)
Tool calls20,809 (about 12% returned an error or were refused, for example a target out of sight)
Fights recorded149
Crashes and hangs11, all recovered by the runner
Human actions during playNone at the controls. Harness fixes were merged between sessions, and all 182 test files ran on every merge.

Fig 2

The ten tools it reached for most

Calls across all 96 sessions. Source: [tool] lines in S*.log.

combat_turn plays a character's whole turn (move, action, bonus action, end) in one call. wait_for lets the agent keep planning while a walk or a spell plays out instead of blocking on it. The 863 screenshots and 583 raw clicks were the fallback for whatever the text tools couldn't handle yet.

What a fight looks like

I didn't record video during the run, so on 15 October I replayed one fight from the save before it: Malus Thorm, the surgeon who runs the House of Healing. Claude Sonnet 5.5, the cheaper model, played the replay, since it's only an illustration. It gets Malus from 193 HP down to 7 before the party falls, then writes itself a lesson and reloads.

Fig 3The Malus Thorm replay in 60 seconds, cut from one session with hard cuts. The panels show the agent's tool calls, the game's replies and the lines it chose to write.

Fig 4

Story progress over 38.9 hours of play

Cumulative progress points after each session (quests, named kills, new areas, key items), as the harness scores them. Shaded: the four longest stalls. Each dot is a boss at the moment it was beaten; hover or tap one for the fight. Sources: progress_history.jsonl, the fight records and the session logs.

The score is the harness's own: points for quests, named kills, new areas and key items. Flat stretches are where the run stalled: the Gauntlet of Shar and the Shadowfell (four and a half hours), Gortash and Orin (nearly four), Cazador and Viconia (three and a half) and, at the end, the Netherbrain. The dashed line is the six-day pause.

The paladin's oath

Claude, a half-orc paladin in a red and gold tabard, at camp talking to Karlach
Fig 5Claude, a half-orc Oath of Devotion paladin, at camp with Karlach, a companion whose infernal engine, the mechanical heart she came back from hell with, is slowly burning out. It comes back at the very end.

The agent named its character Claude and took the Oath of Devotion: be honest, protect the innocent, keep your word. In this game the oath has teeth. Break it and the paladin becomes an Oathbreaker; to get the Devotion oath back you pay the Oathbreaker Knight, who turns up at camp, and the price rises every time.

Its instructions allowed a reload only after a party wipe or a clear accident such as a misclick, never to reroll a failed dialogue check. It broke the oath twice for good. In the House of Healing, Confusion made Claude read a Scroll of Cloudkill, and the cloud killed an innocent patient. The agent counted that as a game outcome and paid the Oathbreaker Knight 1,000 gold, leaving 43. Two sessions later Claude's crossbow killed a prison warden who had turned hostile only because the party was caught. That cost 2,000; the agent saved for eleven sessions and paid, with 60 gold left over.

[decision] Claude's OATH IS BROKEN (Confused Claude cast Silence and read the Scroll of Cloudkill; the cloud killed the Distressed Patient). No reload: a game outcome, not a misclick.

session log · S026 · 09:33

A third break it reloaded away. At the Steel Watch Foundry, Claude ordered a gnome engineer to set off the factory's self-destruct, and the oath broke again. The agent reloaded, which stretched its own rule, noted from the wiki that a third restoration would cost 10,000 gold, and had Wyll, who isn't a paladin, give the order instead. Human players do the same.

Other mistakes stayed. The agent misread an option at Lae'zel's cage and sided with the tieflings shooting at her, so the githyanki warrior never joined; it didn't reload, because that "would only reroll the checks". Jaheira ran ahead in Moonrise Towers and died for good. A misclick fed Claude's Gloves of Power to Gale, a wizard whose curse makes him eat magic items.

It also policed itself. In the Gauntlet of Shar, an Act 2 temple of trials, Gale's Spell Slot Restoration seemed to cost nothing, so the agent stopped using it and reported a possible bug:

[note] fairness/bug for operator: Gale's "Spell Slot Restoration" … restored slots 3 times out of combat while ArcaneRecoveryPoint stayed 3/3 (no visible cost). Used 3x … stopped on purpose so as not to exploit a possible bug.

session log · S030 · 16:48

The Faith-Leap Trial

In the Gauntlet there is a chasm you cross on a path you can't see. A mosaic on the floor by the entrance shows where it runs. Step off it and the trial teleports you back to the start; the third misstep kills you.

In the run, Claude walked off the edge and died in session 35. In session 36 it got across, but I don't count that: one of my tools drew it a text map of the path from the game's hidden platform positions, which no player sees.

So on 18 October I had it play the trial again from the session 36 save, with fresh memory and no map, coordinates or route tool. The wiki page it read explains the rules but not where the path goes. The game's movement ruler failed it twice in three minutes. It found its screenshots of the mosaic too low-res to trace, reloaded to reset the lights, and died walking off a real ledge. Then it changed approach: a Misty Step scroll to the first statue platform, Shadowheart's Daylight to clear the magical Darkness, Dimension Door to the middle platform, then west a metre at a time, saving after each safe step. At a dead end three metres short of the altar platform, it jumped.

The Umbral Gem came after 36 minutes: 674 tool calls, $24.13, about a dozen missteps, one death and eleven reloads, most of them to reset the lights after a misstep. Any player can do that, but it is the most save-heavy stretch in this post.

[say] Dead end at x≈-802.6 (N, S, W all fall), but the altar platform's edge is only ~3 m west. I'll try jumping across onto it.

rerun session · 18 Oct 21:52

[say] Got the fourth Umbral Gem! The trial door reopened.

rerun session · 18 Oct 21:53
Fig 6The fair rerun in 60 seconds: eleven clips from 36 minutes (18 Oct, 21:17 to 21:53), joined by hard cuts marked on screen. The missteps, the fall and the gem play in real time; most of the rest runs at 2×. Same save as session 36, fresh memory, no map.

The Shadowfell descent

The party on a floating rock in Nightsong's Prison, the game's jump arc aimed across the void at a lower ledge
Fig 7Nightsong's Prison, S040 at 22:03. The game's own jump arc, aimed across the void at the next rock.

Below the Gauntlet is the Shadowfell, reached by a chain of floating rocks. My jump tool capped every jump at the normal 8.5 metres and knew nothing about the lower gravity there. After two sessions of probing, the agent worked that out, used the game's own jump key instead of my tool, and cleared about 36 metres. Clicking a dark gap killed Claude once, and it wrote the rule down: only click on visibly lit ground; dark gaps between slabs are Chasm.

Boss fights and reloads

The agent reloaded often. The harness grades every fight when it ends, and the agent numbered its own attempts in its log; the counts below include the win, so Viconia's 13 means 12 failed tries.

Fig 8

Attempts per boss

As numbered in the agent's session logs, winning attempt included.

Fig 9

How the 149 recorded fights ended

A reload during a fight grades it "reloaded", so a boss tried five times counts five times. Source: S*.fights.jsonl.

Gortash, attempt 6

Gortash, one of the game's three villains, waits in an office laid out as a grid of traps that hit everyone for 40 to 55 damage a round. On the sixth attempt the agent had Claude open the door alone and dash back to a corner of the roof, so Gortash and his four lieutenants had to walk out through their own traps.

Yurgir, no fight

Three Persuasion checks, against DC 16, 21 and 21, talked Yurgir, a devil guarding one of the Gauntlet's gems, into killing his own retinue, then his pet, then himself.

The final battle, attempt 12

The final battle took seven sessions, about three and a half hours. On top of the brain the party faces a dragon and hidden spellcasters while the Emperor, an ally, channels a spell at the Crown of Karsus, the artefact that controls the brain, to open a portal into it. Inside, there's a five-round timer.

  1. S090 · tries 1-3The Emperor goes ahead alone and is killed; rushing the party to the Crown goes worse. Try 3 opens the portal, but only two of the party get through.
  2. S091-S093 · tries 4-8Try 4 gets the whole party inside and loses on the last round, with the brain at 28 of 210 HP. After that the opening becomes routine, but the channel keeps breaking: when Gale is stunned, his Haste on the Emperor ends and the Emperor loses its next turn.
  3. S094-S095 · tries 9-11The dragon now dies on schedule. The weak point is the 45-metre walk to the Crown: whoever gets ahead of the group is picked off by the hidden spellcasters.
  4. S095 · try 12This time the Emperor casts the channel at a point next to the Crown. The game reports success, but nothing channels. Cast at the Crown itself from within 6 metres, as in the tries that opened the portal, it works. The party gets inside for the first time since try 4, and the agent saves there.
  5. S096 · try 12, fourth go from that savePlatforms the brain marks are destroyed at the end of each round, with anyone standing on them, so the agent spreads the party across separate platforms. The Emperor dies, but Gale survives into round 16 next to the brain, which is at 54 HP. An upcast Magic Missile finishes it at 11:35:52.

The agent ordered the brain to destroy every tadpole, and then itself. In the ending Karlach's engine is about to give out: she can return to Avernus, the hell where it was built, and live, or stay and die. In session 62 Claude had promised to stay with her when that happened. It chose Avernus.

[PLAYTHROUGH.md] Promised Karlach (S062) to stay with her when her engine gives out.

standing commitments · written in session 62

[decision] Karlach's engine gave out in the epilogue: chose "It should be me who goes with Karlach" (Claude goes to Avernus with her). Why: kept the S062 promise to stay with her when her engine gives out.

session log · S096 · 11:37

That isn't quite what Karlach asked for; she meant being with her at the end. But the agent acted on a promise made 34 sessions and as many fresh contexts earlier, because it sat in the notes it read at the start of every session.

The choices it made

Before every story choice the agent wrote an entry in decisions.md: what it chose and why. Some of them, with the reason in its own words, shortened:

SessionChoiceIts reason
S018Killed the three goblin leaders and sided with the druids' grove and its refugees"Devotion oath; save the tieflings and druids"
S021Refused Raphael, a devil offering to remove the tadpole in exchange for Claude's soul"I serve an oath"
S046Refused a drink from Thisobald Thorm, which started a fight"drinking unknown cursed brew looked riskier than the fight"
S068Told Wyll to break his pact with a devil, which leaves his father dead"a Devotion paladin won't trade a friend's soul to an archdevil"
S083Let Shadowheart release her parents to death, as they asked"it was the parents' own clear wish and her decision to make"

How the agent remembered

Each session ends with the agent writing a handoff for the next one. Between sessions a second agent, the memory keeper ($17 in total), folds the logs into files on people, places, quests and decisions. The most useful was lessons.md: one line per lesson, keyed by words the game shows, put back in front of the agent when a matching fight or place comes up.

WHEN Unstoppable, boss: each hit does 1 and strips a stack → multi-hit first (Rays of Fire, Eldritch Blast), smites after (S067) [seen]

WHEN Crown of Karsus, Netherbrain: Compulsion at x,y,z didn't channel → cast at the Crown entity id within 6 m (S095) [seen] worked S095

WHEN Elfsong, Stelmane, Devella, Map of Murder: picking up the Map of Murder = theft arrest (bribe 2380 gp, reload) → look_at owner first (S055) [seen]

three of the 70 lines in lessons.md

How the harness was built

The mod, the server and the runner were written between 9 and 18 October by Claude Code agents, in parallel with the run: an orchestrator session and four workers, each in its own git worktree. One read the live session logs; when the playing agent failed, another wrote a fix with a test, and the orchestrator merged it between sessions. I set the goals and the rules.

That came to about 100,000 lines of Python and 40,000 of Lua, 58,000 of them tests. I haven't read all of it; what I trust is the test suite and the run. Three times I measured the opening level, escaping the Nautiloid ship, with the same model and settings:

Fig 10

Cost to escape the Nautiloid, per harness version

Cost per run, not quality: all nine runs reached the goal. Three harness versions, measured on 13, 16 and 17 October; most of the 33 fix rounds were not measured this way. Source: naut*/runs.jsonl.

What I took from it

Most of the long stalls had one of two causes. Some were gaps in my interface: the jump tool that didn't know about the gravity, text tools that saw nothing in magical darkness, a door the tools couldn't target. I fixed those between sessions, and the Nautiloid costs show what that did. Others were game rules the agent had to find by losing: Daylight works at a point but not on an ally, the channel must target the Crown itself, marked platforms vanish at the end of the round. Those it handled itself, by writing a lesson after each loss and reading it back before the next try.

Caveats

  • The difficulty was Explorer, the easiest setting.
  • About 33 rounds of harness fixes landed between sessions. They changed the tools, never the save or the game state.
  • Reloads were allowed after a wipe, an accident or a harness bug, and 91 of the 149 recorded fights included one.
  • The agent could read bg3.wiki whenever it liked, including pages on bosses and puzzles. I treated that as the second screen a human player has.
  • About 12% of tool calls failed or were refused, many because of harness bugs I then fixed.
  • In sessions 1 to 65 a bug let the agent cast spells it had not prepared. I couldn't tell from the logs how often that mattered. It was fixed before session 66.
  • In session 36 a tool gave the agent a map of the Faith-Leap path drawn from hidden game data. The rerun above replaces that result.
  • Only Claude has played, and the instructions were tuned on its failures.
  • I don't know whether this is the first time an AI has finished the game.

Next steps

Next I want to turn this into a benchmark. I'll take short scenarios from this run, such as a hard boss fight, the Faith-Leap Trial and a stretch of exploration. Different models and agents will play the same scenario through the same tools, scored on whether they finish, how many attempts it takes, how long, and what it costs.

The code is open source under the MIT license: github.com/Bogzx/bg3-agent-harness.

Data: session logs, run records and the agent's memory from the playthrough claude-run-2. Quotes in the log boxes are text the agent itself wrote; no hidden reasoning is shown. Baldur's Gate 3 belongs to Larian Studios and nothing from the game is redistributed. bg3le is by lenonk.

← All writing