Lab notebook · updated 2 November 2026
A dated record of what the lab started, tested and found. Failed predictions stay in the record next to the ones that held. Each entry says who checked it and, where one exists, links to the full write-up.
How to read this: a failed prediction means a test we wrote down before the run did not pass. In the game-learner entries, it usually means no meaningful gain was shown, not that the learner got worse. Those entries follow one learner that plays the game Terraria; “items” means the different kinds of items it found in one game. The improvement loop (from 1 November) plays games of 9,000 decisions and versions 1 to 6 played games of up to 3,000, so their item counts cannot be compared. Most entries were checked by AI reviewers, and each entry says which. No outside human team has reproduced any of these results yet.
2 Nov 2026Terraria learner
Our automatic improvement loop tested practising on games three times as long (9,000 decisions instead of 3,000). On 13 new worlds, with 78 games for each side, the new recipe found 8.97 different items per game against 9.37 for the current best. That is 0.40 fewer, with a 95% range from 1.27 fewer to 0.32 more (p = 0.40). The rule fixed before the run judged it a fail.
2 Nov 2026Terraria learner
Part of the learner’s reward paid for every unit of an item it gained. That part of its reward grew from 2.6 to 107.5 per game while the number of different items it found stayed flat, which looks like farming the reward rather than discovering things. A small probe by the builder (6 games) was consistent with that explanation but did not settle it. A revised reward that pays only for new highs cut that part’s share of the reward from 0.84 to 0.37. The number of different items found did not change: 10.60 per game against 10.69 (p = 0.86). The planned test judged it a fail.
1 Nov 2026Terraria learner
Version 6 added general signals about its own progress and a reward for reaching new places that shrinks with repeat visits. On 13 new worlds, after matched practice, it led version 5 by 0.28 items per game, with a 95% range from 0.40 fewer to 0.94 more (p = 0.44). That was short of the gain the plan required. Extra practice alone did not help version 5 either. On the same day, a rerun of version 5 with a different random seed did not beat version 2 (7.14 against 7.29 items per game). No later version has yet beaten version 2 by the margin its test required.
1 Nov 2026Remembering vs learning
A pre-registered follow-up to the remembering-vs-learning lab, on its weak point: recovering after the world changes without warning. Over 160 paired games, the revised learner scored 0.073 higher on the lab’s transfer measure, with a 95% range from +0.055 to +0.090. It recovered in 133 of 134 games, against 97 for the original, and needed a median of 6 presses against 20 (counting only games that recovered). The rule for how the world changes was given to the learner. It did not discover it.
1 Nov 2026Terraria learner
The loop trains the current best recipe and a challenger with a change, tests both on 13 brand-new worlds, and keeps whichever wins under a rule fixed in advance. The first two generations were inconclusive. One tried “career mode”, where the character keeps its items from world to world, with a small added reward. The other tried a lower learning rate. Generations 3 and 4, above, were judged fails. The loop scores games of 9,000 decisions, so its item counts cannot be compared with the 3,000-decision results for versions 1 to 6.
31 Oct 2026Terraria learner
Version 5 goes back to an unplayed copy of its practice world after every four games. On 13 more new worlds it averaged 7.7 items per game, against 7.3 for version 2 and 5.0 for its untrained start. It clearly beat its untrained start. Its lead over version 2, 0.4 items per game (p = 0.23; 95% range from 0.4 fewer to 1.2 more), was inconclusive.
29 Oct 2026Terraria learner
Version 4 started out inclined to keep holding its current item. It found 6.3 items per game against 7.6 for version 2, and was ahead in only 2 of 13 new worlds. Training also undid the change, so the idea that holding a tool longer helps is untested rather than refuted. A separate test found the practice worlds wearing out. With version 2’s finished weights held fixed, it averaged 7.3 items per game over the first 8 games on a world and 5.1 over games 17 to 24.
29 Oct 2026Terraria learner
Over 52 games each, version 2 averaged 7.0 different items, against 4.9 for its untrained start and 5.1 for random play. It beat its untrained start in all 13 worlds. Most of the gain is one skill: it picked up dirt in all 52 games, against 12 for its untrained start. A third version, which changed only how it aims, failed both of its tests on the earlier three worlds (5.5 items against 7.5 for version 2), as scored by the rule fixed in advance.
28 Oct 2026Terraria learner
Our first Terraria player was hand-built. In its last 11.8 hours, 5,840 of its 6,188 decisions (94.4%) were the same forced move, and the game refused it every time. We started again with a general learner that has no game-specific rules. Version 1 found 6.67 items per game, against 5.67 untrained and 3.83 for random play. That met its pre-registered mark of one extra item exactly, and its second test failed. Version 2 failed its test on that first small sample.
28 Oct 2026This website
The home page became a board of five investigations, each with an evidence label and notes on what was found, why it matters and what is still open. A full write-up for the intelligence programme went up, and the map of every area and tool moved to Explore.
21 Oct 2026Remembering vs learning
We tested seven learners against nine predictions written in advance, over 960 games, plus 480 more in a learning-to-learn chain. Eight of the nine predictions held. A learner that only memorised did no better than random play. A learner that shared what it learned across controls reached 90% accuracy in 9.1 presses, against 20.5 for one that did not. One prediction failed: after the controls were swapped, recovery reached 0.175, against a predicted 0.8 or more.
16 Oct 2026Canadian Century
Construction: at UBC’s Brock Commons, the peak panel-installation rate was about 3.28 times the starting rate. Electricity: Site C produced 3,574 GWh in fiscal 2026, a commissioning year; its long-run average of 5,100 GWh describes different conditions, so the gap is not a shortfall estimate. Mine haulage: an operator’s investor-day transcript reports more than 20% more material tons per hour in 2024 with a similar fleet; we report the claim, we have not verified it. A production-evidence toolkit makes the first measurements a production claim would need open to inspection. Two of the studies can be rerun on your own computer.
15 Oct 2026Canadian Century
A transparent model built from Statistics Canada industry tables, with its code and data published. One cross-check failed: the GDP and population series we retrieved did not reconcile with the published per-person figure. We report that mismatch openly and publish no national or household gain estimate. Update, 17 October: for one quarter, an earlier population release reconstructs the published per-person figure, which points to a data-vintage explanation; the agency’s exact method is unconfirmed. The Canadian Century plan sets out the questions the work is meant to answer.
15 Oct 2026Design lab
Designs from Anthropic, OpenAI and xAI models, and a search with no AI in it, each beat simple baselines on 4 of 5 synthetic worlds. Small local models beat them on none. The top six designs swapped places on every rerun, so the ranking among them is mostly noise. Update, 1 November (the builder’s own measurement): the test worlds are now saturated. The best design scores +0.225 against a ceiling of +0.243, too close to tell new designs apart.
12 Oct 2026Code repair
A pilot of 32 runs on 8 tasks produced no accepted repairs. In 31 runs the AI never reached the testing step, and the one candidate that did failed six checks. The pilot cannot say whether guidance helps.
Older history: the platform’s earlier activity stream is still on What GLEE is doing. Spotted an error in an entry? Tell us.