Loading the claims…
Every claim tested so far. Each one had its prediction written down before the run, and is measured on many seeds with an interval, not on two or three. Claims that failed stay here, as plainly as the ones that held. The full record of how each was measured is in the lab notebook.
Loading the claims…
A simulation can be made to say almost anything if you look at enough runs and keep the ones you like. The lab's rules are there to stop that, in five habits.
Before a run starts, the claim, the command that reproduces it and what would count as a hit are written
into docs/claims.jsonl and committed. Afterwards the claim is marked confirmed or killed, never
quietly reworded. That is pre-registration, and it is why killed claims are listed above as plainly as the
confirmed ones.
A rule is compared with no rule on the same twenty seeds: the same city, the same founders, the same luck at the start, changed only by the rule. Each seed gives one difference, so the question is how those twenty differences behave, and not whether two averages of very noisy runs happen to differ.
For a rule, the mean change across seeds comes with a 95% interval from a bootstrap: the twenty seeds are resampled with replacement 4000 times and the middle 95% of the resulting means is kept. If the interval includes zero, the effect is not distinguishable from nothing, and the bar says so. For a proportion, such as how many replays the same family wins, the interval is a Wilson interval, which stays sensible near 0% and 100% and with few replays.
"The biggest family stays biggest" explains a lot of the monopoly results, so the replay study tests the leader against its own share of the cars before it claims any head start. That claim was killed. A result that survives with the drivers' evolution frozen, or with a hand-written driver that cannot evolve, is about the economy and not about evolution; one that disappears needed the evolving drivers.
Results that rested on two to four seeds flipped when seeds were added, and the README says so. A result is corrected where it was first stated, and the lab notebook keeps the original numbers and the reason for each correction. A tool that counted replays where the family merely survived, and not replays where it won, gave wrong tables twice; since then a win means ending with over half the cars, read straight from the tool's printed summary.