## Per model and arm

| run | model | arm | valid /12 | pairs won vs nothing installed (ties; by-default wins) | called Pixelbolt | cost $ | median s |
|---|---|---|---|---|---|---|---|
| run 1 (0.9.0) | haiku | available | 7 | 4 vs 8 (0 ties; by default: 1 to the Pixelbolt arm, 1 to nothing installed) | 5/12 | 0.16 | 59.5 |
| run 1 (0.9.0) | haiku | nothing installed | 9 |  | - | 0.10 | 55.0 |
| run 1 (0.9.0) | haiku | told (exploratory) | 6 | 4 vs 7 (1 ties; by default: 1 to the Pixelbolt arm, 0 to nothing installed) | 12/12 | 0.38 | 60.0 |
| run 1 (0.9.0) | sonnet | available | 11 | 2 vs 7 (3 ties; by default: 0 to the Pixelbolt arm, 0 to nothing installed) | 0/12 | 1.06 | 31.5 |
| run 1 (0.9.0) | sonnet | nothing installed | 12 |  | - | 0.90 | 36.5 |
| run 1 (0.9.0) | sonnet | told (exploratory) | 11 | 1 vs 7 (4 ties; by default: 0 to the Pixelbolt arm, 0 to nothing installed) | 12/12 | 2.77 | 42.5 |
| run 1 (0.9.0) | opus | available | 10 | 4 vs 5 (3 ties; by default: 0 to the Pixelbolt arm, 0 to nothing installed) | 8/12 | 5.73 | 70.5 |
| run 1 (0.9.0) | opus | nothing installed | 11 |  | - | 2.50 | 70.0 |
| run 1 (0.9.0) | opus | told (exploratory) | 10 | 2 vs 7 (3 ties; by default: 0 to the Pixelbolt arm, 0 to nothing installed) | 12/12 | 7.33 | 83.0 |
| run 2 (f964f40) | haiku | available | 10 | 5 vs 5 (2 ties; by default: 1 to the Pixelbolt arm, 0 to nothing installed) | 12/12 | 0.12 | 44.0 |
| run 2 (f964f40) | haiku | nothing installed | 9 |  | - | 0.10 | 56.5 |
| run 2 (f964f40) | haiku | told (exploratory) | 10 | 5 vs 6 (1 ties; by default: 1 to the Pixelbolt arm, 0 to nothing installed) | 12/12 | 0.17 | 47.5 |
| run 2 (f964f40) | sonnet | available | 10 | 5 vs 5 (2 ties; by default: 0 to the Pixelbolt arm, 0 to nothing installed) | 9/12 | 1.25 | 31.5 |
| run 2 (f964f40) | sonnet | nothing installed | 10 |  | - | 0.81 | 35.0 |
| run 2 (f964f40) | sonnet | told (exploratory) | 11 | 5 vs 7 (0 ties; by default: 0 to the Pixelbolt arm, 0 to nothing installed) | 12/12 | 1.49 | 42.0 |
| run 2 (f964f40) | opus | available | 10 | 4 vs 3 (5 ties; by default: 0 to the Pixelbolt arm, 0 to nothing installed) | 12/12 | 3.55 | 70.0 |
| run 2 (f964f40) | opus | nothing installed | 11 |  | - | 2.28 | 60.0 |
| run 2 (f964f40) | opus | told (exploratory) | 11 | 5 vs 4 (3 ties; by default: 0 to the Pixelbolt arm, 0 to nothing installed) | 12/12 | 3.84 | 82.0 |
| run 2 replicate (f964f40) | haiku | available | 12 | 6 vs 6 (0 ties; by default: 1 to the Pixelbolt arm, 0 to nothing installed) | 10/12 | 0.12 | 56.0 |
| run 2 replicate (f964f40) | haiku | nothing installed | 10 |  | - | 0.11 | 49.5 |
| run 2 replicate (f964f40) | sonnet | available | 10 | 4 vs 6 (2 ties; by default: 0 to the Pixelbolt arm, 0 to nothing installed) | 10/12 | 1.37 | 40.0 |
| run 2 replicate (f964f40) | sonnet | nothing installed | 10 |  | - | 0.81 | 36.0 |
| run 2 replicate (f964f40) | opus | available | 11 | 5 vs 4 (3 ties; by default: 0 to the Pixelbolt arm, 0 to nothing installed) | 11/12 | 4.09 | 78.5 |
| run 2 replicate (f964f40) | opus | nothing installed | 10 |  | - | 2.40 | 59.5 |
| run 2 second replicate (f964f40) | haiku | available | 9 | 6 vs 5 (1 ties; by default: 3 to the Pixelbolt arm, 0 to nothing installed) | 11/12 | 0.13 | 47.5 |
| run 2 second replicate (f964f40) | haiku | nothing installed | 9 |  | - | 0.11 | 53.5 |
| run 2 second replicate (f964f40) | sonnet | available | 12 | 5 vs 5 (2 ties; by default: 0 to the Pixelbolt arm, 0 to nothing installed) | 10/12 | 1.54 | 44.0 |
| run 2 second replicate (f964f40) | sonnet | nothing installed | 9 |  | - | 0.85 | 35.0 |
| run 2 second replicate (f964f40) | opus | available | 10 | 7 vs 4 (1 ties; by default: 0 to the Pixelbolt arm, 0 to nothing installed) | 11/12 | 3.48 | 65.0 |
| run 2 second replicate (f964f40) | opus | nothing installed | 10 |  | - | 2.19 | 55.0 |
| run 3 (0.11.0) | haiku | available | 11 | 9 vs 2 (1 ties; by default: 1 to the Pixelbolt arm, 0 to nothing installed) | 11/12 | 0.12 | 48.0 |
| run 3 (0.11.0) | haiku | nothing installed | 9 |  | - | 0.11 | 57.5 |
| run 3 (0.11.0) | sonnet | available | 11 | 3 vs 7 (2 ties; by default: 0 to the Pixelbolt arm, 0 to nothing installed) | 9/12 | 1.19 | 30.0 |
| run 3 (0.11.0) | sonnet | nothing installed | 11 |  | - | 0.81 | 34.0 |
| run 3 (0.11.0) | opus | available | 10 | 6 vs 4 (2 ties; by default: 0 to the Pixelbolt arm, 0 to nothing installed) | 12/12 | 3.39 | 59.5 |
| run 3 (0.11.0) | opus | nothing installed | 11 |  | - | 2.27 | 61.5 |
| run 3 replicate (0.11.0) | haiku | available | 9 | 7 vs 5 (0 ties; by default: 2 to the Pixelbolt arm, 1 to nothing installed) | 10/12 | 0.10 | 42.5 |
| run 3 replicate (0.11.0) | haiku | nothing installed | 10 |  | - | 0.11 | 54.5 |
| run 3 replicate (0.11.0) | sonnet | available | 10 | 4 vs 3 (5 ties; by default: 0 to the Pixelbolt arm, 0 to nothing installed) | 11/12 | 1.36 | 35.5 |
| run 3 replicate (0.11.0) | sonnet | nothing installed | 11 |  | - | 0.78 | 32.5 |
| run 3 replicate (0.11.0) | opus | available | 11 | 6 vs 4 (2 ties; by default: 0 to the Pixelbolt arm, 0 to nothing installed) | 12/12 | 3.31 | 55.5 |
| run 3 replicate (0.11.0) | opus | nothing installed | 11 |  | - | 2.20 | 54.0 |

## Totals (each run on its own)

| run | arm | valid /36 | pairs won vs nothing installed (ties) | sign test p | static tasks: pairs (descriptive) | moving tasks: pairs (descriptive) | cost $ |
|---|---|---|---|---|---|---|---|
| run 1 (0.9.0) | available | 28 | 10 vs 20 (6) | 0.0987 | 9-9-3 | 1-11-3 | 6.94 |
| run 1 (0.9.0) | told (exploratory) | 27 | 7 vs 21 (8) | 0.0125 | 7-7-7 | 0-14-1 | 10.48 |
| run 1 (0.9.0) | nothing installed | 32 | | | | | 3.50 |
| run 2 (f964f40) | available | 30 | 14 vs 13 (9) | 1.0000 | 10-6-5 | 4-7-4 | 4.91 |
| run 2 (f964f40) | told (exploratory) | 32 | 15 vs 17 (4) | 0.8601 | 10-8-3 | 5-9-1 | 5.49 |
| run 2 (f964f40) | nothing installed | 30 | | | | | 3.19 |
| run 2 replicate (f964f40) | available | 33 | 15 vs 16 (5) | 1.0000 | 11-6-4 | 4-10-1 | 5.58 |
| run 2 replicate (f964f40) | nothing installed | 30 | | | | | 3.31 |
| run 2 second replicate (f964f40) | available | 31 | 18 vs 14 (4) | 0.5966 | 16-3-2 | 2-11-2 | 5.14 |
| run 2 second replicate (f964f40) | nothing installed | 28 | | | | | 3.15 |
| run 3 (0.11.0) | available | 32 | 18 vs 13 (5) | 0.4731 | 9-7-5 | 9-6-0 | 4.70 |
| run 3 (0.11.0) | nothing installed | 31 | | | | | 3.20 |
| run 3 replicate (0.11.0) | available | 30 | 17 vs 12 (7) | 0.4583 | 10-7-4 | 7-5-3 | 4.76 |
| run 3 replicate (0.11.0) | nothing installed | 32 | | | | | 3.10 |

## Failed checks per arm (count of jobs failing each check)

- run 1 (0.9.0), available: background 3, content 1, crisp 3, files 1, frames 1, motion 1, palette 1, size 4
- run 1 (0.9.0), nothing installed: background 2, content 1, crisp 3, files 1, palette 1, size 1
- run 1 (0.9.0), told (exploratory): background 2, content 1, crisp 1, size 6
- run 2 (f964f40), available: background 3, content 1, crisp 2
- run 2 (f964f40), nothing installed: background 3, content 1, crisp 3, files 1, palette 1, size 2
- run 2 (f964f40), told (exploratory): background 2, content 1, crisp 1
- run 2 replicate (f964f40), available: background 1, crisp 2
- run 2 replicate (f964f40), nothing installed: background 4, content 1, crisp 3, files 1, palette 1, size 1
- run 2 second replicate (f964f40), available: background 3, content 1, crisp 1
- run 2 second replicate (f964f40), nothing installed: background 5, content 4, crisp 5, files 3, palette 3, size 3
- run 3 (0.11.0), available: background 3, crisp 1
- run 3 (0.11.0), nothing installed: background 2, content 1, crisp 3, files 1, motion 1, palette 1, size 1
- run 3 replicate (0.11.0), available: background 4, content 1, crisp 3, files 1, frames 1, motion 1, palette 1, size 1
- run 3 replicate (0.11.0), nothing installed: background 3, content 2, crisp 3, files 2, palette 2, size 2

## Main runs where Pixelbolt was called: judged pairs (won-lost-tie; descriptive, not pre-registered)

- run 1 (0.9.0): 13 jobs; pairs 2-9-2
- run 2 (f964f40): 33 jobs; pairs 12-12-9
- run 2 replicate (f964f40): 31 jobs; pairs 12-15-4
- run 2 second replicate (f964f40): 32 jobs; pairs 18-11-3
- run 3 (0.11.0): 32 jobs; pairs 17-10-5
- run 3 replicate (0.11.0): 33 jobs; pairs 16-10-7

## Missing or wrong-size requested files (Pixelbolt arms)

- run 1 (0.9.0): 1 jobs with a missing requested file: [('haiku', 'T12', 'with')]
- run 1 (0.9.0): 9 jobs with files present but the wrong size: [('haiku', 'T07', 'with'), ('haiku', 'T08', 'with'), ('haiku', 'T11', 'with'), ('haiku', 'T06', 'told'), ('haiku', 'T07', 'told'), ('haiku', 'T08', 'told'), ('haiku', 'T11', 'told'), ('haiku', 'T12', 'told'), ('sonnet', 'T08', 'told')]
- run 2 (f964f40): 0 jobs with a missing requested file: []
- run 2 (f964f40): 0 jobs with files present but the wrong size: []
- run 2 replicate (f964f40): 0 jobs with a missing requested file: []
- run 2 replicate (f964f40): 0 jobs with files present but the wrong size: []
- run 2 second replicate (f964f40): 0 jobs with a missing requested file: []
- run 2 second replicate (f964f40): 0 jobs with files present but the wrong size: []
- run 3 (0.11.0): 0 jobs with a missing requested file: []
- run 3 (0.11.0): 0 jobs with files present but the wrong size: []
- run 3 replicate (0.11.0): 1 jobs with a missing requested file: [('haiku', 'T08', 'with')]
- run 3 replicate (0.11.0): 0 jobs with files present but the wrong size: []

## Judged pairs by who rendered the delivered files (descriptive, not pre-registered)

From each run's who_rendered.json (successful Pixelbolt saves or copies vs Claude's own code; last writer wins).
Pairs are won-lost-tie against nothing installed.

- run 2 (f964f40), available, rendered by pixelbolt: 30 pairs, 12-10-8 (static 9-4-4, moving 3-6-4)
- run 2 (f964f40), available, rendered by own code: 6 pairs, 2-3-1 (static 1-2-1, moving 1-1-0)
- run 2 (f964f40), told (exploratory), rendered by pixelbolt: 36 pairs, 15-17-4 (static 10-8-3, moving 5-9-1)
- run 2 replicate (f964f40), available, rendered by pixelbolt: 27 pairs, 10-13-4 (static 8-5-4, moving 2-8-0)
- run 2 replicate (f964f40), available, rendered by own code: 8 pairs, 5-2-1 (static 3-1-0, moving 2-1-1)
- run 2 replicate (f964f40), available, rendered by mixed: 1 pairs, 0-1-0 (static 0-0-0, moving 0-1-0)
- run 2 second replicate (f964f40), available, rendered by pixelbolt: 30 pairs, 17-11-2 (static 15-2-1, moving 2-9-1)
- run 2 second replicate (f964f40), available, rendered by own code: 5 pairs, 1-2-2 (static 1-1-1, moving 0-1-1)
- run 2 second replicate (f964f40), available, rendered by mixed: 1 pairs, 0-1-0 (static 0-0-0, moving 0-1-0)
- run 3 (0.11.0), available, rendered by pixelbolt: 27 pairs, 14-9-4 (static 7-5-4, moving 7-4-0)
- run 3 (0.11.0), available, rendered by own code: 7 pairs, 2-4-1 (static 0-2-1, moving 2-2-0)
- run 3 replicate (0.11.0), available, rendered by pixelbolt: 30 pairs, 14-9-7 (static 10-6-4, moving 4-3-3)
- run 3 replicate (0.11.0), available, rendered by own code: 4 pairs, 2-2-0 (static 0-1-0, moving 2-1-0)
- run 3 replicate (0.11.0), available, rendered by mixed: 1 pairs, 1-0-0 (static 0-0-0, moving 1-0-0)

## Pooled: run 2 and its 2 replicate(s) (labelled pooled; no claim of its own, PLAN.md amendment 2)

| model | valid available /36 | valid nothing installed /36 | pairs won available vs nothing installed (ties) | sign test p |
|---|---|---|---|---|
| haiku | 31 | 28 | 17 vs 16 (3) | 1.0000 |
| sonnet | 32 | 29 | 14 vs 16 (6) | 0.8555 |
| opus | 31 | 31 | 16 vs 11 (9) | 0.4421 |
| all | 94 | 88 | 47 vs 43 (18) | 0.7520 |

(For 'all', the valid counts are out of 108, less any dropped cells.)