A side finding: Haiku 5.5

16 October 2026

this page is out of scope for graphify-benchmark, so it lives here. the plan was to test whether a cheaper model plus graphify could match sonnet 5.5. the first step made that plan pointless.

The plan

1. run haiku 5.5 with plain grep on six questions, one trial each.
2. if haiku scored F1 0.95 or more, stop: haiku has no gap for graphify to close.
3. otherwise, run haiku with the new hook, and compare it with sonnet 5.5.

this plan was written down before the first haiku trial.

The result

haiku 5.5 scored above 0.95, so the run stopped after step 1.

the sonnet numbers are the main run's plain-grep trials, three per question. the haiku numbers are one trial per question.

What this means

haiku 5.5 matched sonnet 5.5 on accuracy. haiku also cost about 2.2 times as much per trial. haiku made more calls and read more text, and that outweighed its lower price per token.

so a cheaper model is not cheaper on these questions. if graphify can help haiku, it must cut haiku's extra calls. accuracy leaves no room.

Limits

• six questions, one trial each.
• the questions are the same six as in graphify-benchmark-new-hook, chosen by the same rule.
• the haiku-with-new-hook run never happened, so this page says nothing about graphify on haiku.

Where to look

back to graphify-benchmark.