Where graphify might help

16 October 2026

this page is speculation. the benchmark in graphify-benchmark found where graphify did not clearly help. it did not map where graphify does help. each idea below is a guess, labelled with how much evidence supports it.

the words "agent", "call", "note", "hook", "new hook", "plain grep" and "gold" mean what the word list in graphify-benchmark says.

1. Caller questions that a note answers in full

the guess. when a note lists every caller with its calling line, the agent can answer without searching further.

why. on question E01, the new hook answered in 1 and 3 calls. the old hook and plain grep both needed about 15 calls. the first small test also showed caller coverage rising from 0.77–0.83 to 1.00 with a hook.

the doubt. E01 is one question, with two trials. the first small test used 9 to 15 trials per setup, on one small repository.

the test. run the new hook against plain grep on many caller questions, with three repeats each. count calls and tokens, and check accuracy.

2. A cheaper model with the new hook

the guess. a cheaper model searches more, so a note that settles a question saves it more calls.

why. haiku 5.5 with plain grep matched sonnet 5.5 on accuracy. haiku made 41 calls per trial against sonnet's 28. haiku used 642k tokens per trial against 291k, and cost about 2.2 times as much. those extra calls are the work a note could replace.

the doubt. no haiku trial used the new hook. the cost gap may come from how haiku reads files, which a note may not change.

the test. run haiku 5.5 with the new hook and with plain grep, on the same questions. compare calls, tokens and cost. graphify-benchmark-haiku has the plain-grep numbers.

3. Chains of calls

the guess. a chain question asks "what reaches this function through two calls?". the agent must rebuild the chain one search at a time. the graph already holds each link.

why. on the two chain questions in the replay, the notes named 44% and 56% of the gold items. the agent still spent 20 or more calls rebuilding the chains.

the doubt. the old note listed single links, not whole chains. the agent checked those links anyway.

the test. give the note the full chains, each link with its calling line. then compare calls on chain questions against plain grep.

4. Whole-system questions

the guess. questions such as "what are the main parts of this codebase?" or "what does everything depend on?" suit a graph. grep cannot answer them in a few searches.

why. graphify groups code into clusters and finds the most-connected parts. an agent with grep would have to read widely to build the same picture.

the doubt. nothing measured this. such answers have no exact gold list, so scoring them needs a written checklist of facts.

the test. write a few whole-system questions, each with a checklist of facts built from the source. score answers with plain grep and with graph tools.

5. Documents and other non-code inputs

the guess. graphify also builds graphs from documents, papers and images. there, an agent has no function names to search for.

why. grep finds exact words. a question about ideas across many documents may share no words with the answer.

the doubt. nothing measured this. every question in the benchmark was about code.

the test. build a graph over a set of documents. ask questions whose answers span several files, and compare with plain grep.

6. Long sessions with many questions

the guess. a graph is built once and answers many questions. in a long session, that one-time cost spreads out.

why. every benchmark trial was one question in a fresh session. so each trial paid the full cost of learning the codebase from zero.

the doubt. the benchmark did not count graph build cost against graphify at all. so this idea mostly matters for comparing total cost, not per-question cost.

the test. run sessions of ten related questions each. count total tokens per session, with and without graphify.

Where to look

back to graphify-benchmark.