The benchmarks measure the mechanism. The real test is whether the graft map helps an agent ship real changes on code people actually run, not just answer questions. So we benchmark it on popular open-source repos: 15 tasks each, 10 real developer questions plus 5 actual implementation tasks (real merged pull requests, each re-implemented from its base commit and scored against the files the maintainers actually changed). Same agent (Claude Opus), same file tools; the only difference is whether the graft map is wired in.

On PocketBase the graft map ran 21% cheaper and 14% faster across the 15 tasks, and 2.6× cheaper on the hardest cross-file question, with no loss of correctness: it reproduced the real merged PRs by touching the same files the maintainers did.

PocketBase (Go, ~350 files)

| Aggregate over 15 tasks | Standard Claude Code | With the graft map | |---|---|---| | Cost | $13.91 | $11.02 (−21%) | | Wall-clock | 2,044s | 1,762s (−14%) | | PRs reproduced | 5 / 5 | 5 / 5 (same files as the maintainers) |

Cheaper and faster with no loss of correctness: the graft map reproduced all five merged PRs, touching the same files the maintainers did. The gap is widest on cross-file understanding: "how does auth work across OAuth2 providers" dropped from $2.19 to $0.84.

<details> <summary><b>The 10 questions we asked</b></summary>

1. Orientation: Give me a map of PocketBase's architecture: the main subsystems and how an HTTP request flows through to the database.
2. Entry-point trace: Trace end-to-end what happens when a client creates a record via the REST API, from route handler to database write.
3. Feature location: I want to add a brand-new collection field type. Where do I hook it in, and which pieces must change?
4. Bug localization: Realtime subscriptions silently stop delivering events after a while. Where would you start looking, and why?
5. Blast radius: If I change the signature of the record-validation logic, what depends on it and what could break?
6. Cross-file synthesis: How does auth work across OAuth2 providers: where are tokens issued, validated, stored, and refreshed?
7. Extensibility: How do I use PocketBase as a Go framework to register a custom route plus an on-record-create hook?
8. Security discovery: Where is user input validated, and where are collection API access rules enforced before a query runs?
9. Public API: As an external app, how do I authenticate and then list and filter records over the REST API?
10. Test verification: Where are the tests for the record CRUD API, and what do they assert about access rules?

</details> <details> <summary><b>The 5 merged PRs we re-implemented</b></summary>

Each PR was reset to its base commit; the agent's diff was scored against the files the merged PR changed.

| PR | Type | What it does | Files the maintainers touched | |---|---|---|---| | #6744 | feat | Generate & serve WebP thumbnails | apis/file.go, tools/filesystem/filesystem.go | | #6947 | fix | Uniform char distribution in regex random strings | tools/security/random_by_regex.go | | #6690 | refactor | Patreon OAuth2 to use x/oauth2/endpoints | tools/auth/patreon.go | | #2726 | perf | Drop a redundant admin-count query on a hot middleware path | apis/middlewares.go | | #3192 | fix | Restore prior API rules on automigration rollback | plugins/migratecmd/templates.go |

</details> <details> <summary><b>Method</b></summary>

Two clones of PocketBase at the same commit: one wired with graft init, one untouched and verified to have no graft map in it. Each task run headless (claude -p, Claude Opus) with an empty MCP config. Understanding questions were graded by whether the answer pointed to the right files and functions; PR tasks were scored on whether the agent's diff touched the same files as the merged PR. Every transcript was audited to confirm the graft map was actually used in one arm and absent from the other.

</details>

---