ProtoMap benchmarks: Claude Code on 9 gRPC repos

Per prompt

Every prompt, with and without ProtoMap

The median of 2 runs per prompt (their mean), on a log scale. Time is the whole session's wall-clock time; cost and tokens are what Claude Code reports at API prices.

Time to Answer Cost Tokens

Services in a repo 1m 12s 39s 1.8×

Most-connected services 23m 56s 12s 121×

Dependents of a service 6m 36s 18s 22×

Callers of a gRPC method 3m 19s 24s 8.3×

Field blast radius 6m 27s 21s 18×

Changing a request field 3m 24s 1m 09s 3.0×

Response field source 25s 18s 1.4×

Two-hop field lineage 49s 24s 2.0×

Removing a method 8m 13s 27s 18×

Downstream dependencies 53s 22s 2.4×

Service outage impact 7m 18s 16s 28×

Shortest call chain 5m 09s 8s 39×

Callers outside handlers 4m 59s 28s 11×

Service API docs 1m 54s 22s 5.3×

Dead gRPC methods 2m 49s 2m 41s 1.1×

Personal data flow 7m 12s 29s 15×

Uses of a response field 4m 43s 25s 11×

What a handler calls 2m 32s 36s 4.2×

Cross-domain calls 2m 54s 20s 8.7×

Proto change impact 2m 11s 24s 5.4×

Services in a repo $0.58 $0.31 47%

Most-connected services $9.86 $0.15 98%

Dependents of a service $1.81 $0.19 90%

Callers of a gRPC method $1.13 $0.19 83%

Field blast radius $9.00 $0.22 98%

Changing a request field $1.69 $0.65 62%

Response field source $0.24 $0.21 13%

Two-hop field lineage $0.47 $0.20 57%

Removing a method $4.46 $0.20 96%

Downstream dependencies $0.49 $0.19 61%

Service outage impact $1.97 $0.13 93%

Shortest call chain $1.55 $0.076 95%

Callers outside handlers $8.70 $0.20 98%

Service API docs $1.06 $0.23 78%

Dead gRPC methods $0.97 $0.51 47%

Personal data flow $1.45 $0.30 79%

Uses of a response field $2.48 $0.31 88%

What a handler calls $0.34 $0.39 −15%

Cross-domain calls $0.92 $0.19 79%

Proto change impact $1.51 $0.31 79%

Services in a repo 411K 109K 73%

Most-connected services 15.8M 71K 99%

Dependents of a service 2.7M 73K 97%

Callers of a gRPC method 1.2M 98K 92%

Field blast radius 16.8M 107K 99%

Changing a request field 1.9M 693K 64%

Response field source 172K 161K 6%

Two-hop field lineage 378K 121K 68%

Removing a method 7.0M 130K 98%

Downstream dependencies 473K 84K 82%

Service outage impact 3.4M 68K 98%

Shortest call chain 2.0M 62K 97%

Callers outside handlers 17.1M 100K 99%

Service API docs 1.3M 127K 90%

Dead gRPC methods 1.1M 502K 54%

Personal data flow 2.4M 178K 93%

Uses of a response field 3.5M 164K 95%

What a handler calls 261K 372K −43%

Cross-domain calls 1.0M 86K 91%

Proto change impact 1.6M 139K 91%

Opus 5.5, high effort
Prompt	Claude Code time	Claude Code + ProtoMap	Faster	Claude Code cost	Claude Code + ProtoMap	Cheaper	Claude Code tokens	Claude Code + ProtoMap
Services in a repo	1m 12s	39s	1.8×	$0.58	$0.31	1.9×	411K	109K
Most-connected services	23m 56s	12s	121×	$9.86	$0.15	64×	15.8M	71K
Dependents of a service	6m 36s	18s	22×	$1.81	$0.19	9.8×	2.7M	73K
Callers of a gRPC method	3m 19s	24s	8.3×	$1.13	$0.19	6.0×	1.2M	98K
Field blast radius	6m 27s	21s	18×	$9.00	$0.22	41×	16.8M	107K
Changing a request field	3m 24s	1m 09s	3.0×	$1.69	$0.65	2.6×	1.9M	693K
Response field source	25s	18s	1.4×	$0.24	$0.21	1.2×	172K	161K
Two-hop field lineage	49s	24s	2.0×	$0.47	$0.20	2.4×	378K	121K
Removing a method	8m 13s	27s	18×	$4.46	$0.20	23×	7.0M	130K
Downstream dependencies	53s	22s	2.4×	$0.49	$0.19	2.7×	473K	84K
Service outage impact	7m 18s	16s	28×	$1.97	$0.13	15×	3.4M	68K
Shortest call chain	5m 09s	8s	39×	$1.55	$0.076	20×	2.0M	62K
Callers outside handlers	4m 59s	28s	11×	$8.70	$0.20	44×	17.1M	100K
Service API docs	1m 54s	22s	5.3×	$1.06	$0.23	4.6×	1.3M	127K
Dead gRPC methods	2m 49s	2m 41s	1.1×	$0.97	$0.51	1.9×	1.1M	502K
Personal data flow	7m 12s	29s	15×	$1.45	$0.30	4.8×	2.4M	178K
Uses of a response field	4m 43s	25s	11×	$2.48	$0.31	8.1×	3.5M	164K
What a handler calls	2m 32s	36s	4.2×	$0.34	$0.39	0.9×	261K	372K
Cross-domain calls	2m 54s	20s	8.7×	$0.92	$0.19	4.8×	1.0M	86K
Proto change impact	2m 11s	24s	5.4×	$1.51	$0.31	4.9×	1.6M	139K
All prompts	96m 55s	10m 42s		$50.69	$5.13		80.5M	3.4M

All the figures, as a table

Prompt	Baseline time	ProtoMap time	Faster	Baseline cost	ProtoMap cost	Cheaper	Baseline tokens	ProtoMap tokens	ProtoMap calls
Q01 · list services	1m 12s	39s	1.8×	$0.58	$0.31	1.9×	411K	109K	3
Q02 · most connected	23m 56s	12s	121×	$9.86	$0.15	64×	15.8M	71K	2
Q03 · service dependents	6m 36s	18s	22×	$1.81	$0.19	9.8×	2.7M	73K	2
Q04 · rpc callers	3m 19s	24s	8.3×	$1.13	$0.19	6.0×	1.2M	98K	1
Q05 · field blast radius	6m 27s	21s	18×	$9.00	$0.22	41×	16.8M	107K	4
Q06 · request field change	3m 24s	1m 09s	3.0×	$1.69	$0.65	2.6×	1.9M	693K	4
Q07 · response provenance	25s	18s	1.4×	$0.24	$0.21	1.2×	172K	161K	1
Q08 · two hop lineage	49s	24s	2.0×	$0.47	$0.20	2.4×	378K	121K	5
Q09 · remove rpc	8m 13s	27s	18×	$4.46	$0.20	23×	7.0M	130K	2
Q10 · outbound dependencies	53s	22s	2.4×	$0.49	$0.19	2.7×	473K	84K	6
Q11 · transitive impact	7m 18s	16s	28×	$1.97	$0.13	15×	3.4M	68K	2
Q12 · call chain	5m 09s	8s	39×	$1.55	$0.076	20×	2.0M	62K	1
Q13 · non handler callers	4m 59s	28s	11×	$8.70	$0.20	44×	17.1M	100K	1
Q14 · api docs	1m 54s	22s	5.3×	$1.06	$0.23	4.6×	1.3M	127K	3
Q15 · dead rpcs	2m 49s	2m 41s	1.1×	$0.97	$0.51	1.9×	1.1M	502K	4
Q16 · pii flow	7m 12s	29s	15×	$1.45	$0.30	4.8×	2.4M	178K	9
Q17 · response consumers	4m 43s	25s	11×	$2.48	$0.31	8.1×	3.5M	164K	4
Q18 · handler fanout	2m 32s	36s	4.2×	$0.34	$0.39	0.9×	261K	372K	2
Q19 · domain dependencies	2m 54s	20s	8.7×	$0.92	$0.19	4.8×	1.0M	86K	2
Q20 · proto change repos	2m 11s	24s	5.4×	$1.51	$0.31	4.9×	1.6M	139K	8
All prompts	96m 55s	10m 42s	9.1×	$50.69	$5.13	9.9×	80.5M	3.4M

Accuracy

100% with ProtoMap, 98% without

Each answer scores the share of the facts it must state, such as every calling service, that it names. The facts come from the fixtures' oracles (expected.json, written by the generator of the code, not by ProtoMap) and were checked by hand where ProtoMap and the oracle disagreed; the agents could not read the oracles, which were moved out of the workspace. Mean of 2 runs.

Prompt	Baseline	ProtoMap	Facts graded
Q01 · list services	100%	100%	110
Q02 · most connected	80%	100%	top 10
Q03 · service dependents	100%	100%	16
Q04 · rpc callers	96%	100%	14
Q05 · field blast radius	100%	100%	12
Q06 · request field change	100%	100%	17
Q07 · response provenance	100%	100%	4
Q08 · two hop lineage	100%	100%	5
Q09 · remove rpc	100%	100%	17
Q10 · outbound dependencies	96%	100%	12
Q11 · transitive impact	100%	100%	18
Q12 · call chain	100%	100%	4
Q13 · non handler callers	100%	100%	25
Q14 · api docs	100%	100%	21
Q15 · dead rpcs	100%	100%	4 dead of 9
Q16 · pii flow	96%	100%	12
Q17 · response consumers	100%	100%	8
Q18 · handler fanout	100%	100%	6
Q19 · domain dependencies	100%	100%	43
Q20 · proto change repos	100%	100%	6

The prompts

20 questions, as given

Five are the chat panel's quickstart prompts with real names filled in; the other 15 need several services, several repositories, or several protobuf messages to answer. Every prompt was given as written, after the same one-line preamble:

Answer this question about the code in the current directory, which holds Acme's service repositories. Work read-only: do not modify any files. Be concise and exact: give full names, and file:line where asked.

Q01 Quickstart
List all the gRPC services implemented in the Swift repo (protomap-test-swift). Give the fully qualified name of each and how many RPCs it has.
Q02 Quickstart
Show the 10 most connected gRPC services across all our repos: the ones with the most distinct other services calling them or being called by them. Give the inbound and outbound service counts for each.
Q03 Quickstart
Which services depend on acme.loyalty.v1.LoyaltyMemberQueryService? For each dependent service, tell me which repo it lives in and which RPCs it calls.
Q04 Quickstart
Which services call acme.customers.v1.CustomerService/ListCustomers? List every call site with file:line and the enclosing function, across all repos.
Q05 Quickstart
What is the blast radius of changing acme.ledger.v1.LedgerAccount.currency_code? Which services actually use this field's data (read it and pass it along to another RPC or into a response), and through which RPCs?
Q06 Cross-service
We're replacing the `customer` field of acme.customers.v1.UpdateCustomerRequest with a new message. Which code, in which repos, sets that field when calling acme.customers.v1.CustomerService/UpdateCustomer, and to which other RPCs does the UpdateCustomer handler pass the field on?
Q07 Cross-service
Where does the `cart` in the response of acme.cart.v1.CartCalculationService/SimulateCart come from? Trace it back to the upstream RPC responses or request fields it is built from.
Q08 Cross-service
Trace the `payout` field of acme.payments.v1.SchedulePayoutExpiryRequest two hops downstream: which RPCs does the SchedulePayoutExpiry handler send it to, and where do the handlers of those RPCs send it next? Name the services and repos at each hop.
Q09 Cross-service
We want to remove the RPC acme.documents.v1.DocumentQueryService/ResolveDocuments. Find every place in every repo that calls it, including indirect calls through helpers, method names built from strings, registries, or reflection, and give file:line and the calling service for each so we can migrate them.
Q10 Cross-service
What does acme.payments.v1.PaymentCalculationService depend on? List every downstream service it calls, which RPCs, and which repo owns each downstream service.
Q11 Cross-service
If acme.pricing.v1.PriceListQueryService goes down, which services are affected? Include direct callers and services that depend on it transitively through the RPCs that call it, up to 3 hops away, and say how many hops away each one is.
Q12 Cross-service
Can a request to acme.ticketing.v1.TicketTransferWorkflowService end up calling acme.ledger.v1.LedgerAccountValidationService? Show the shortest call chain hop by hop: the RPC called at each hop, and the repo and file:line where each call is made.
Q13 Cross-service
Which code outside of gRPC handlers (event workers, batch jobs, CLIs, server startup) calls the ledger services (acme.ledger.v1.*)? List each caller, its repo, and the ledger RPCs it calls.
Q14 Cross-service
Write API reference docs for acme.giftcards.v1.GiftCardService: for each RPC, its request and response messages, the file that implements it, and which services call it.
Q15 Cross-service
Which RPCs of acme.checkout.v1.CheckoutSessionService are never called by any code in any of our repos? We want to delete dead endpoints, so don't list an RPC that is actually called somewhere. End your answer with one line of the form `UNUSED: <comma-separated RPC names>`.
Q16 Cross-service
For a privacy review: which services' code actually reads a patient's national ID (acme.patients.v1.Patient.national_id) and passes it on, and through which RPCs does it travel?
Q17 Cross-service
If acme.customers.v1.CustomerCalculationService/ExplainCustomerCalculation stopped returning the `result` field of its response, which callers would be affected? For each caller, which RPC requests or responses does that data flow into?
Q18 Cross-service
What does the handler of acme.shipping.v1.ShipmentCalculationService/CalculateShipment call downstream, and which fields of its request end up in each downstream call?
Q19 Cross-service
Which other domains does the checkout domain (acme.checkout.v1) call into, and which domains call checkout? Summarize the dependencies in both directions.
Q20 Cross-service
We're making a breaking change to the acme.subscriptions.v1 protos. Which repos call or implement subscriptions services and would need to update and redeploy? For each repo, which subscriptions services does it use?

The codebase

Nine repositories, 5 million lines

Acme's nine repositories, generated from one catalog of APIs and each in a different language, call each other's services and copy each other's protobuf files, as an enterprise's do. Their code moves data through patterns that are hard to follow statically: method paths built from strings, registries, reflection, futures, callbacks, event buses, and framework dispatch. The indexing time is a fresh protomap scan of each repository on one machine.

Repository	Language	Lines (tracked)	Services	Calls	Field flows	Index time
protomap-test-csharp	C#	400,958	340	7,653	27,397	5 s
protomap-test-golang	Go	2,590,754	480	6,923	37,747	9 s
protomap-test-java	Java	590,063	400	6,027	32,215	4 s
protomap-test-javascript	JavaScript	166,210	180	2,567	13,730	3 s
protomap-test-kotlin	Kotlin	235,041	190	4,420	15,542	2 s
protomap-test-python	Python	298,060	240	3,726	19,177	5 s
protomap-test-rust	Rust	242,115	150	1,996	10,883	2 s
protomap-test-swift	Swift	152,493	110	1,974	9,690	2 s
protomap-test-typescript	TypeScript	314,723	290	4,226	22,873	4 s
All nine	7 languages + protobuf	4,990,417	2,380	39,512	189,254	35 s

4.99M tracked lines: 1.64M of service code, 1.23M of protobuf (each repository holds copies of the files of the services it calls), and 2.12M of generated Go bindings, which the Go repository checks in. Calls and flows are the oracles' counts of call sites and field-level dataflows the code makes.

How it was run

The same agent, tools, and prompt, with and without ProtoMap

Each run is a fresh claude -p session (Claude Code 2.1.306) in ~/protomap-test, which holds only the nine repositories, with nothing the agents could take answers from: the fixtures' answer keys, build outputs, and other projects were moved out for the runs.
Both arms have the same read-only tools: Read, Grep, Glob, subagents, and read-only shell commands (rg, grep, find, git grep, awk, jq, and the like); anything else is denied. The ProtoMap arm adds the ProtoMap MCP server (Streamable HTTP, the shared graph of the nine repositories); the baseline arm has no MCP servers.
User settings, CLAUDE.md files, and hooks were left out (--setting-sources project,local), so only the prompt differs.
Opus 5.5 ran at high effort. Every prompt ran twice per arm (80 runs; a third round was cut short by usage limits and discarded); the charts show the median, which for two runs is their mean, and accuracy is the mean.
Time is wall-clock time of the whole session; cost and tokens are what Claude Code reports at API prices, subagents and cache reads included. A run is stopped after 45 minutes and counts its 45 minutes and the cost it had reached.
Grading is by code, with no model as judge: each answer must name the services, methods, repositories, or fields of its prompt's answer key (by full or short name, or the file its implementation is in). The dead-RPC prompt scores the F1 of the list the answer ends with, so a wrong RPC costs as much as a missed one.

Try it on your code.

Install the self-hosted version for free. It runs on your machine with Docker, and your source code stays in your environment.

Book a Demo Start Free

Paste into your coding agent

Set up ProtoMap and show me what it can do. ProtoMap maps gRPC services and protobuf messages across repositories, and gives coding agents that graph over MCP. Docs: https://docs.protomap.ai/quickstart/

Tell me what you are about to do before each step that installs something.

1. Check that this computer has git, Docker with Compose 2.23.1 or newer, at least 4 GB of RAM, and 20 GB of disk space. If anything is missing, tell me and stop.

2. Install ProtoMap Server, in a folder outside this repository:

git clone https://github.com/hounddogai/protomap.git
cd protomap
./self-hosted/scripts/install.sh

The installer asks questions. Choose Trial, which runs PostgreSQL locally and logs the CLI in, and keep the default port.

3. Show me the address and the one-time setup key the installer prints. Ask me to open the address, enter the key, and create my organization and owner account. Wait until I say that's done.

4. Scan this repository, from a new shell so the CLI is on the PATH:

protomap scan .

If it has no gRPC services in a supported language, tell me, and offer to scan a public example instead:

git clone https://github.com/GoogleCloudPlatform/microservices-demo.git
protomap scan microservices-demo

5. From the scanned repository's folder, add ProtoMap to yourself as an MCP server:

Claude Code: claude mcp add protomap -- protomap mcp serve
Codex: codex mcp add protomap -- protomap mcp serve
Antigravity: add "protomap": { "command": "protomap", "args": ["mcp", "serve"] } under mcpServers in ~/.gemini/config/mcp_config.json
Others: https://docs.protomap.ai/coding-agents/

Tell me if I need to restart you for it to take effect.

6. Tell me what the scan found, then use ProtoMap's tools to answer one question about my services, such as which services call a method and where. Suggest two more questions I could ask.

13× fewer tokens, 7.7× faster answers for Claude Code with ProtoMap

Token savings, speed, cost, and accuracy