Your service topology is a graph. Stop embedding it.

The default RAG recipe is well known enough to be a reflex: take your documents, chunk them, embed the chunks, retrieve by cosine similarity, stuff the top k into a prompt. It works surprisingly well for corpora of prose.

It works badly for a service dependency map, and the reason is worth being precise about.

Similarity is not adjacency

If I ask "what does checkout-api depend on", the correct answer is a set of edges. It is exact, it is enumerable, and it is already stored somewhere as structured data.

A vector store answers a different question: which chunks of text are most similar to my query. Those two answers overlap enough to be dangerous. You will get back the section of the architecture doc that mentions checkout-api, plus a section about checkout-worker because the names are close, plus a runbook paragraph that happens to use similar vocabulary. You will not reliably get the two-hop dependency that has no lexical overlap with your query at all — and in an incident, the two-hop dependency is very often the thing that broke.

Worse, you get no signal about what is missing. A traversal that returns four neighbours is telling you there are four. A similarity search that returns four chunks is telling you nothing about whether there was a fifth.

What we actually did

Retrieval in the incident agent runs in two stages, and the order matters.

Stage one is a traversal. Start from the alerting service, expand one to two hops through the dependency graph, and collect the resulting node set. This is a graph query. It is exact, it is fast, and it cannot hallucinate a neighbour that does not exist. It also picks up things a text search would never connect: the shared intermediate certificate, the node pool that joined the mesh two days ago, the change record attached to a topology edge.

Stage two is scoped retrieval. Now use embeddings — over runbooks, prior postmortems, and telemetry summaries — but restricted to the node set from stage one. Semantic search is excellent at "find me the postmortem that describes this symptom", which is a genuinely fuzzy question. It is being used for the thing it is good at, on a candidate set that structure has already narrowed.

The graph decides what is relevant. The embeddings decide what is similar within it.

The part that surprised me

I expected the win to be precision. It was, but the larger win was that the agent's output became auditable.

When retrieval is a similarity score, "why did you pull this document" has no satisfying answer — 0.83 is not a reason. When the first stage is a traversal, the answer is a path: checkout-api → payments-ledger → pgbouncer, plus the change record on that edge. An engineer can look at the path and immediately tell you whether the agent was looking in the right place, before reading a word of the generated analysis.

That turned out to matter more for adoption than any accuracy number. Engineers do not trust a system because it is right; they trust it because they can check it quickly and it holds up. A retrieval path is checkable in two seconds. A relevance score is not checkable at all.

When the default recipe is fine

None of this is an argument against vector search. If your corpus is genuinely unstructured — support tickets, research papers, documentation with no reliable schema — then similarity is the only handle you have, and chunk-and-embed is the right call.

The mistake is applying it to data that already has structure, because the recipe is familiar. If you have a graph, query the graph. You paid to build it. Flattening it into embeddings is throwing away the expensive part and keeping the cheap part.