ADR 0042: Themes are recomputed, never stored

Status: accepted.

Why this is an ADR: it adds a thematic read surface above the entity —

the capability BACKLOG B142 filed and explicitly refused to let anyone

start in code. Four questions had to be answered before a line was worth

writing: which layer holds it, whether communities are persisted, what a

community is in a multi-tenant store, and whether clustering is a

GraphStore capability or a caller-side algorithm.

Context

Retriever.retrieve fuses two channels and returns k scored entities.

That is the entire read surface. "What are the main themes in this corpus" has

no answer and no composition of the existing ports produces one: neighbors

walks out from an entity you already know, and both retrieval channels rank

individual entities against a query string.

Microsoft GraphRAG's answer is two steps — cluster the graph, then write one

report per cluster with a model call. The cost shape is why it is worth

having at all: model calls scale with the number of communities, that is

with the corpus's structure, not with its length. It is the one place in that

design where a model call does not scale with tokens ingested.

0023 is what makes

the second step tractable here. GraphRAG's fast pipeline summarises a

community from its source text units rather than from entity descriptions,

because in that pipeline the descriptions are empty. We store passages now, so

that variant is available and B117's description-quality problem does not

block this.

Decision

Communities are recomputed on every call and never persisted

A community is a function of the whole graph at one instant. Every

DocumentExtracted and every EntitiesMerged can move a node between

clusters, and can do so for nodes the new document never mentioned — a single

edge joining two previously separate regions re-partitions both. So a stored

community is stale the moment the next document lands, and there is no

invalidation smaller than "recompute everything".

That is the difference from every other judgement in this library.

0004 says a judgement emits an event,

and a community report is a judgement — but the rule's premise is that the

judgement is about something durable enough for the record to remain true.

A merge is about two entity ids and stays true. A community is about a

partition of a graph that no longer exists.

So there is no CommunityId, no community event, no community store, and

nothing new for a projection to write. summarize_themes returns its reports

to the caller, who keeps them for exactly as long as they are useful. This is

the same shape as PipelineResult: computed, returned, written nowhere.

The route back, when a caller needs community identity stable across

calls — cross-call diffing, a UI that pins a theme, incremental reporting —

is not "persist what we compute now". It is a decision about what makes two

partitions' communities the same community, which is a harder question than

storage and has to be answered first. Filed as B147.

Clustering is a caller-side algorithm over the existing port

ADR 0016 says the port is

capabilities, and "read the whole tenant's topology" is not one of the five.

It does not have to become one: find_entities already pages over a total

order with a resumable cursor, and get_relationships_for takes a batch of

ids. Page the entities, batch their edges, and the whole topology arrives

through capabilities that already exist, with no port change and no adapter

obliged to grow a method.

A bulk-topology read would be faster and is deliberately not being added.

Nothing has measured the paged read as a cost centre, and widening a port that

two adapters and a published compliance suite implement is not a speculative

change. Filed as B148 with what would justify it.

The clustering itself is a pure function in `domain`

domain/community.py takes node ids and weighted edges and returns a

partition. No store, no I/O, no model — so it is testable against

hand-built graphs where the right answer is known, which is the property

that matters most for an algorithm whose output is otherwise hard to

falsify.

Modularity optimisation in pure Python, not Leiden via leidenalg.

igraph/leidenalg are a C dependency with wheels that a consumer of a

library like this one should not be made to acquire for a capability they may

not use, and per CLAUDE.md a new third-party client owes a confinement row in

the same commit. The greedy modularity pass here is the first half of Louvain

and gives the same kind of answer; where they differ is on quality at scale,

which is a measurement nobody here has taken. Swapping the implementation

behind this function later changes no signature — filed as B149.

Determinism is part of the contract. Modularity optimisation is

order-sensitive, and a clustering that varies run to run makes every

downstream report unreproducible and every test a coin flip. Nodes are visited

in ascending id order and every tie is broken by the same order, so two runs

over the same graph return the identical partition — asserted, not assumed.

The composition is a new module in `composition`

themes.py joins graph, llm and chunks. Those are three siblings in the

band, forbidden from importing each other, so no lower layer can hold all

three — which is the test CLAUDE.md sets for admission to this layer, and

retrieval.py passes it the same way for a different triple.

It holds the narrowest ports it uses: EntityReader and

RelationshipStore, not the composed GraphStore. A theme summariser that

could wipe a tenant is a fact worth keeping out of a signature.

A community is per-tenant by construction

Every read the summariser makes is tenant-scoped, so no cross-tenant edge can

enter the partition and the question does not arise at the algorithm's level

at all. domain/community.py never sees a TenantId, which is the point:

the isolation is enforced where every other read enforces it, not re-derived

by a second mechanism that could disagree.

Consequences