ADR 0038: The chunk's vector lives on the chunk

Status

Accepted. Amended by 0044 a chunk id is derived, not supplied, which closes the executable half this ADR left open below.

Amends 0023 the chunk corpus in its Consequences: the semantic search it deferred is now built, and the port docstring that said there was no such method is corrected. No decision 0023 made is reversed. Amends 0026 ChunkStore and Cache are composed from capabilities, like GraphStore: a fifth capability, SemanticCandidateSource, joins the four it named.

0002 two store ports, 0006 the public surface is gated, 0012 no ANN index in a multi-tenant vector store, 0017 the embedding provider port, 0022 the lexical channel is not BM25 and 0024 BM25 over the chunk corpus stand.

Context

ChunkStore has a lexical recall channel, LexicalCandidateSource, and no semantic one. A caller who has embedded a corpus cannot ask "which chunks are nearest this vector" without going around the port. VectorStore already answers a structurally identical question for entities, so the first question is whether a chunk's embedding belongs there instead of on the chunk store.

It does not. VectorStore is keyed on an arbitrary vector id and carries metadata a caller filters on; it has no notion of a chunk's other fields โ€” source_id, chunk_index, entity_ids, the text itself โ€” and giving it one would mean either duplicating StoredChunk's identity into a second store (a new instance of the divergence .claude/rules/recurring-defects.md ยง1 exists to catch โ€” two records of one fact, kept in sync by nothing) or making ChunkStore responsible for handing the vector store the same id on every write, which is a distributed-transaction problem this library has consistently avoided by keeping a fact in one store. The vector lives where the chunk lives: as a nullable column on StoredChunk, read back by the same get/get_by_source/get_by_entity methods that already return the chunk, and searched by a new capability on the same port.

0024 moved BM25 scoring into the domain because two adapters implementing a ranking formula could compute different numbers from the same recall set and disagree silently while agreeing on which chunks matched โ€” the ranking diverges, not the membership. Cosine similarity does not have that failure mode: there is one definition, and VectorStore.search already relies on each adapter computing it correctly. Requiring the domain to score would mean shipping every candidate's full vector across the port so it could be dotted in Python, which is a real cost for no correctness gain โ€” the thing 0024 protects against, formula divergence, cannot happen here. What the two adapters can still disagree about is order and cutoff: which chunks a tie in score resolves to, and which candidates a limit truncates. So the port states a total order โ€” score descending, ties by id ascending โ€” and the compliance suite (a later task) asserts both adapters agree on it, exactly as lexical_candidates is pinned today.

A chunk store serves callers with different embedding models, and a search issued with a vector of the wrong width is not an error the store can detect by inspecting the vector alone โ€” it needs to know its own configured width. VectorStore already carries this as a constructor argument; ChunkStore gains the same requirement through SemanticCandidateSource.dimension, declared once at construction rather than inferred from whichever chunk happens to be embedded first. An optional width โ€” leaving semantic_candidates raise until the first embedded write told the store what to expect โ€” would let a store satisfy the capability's structure without satisfying its behaviour, which is worse than requiring the argument.

Something has to call the embedding provider and the chunk writer in the same breath, and neither extraction nor chunks may. Extraction writes to no store, by the library's standing rule; chunks is a sibling of llm in the layered contract and may not import an embedding provider. composition is the layer built to hold exactly this kind of pair โ€” it already drives ChunkWriter from index_documents โ€” so embedding a corpus is a composition-layer decision, made by passing an optional EmbeddingProvider through, not a capability either sibling package acquires.

Decision

Add SemanticCandidateSource as a fifth capability composed into ChunkStore, alongside ChunkWriter, ChunkReader, LexicalCandidateSource and ChunkPurge:


class SemanticCandidateSource(AsyncClosable, Protocol):
    @property
    def dimension(self) -> int: ...

    async def semantic_candidates(
        self,
        vector: Sequence[float],
        tenant_id: TenantId,
        limit: int,
        *,
        min_score: float | None = None,
    ) -> list[SemanticCandidate]: ...

The adapter computes the similarity score. This is a stated exception to 0024's "the domain scores" rule, not a reversal of it: 0024's argument was about a ranking formula that two adapters could implement differently, which is a property of BM25 and not of cosine similarity. The port pins the one thing that can still diverge โ€” total order, score descending then id ascending โ€” so the compliance suite still asserts the adapters agree on results even though it does not recompute their arithmetic.

The store declares its width at construction. dimension is a required property, not an optional one a store discovers from its first write. Both adapters (a later task) take it as a constructor argument, the same shape VectorStore already uses.

Composition embeds; extraction and chunks still do not. The pipeline module that drives ChunkWriter gains an optional EmbeddingProvider argument (a later task) rather than either sibling package acquiring a new import. A corpus indexed without one is stored exactly as today, with embedding=None; chunks with embedding is None are not semantic candidates โ€” skipped, not scored zero, for the same reason an absent component score means "unranked" rather than "ranked at zero" on ScoredChunk.

No ANN index. 0012 already argues that an approximate index over a multi-tenant table either crosses tenant boundaries or is built per tenant, and that argument does not depend on which table carries the vector. The chunk-corpus scan is exact and linear in the tenant's corpus for the same reason the entity vector store's is, and 0012 governs this instance without being relitigated.

Consequences

The port composes five capabilities instead of four; the public export surface grows by SemanticCandidateSource. ChunkStore's docstring no longer claims there is no semantic method here, correcting the sentence 0023 wrote before this work existed.

The two adapters implementing SemanticCandidateSource, and the compliance cases pinning its ordering, min_score, limit = 0 and negative limit, unembedded-chunk exclusion, and the two isolation cases the coverage gate names by convention, are later work. Until they land, InMemoryChunkStore and PostgresChunkStore satisfy ChunkWriter, ChunkReader, LexicalCandidateSource and ChunkPurge but not the composed ChunkStore, which is a correct and visible consequence of widening the port ahead of its adapters rather than a defect to paper over.

Content addressing on ChunkWriter.upsert_many is stated as prose here because the new column inherits it: embedding is derived from a chunk's text the same way the lexical term index and doc_length are, so a write that reuses an id for different text is outside the contract for the same reason those two are. The executable half stayed open at the time this ADR was written; see 0044, which closes it by making the id a computed field instead of adding a compliance case for a state a caller can no longer construct.