ADR 0009: The extraction fold resolves endpoints through the alias table

Status: accepted (slice 7 — the slice that began emitting EntitiesMerged).

Why this is an ADR. ADR 0002 gestures at this decision from the outside: it lists "deleting would break replay" as reason 2 for the absence of GraphStore.delete_entity, and moves on. The decision it gestures at is load-bearing on its own, and it has to answer two questions 0002 does not:

The answers are no and no, and both matter to anyone touching projections/graph.py.

Context: what the fold used to do

GraphProjection._apply_extraction wrote a document's entities and then its relationships, by id, with the endpoints extraction had found. That is the obvious fold for DocumentExtracted, and it is correct as long as the ids extraction emits still name the entities the graph holds.

The failure: a re-extraction after a merge silently reverted the merge

They stop naming the same entities the moment a merge lands:


DocumentExtracted(doc-1, model=A)   entities e0,e1,e2 and edge e1->e2
EntitiesMerged(e1 into e0)          edge redirected to e0->e2
DocumentExtracted(doc-1, model=B)   the same edge, endpoints e1->e2 again

The third event upserts the edge over the id it shares with the redirected one, restoring the pre-merge endpoints. The merge is gone from the read model and nothing raises: every write succeeded, and e1 still exists, so upsert_relationship has no missing endpoint to complain about.

It was not a redelivery hazard, and no race was required

This is the part that took a slice to see. The log above is in strict order, each event delivered exactly once, one projection, one process. Re-extraction is not a pathology either — Document.record_extraction is keyed on the model version precisely so a newer model can re-run a document already extracted.

Redelivery was the milder half of the same backlog item, and there the fold was already safe: every write is an upsert or an idempotent delete, so a checkpointed feed redelivering a contiguous suffix still ends at the log's state.

The assumption the fold was actually making, and why no bus can supply it

Stated precisely, the old fold assumed "no DocumentExtracted ever follows a merge that touched its entities." That is a claim about what the write side emits, not about delivery. No ordering guarantee, no exactly-once bus, and no amount of checkpoint discipline can supply it, which is why the fix could not live in the delivery layer.

Decision: the store records merges, and the extraction fold reads that record

Root cause was `GraphStore`'s shape, not the handler's logic

The handler could not distinguish "this edge belongs to an entity that has since been absorbed" from "this edge is new", because the store had nowhere to record that a merge had happened. Any fix written inside the handler would have had to reconstruct that fact, and the only place to reconstruct it from is the event log — which is to say, from a projection the projection does not have.

`upsert_alias` / `remove_alias` / `resolve_entity_ids` are that somewhere

GraphStore gained three alias operations (ports/graph_store.py). upsert_alias and remove_alias are how the merge and undo folds maintain the record; resolve_entity_ids is the read a fold makes before writing an edge. An id that is not an alias maps to itself, so every requested id appears in the result and the caller looks up unconditionally.

GraphProjection._resolved is the reader. It is not consolidation logic leaking into the store: the store is being given somewhere to put a fact that already happened.

One batch resolution per document, not one call per edge

_resolved collects both endpoints of every relationship in the event, makes a single resolve_entity_ids call, and rewrites the edges from the returned map. The port's batch shape exists for this caller — per edge it would be two round trips each, against a store that may be Neo4j across a network.

Consequence: resolution is transitive

Chains are legal, so resolution must follow them

ConsolidationLog refuses to merge into an alias — that is what stops cycles — but it does not refuse to merge a canonical entity away. So B -> A followed by A -> C is a legal pair of merges, and an edge on B belongs on C. resolve_entity_ids is therefore transitive.

Why that terminates, and why `find_aliases` stays direct-only

A cycle would need some merge to name an alias as its canonical, which the write model refuses. Adapters are required to bound the walk anyway, and the port says so rather than leaving it to taste: this is a loop whose exit depends on adapter-supplied data, and an unbounded one turns a corrupt row into a hang — which in CI reads as infrastructure trouble and gets retried instead of investigated. Both adapters detect the cycle and raise AliasCycleError naming the id and tenant, but they get there by different means, and the difference is worth knowing before you write a third adapter:

The shared rule for any future adapter is the one the port states: never let a malformed row decide how long the walk runs, and never let "cycle" and "not an alias" produce the same value.

find_aliases deliberately does not follow chains: find_aliases(A) after B -> A and C -> B returns B alone, not [B, C]. The port says so, both adapters implement it that way — the Cypher is one hop, not *1.. — and tests/compliance/graph_store.py::test_find_aliases_is_direct_and_ordered pins it for every adapter.

The asymmetry is not an inconsistency, because the two methods answer different questions. resolve_entity_ids asks "which entity now stands for this id", and only the end of the chain does. find_aliases asks "which entities did this merge absorb" — the question an undo asks, and the one tests/unit/consolidation/oracle.py reconstructs a merge from. A transitive find_aliases would make that unanswerable: undoing C -> B would find A's absorbed set contaminated with an entity A's merge never touched, and the undo would restore endpoints belonging to a merge still in force.

Consequence: an edge that collapses to a self-loop is deleted, not written

There is no value `_resolved` could return meaning "and this one must go"

An edge can have both endpoints absorbed by the same merge — first -> second where both were merged into canonical. Resolution maps both to the same id, and the edge collapses.

_resolved deletes it through delete_relationship and continues, rather than returning a marker the caller must interpret. That is not a style choice: Relationship._reject_self_loops (domain/relationship.py) refuses at construction to build one whose endpoints are equal, so there is no Relationship value carrying the outcome. The return type is list[Relationship], and the collapsed edge has no representative in it.

The three alternatives were all worse. Returning the edge unresolved would write pre-merge endpoints back, which is the defect this ADR exists to fix. Widening the return type to list[Relationship | Something] pushes the same decision onto every future caller of a private helper with one caller. Constructing the self-loop and letting pydantic raise sends a perfectly ordinary DocumentExtracted to the DLQ — a re-extraction of an already-consolidated document would fail, which is exactly the workflow Document.record_extraction is keyed by model version to support.

The originating merge already recorded `after=None` for that edge

Deleting is not a new decision, it is agreement with one already taken on the write side. plan_redirections (consolidation/planning.py) drops an edge whose endpoints are both absorbed, because it "would be a self-loop on A", and records that as a RelationshipRedirection with after=None rather than by omitting it — an omitted edge is indistinguishable from one the merge never saw, and undo would have nothing to recreate it from. _apply_merge deletes it from the store when it applies the event.

So by the time the re-extraction arrives, the edge is already gone from the graph and already recoverable from the log. The fold deleting it again is idempotent agreement, not a second policy. (The same after=None representation covers a second case, parallel-edge deduplication, which resolution never reaches: those edges do not collapse, they duplicate.)

tests/unit/projections/test_aliases_survive_re_extraction.py::TestAnEdgeThatCollapsesOntoOneEntity pins it, in two tests that are both load-bearing. The first is a control which projects the log without the re-extraction: without it, the second could pass because the edge was never written rather than because it was dropped and stayed gone. The second asserts report.failed == 0 alongside the empty edge set, because "deleted" and "poisoned" are otherwise indistinguishable — a fold that tried to construct the self-loop would fail the event, and an absent edge looks the same either way.

Replay is pinned separately, by the dropping-merge and undo-of-dropping-merge scenarios in the PINNED set of tests/unit/projections/test_replay_equivalence.py. It needs its own coverage because this is the one path through _resolved that does not end in an upsert, and a delete is the write that a replay can most easily disagree about.

Consequence: alias ids are `uuid5`-derived

Why not `uuid4` — replay must produce the same alias rows for the same log

_alias_id (projections/graph.py) is a pure function of the tenant and the absorbed entity id:


def _alias_id(tenant_id: TenantId, alias_entity_id: EntityId) -> UUID:
    return uuid5(NAMESPACE_OID, f"redstring:alias:{tenant_id}:{alias_entity_id}")

The obvious alternative is uuid4() at the point _apply_merge builds the Alias, and nothing in the fold would notice: the id is written, never read back, and no query joins on it. What notices is a second run of the fold. Projecting the same log twice — a rebuild, a resume from a checkpoint, a wipe and replay — would mint a new id each time, so the graph after a replay would differ from the graph before it in a value that is supposed to be a function of the log. That is precisely the property rebuilding a projection tells operators they have, and it is what the replay-equivalence suite exists to forbid.

The reason a uuid4 is tempting here and correct in most places is worth naming: a uuid4 is fine for a value the write side mints once and the log then carries — an event id, an entity id — because the log preserves it. It is wrong for a value the read side mints, because the read side is rebuilt on demand and its output must be a function of its input. Alias is a read-model row, so its id is derived rather than drawn.

Why the merge event id is deliberately excluded from the hash — the row is keyed `(tenant_id, alias_entity_id)` in every adapter

_apply_merge has the EntitiesMerged event in hand and could hash its id in. It does not, and the reason is that the pair (tenant_id, alias_entity_id) already is the row's identity in both adapters:

That "at most one canonical parent" is the same fact ConsolidationLog enforces on the write side by refusing to merge an entity that has already been merged away. Since the pair is the identity, a second merge naming the same absorbed entity must overwrite the same logical row — and hashing the event id in would give that row a different id depending on which merge last wrote it, while both adapters continued to treat it as one row. The result is not a duplicate row but something worse: one row whose surrogate id disagrees with itself across replays, and across adapters if their upsert paths ever differ in whether they rewrite the property.

Derive the id from the key, in other words, or do not have one.

What `test_two_halves_project_to_the_same_state_as_one_whole` would catch

The failure a merge-id-derived alias id produces is not the one you would guess. It survives a single-log replay: the same log carries the same event ids, so hashing them in still yields the same alias ids on every pass, and the replay-equivalence scenarios stay green. What breaks is comparing two independently built logs of the same scenario — the ids differ because the event ids differ.

tests/unit/projections/test_checkpoints.py::test_two_halves_project_to_the_same_state_as_one_whole does exactly that comparison, deliberately: it builds the merge scenario twice so the two rigs have independent checkpoint state, projects one whole and the other in two passes, and asserts the dumps match. The scenario fixes every entity id, so the only value that legitimately differs between the two logs is wall-clock time — which is why the helper it compares through drops the alias merged_at and nothing else. An alias id that varied with the merge event id would be a second such value, and the test would fail without that being what anyone had changed.

Two things follow for anyone editing that test. Widening the drop-list to make a failure go away would disable this check, so a new dropped field needs the same kind of justification the merged_at comment carries. And the single-log replay-equivalence scenarios cannot stand in for this test: they hold the event ids fixed, which is the assumption under examination.

What this decision does *not* change

Three things about the fold look as though this decision should have touched them. None of them changed, and each is worth stating, because the natural assumption in every case is the wrong one.

`MergeUndone` does not resolve, and the order of its two steps is not load-bearing (checked, not assumed)

_apply_undo removes the aliases and then upserts the restored relationships. Nothing resolves on that path, and it must not: restored_relationships carry the pre-merge endpoints deliberately, since restoring them is the whole point of an undo. Resolving them would map each one straight back onto the canonical the undo is retracting, and the undo would be a no-op. Only _apply_extraction resolves, because only it handles data that predates a merge without knowing about it — extraction emits the ids it found, and it has no way to know a merge has happened since.

The order of the two steps therefore does not matter, which is worth saying because it looks as though it should. An alias still in place while the pre-merge endpoints are written seems to contradict them; it does not, because nothing on this path consults the alias table.

That claim was checked rather than assumed: swapping the two statements by hand left the whole of tests/unit/consolidation passing. The comment in _apply_undo records the same experiment, with the count it was run at. An earlier version of that comment asserted the order was load-bearing, which is how a later reader comes to believe the fold resolves here too — a comment asserting a constraint that does not exist is worse than no comment, because nobody re-derives it.

Idempotency was already a port property; no second dedupe layer in the projection

Every write in the fold is an upsert or an idempotent delete. That is a property of GraphStore itself, stated at the top of ports/graph_store.py ("Every write is idempotent. Projection handlers replay."), and it was made so precisely so that projections would not need a dedupe layer of their own.

GraphProjection has none — no seen-event set, no sequence-number check, no write-time comparison — and this ADR does not add one. Resolution changes which endpoints an edge is written with; it does not change how many times writing it is safe.

The property is spread across the port's methods rather than asserted once, and each spelling matters to a different write in the fold. upsert_entity and upsert_alias are "idempotent, last-write-wins", so a second application leaves exactly one row holding the later value. delete_relationship and remove_alias return False for something already absent rather than raising, which is what makes the self-loop delete on the resolution path replayable at all. And upsert_relationships was explicitly not atomic when this was written — a MissingEntityError part-way through left earlier elements written — which the port could afford only because each element is individually idempotent, so a retry converges on the same final state instead of needing a rollback the GraphStore interface does not offer.

That promise has since been strengthened to all-or-nothing (0019), because the two adapters never agreed on it. The argument above is unaffected and slightly strengthened: it rests on per-element idempotency making a retry converge without a rollback, which is still true, and there is now less to converge from. The fold's own partial application — entities written, then the relationship call refused — is a property of making two calls and is unchanged.

Any "have I seen this event already" bookkeeping added to the projection would duplicate a guarantee the port already owes, in the one place it cannot be tested against every adapter: the compliance suite exercises stores, so a projection-level dedupe would be checked against InMemoryGraphStore and nothing else, while the guarantee it duplicates is checked against all of them.

What idempotency does not buy is worth stating in the same breath, because conflating the two is how the original defect survived review. Idempotency makes redelivery safe; it does not make reordering safe. A checkpointed feed redelivers a contiguous suffix in order, so the last occurrence of each event is still in log order and the final state is the log's — see rebuilding a projection. A bus that could deliver e1, e2, e1 would break this fold no matter how idempotent each write is, and so would the extraction-after-merge sequence at the top of this ADR, which is in perfect order and needed the alias table rather than a stronger delivery guarantee.

A missing endpoint remains a poison event routed to the DLQ

upsert_relationship still raises MissingEntityError for an edge pointing at an entity the tenant does not hold — dangling edges are not permitted, and that is unchanged. It happens when a document references an entity from a document not yet projected. The event goes to the DLQ and the projection carries on: not retried into a wedge, and not silently dropped.

Resolution does not paper over it, and the distinction is exactly the one to keep: resolution answers "has this id been merged away", not "does this id exist". resolve_entity_ids says so in its own docstring, and an id the tenant has never seen maps to itself for the same reason a non-alias does — every requested id appears in the result so the caller can look up unconditionally. The edge is then written with the ids extraction found, and fails the upsert exactly as it did before this decision.

The alternative — having _resolved drop edges whose endpoints do not resolve to anything the store holds — is the one to refuse. It would need resolution to distinguish "unknown" from "not an alias", which is a second question the port deliberately does not answer, and it would trade a loud, diagnosable poison event for a graph silently missing relationships. Missing edges are the harder failure to notice and the harder one to attribute later.

Nothing in this ADR touches how that poison event is handled, and the handling is not local to GraphProjection. StoreProjection passes dlq_repo and retry_policy straight through to eventsource's DeclarativeProjection, which retries and writes the DLQ record; replay_all then counts the event in ReplayReport.failed and keeps reading rather than re-raising, because re-raising is what would wedge a rebuild on one bad event. So the operator signal for this case is a non-zero failed count plus a DLQ entry — see rebuilding a projection.

Relationship to ADR 0002

0002 lists this as reason 2 for the absence of `delete_entity`; the dependency runs the other way too

ADR 0002 reaches this decision from the port's side. Its reason 2 for having no delete_entity is the failure at the top of this document: a DocumentExtracted folded after an EntitiesMerged writes the pre-merge endpoints back, and the fix requires the alias to still be there. From 0002's angle, this ADR is the justification for a method it declines to add.

The dependency runs the other way too, and that is the half worth stating here: resolution requires the absorbed entity's alias row to survive, so the fold depends on the port's restraint exactly as much as the restraint depends on the fold. A delete_entity would not break the fold by being called on an arbitrary entity; it would break it by making "remove the entity a merge absorbed" an expressible operation, and the alias row is what that operation would take with it. Neither decision is safe alone. If the port grew a delete and the fold kept resolving, the fold would resolve against rows something else was entitled to remove; if the port stayed as it is and the fold stopped resolving, the alias rows would still be there and nothing would read them.

That mutual hold is why 0002's Status records the alias surface as an extension of the same decision rather than a later amendment. The no-delete argument was always conditional on something keeping the merge fact durable, and it was aspirational until something depended on it: a projection that quietly dropped the absorbed row would pass every test about entities and relationships, because the damage only shows up in the next extraction fold. Resolution is what makes the dependency structural instead — _resolved cannot write an edge without calling resolve_entity_ids, resolve_entity_ids cannot answer without the alias rows, and an adapter that discards them fails the alias tier of tests/compliance/graph_store.py rather than a code review.

So the two documents are one decision written from two sides, and the division of labour between them is deliberate. 0002 owns the port: which methods exist, why delete_entity is not among them, what an adapter owes — see implementing a store adapter for the implementer's side of resolve_entity_ids. This ADR owns the fold: which handler resolves and which deliberately does not, that resolution is batched per document, and that a collapsed edge is deleted rather than upserted. A change to either side has to be checked against the other; in particular, anything that would let an absorbed entity's row disappear is a change to 0002, not a local edit to projections/graph.py.

Consequences and open items