Correctness Fixes Slice β€” Design

Status: Design complete, ready for planning Source of scope: BACKLOG.md wrongness-class entries accumulated during the clean-architecture rings campaign (PR #88). Exactly ten items; every other backlog entry β€” including all ring-residue refactors and the "Re-benchmark pg catch-up horizon predicate at scale" entry β€” is out of scope. Β§7 states explicitly how this slice changes that re-bench entry's premise.

Each section below is self-contained: the defect with citations, the ruling with rationale and rejected alternatives, the fix precisely enough specified for red/green TDD tasks, the failing test that must exist first, acceptance criteria, and docs/CHANGELOG obligations. Β§11 is the ADR impact ledger; Β§12 the suggested task grouping.

House rule applied throughout: where an item was genuinely two-sided, the fail-closed / consistent-with-documented-invariants side wins.


1. SQLite outbox adapter incompatible with its shipped migration schema (P1)

Defect

migrations/templates/sqlite/outbox.sql:27 declares id INTEGER PRIMARY KEY AUTOINCREMENT. INTEGER PRIMARY KEY is SQLite's strictly-typed rowid alias β€” the only column type SQLite actually enforces β€” and SQLiteOutboxRepository.add_event inserts str(uuid4()) into it (adapters/sqlite/outbox.py:107-131), so every insert against the shipped schema raises sqlite3.IntegrityError: datatype mismatch. The same broken table body is embedded in the combined migrations/schemas/sqlite_all.sql:61, which SQLiteEventStore._conn() applies on every first connection (adapters/sqlite/store.py:141) β€” so every store-provisioned SQLite database carries the unusable table.

The defect was masked because tests/conftest.py::sqlite_outbox_repo (tests/conftest.py:636-677) hand-rolls an id TEXT PRIMARY KEY table instead of using get_schema(). The real-schema conformance binding exists but is xfail(strict=False) (tests/unit/adapters/test_sqlite_conformance.py:178-201).

Ruling

Replace the schema in place: id TEXT PRIMARY KEY in both migrations/templates/sqlite/outbox.sql and the event_outbox block of migrations/schemas/sqlite_all.sql. This is the sanctioned exception to the append-only-by-file rule, and the justification must be understood precisely:

TEXT (36-char hyphenated UUID) matches the PostgreSQL outbox (migrations/templates/outbox.sql:28: id UUID PRIMARY KEY DEFAULT gen_random_uuid()), the port contract (OutboxEntry.id: UUID), and the file's own stated convention ("UUID stored as TEXT"). No DEFAULT on the SQLite side β€” the adapter always supplies the id, and SQLite has no native UUID generator to mirror gen_random_uuid().

Rejected alternatives

Fix specification

  1. Edit migrations/templates/sqlite/outbox.sql: id TEXT PRIMARY KEY, comment updated to "UUID as TEXT (36 characters, hyphenated)"; delete the "Auto-incrementing ID for ordering" comment (ordering comes from created_at, which get_pending_events already orders by).
  2. Edit migrations/schemas/sqlite_all.sql event_outbox block identically. No other table in either file changes.
  3. Retarget tests/conftest.py::sqlite_outbox_repo onto get_schema("outbox", backend="sqlite") via executescript, deleting the hand-rolled DDL.
  4. Remove the @pytest.mark.xfail decorator from TestSQLiteOutboxRepository in tests/unit/adapters/test_sqlite_conformance.py.
  5. No adapter change.

Failing-test-first requirement

The failing test already exists: the xfail'd conformance binding. Red step = delete the xfail decorator and run uv run pytest tests/unit/adapters/test_sqlite_conformance.py -k Outbox β€” every insert-path test must fail with IntegrityError: datatype mismatch. Green step = the two schema edits. The fixture retarget (step 3) then puts the migration under the rest of the outbox unit suite as well.

Acceptance criteria

Docs / CHANGELOG obligations


2. Cutover can switch routing with up to `cutover_max_lag_events` missing (P2)

Defect

MigrationConfig.cutover_max_lag_events defaults to 100 (migration/models.py:391, and again in the from_dict fallback at :451). cutover.py:321 allows cutover when lag.is_within_threshold(config.cutover_max_lag_events) β€” i.e. with up to 100 source events provably absent from the target. Writes are paused for the entire cutover (cutover.py:306), and nothing in the cutover sequence copies the residue β€” Step 5 (:350-354) only sleeps ≀10 ms to drain in-flight mirror appends. So any lag remaining at the routing switch (:389-395) is events the target never receives while it becomes authoritative: a real loss window, caught only by the non-fatal post-cutover consistency check.

This directly contradicts the invariant dual_write.py:356-360 documents and the campaign final review verified: "the accepted failure mode here is stuck-until-recopied, never a cutover over missing data." The lag being counted is not optimistic slack β€” safe_lag_anchor guarantees every counted event is genuinely not in the target.

Ruling

Default cutover_max_lag_events to 0 (strict). The knob survives: a nonzero value becomes an explicit, documented operator acceptance of bounded event loss at the switch, never a silent default.

Rationale: zero is achievable on the healthy path β€” writes pause before the lag check, so a working mirror has already drained to exactly zero by Step 2; the interceptor's whole anchor machinery exists so that lag counts only provably-missing events. And zero is recoverable on the unhealthy path once Β§5's in-phase resync exists: a clamped anchor is cleared by run_resync_pass, not by tolerating the hole. Items 2 and 5 are designed together for exactly this reason β€” strict-0 without resync means "stuck-until-abort"; with resync it means "run one pass, then cut over."

Rejected alternatives

Fix specification

  1. migration/models.py:391 β†’ cutover_max_lag_events: int = 0; :451 from_dict fallback β†’ 0; attribute docstring (:373) rewritten: "Max lag allowed before cutover (default 0 β€” strict). Any nonzero value permits cutover while that many source events are absent from the target."
  2. Update in-tree examples that model the old default: sync_lag_tracker.py:22 and :140 docstring examples, and any cutover_max_lag_events=100 occurrences found by grep across src/ and docs/.
  3. No change to cutover.py, is_within_threshold, or calculate_lag β€” with threshold 0 the tracker reads limit=1 and the existing bounded-count logic behaves correctly.

Failing-test-first requirement

Two red tests:

Existing tests asserting the old default (tests/unit/migration/test_models.py default-values test, any cutover tests relying on slack) flip in the green step.

Acceptance criteria

Docs / CHANGELOG obligations


3. Live-phase lag has no signal (P2)

Defect

Subscription.lag is max(0, _events_seen - _events_delivered) (subscriptions/subscription.py:493), with a documented invariant that callers keep the two counters symmetric across any boundary (:484-488). Only the catch-up runner honors it: catchup.py:309 records seen per read batch and :374-376 reconciles abandoned tails. The live runner never records seen β€” _handle_live_event (runners/live.py:220-254) counts receipts only into its private stats, while _process_live_event increments _events_delivered via record_event_processed (:311, :339, :349).

Consequences: lag is structurally 0 during LIVE (a stalled subscriber with events arriving is indistinguishable from a healthy idle one), and the accumulated delivered-surplus makes seen - delivered deeply negative, so a later transition back to catch-up under-reports real backlog until the surplus is burned off.

Ruling

Make the accounting phase-symmetric; the bus delivery receipt is the live seen-point. _handle_live_event records record_events_seen(1) for every received event before branching (transition buffer, pause buffer, or direct processing). Every terminal path in _process_live_event that does not call record_event_processed compensates with record_events_unseen(1):

Path Today After
delivered (with or without position, :339/:349) delivered+1 seen+1 at receipt, delivered+1 β€” balanced
filtered, position known (:311) delivered+1 balanced as above
duplicate skip (:285-296) nothing seen+1 then unseen+1 β€” net zero
filtered, no position (:299-316) nothing net zero via unseen
failure swallowed under continue_on_error (:355-379) failed+1 unseen+1 β€” the event is terminally disposed (possibly DLQ'd); failure counters carry that signal, lag must not report it as outstanding work forever
failure re-raised (continue_on_error=False) raises seen stays +1 β€” honest: the subscription enters ERROR with one genuinely unprocessed event

Buffered events (catch-up→live transition buffer, pause buffer) are seen at receipt and drain through _process_live_event on process_buffer / process_pause_buffer — so live lag now equals queue depth plus in-flight count. That is precisely the operator signal the backlog item asks for: a paused or stalled subscription shows growing lag; a healthy one shows ~0.

Rejected alternatives

Fix specification

  1. runners/live.py::_handle_live_event: first statement await self.subscription.record_events_seen(1).
  2. _process_live_event: add await self.subscription.record_events_unseen(1) on the duplicate-skip return, the filtered-without-position return, and in the swallowed-failure branch (after record_event_failed, only when not re-raising).
  3. Subscription.lag and record_events_seen docstrings updated to describe the phase-symmetric semantics (seen = read from feed or received from bus; live lag β‰ˆ received-not-yet-delivered including buffers).
  4. No catch-up runner change; no public API change.

Failing-test-first requirement

Acceptance criteria

Docs / CHANGELOG obligations


4. Wire `position_mapper` into BulkCopier's coordinator call site (P2)

Defect

MigrationCoordinator accepts and stores a position_mapper (migration/coordinator.py:203, :261) and MigrationConfig documents position_mapping_enabled: bool = True ("Whether to record position mappings", models.py:375, :393) β€” but the flag is read nowhere in src/, and the coordinator's BulkCopier construction (coordinator.py:815-820) omits position_mapper=. So a default-path migration records no mappings, and subscription checkpoint translation β€” migrate_subscriptions: bool = True is also the default β€” silently skips (coordinator.py:1141-1153) or raises when called directly (:1733-1751). Two documented-True defaults are dead on arrival; the only code exercising the mapper path is tests that construct BulkCopier with a mapper by hand.

Ruling

Wire it, gated by the config flag: the coordinator builds its copier with position_mapper=self._position_mapper if migration.config.position_mapping_enabled else None. This makes both documented defaults true when the coordinator was given a mapper, and makes position_mapping_enabled=False a real switch.

Extract the construction into a private helper β€” _build_copier(migration) β€” used by _run_bulk_copy and by Β§5's run_resync_pass, so the two copy entry points can never diverge on wiring. This is the deliberate joint seam with item 5.

Performance consequence, stated honestly: with a mapper attached the copier appends one event at a time (bulk_copier.py:540, :586) so each target position can be recorded β€” the unmeasured cost named as risk 2 in the legacy-store-retirement spec Β§11 now runs on the default path when a mapper was provided. A coordinator constructed without a mapper (the parameter defaults to None) keeps batched appends, as does position_mapping_enabled=False. The guide documents the trade.

Rejected alternatives

Fix specification

  1. Add MigrationCoordinator._build_copier(self, migration: Migration) -> BulkCopier: constructs with source, target-for-that-migration, self._migration_repo, enable_tracing=self._enable_tracing, and the gated position_mapper above. (_run_bulk_copy already has target_store in scope; pass it as a parameter or resolve inside β€” implementer's choice, but one helper, two callers.)
  2. _run_bulk_copy (coordinator.py:815-820) uses _build_copier.
  3. No BulkCopier change; no config change.

Failing-test-first requirement

Acceptance criteria

Docs / CHANGELOG obligations


5. In-phase resync for dual-write mirror failures after bulk copy (P2)

Defect

safe_lag_anchor's failure clamp (dual_write.py:311-324, :393-398) is released only by mark_copy_pass_complete, and the sole caller is the bulk-copy loop (coordinator.py:835). Once the migration is in DUAL_WRITE, no code path can run an absorbing copy pass: a transient mirror failure late in a long migration clamps the anchor permanently, cutover refuses forever (correctly β€” the data really is missing), and today's only remedy is abort-and-restart, as docs/guides/live-migration.md:143-145 documents. The same dead end applies when _MAX_CATCHUP_ROUNDS is exhausted (coordinator.py:838-848).

Ruling

Add an operator-triggered resync entry point:


async def run_resync_pass(self, migration_id: UUID) -> int:
    """Run one bounded catch-up copy pass while in DUAL_WRITE.

    Returns the number of unabsorbed mirror failures remaining
    (0 means the lag anchor is unclamped and cutover can proceed
    once lag drains).
    """

Semantics, each load-bearing:

None of the dual-write invariants weaken: the clamps, the never-advance-over-a-hole rule, and the saturation fail-closed path are untouched β€” this adds the sanctioned release path to a phase that lacked it. Re-copying is already safe by design (the copier treats already-present events as copied, _run_bulk_copy docstring coordinator.py:786-789).

Interaction with Β§2: strict-0 cutover makes this API the standard remedy β€” the pairing turns "stuck-until-recopied" from a dead end into a runbook step.

Rejected alternatives

Fix specification

As specified in the ruling. Public API: one new coordinator method β€” additive only, exported implicitly via the class; no __init__.py change needed (MigrationCoordinator is already exported).

Failing-test-first requirement

Red test (unit, in-memory stores): drive a migration into DUAL_WRITE with one mirror failure (target store double that fails one append during dual-write, then recovers); assert is_cutover_ready is False with the anchor clamped; call run_resync_pass β†’ returns 0; assert the anchor advanced (_lag_anchor past the failure position) and is_cutover_ready becomes True once lag is 0. Fails today: the method does not exist. Guard tests: wrong phase raises MigrationStateError; concurrent active copier raises MigrationError; unknown id raises MigrationNotFoundError; restart-shaped case (no interceptor registered) completes and returns 0.

Acceptance criteria

Docs / CHANGELOG obligations


6. Reconcile `events.tenant_id` schema drift (P2)

Defect

The canonical schema declares tenant_id UUID (migrations/schemas/events.sql:35; identically migrations/templates/events.sql:35), matching the type system end to end: DomainEvent.tenant_id: UUID | None (events/base.py:118), TenantId = UUID | None (types.py:14), FeedReadOptions.tenant_id: UUID | None (ports/envelopes.py:96), and the ports PostgreSQL adapter binds uuid.UUID values directly (adapters/postgresql/store.py:312, :501-503).

tests/integration/conftest.py:280 hand-rolls tenant_id VARCHAR(255) into the shared session postgres_engine fixture (:407-433), alongside three more hand-rolled blocks (EVENTS_SCHEMA_STATEMENTS, CHECKPOINTS_SCHEMA_STATEMENTS, DLQ_SCHEMA_STATEMENTS, OUTBOX_SCHEMA_STATEMENTS, :272-376) that drift from the canonical files in columns, constraints, and indexes. The drift is why the ports conformance tests had to retreat to a private ports_conformance database (tests/integration/adapters/conftest.py:1-19, whose docstring still attributes the VARCHAR side to the now-deleted legacy store). Consequence: the main integration suite exercises a schema no production deployment has, and the canonical schema's only full-suite consumer is the private database.

Ruling

The canonical migrations schema is authoritative; the hand-rolled test DDL dies. The postgres_engine fixture provisions from eventsource.migrations.get_schema(...) (events, checkpoints, dlq, outbox β€” or get_all_schemas()), executed through a raw asyncpg connection, not sqlalchemy text() β€” the canonical files are multi-statement scripts with COMMENT ON, which is exactly why the snapshots schema already goes through the raw connection in this fixture (commit ce6d15a). Any integration test that then fails by binding str(tenant_id) into events gets fixed to bind UUID β€” that is the reconciliation the backlog item asks for.

The private ports_conformance database stays (its DROP+recreate isolation is independently valuable), but the stale docstring in tests/integration/adapters/conftest.py is rewritten: the drift it documents as out-of-scope is now fixed, and the remaining justification is isolation from table-recreating tests. The stale comment at tests/integration/adapters/test_postgresql_conformance.py:159 is updated in the same pass.

Rejected alternatives

Fix specification

  1. Delete the four *_SCHEMA_STATEMENTS blocks from tests/integration/conftest.py; provision via get_schema() through the raw asyncpg driver connection in postgres_engine (pattern already in the file for snapshots).
  2. Run the full -m postgres integration suite; fix any test that assumed VARCHAR semantics (string tenant ids, missing uq_events_aggregate_version constraint name, index differences) to match the canonical schema.
  3. Update the two stale comment/docstring sites named above.

Failing-test-first requirement

This is test-infrastructure reconciliation, so the red step is the fixture flip itself: swap the provisioning first, run the -m postgres suite, and treat every failure as the reconciliation worklist. No failure may be resolved by editing schema β€” only by fixing the test (or, if one surfaces, an adapter bug, which then gets its own red test). Record in the PR description which tests failed on the flip; "none" is a legitimate and reportable outcome.

Acceptance criteria

Docs / CHANGELOG obligations


7. Make the postgres safe-horizon predicate wraparound-safe (P2)

Defect

adapters/postgresql/store.py:80:


_HORIZON_PREDICATE = "xmin::text::bigint < pg_snapshot_xmin(pg_current_snapshot())::text::bigint"

used in read_all's feed queries at :493 and :524. xmin is a 32-bit xid whose textual value wraps at 2^32; pg_snapshot_xmin returns an epoch-extended 64-bit xid8. Once a cluster crosses its first xid epoch (~4 billion transactions), the right side exceeds 2^32 permanently while the left side stays below it: the predicate becomes universally true and the no-skip protection β€” deferring rows whose inserting transaction may still be uncommitted, so a resuming reader can't leap over them β€” silently vanishes. High-write-volume deployments are exactly the ones that both cross epochs and need the guard. The code comment (:77-79) already names the fix it defers: "revisit if a xid8 column is ever added."

Ruling

Add the xid8 column and restructure the predicate.

Schema (within append-only rules, via the existing mechanisms):

NULL semantics: a NULL txid row predates the ALTER. ALTER TABLE takes ACCESS EXCLUSIVE, so any transaction that inserted a NULL row finished before every post-migration snapshot β€” NULL rows are always definitely-committed and safe to read.

Predicate: compute the horizon once per read_all batch with a separate scalar query β€” SELECT pg_snapshot_xmin(pg_current_snapshot()) β€” and bind it:


(txid IS NULL OR txid < :txid_horizon)

Native xid8 comparison (64-bit, does not wrap on any human timescale), no casts, and a plain bound-parameter filter in place of the volatile inline expression. The adapter's INSERT does not name the column; the DEFAULT covers it. Version floor: pg_current_xact_id/xid8 require PostgreSQL 13 β€” the same floor pg_current_snapshot() already imposes, so no requirement change.

Failure mode for un-migrated databases: the feed query errors naming the missing column β€” loud by design (see rejected alternatives).

Effect on the deferred re-bench backlog entry: its premise changes and the entry must be updated, not closed. It benchmarked the inline xmin-cast predicate that EXPLAIN showed defeating the global_position index; that predicate is deleted here, and the once-per-batch-horizon restructure is the very mitigation the entry proposed. The re-bench remains warranted β€” at 1M+ rows with concurrent writers, now against (txid IS NULL OR txid < $n) β€” and its methodology section must be rewritten against the new shape.

Rejected alternatives

Fix specification

  1. The two migration artifacts and registry entries above.
  2. adapters/postgresql/store.py: delete _HORIZON_PREDICATE's xmin form; add the per-batch horizon fetch in the read_all path (one scalar query per batch, same connection) and the bound-parameter predicate at both call sites (:493, :524); rewrite the explanatory comment (:72-79) including the NULL-rows rationale.
  3. tests/integration/adapters/ no-skip and conformance tests run unchanged against the now-fragment-carrying canonical schema.

Failing-test-first requirement

Acceptance criteria

Docs / CHANGELOG obligations


8. Reconcile DLQ `delete_resolved_events` cutoff semantics (P3)

Defect

adapters/memory/dlq.py:330-332 truncates now-UTC to midnight before subtracting older_than_days; adapters/sql/dlq.py:386 subtracts from now() directly. At older_than_days=0 a moments-ago resolution is deleted by both SQL dialects but kept by the memory adapter until midnight rolls over. The port docstring (ports/dlq.py:186-197) specifies nothing, which is why two conforming-looking adapters diverged; the divergence is currently institutionalized β€” excluded from the shared conformance suite by its module docstring (testing/conformance_ports/dlq.py:4-6) and pinned by per-backend tests (tests/unit/adapters/test_sqlite_conformance.py:155-176, tests/unit/adapters/test_memory_dlq_conformance.py:17-34).

Ruling

The SQL semantic wins: cutoff is exactly datetime.now(UTC) - timedelta(days=older_than_days); an entry is deleted iff status == "resolved" and resolved_at < cutoff; older_than_days=0 therefore deletes every already-resolved entry. Rationale: it is the plain-language reading of "older than N days"; it is what two of the three backends (the SQL adapter serves both dialects) already do; the memory adapter's truncation is self-described as "simplified" (memory/dlq.py:331); and it matches the rolling-cutoff shape of the outbox's SQL cleanup. The memory adapter conforms to the port, not the port to an implementation shortcut.

Rejected alternatives

Fix specification

  1. adapters/memory/dlq.py::delete_resolved_events: delete the .replace(hour=0, minute=0, second=0, microsecond=0) truncation; cutoff becomes datetime.now(UTC) - timedelta(days=older_than_days).
  2. ports/dlq.py::delete_resolved_events docstring: state the cutoff formula, the strict < comparison against resolved_at, and the day-zero consequence explicitly.
  3. testing/conformance_ports/dlq.py: add the shared day-zero test (add β†’ resolve β†’ delete_resolved_events(older_than_days=0) deletes it; the existing failed-entries-intact test at :230 stays); rewrite the module docstring removing the "intentionally excluded" carve-out.
  4. Delete the two per-backend divergence tests (test_sqlite_delete_resolved_events_removes_past_cutoff_entries, test_memory_delete_resolved_events_cutoff_is_truncated_to_midnight) β€” their content now lives in the shared suite.

Failing-test-first requirement

The new shared conformance day-zero test, run via the existing memory binding (tests/unit/adapters/test_memory_dlq_conformance.py) β€” red today (memory keeps the entry), already green on the SQL bindings. That asymmetry is the proof the test pins the right thing.

Acceptance criteria

Docs / CHANGELOG obligations


9. Catch-up can end early with `completed=False` on an all-filtered batch (P3)

Defect

subscriptions/runners/catchup.py::_process_batch returns events_in_batch β€” delivered events only. Envelopes excluded by the event-type filter advance position and the reconciliation counter (:335-345) but not events_in_batch. The outer loop breaks on a zero return (:239-241), so a batch whose envelopes are all filtered ends catch-up with self._reached_target still False β†’ completed=False (:243), even though the store position advanced and the feed has more. The loop conflates "empty read" (genuinely caught up; sets _reached_target at :306-308) with "all filtered" (nothing matched this batch). The live runner happens to cover the gap end-to-end, but catch-up's own completion signal is unreliable for heavily-filtered subscriptions.

Ruling

Split the return contract and delete the delivered-count break. _process_batch returns a small module-private frozen dataclass:


@dataclass(frozen=True)
class _BatchOutcome:
    envelopes_read: int
    events_delivered: int

The outer loop accumulates total_processed += outcome.events_delivered and has no zero-break at all: termination is exactly _reached_target (already set on empty read :306-308 and on target overshoot :325-327), _stop_requested, and not self._running β€” all already in the while condition (:224). Progress is guaranteed without the break: every counted envelope (delivered or filtered) calls record_event_processed, advancing last_processed_position, and the next read starts strictly after it; the position-None guard (:325) already converts the only no-progress pathology into _reached_target.

CatchUpResult.events_processed keeps its current meaning β€” events delivered to the subscriber β€” which is why filtered envelopes must not be folded into a single return count (see rejected alternatives).

Rejected alternatives

Fix specification

As in the ruling: _BatchOutcome in catchup.py, _process_batch returns it (envelopes_read = len(envelopes), events_delivered = events_in_batch), the if batch_result == 0: break block (:239-241) is deleted, and the docstrings of run_until_position/_process_batch updated. No public API change; CatchUpResult untouched.

Failing-test-first requirement

Red test: subscription filtered to event type B over a feed containing only type-A events spanning at least two read batches, target at the feed head β†’ run_until_position must return completed=True, events_processed == 0, and final_position at the last A envelope. Fails today: returns after the first batch with completed=False. Companion test: alternating A/B batches where an interior batch is all-A still reaches the target with the correct delivered count.

Acceptance criteria

Docs / CHANGELOG obligations


10. SQLite adapter: reads share the writer's connection (P3)

Defect

All read paths β€” _do_read_stream (adapters/sqlite/store.py:306), _do_read_all (:370), _do_read_category (:418), get_stream_version (:341), event_exists (:354), current_position (:402) β€” execute on the same aiosqlite connection as append but outside self._lock, which only append holds (:206). append is multi-statement: per-event INSERTs then commit() (:233-260). aiosqlite serializes individual statements on one background thread, but each INSERT is separately awaited, so the event loop can schedule a reader's execute between two of them β€” and a same-connection statement runs inside the append's open implicit transaction. A read interleaved mid-append observes uncommitted rows of the in-flight batch: read_all can yield a torn batch, and current_position can mint a Position for a row that is then rolled back on IntegrityError (:263-296) β€” a position for an event that never existed.

Ruling

Take reads under the existing write lock β€” not a second connection, and not WONTFIX. The shared single connection is load-bearing, exactly as the task anticipated: the class docstring (:71-74) pins it as required for ":memory:" databases, whose contents live only as long as the creating connection (a second connection sees a different, empty database; shared-cache URIs change semantics and WAL does not apply to in-memory). That rules out the separate-read-connection design β€” but it does not justify a documented no-op, because the cheap fix exists: every read helper already fetches eagerly (fetchall() before any yield), so wrapping execute + fetch in async with self._lock: gives bounded critical sections and never holds the lock across consumer iteration. Concurrency cost is ~nil β€” aiosqlite already serializes per-statement on one thread; the lock only adds ordering at transaction granularity, which is the correctness requirement.

Deadlock safety: asyncio.Lock is non-reentrant; append's internal SELECTs (:208-217, :273-282) use the connection directly and must stay that way β€” the fix must not route append internals through the public read helpers. No current code path acquires the lock twice.

Rejected alternatives

Fix specification

Wrap the conn.execute(...) + fetchall()/fetchone() pair of each of the six read paths listed above in async with self._lock:. Yielding of envelopes stays outside the lock. append unchanged. Update the module/class docstring to state the discipline: all statements on the shared connection run under self._lock; readers therefore never observe an open append transaction.

Failing-test-first requirement

Deterministic interleave test (unit, tests/unit/adapters/): obtain the live connection via await store._conn(), wrap its execute so that after the first INSERT into events it sets an asyncio.Event and waits on a second one; start append(stream, [e1, e2], ...) as a task; when the first INSERT has landed, run read_all (and current_position) from the test task, then release the append. Red today: the read observes 1 of 2 events / a position for an uncommitted row. Green after: the read blocks until the append commits and observes 0-or-2, never 1. (White-box interception of the store's connection is acceptable here; the wrapping must be transparent to the lock itself β€” intercept below the adapter, not around _lock.)

Acceptance criteria

Docs / CHANGELOG obligations


11. ADR Impact

ADR Verdict
0019 (clean architecture store ports) Stands. Positions stay opaque; Β§3 and Β§9 explicitly refuse position arithmetic; Β§7 changes an adapter's internal predicate, not the port.
0024 (projection persistence ports) Stands. Β§8 tightens the DLQ port's docstring within 0024's structure; no interface change.
0025 (legacy store retirement) Stands. Β§7 replaces the safe-horizon mechanism the ports adapter shipped with, but the decision 0025 records (retire legacy stores onto the ports adapters, with no-skip protection) is unchanged; ADR 0027 cross-references it.
0026 (outbox ring migration) Stands. Β§1 corrects the SQLite schema 0026's split exposed; the per-backend adapter shape and payload contract are untouched.
0027 (new) Schema correctness: (a) in-place replacement of the SQLite outbox schema and the exception criteria to append-only-by-file (a shipped schema that provably never worked with its only writer); (b) the events.txid xid8 additive column, the NULL-is-safe argument, the per-batch horizon predicate, and the PostgreSQL 13 floor; (c) canonical-schema authority for events.tenant_id (UUID) and the retirement of hand-rolled test DDL.
0028 (new) Cutover strictness and recovery: cutover_max_lag_events defaults to 0 (cutover never proceeds over provably-missing data by default; nonzero is explicit operator-accepted loss), paired with the run_resync_pass in-phase recovery API and the shared _build_copier seam that keeps position-mapper wiring uniform across copy entry points.

Items 3, 8, 9, 10 are invariant-restoring bugfixes: CHANGELOG entries, no ADR.

12. Suggested task grouping for the implementation plan

Task Items Rationale
T1 Β§1 (sqlite outbox schema) P1, fully standalone; ships the xfail-drop + fixture retarget + two SQL edits + ADR 0027 draft (part a). First.
T2 Β§7 (xid8 horizon) Isolated to the pg adapter + migrations machinery; needs -m postgres; ADR 0027 part (b); rewrites the re-bench backlog entry.
T3 Β§6 (tenant_id reconciliation) Blocked by T2 β€” the fixture must provision the fragment-carrying schema. ADR 0027 part (c).
T4 Β§4 + Β§5 (mapper wiring + resync API) One task: they share the _build_copier seam and the dual-write test scaffolding; designed together so neither forecloses the other. ADR 0028 draft.
T5 Β§2 (cutover strict-0) Blocked by T4 β€” the strict default's story depends on run_resync_pass existing; the joint end-to-end test (resync β†’ strict cutover) lives here. Finalizes ADR 0028 and the guide updates.
T6 Β§3 (live lag) Standalone; subscriptions/runners/live.py + subscription.py docstrings.
T7 Β§9 (catch-up batch outcome) Standalone; subscriptions/runners/catchup.py only. Parallel-safe with T6 (different runner files; both add tests under the same test dir β€” coordinate file names only).
T8 Β§8 (DLQ cutoff) Standalone; memory adapter + port docstring + conformance suite.
T9 Β§10 (sqlite read lock) Standalone; adapters/sqlite/store.py + one interleave test module.

Dependency edges: T2 β†’ T3; T4 β†’ T5. All else parallelizable. CHANGELOG entries accumulate per task; a final assembly pass (or T5, as the last serialized task) verifies the CHANGELOG carries all nine user-facing changes (Β§1, Β§2, Β§3, Β§4, Β§5, Β§7, Β§8, Β§9, Β§10 β€” Β§6 only if it surfaced an adapter fix) and that ADRs 0027/0028 are complete with index.md updated.