Legacy Store Retirement β€” Slice (c): Migration

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Move src/eventsource/migration/ off the legacy EventStore ABC and onto the ports surface, and make every position it holds, compares, persists, or maps an opaque Position. This includes retiring the slice-(c) seam that slice (b) deliberately left behind: migration/subscription_migrator.py imports adapters/_sql/positions.IntPositionCodec and both it and MigrationCoordinator take required keyword-only source_store_id / target_store_id / source_position_store_id / target_position_store_id parameters whose docstrings name this slice as the place they die. Nothing is deleted: src/eventsource/stores/ stays in place and green, and slice (d) still owns the deletion, the MemoryEventStore rename, and the public-API swap.

Architecture: Migration stops being a legacy-ABC citizen. TenantStoreRouter and DualWriteInterceptor stop subclassing EventStore and become ports-shaped structural FullEventStore wrappers (a routing wrapper and a source-then-best-effort-target wrapper); BulkCopier reads the source GlobalEventFeed and appends through EventAppender; ConsistencyVerifier compares EventEnvelopes; SyncLagTracker stops subtracting two stores' positions β€” which is not merely banned but meaningless, since positions from different stores raise PositionForeignError on comparison β€” and reports a bounded count of source events not yet copied. The migration_position_mappings table becomes token-keyed, and once it is, the int↔token codecs at the subscription-migrator boundary have nothing left to convert: positions flow from the checkpoint table through the mapper to the checkpoint table as tokens, end to end. That is what retires the seam, and it is why neither the migrator nor the coordinator needs a declared store_id afterwards.

Tech Stack: Python 3.13, pydantic v2, sqlalchemy 2 (adapters and migration repositories only), asyncpg, aiosqlite, pytest + pytest-asyncio + hypothesis, mypy strict, ruff, import-linter, mutmut.

Spec: docs/superpowers/specs/2026-07-31-legacy-store-retirement-design.md (slice (c): Β§1.1, Β§1.2, Β§1.3, Β§1.5, Β§1.6, Β§2, Β§3 migration/, Β§6 slice (c), Β§7 slice (c), Β§11 risks 2, 4 and 5)

Sibling slices: docs/superpowers/plans/2026-07-31-store-retirement-a-leaf-consumers.md and docs/superpowers/plans/2026-07-31-store-retirement-b-subscriptions.md have both LANDED. AggregateStore exists in ports/store.py; harness/sync/bench are on the ports; subscriptions and the checkpoint adapters hold opaque Positions; SubscriptionPositions.get_position already returns Position | None; the additive-fragment composition path exists in src/eventsource/migrations/__init__.py (_ADDITIVE_FRAGMENTS, _ADDITIVE_DIR, get_schema(..., additive=...)) and Task 1 below reuses it exactly as it was designed to be reused. Do not redo any of it.

Global Constraints

`` Co-Authored-By: Claude Fable 5 ``

Behavior deltas this slice makes visible

Spec decisions (Β§1.1, Β§1.3, Β§1.5, Β§1.6, Β§2, Β§6 slice (c)), not implementation choices. Repeated in the task that first encounters each.

Delta Legacy Ports Who feels it
AppendResult.position position of the last appended event position of the first appended event Task 4: BulkCopier._write_batch's result.global_position - len(agg_events) + i + 1 arithmetic is invalid in both directions and is replaced, not adjusted
Duplicate event_id legacy in-memory/postgresql silently skip an already-stored id adapters raise DuplicateEventError Task 4 BulkCopier: catch and count as already-copied; this is what makes a resumed copy converge, and Task 5 is its property test
Empty append batch returns AppendResult.successful(expected_version) raises ValueError Task 4: _write_batch groups by aggregate and can only produce non-empty groups β€” the guard is re-verified, not assumed
Append conflict AppendResult.success=False / .conflict OptimisticLockError raised Task 4: the if not source_result.success branch in dual_write.py:369-374 and every result.conflict read is deleted, not translated; test doubles returning AppendResult.conflicted(...) become raising doubles (spec Β§11 risk 5)
Empty store get_global_position() returns 0 current_position() returns None Task 4: SyncLagTracker, cutover.py:367
Cross-store position math source_position - target_position PositionForeignError on the comparison Task 4: lag becomes a bounded count-behind on the source feed
Cross-type lookup get_events(aggregate_id, aggregate_type=None) no port; capability dropped (spec Β§2) Task 4: TenantStoreRouter.get_events and DualWriteInterceptor.get_events cease to exist; tests/unit/migration/test_dual_write.py:731 is rewritten against the ports shape
Category read filters/orders on event.occurred_at, exclusive > filters/orders on stored_at, inclusive >=, position tie-break Task 4: read_category routing in TenantStoreRouter
Feed event type StoredEvent (.global_position, .stream_position, .aggregate_id) EventEnvelope (.position, .stream_version, .stream_id.aggregate_id) Tasks 4 (BulkCopier, ConsistencyVerifier)
PostgreSQL feed horizon legacy feed had none adapter feed is bounded to the xmin safe horizon: current_position() can lag a just-appended position under concurrent writers Task 4: lag and convergence are advisory signals, never assertions about what the store contains
Store spans legacy stores emit postgresql_event_store.* etc. adapters emit none (ADR 0016 amendment) Any migration test asserting a store span must be deleted, not retargeted

Task 1: Token columns for the migration schema

Files:

Interfaces:

This task resolves spec Β§11 open risk 4. The reconnaissance was already performed against this tree by the plan author; Step 1 re-verifies it rather than rediscovering it, because the rest of the slice rests on it.

Run each and confirm the stated result. If any differs, STOP and report.


grep -rln "migration_position_mappings" --include='*.sql' src/

Expected: exactly src/eventsource/migrations/templates/migration.sql. The migration tables live in a template, not in schemas/all.sql.


grep -c "migration_position_mappings" src/eventsource/migrations/schemas/all.sql src/eventsource/migrations/schemas/sqlite_all.sql

Expected: 0 for both. The migration tables are deliberately not part of the combined schema β€” they are opt-in operator DDL.


ls src/eventsource/migrations/templates/sqlite/

Expected: no migration.sql. The migration schema is PostgreSQL-only; get_schema("migration", backend="sqlite") raises ValueError today and must keep doing so. Register no sqlite fragment.


grep -rn "^class \|PositionMappingRepository\|MigrationRepository" src/eventsource/migration/repositories/*.py | grep "^.*:class "

Expected: each repository has exactly a Protocol and a PostgreSQL* implementation β€” no in-memory or SQLite variant. Only PostgreSQL DDL is in play.

Conclusion to record in the commit body: the migration DDL is templates/migration.sql, PostgreSQL-only, applied by operators and by tests/integration/migrations/test_migration_schema_postgresql.py; unit tests exercise the repositories against mocks. The token columns therefore arrive through the same additive-fragment mechanism slice (b) built, registered for ("migration", "postgresql") only.

Extend tests/unit/migrations/test_additive_schema.py with a class covering the migration fragment:

Run: uv run pytest tests/unit/migrations/test_additive_schema.py -q β€” Expected: the reach cases FAIL, the base-file and sqlite cases PASS.

src/eventsource/migrations/additive/migration_position_tokens.sql, in the comment style of additive/checkpoints_position_token.sql. The legacy global_position checkpoint column is now frozen (adapters write only position_token) β€” this fragment work is the natural place to mark or drop it.


-- Additive fragment: opaque position tokens for migration bookkeeping.
-- Appended to the migration schema at composition time
-- (eventsource.migrations.get_schema). Idempotent on PostgreSQL.
--
-- The legacy BIGINT position columns are left in place and are neither
-- written nor read by the library after slice (c); they die with their
-- own schema revision, not this one.
ALTER TABLE migration_position_mappings
ADD COLUMN IF NOT EXISTS source_position_token TEXT;

ALTER TABLE migration_position_mappings
ADD COLUMN IF NOT EXISTS target_position_token TEXT;

-- Each source position maps to exactly one target position, as the legacy
-- UNIQUE (migration_id, source_position) constraint expressed for ints.
CREATE UNIQUE INDEX IF NOT EXISTS uq_position_mappings_source_token
ON migration_position_mappings (migration_id, source_position_token)
WHERE source_position_token IS NOT NULL;

-- Ordering for checkpoint translation is by surrogate id (mappings are
-- recorded in ascending source-position order); these indexes serve the
-- exact-token lookups.
CREATE INDEX IF NOT EXISTS idx_position_mappings_source_token
ON migration_position_mappings (migration_id, source_position_token);

CREATE INDEX IF NOT EXISTS idx_position_mappings_target_token
ON migration_position_mappings (migration_id, target_position_token);

ALTER TABLE tenant_migrations
ADD COLUMN IF NOT EXISTS last_source_position_token TEXT;

ALTER TABLE tenant_migrations
ADD COLUMN IF NOT EXISTS last_target_position_token TEXT;

The legacy NOT NULL on source_position / target_position is a problem for new inserts that carry only tokens: drop the NOT NULL in the same fragment (ALTER TABLE migration_position_mappings ALTER COLUMN source_position DROP NOT NULL; and the same for target_position) rather than writing a fabricated int. Re-read templates/migration.sql:149-170 before writing this to confirm the constraint names and nullability you are relaxing; if the real DDL differs from what this plan quotes, follow the real DDL and report the difference.

Also create the operator-facing updates/003_add_migration_position_tokens.sql, matching the shape and comment style of updates/002_add_position_token.sql: the same body plus COMMENT ON COLUMN lines for the four new columns. It is not applied by library code; it exists so operators can migrate an existing database.

In src/eventsource/migrations/__init__.py, add one entry to _ADDITIVE_FRAGMENTS:


    ("migration", "postgresql"): ("migration_position_tokens",),

No other change is needed: get_schema already resolves the backend additive directory, appends each registered fragment, and raises FileNotFoundError for a missing one. Do not add a ("migration", "sqlite") entry β€” there is no SQLite migration schema to compose against, and a registry entry would turn today's clean ValueError into a confusing FileNotFoundError.

tests/unit/migrations/test_migration_schema.py asserts on the text of get_schema("migration") (column presence, index presence, comment presence). Its existing assertions stay true β€” the fragment only appends. Add a case asserting the four token columns are present via the composed schema, and confirm no existing case asserts the absence of anything the fragment adds or counts occurrences of ALTER TABLE.


git status --porcelain src/eventsource/migrations/

Expected: only added files under additive/ and updates/, plus a modified __init__.py. If any schemas/.sql or templates/.sql shows as modified, revert it and report.

Run: uv run pytest tests/unit/migrations/ -q β€” Expected: PASS. Run: uv run mypy src/eventsource/migrations/ --config-file=pyproject.toml β€” Expected: clean. Run: uv run ruff check src/eventsource/migrations/ tests/unit/migrations/ β€” Expected: clean. Run: uv run lint-imports β€” Expected: clean.


git add src/eventsource/migrations/ tests/unit/migrations/
git commit -m "feat: additive position token columns for the migration schema"

Record in the commit body: the four reach points from Step 1, that the migration schema is PostgreSQL-only, and that no existing schema file was edited.


Task 2: The position-mapping table goes token-keyed

Files:

Interfaces:


# models.py
@dataclass(frozen=True)
class PositionMapping:
    ...
    source_position: Position
    target_position: Position

# position_mapper.py
@dataclass(frozen=True)
class TranslationResult:
    source_position: Position
    target_position: Position
    is_exact: bool
    nearest_source_position: Position | None = None

class PositionMapper:
    async def record_mapping(self, migration_id: UUID, source_position: Position,
                             target_position: Position, event_id: UUID) -> None: ...
    async def record_mappings_batch(
        self, migration_id: UUID,
        mappings: list[tuple[Position, Position, UUID]]) -> int: ...
    async def translate_position(self, migration_id: UUID, source_position: Position,
                                 *, use_nearest: bool = True) -> TranslationResult: ...

ReverseTranslationResult and the reverse-translation path retype symmetrically.

The one genuinely hard problem in this task is find_nearest_source_position. Today it is WHERE source_position <= :p ORDER BY source_position DESC LIMIT 1 β€” SQL ordering over a BIGINT. A Position is an opaque token whose to_str() is JSON; lexicographic ordering of that text is not position ordering, so no ORDER BY source_position_token can be correct, and inventing an int sort key would be exactly the arithmetic the ports forbid.

Decision: order by the surrogate id, compare tokens in Python, and find the nearest by binary search over the row ordinal. Mappings for one migration are recorded in ascending source-position order by a single writer (BulkCopier streams the source feed in position order; DualWriteInterceptor records in append order), so id order is source-position order. Under that precondition the nearest-match query is a binary search: O(log n) single-row LIMIT 1 OFFSET k reads against ORDER BY id, each comparing one Position against the target in Python. Exact lookups stay a single indexed query on the canonical token string (Position.to_str() is deterministic β€” fixed separators, fixed key order β€” so string equality is position equality within a store). The rejected alternative is loading every mapping and scanning in Python: correct, but unbounded memory on a large migration.

The monotonicity precondition must be stated in the repository docstring as a contract, not left implicit, and the batch writer must preserve it.

In src/eventsource/migration/models.py, retype PositionMapping.source_position and .target_position to Position (import from eventsource.ports.positions). In src/eventsource/migration/exceptions.py, PositionMappingError.source_position becomes Position | None and its message renders to_str().

In tests/unit/migration/test_position_mapping_repository.py, rewrite the cases: every source_position=1000 becomes a Position. Build them with a small module-level helper (def pos(n: int, store: str = "src") -> Position: return Position(store_id=store, key=(n,))) so ordering intent stays readable. Add the cases the new design needs:

Run: uv run pytest tests/unit/migration/test_position_mapping_repository.py -q β€” Expected: FAIL.


        # Mappings are recorded in ascending source-position order by a single
        # writer, so `id` order is source-position order. Positions are opaque
        # tokens: they cannot be ordered in SQL, so the nearest match is a
        # binary search over the row ordinal with the comparison in Python.
        total = await self.count_by_migration(migration_id)
        if total == 0:
            return None
        lo, hi = 0, total - 1
        best: PositionMapping | None = None
        while lo <= hi:
            mid = (lo + hi) // 2
            candidate = await self._get_by_ordinal(migration_id, mid)
            if candidate is None:
                break
            if candidate.source_position <= source_position:
                best = candidate
                lo = mid + 1
            else:
                hi = mid - 1
        return best

_get_by_ordinal is SELECT ... WHERE migration_id = :m ORDER BY id LIMIT 1 OFFSET :k. The <= comparison raises PositionForeignError if a caller passes a position from the wrong store β€” that is the correct failure and must not be caught here.

translate_position / translate_position_reverse take and return Position. record_mapping / record_mappings_batch take Position. Every log line and span attribute that carried %d positions renders to_str() β€” span attributes must be primitives, so pass the string, never the VO. The use_nearest behavior, the is_exact flag, and the caching (if the module caches β€” check before changing it) are unchanged in shape.


grep -rn "source_position\|target_position" src/eventsource/migration/position_mapper.py src/eventsource/migration/repositories/position_mapping.py | grep -E "int\b|[-+*/]|min\(|max\("

Expected: no matches (no int annotations, no arithmetic).

Run: uv run pytest tests/unit/migration/test_position_mapping_repository.py tests/unit/migration/test_position_mapper.py tests/unit/migration/test_models.py -q β€” Expected: PASS. Run: uv run mypy src/eventsource/ --config-file=pyproject.toml β€” Expected: clean. Run: uv run ruff check src/eventsource/migration/ tests/unit/migration/ β€” Expected: clean. Run: uv run lint-imports β€” Expected: clean.


git add src/eventsource/migration/repositories/position_mapping.py \
        src/eventsource/migration/position_mapper.py \
        src/eventsource/migration/models.py src/eventsource/migration/exceptions.py \
        tests/unit/migration/test_position_mapping_repository.py \
        tests/unit/migration/test_position_mapper.py tests/unit/migration/test_models.py
git commit -m "refactor: key position mappings by opaque position tokens"

Record in the commit body: the monotonicity precondition and why nearest-match became a binary search.


Task 3: Retire the slice-(c) seam

Files:

Interfaces:


class SubscriptionMigrator:
    def __init__(self, position_mapper: PositionMapper,
                 checkpoint_repo: CheckpointRepository, *,
                 tracer: Tracer | None = None,
                 enable_tracing: bool = True) -> None: ...

β€” no source_store_id, no target_store_id, no IntPositionCodec import. MigrationCoordinator.__init__ loses source_position_store_id and target_position_store_id; source_store_id survives as what it always was, a routing label.

Why the seam dies here without any new port. Slice (b)'s docstrings anticipated deriving the ids "from the ports stores' own store_id attributes". Read src/eventsource/ports/store.py before implementing: FullEventStore declares no store_id β€” it is an attribute of the concrete adapters, not of the port. Deriving the ids from it would mean widening a port to carry an adapter detail. That turns out to be unnecessary: once Task 2 makes the mapping table token-keyed, the migrator never constructs a position from an int and never reduces one to an int. It reads a Position from the checkpoint repository, hands it to the mapper, receives a Position back, and writes it to the checkpoint repository. There is nothing left to name a store_id for. Do not add store_id to any port. If an implementer finds a call that genuinely still needs one, STOP and report β€” that is a design change, not an implementation detail.


grep -rn "SubscriptionMigrator(\|source_position_store_id\|target_position_store_id\|IntPositionCodec" src/ tests/ bench/

Record the full list. In each test construction site, delete the source_store_id= / target_store_id= / *_position_store_id= arguments. In tests/unit/migration/test_subscription_migrator.py, every checkpoint value and every mapper stub currently traffics in ints at this boundary β€” retype them to Position, and add a case asserting that a checkpoint position and its translated target position round-trip through save_position as tokens, with no int appearing anywhere in the path.

Run: uv run pytest tests/unit/migration/test_subscription_migrator.py -q β€” Expected: FAIL (TypeError: unexpected keyword argument is not the failure you want to stop at; the meaningful red is the int-vs-Position mismatch once the constructor accepts the call).

In src/eventsource/migration/subscription_migrator.py:

In src/eventsource/migration/coordinator.py:


grep -rn "IntPositionCodec\|position_store_id\|slice-(c) seam\|slice (c)" src/eventsource/migration/

Expected: no matches. This is the acceptance for the task.


grep -rn "from eventsource.adapters" src/eventsource/migration/

Expected: no matches. migration/ is a use-case-ring package and must not import an adapter.

Run: uv run pytest tests/unit/migration/test_subscription_migrator.py tests/unit/migration/test_coordinator_subscriptions.py -q β€” Expected: PASS. Run: uv run mypy src/eventsource/ --config-file=pyproject.toml β€” Expected: clean. Run: uv run ruff check src/eventsource/migration/ tests/unit/migration/ β€” Expected: clean. Run: uv run lint-imports β€” Expected: clean. If a contract was previously allowing migration β†’ adapters as recorded debt, remove that allowance now and re-run; if no such allowance exists, say so in the commit body.


git add src/eventsource/migration/subscription_migrator.py src/eventsource/migration/coordinator.py \
        tests/unit/migration/test_subscription_migrator.py \
        tests/unit/migration/test_coordinator_subscriptions.py
git commit -m "refactor: retire the int position codec seam in subscription migration"

Record in the commit body: that the seam died by making the mapping table token-keyed rather than by widening a port with store_id, and that MigrationCoordinator.source_store_id survives as a routing label only.


Task 4: The flip β€” migration moves onto the ports store surface

Why this task is large. Every commit must be type-clean (pre-commit runs mypy) and no shims are permitted. MigrationCoordinator holds the stores and hands the same objects to TenantStoreRouter, DualWriteInterceptor, BulkCopier, ConsistencyVerifier, SyncLagTracker and CutoverManager. A legacy EventStore ABC instance does not structurally satisfy FullEventStore (the method names differ: append_events vs append), so the moment any one of those collaborators retypes, the coordinator's call site is red β€” and the moment the coordinator retypes, all of them are. There is no intermediate state that is both green and shim-free, so the flip is one commit. The steps below are ordered so an implementer can work outside-in.

The orchestrator may dispatch Steps 1-7 and Steps 8-11 as separate work units against the same uncommitted tree, but there is exactly one commit at the end.

Files:

Interfaces:


class TenantStoreRouter:                     # structural FullEventStore, no ABC base
    def __init__(self, default_store: FullEventStore, routing_repo, *,
                 stores: dict[str, FullEventStore] | None = None,
                 default_store_id: str = "default", ...) -> None: ...
    async def append(self, stream: StreamId, events: Sequence[DomainEvent],
                     expected: ExpectedVersion) -> AppendResult: ...
    def read_stream(self, stream, options=None) -> AsyncIterator[EventEnvelope]: ...
    async def get_stream_version(self, stream: StreamId) -> int: ...
    async def event_exists(self, event_id: UUID) -> bool: ...
    def read_all(self, from_position=None, options=None) -> AsyncIterator[EventEnvelope]: ...
    async def current_position(self) -> Position | None: ...
    def read_category(self, category, options=None) -> AsyncIterator[EventEnvelope]: ...

class DualWriteInterceptor:                  # structural FullEventStore, no ABC base
    ...same eight members; reads delegate to the source verbatim

@dataclass(frozen=True)
class FailedWrite:
    ...
    source_position: Position | None

@dataclass
class SyncLag:
    events: int                              # exact up to the bound, then ">bound"
    source_position: Position | None
    target_position: Position | None
    timestamp: datetime

@dataclass
class Migration:
    ...
    last_source_position: Position | None = None
    last_target_position: Position | None = None

tests/unit/migration/test_router.py and test_dual_write.py are the specification of the two wrappers. Rewrite them first:

Run: uv run pytest tests/unit/migration/test_router.py tests/unit/migration/test_dual_write.py -q β€” Expected: FAIL.

src/eventsource/migration/router.py: drop the EventStore base class and the stores.interface import; type _stores, _default_store, _dual_write_interceptors and every accessor as FullEventStore. Translate method by method per spec Β§1:

src/eventsource/migration/dual_write.py: drop the ABC base and the stores.interface import.

src/eventsource/migration/sync_lag_tracker.py. Today calculate_lag (:266-270) reads both stores' get_global_position() and subtracts. With opaque tokens that subtraction is not merely banned, it is impossible: the two positions come from different stores, and comparing them raises PositionForeignError.

New shape:


    async def calculate_lag(self, *, since: Position | None = None) -> SyncLag:
        """Count source events not yet copied, bounded by the sync threshold.

        `since` is the last source position the target has copied (the
        migration's `last_source_position`); None means nothing has been
        copied and the count starts at the head of the source feed. The
        count is exact up to `cutover_max_lag_events + 1`; beyond that it
        reports the bound and stops reading, which is all a convergence
        decision needs.
        """

Callers must supply since. MigrationCoordinator has it on the Migration record (last_source_position); wire it through in Step 7.

src/eventsource/migration/bulk_copier.py:


                stream = StreamId(aggregate_id=aggregate_id, category=aggregate_type)
                try:
                    current_version = await self._target.get_stream_version(stream)
                except Exception:
                    current_version = 0

Then, per spec Β§6 slice (c) item 3:

Spec Β§11 risk 2 (per-event append cost with a mapper) is a Slice Completion Criterion, not an implementer's call. Implement the per-event path as specified and let the orchestrator measure.

src/eventsource/migration/consistency.py: StoredEvent β†’ EventEnvelope throughout (:526, :537, :547-568, :575-576, :721-724, :760); ReadOptions β†’ FeedReadOptions at the collection site; .stream_position β†’ .stream_version (:568, :608-609, :674-693, :780); the grouping key comes from envelope.stream_id.render() where the code used the legacy stream_id string, keeping the same wire format (domain/stream_id.py). Event hashes are payload-based and their inputs are unchanged apart from stream_version replacing stream_position β€” the same number under a new name, so hash values do not move.

Per spec Β§7 slice (c). Work in this order, cheapest first:


grep -rn "stores\.interface\|StoredEvent\|ReadOptions\|append_events\|get_events_by_type\|get_global_position\|global_position" src/eventsource/migration/

Expected: no matches.


grep -rn "position" src/eventsource/migration/ | grep -E "[-+*/] *(source|target|last)_position|min\(.*position|max\(.*position"

Expected: no matches. Inspect every hit; a max() over a count is fine, a max() over positions is not.


grep -rn "\.success\|\.conflict" src/eventsource/migration/ tests/unit/migration/

Expected: no hits on an AppendResult. Other success fields (SubscriptionMigrationResult.success, BulkCopyResult.success) are unrelated and stay β€” check each hit rather than assuming.


grep -rln "stores\.interface\|append_events" src/eventsource/__init__.py src/eventsource/stores/

Expected: several matches β€” slice (d) still owns those. An empty result means this slice overreached.


git diff main --stat -- src/eventsource/stores/ src/eventsource/subscriptions/

Expected: empty. Both are frozen in this slice.

Run: uv run pytest tests/unit/migration/ -q β€” Expected: PASS. Run: uv run pytest tests/unit/test_public_api.py -q β€” Expected: PASS (this slice adds and removes no public name). Run: uv run mypy src/eventsource/ --config-file=pyproject.toml β€” Expected: clean. Run: uv run ruff check src/eventsource/migration/ tests/unit/migration/ β€” Expected: clean. Run: uv run lint-imports β€” Expected: clean.


git add src/eventsource/migration/ tests/unit/migration/
git commit -m "refactor: retype migration onto the ports store surface"

Record in the commit body: the dropped aggregate_type=None capability (spec Β§2), the per-event append decision and why the old target-position estimate could not be preserved, the DuplicateEventError-as-already-copied rule, the count-behind lag, and the number of deleted store-span cases.


Task 5: The bulk-copy resume idempotency property

Files:

Interfaces:

Against two real MemoryEventStores and a PositionMapper over an in-memory double of PositionMappingRepository (there is no in-memory implementation in the tree β€” write a minimal one in the test module, holding a list and honouring the ascending-source-order precondition, or drive BulkCopier without a mapper if wiring the double proves to dominate the test; prefer the double, since the mapper path is the one this slice redesigned):


@settings(max_examples=50, deadline=None, suppress_health_check=[HealthCheck.too_slow])
@given(
    streams=st.lists(st.integers(min_value=1, max_value=5), min_size=1, max_size=6),
    batch_size=st.integers(min_value=1, max_value=4),
    crash_after=st.integers(min_value=0, max_value=20),
)
@pytest.mark.asyncio
async def test_bulk_copy_resumes_to_a_source_equal_target(
    streams: list[int], batch_size: int, crash_after: int
) -> None:
    """Random stream shapes, batch sizes and crash points: a resumed copy
    converges on a target whose streams equal the source's.

    A resume that re-reads from an inclusive position re-appends events the
    target already has; the copy is only correct because that re-append
    raises DuplicateEventError and is counted as already-copied. Remove that
    handler and this property fails immediately.
    """

Shape: append sum(streams) events across that many distinct streams into the source; run a BulkCopier and cancel it after crash_after events have been copied; construct a second copier over the same stores, resuming from the persisted last_source_position; run it to completion. Assert with ConsistencyVerifier that the target is source-equal, and separately that no event id appears twice in the target's feed.

Clamp crash_after to the total rather than discarding examples with hypothesis.assume.

Run: uv run pytest tests/unit/migration/test_bulk_copy_resume_property.py -q β€” Expected: PASS. Run: uv run pytest tests/unit/migration/test_bulk_copy_resume_property.py -q -p no:randomly --hypothesis-seed=0 β€” Expected: PASS (a deterministic second opinion; if the two disagree, the property has a state leak between examples β€” fix the fixture, do not loosen the property). Run: uv run ruff check tests/unit/migration/test_bulk_copy_resume_property.py β€” Expected: clean.


git add tests/unit/migration/test_bulk_copy_resume_property.py
git commit -m "test: property test for bulk copy resume idempotency"

Task 6: Mutation config, docs, changelog, final sweep

Files:

Interfaces:

Concurrency caveat: other agents are editing pyproject.toml. Locate the [tool.mutmut] block by its TOML header, not by line number, and re-read the file immediately before editing. If an edit is already present, verify rather than duplicate and say so in the commit body.

Spec Β§7: slice (c) adds src/eventsource/migration/bulk_copier.py to only_mutate and tests/unit/migration/test_bulk_copier.py to pytest_add_cli_args_test_selection β€” the resume and duplicate-skip logic is exactly the off-by-one surface mutmut exists for. The spec marks this "optionally"; take it, and add src/eventsource/migration/position_mapper.py with tests/unit/migration/test_position_mapper.py alongside it, since Task 2's binary search is a second such surface.

Run: uv run python -c "import tomllib,pathlib; c=tomllib.loads(pathlib.Path('pyproject.toml').read_text()); print(c['tool']['mutmut'])" and confirm both keys read back as intended. Do not run mutmut itself β€” that is the orchestrator's call.


grep -rn "get_global_position\|append_events\|source_position\|target_position\|position_store_id" docs/guides/live-migration.md docs/api/migration-schema.md docs/guides/database-schema.md docs/explanation/schema-design.md docs/ examples/ README.md

For each hit in prose or example code about migration: int positions become tokens; get_global_position() becomes current_position(); append_events(...) becomes append(StreamId(...), events, ExpectedVersion...); the migration schema pages gain the four token columns and the note that the legacy BIGINT columns are no longer written. Do not touch documentation of the legacy stores themselves, of subscriptions/, or docs/adrs/ β€” those are slice (d)'s and slice (b)'s respectively. ADR 0025 is written in slice (d) (spec Β§9); do not create it here.

Add a CHANGELOG.md entry for this slice naming the deltas loudly: migration runs on the ports store surface; cross-type get_events(aggregate_type=None) is gone; a duplicate append during bulk copy is counted as already-copied instead of silently skipped; sync lag is a bounded count of uncopied source events rather than a position delta; position mappings and migration progress are persisted as opaque tokens; MigrationCoordinator and SubscriptionMigrator no longer take position-store ids.


grep -rn "EventStore\b" src/eventsource/migration/

Expected: no matches other than prose in docstrings that means "event store" generically β€” inspect each and rewrite the prose where it names the dead ABC.


grep -rn "IntPositionCodec\|position_store_id" src/ tests/ bench/

Expected: no matches outside src/eventsource/adapters/_sql/positions.py and its own tests (the codec itself stays; it is the SQL adapters' internal encoding).


git diff main --stat -- src/eventsource/stores/ src/eventsource/subscriptions/ src/eventsource/__init__.py

Expected: empty.


git diff main --stat -- src/eventsource/migrations/

Expected: additions under additive/ and updates/ plus a modified __init__.py β€” no schemas/ or templates/ file modified.


git add pyproject.toml CHANGELOG.md docs/
git commit -m "docs: document opaque positions in migration and extend mutation selection"

Slice Completion Criteria

The orchestrator runs these; implementers do not.

Plan Self-Review

Spec Gaps Found (report, do not silently deviate)

  1. The spec's plan for retiring the seam does not survive contact with ports/store.py. Slice (b)'s docstrings (and the dispatch brief) say the store ids will "derive from the ports stores' own store_id attributes once migration holds ports stores". FullEventStore declares no store_id β€” it is a concrete-adapter attribute. Deriving from it would require widening a port to carry an adapter detail, which ADR 0019's segregation forbids. Resolved instead by making the position-mapping table token-keyed (Task 2), after which the migrator has nothing to convert and needs no store_id at all (Task 3). This is a better outcome than the anticipated one and it removes rather than adds port surface, but it is not the mechanism the earlier slice predicted.
  2. Β§6 slice (c) item 7 says the mapping rows "gain token TEXT columns" and stops there. It does not address that find_nearest_source_position β€” the whole reason the table exists β€” is an ORDER BY source_position DESC LIMIT 1 over an orderable integer, and that no ORDER BY over an opaque token can replace it (Position.to_str() is JSON; its lexicographic order is not position order). Resolved in Task 2 with a binary search over the surrogate id and Python-side Position comparison, resting on a documented monotonicity precondition (mappings are recorded in ascending source order by a single writer). The rejected alternative β€” load all mappings and scan β€” is correct but unbounded. This is a real algorithmic change the spec does not anticipate, and it deserves a line in ADR 0025 when slice (d) writes it.
  3. The migration_position_mappings legacy columns are NOT NULL and the spec's additive-only rule does not cover relaxing them. Token-only inserts violate source_position NOT NULL. Task 1 drops the NOT NULL in the additive fragment rather than fabricating an int, which is still a fragment (no base-file edit) but is a constraint change, not purely an addition. Flagging because "append-only" could be read as forbidding it; the alternative β€” writing a derived int β€” would reintroduce exactly the arithmetic this slice removes.
  4. SyncLagTracker has no source anchor to count from. Β§6 slice (c) item 5 says to "read the source feed strictly after the target's last-copied source position", but the tracker holds only two stores; the last-copied source position lives on the Migration record, which the coordinator holds. Resolved by making it a calculate_lag(*, since: Position | None) parameter supplied by the caller (Task 4 Steps 4 and 7), with None meaning "nothing copied yet". The alternative β€” giving the tracker a MigrationRepository β€” would widen its dependencies for one field.
  5. SyncLag.events becomes ambiguous at the bound. A bounded count that stops at threshold + 1 cannot distinguish "exactly threshold+1 behind" from "far behind". Convergence decisions are unaffected (both are over threshold), but the reported number is no longer exact at the top of its range and the spec presents it as exact. Task 4 Step 4 adds a flag to SyncLag so the reported number stays honest, with an explicit fallback and a report-instead-of-guess instruction if that field churns models.py consumers.
  6. Neither the Migration record's progress columns nor BulkCopyError.last_position appear in the spec's Β§6 slice (c) file list, but tenant_migrations.last_source_position / .last_target_position are the bulk copier's resume state and BulkCopyError carries a position in its message. They are in scope by necessity (Task 1's fragment, Task 4 Steps 5 and 7). Likewise cutover.py:367 calls get_global_position() and migration/exceptions.py types PositionMappingError.source_position as int | None; both are covered here and neither is listed in the spec.