Broker Backend God-Class Decomposition Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Decompose RabbitMQEventBus (~3,570-line class) and KafkaEventBus (~2,195-line class) into backend subpackages of state-owning collaborators behind unchanged facades, shipped as 0.7.0.

Architecture: Each backend flat module becomes a package (bus/rabbitmq/, bus/kafka/) whose __init__.py re-exports everything the old module exported. Collaborators (ConnectionManager, Topology, Publisher, Consumer, DLQAdmin, plus pure serialization/death-header modules) own their state; the *EventBus class becomes a facade that composes them and keeps every public signature. Extraction is leaf-first: package conversion β†’ pure modules β†’ stateful collaborators β†’ facade slim-down.

Tech Stack: Python 3.11+, pydantic v2, aio-pika (RabbitMQ), aiokafka (Kafka), pytest + pytest-asyncio, hypothesis, mutmut, uv.

Spec: docs/superpowers/specs/2026-07-29-bus-god-class-decomposition-design.md β€” read the spec section named in a task before implementing it.

Global Constraints


Task 1: Convert `bus/rabbitmq.py` to a package

Files:

Interfaces:


mkdir -p src/eventsource/bus/rabbitmq
git mv src/eventsource/bus/rabbitmq.py src/eventsource/bus/rabbitmq/bus.py

"""RabbitMQ event bus backend.

Public import surface is identical to the pre-0.7.0 flat module
``eventsource.bus.rabbitmq``. Internal collaborators live in sibling
modules and are not part of the public API.
"""

from eventsource.bus.rabbitmq.bus import (
    OTEL_AVAILABLE,
    RABBITMQ_AVAILABLE,
    BatchPublishError,
    DLQMessage,
    HealthCheckResult,
    QueueInfo,
    RabbitMQEventBus,
    RabbitMQEventBusConfig,
    RabbitMQEventBusStats,
    RabbitMQNotAvailableError,
    ShutdownError,
)

__all__ = [
    "BatchPublishError",
    "DLQMessage",
    "HealthCheckResult",
    "OTEL_AVAILABLE",
    "QueueInfo",
    "RabbitMQEventBus",
    "RabbitMQEventBusConfig",
    "RabbitMQEventBusStats",
    "RabbitMQNotAvailableError",
    "RABBITMQ_AVAILABLE",
    "ShutdownError",
]

Keep __all__ in bus.py too for now (it moves out as later tasks strip the file).

tests/unit/bus/rabbitmq/test_import_guard.py (create tests/unit/bus/rabbitmq/__init__.py as an empty file first):


"""The optional-dependency guard must behave identically to the flat module."""

import importlib
import sys
from unittest import mock

import pytest


def _purge(prefix: str) -> None:
    for name in [m for m in sys.modules if m == prefix or m.startswith(prefix + ".")]:
        del sys.modules[name]


def test_rabbitmq_available_true_with_driver_installed() -> None:
    mod = importlib.import_module("eventsource.bus.rabbitmq")
    assert mod.RABBITMQ_AVAILABLE is True


def test_import_succeeds_and_flag_false_without_aio_pika() -> None:
    saved = {
        m: sys.modules[m]
        for m in list(sys.modules)
        if m == "aio_pika" or m.startswith("aio_pika.")
        or m == "eventsource.bus.rabbitmq"
        or m.startswith("eventsource.bus.rabbitmq.")
    }
    try:
        _purge("eventsource.bus.rabbitmq")
        _purge("aio_pika")
        with mock.patch.dict(sys.modules, {"aio_pika": None}):
            mod = importlib.import_module("eventsource.bus.rabbitmq")
            assert mod.RABBITMQ_AVAILABLE is False
            with pytest.raises(mod.RabbitMQNotAvailableError):
                mod.RabbitMQEventBus(mod.RabbitMQEventBusConfig(rabbitmq_url="amqp://x"))
    finally:
        _purge("eventsource.bus.rabbitmq")
        _purge("aio_pika")
        sys.modules.update(saved)

Note: RabbitMQEventBusConfig may require more constructor args β€” check its __post_init__ (bus.py:130-377) and pass the minimal valid set. If constructing the config itself requires aio-pika (it should not β€” it is a plain dataclass), construct it before the patch.


uv run pytest tests/unit/bus/ -x -q
uv run pytest tests/unit/ -q
uv run mypy src/eventsource/ --config-file=pyproject.toml
uv run ruff check src/ tests/

Expected: all pass. If mock.patch.dict on sys.modules proves flaky with pytest plugins, an equivalent approach: insert a importlib.abc.MetaPathFinder that raises ImportError for aio_pika β€” either is acceptable; the assertion set must not change.


git add -A src/eventsource/bus/rabbitmq tests/unit/bus/rabbitmq
git commit -m "refactor: convert bus/rabbitmq.py to a package with identical import surface"

Task 2: Extract `death_headers.py` (pure functions) with property tests

Files:

Interfaces:

Port any existing death-header assertions from current tests (grep get_death_count\|is_from_dlq\|get_death_info under tests/) and add coverage for every function. Representative shape (adjust arg construction to the real signatures β€” they take a headers mapping or an AbstractIncomingMessage; mirror what bus.py does):


"""Unit + property tests for the pure death-header functions."""

from hypothesis import given
from hypothesis import strategies as st

from eventsource.bus.rabbitmq import death_headers


def _headers_with_death(count: int, queue: str = "q1", reason: str = "rejected",
                        exchange: str = "ex", routing_keys: list[str] | None = None) -> dict:
    return {
        "x-death": [
            {
                "count": count,
                "queue": queue,
                "reason": reason,
                "exchange": exchange,
                "routing-keys": routing_keys or ["events.order.created"],
            }
        ]
    }


def test_death_count_zero_when_no_header() -> None:
    assert death_headers.get_death_count({}) == 0


def test_death_count_reads_first_entry() -> None:
    assert death_headers.get_death_count(_headers_with_death(3)) == 3


def test_is_from_dlq_false_without_header() -> None:
    assert death_headers.is_from_dlq({}) is False


@given(count=st.integers(min_value=0, max_value=10_000))
def test_death_count_roundtrips_any_count(count: int) -> None:
    assert death_headers.get_death_count(_headers_with_death(count)) == count


@given(queue=st.text(min_size=1, max_size=50))
def test_first_death_queue_returns_recorded_queue(queue: str) -> None:
    assert death_headers.get_first_death_queue(_headers_with_death(1, queue=queue)) == queue

Add one test per remaining function (get_first_death_reason, get_first_death_exchange, get_original_routing_key, get_death_info) asserting the happy path and the missing-header default. Match the real return types (int, str | None, dict | None β€” read the signatures).

In bus.py, delete the seven method bodies and add inside the class body:


    # Permanent public aliases: the pure implementations live in death_headers.py.
    get_death_count = staticmethod(death_headers.get_death_count)
    get_first_death_queue = staticmethod(death_headers.get_first_death_queue)
    get_first_death_reason = staticmethod(death_headers.get_first_death_reason)
    get_first_death_exchange = staticmethod(death_headers.get_first_death_exchange)
    get_original_routing_key = staticmethod(death_headers.get_original_routing_key)
    is_from_dlq = staticmethod(death_headers.is_from_dlq)
    get_death_info = staticmethod(death_headers.get_death_info)

Update in-class callers (grep self.get_death_count\|self.is_from_dlq etc. in bus.py) β€” they keep working via the aliases; leave them as-is.


Task 3: Extract Rabbit `config.py`, `models.py`, `serialization.py`

Files:

Interfaces:


import pytest
from hypothesis import given

from eventsource.bus.rabbitmq import serialization
from eventsource.bus.rabbitmq.config import RabbitMQEventBusConfig
# reuse the event strategies/fixtures test_serialization_properties.py uses


def test_roundtrip_preserves_event_identity(sample_event, config) -> None:
    body = serialization.serialize_event(sample_event)
    restored = serialization.deserialize_event(
        body, headers={}, resolve_event_class=lambda name: type(sample_event)
    )
    assert restored == sample_event

Drive hypothesis-based async bodies (if any) with asyncio.run from sync tests β€” @given on async def passes vacuously.


Task 4: Extract `RabbitMQConnectionManager`

Files:

Interfaces:


class RabbitMQConnectionManager:
    def __init__(self, config: RabbitMQEventBusConfig, stats: RabbitMQEventBusStats) -> None: ...

    @property
    def is_connected(self) -> bool: ...
    @property
    def is_reconnecting(self) -> bool: ...
    @property
    def connection(self) -> AbstractRobustConnection | None: ...
    @property
    def channel(self) -> AbstractChannel | None: ...

    def require_channel(self) -> AbstractChannel:
        """Return the live channel or raise RuntimeError('Not connected...')
        with the same message the current code uses."""

    def on_reconnect(self, callback: Callable[[], Awaitable[None]]) -> None:
        """Register an async callback fired after a successful reconnect,
        in registration order."""

    async def connect(self) -> None: ...
    async def disconnect(self) -> None: ...
    async def force_disconnect(self) -> None: ...

Full test skeleton:


from unittest import mock

import pytest

from eventsource.bus.rabbitmq.config import RabbitMQEventBusConfig
from eventsource.bus.rabbitmq.connection import RabbitMQConnectionManager
from eventsource.bus.rabbitmq.models import RabbitMQEventBusStats


@pytest.fixture
def manager() -> RabbitMQConnectionManager:
    return RabbitMQConnectionManager(
        config=RabbitMQEventBusConfig(rabbitmq_url="amqp://guest:guest@localhost:5672/"),
        stats=RabbitMQEventBusStats(),
    )


@pytest.mark.asyncio
async def test_connect_sets_connected_and_channel(manager) -> None:
    fake_channel = mock.AsyncMock()
    fake_conn = mock.AsyncMock()
    fake_conn.channel.return_value = fake_channel
    with mock.patch(
        "eventsource.bus.rabbitmq.connection.aio_pika.connect_robust",
        new=mock.AsyncMock(return_value=fake_conn),
    ):
        await manager.connect()
    assert manager.is_connected
    assert manager.channel is fake_channel


@pytest.mark.asyncio
async def test_reconnect_callbacks_fire_in_order(manager) -> None:
    calls: list[str] = []

    async def cb_a() -> None:
        calls.append("a")

    async def cb_b() -> None:
        calls.append("b")

    manager.on_reconnect(cb_a)
    manager.on_reconnect(cb_b)
    await manager._run_reconnect_callbacks()  # the internal hook the aio-pika callback awaits
    assert calls == ["a", "b"]

(Adapt mock plumbing to the real connect body β€” e.g. QoS setup, publisher confirms β€” after reading it. The assertions above are the contract; add mocks until they hold.)


Task 5: Extract `RabbitMQTopology`

Files:

Interfaces:


class RabbitMQTopology:
    def __init__(self, config: RabbitMQEventBusConfig, connection: RabbitMQConnectionManager) -> None: ...

    @property
    def exchange(self) -> AbstractExchange | None: ...
    @property
    def dlq_exchange(self) -> AbstractExchange | None: ...
    @property
    def consumer_queue(self) -> AbstractQueue | None: ...
    @property
    def dlq_queue(self) -> AbstractQueue | None: ...

    async def declare_all(self) -> None:
        """Declare exchange, consumer queue, bindings, and DLQ per config flags β€”
        exactly what connect() declares today, in the same order."""

    async def redeclare(self) -> None:
        """Reconnect path: re-declare everything (current _on_reconnect behavior)."""

    async def bind_event_type(self, event_type: type[DomainEvent]) -> None: ...
    async def bind_routing_key(self, routing_key: str) -> None: ...

Task 6: Extract `RabbitMQPublisher`

Files:

Interfaces:


class RabbitMQPublisher:
    def __init__(
        self,
        config: RabbitMQEventBusConfig,
        connection: RabbitMQConnectionManager,
        topology: RabbitMQTopology,
        stats: RabbitMQEventBusStats,
        tracer: Tracer | None,
        enable_tracing: bool,
    ) -> None: ...

    async def publish_one(self, event: DomainEvent) -> None:
        """Current _publish_single body: build message via serialization.create_message_with_tracing,
        publish to topology.exchange with the derived routing key, update stats."""

    async def publish_batch(self, events: Sequence[DomainEvent]) -> dict[str, int]:
        """Current batch entry: dispatch to sequential/concurrent strategy per config,
        raising BatchPublishError exactly as today."""

(Internal strategy methods _publish_chunk_concurrent etc. move as private methods of the publisher, bodies verbatim.)


Task 7: Extract `RabbitMQConsumer` (consume loop + retry/DLQ write path)

Files:

Interfaces:


class RabbitMQConsumer:
    def __init__(
        self,
        config: RabbitMQEventBusConfig,
        connection: RabbitMQConnectionManager,
        topology: RabbitMQTopology,
        stats: RabbitMQEventBusStats,
        retry_policy: RetryPolicy,
        handlers_for: Callable[[type[DomainEvent]], tuple[HandlerAdapter, ...]],
        resolve_event_class: Callable[[str], type[DomainEvent] | None],
        tracer: Tracer | None,
        enable_tracing: bool,
    ) -> None: ...

    @property
    def is_consuming(self) -> bool: ...

    async def start(self) -> None: ...          # current start_consuming body
    async def stop(self) -> None: ...           # current stop_consuming body
    def start_in_background(self) -> asyncio.Task[None]: ...
    async def resume_if_was_consuming(self) -> None:
        """Reconnect path: restart consuming iff it was active before the drop
        (current _was_consuming logic)."""
    async def stop_gracefully(self, timeout: float) -> None: ...
    async def drain_in_flight(self, timeout: float) -> None: ...

Task 8: Extract `RabbitMQDLQAdmin`

Files:

Interfaces:


class RabbitMQDLQAdmin:
    def __init__(
        self,
        config: RabbitMQEventBusConfig,
        connection: RabbitMQConnectionManager,
        topology: RabbitMQTopology,
        stats: RabbitMQEventBusStats,
    ) -> None: ...

    async def get_messages(...) -> list[DLQMessage]: ...   # copy the exact public signatures
    async def get_message_count(self) -> int: ...          # from bus.py before moving
    async def replay_message(...) -> ...: ...
    async def purge(self) -> ...: ...

Copy each public signature verbatim from bus.py (defaults included) β€” the facade delegations must be signature-identical to today's methods.


Task 9: Slim the Rabbit facade; shutdown/health composition; backfill wiring tests

Files:

Interfaces:


@pytest.mark.asyncio
async def test_reconnect_redeclares_topology_and_resumes_consumer(bus_with_mocks) -> None:
    """Fire the connection manager's reconnect callbacks; assert topology.redeclare
    and consumer.resume_if_was_consuming were awaited, in that order."""


@pytest.mark.asyncio
async def test_shutdown_ordering(bus_with_mocks) -> None:
    """shutdown() stops the consumer, drains in-flight, drains background tasks,
    then disconnects β€” assert relative call order via a shared call log."""


@pytest.mark.asyncio
async def test_health_check_composes_slices(bus_with_mocks) -> None:
    """health_check() returns HealthCheckResult reflecting mocked slice values."""


@pytest.mark.asyncio
async def test_stats_accumulate_across_collaborators(bus_with_mocks) -> None:
    """The stats object handed to publisher/consumer is the same object the
    facade's get_stats() returns (identity check + counter increment visible)."""

Build bus_with_mocks as a fixture: construct a real RabbitMQEventBus, then replace _connection_manager, _topology, _consumer, _publisher, _dlq_admin with mocks wired to a shared calls: list[str] log. Write the docstring-described assertions as real code.


Task 10: Convert `bus/kafka.py` to a package

Files:

Interfaces:


Task 11: Extract Kafka `config.py`, `models.py`, `serialization.py`

Files:


Task 12: Extract Kafka `metrics.py`; move gauge registration out of `connect()`

Files:

Interfaces:


class KafkaEventBusMetrics:  # moved verbatim
    ...

def register_connection_gauge(
    meter: Any, metrics: KafkaEventBusMetrics, is_connected: Callable[[], bool]
) -> bool:
    """Register the connection gauge; return True if registered (current
    _connection_gauge_registered semantics). No-op returning False when meter is None."""

def register_consumer_lag_gauge(
    meter: Any, metrics: KafkaEventBusMetrics, lag_supplier: Callable[[], Any]
) -> bool: ...

Adjust the callable params to what the current gauge callbacks actually read off self β€” every self.X becomes an injected callable/value.


Task 13: Extract `KafkaConnectionManager`; move `KafkaRebalanceListener`; deprecate `record_*`

Files:

Interfaces:


class KafkaConnectionManager:
    def __init__(
        self,
        config: KafkaEventBusConfig,
        stats: KafkaEventBusStats,
        metrics: KafkaEventBusMetrics,
    ) -> None: ...

    @property
    def producer(self) -> AIOKafkaProducer | None: ...
    @property
    def consumer(self) -> AIOKafkaConsumer | None: ...
    @property
    def is_connected(self) -> bool: ...

    def require_producer(self) -> AIOKafkaProducer: ...
    async def connect(self) -> None: ...
    async def disconnect(self) -> None: ...
    async def cleanup(self) -> None: ...
    async def reconnect_consumer(self) -> None: ...

    def record_reconnection(self) -> None: ...   # real logic (stats + metrics), moved from the facade
    def record_rebalance(self, ...) -> None: ...  # copy exact current signature

    def record_reconnection(self) -> None:
        """Deprecated since 0.7.0; scheduled for removal in 0.8.0."""
        warnings.warn(
            "KafkaEventBus.record_reconnection() is deprecated and will be "
            "removed in 0.8.0; it was only ever intended for internal use.",
            DeprecationWarning,
            stacklevel=2,
        )
        self._connection_manager.record_reconnection()

(record_rebalance shim mirrors this, forwarding its args.)


Task 14: Extract `KafkaPublisher` (split-phase send/ack)

Files:

Interfaces:


class KafkaPublisher:
    def __init__(
        self,
        config: KafkaEventBusConfig,
        connection: KafkaConnectionManager,
        serializer: EventSerializer,
        stats: KafkaEventBusStats,
        metrics: KafkaEventBusMetrics,
        tracer: Tracer | None,
        enable_tracing: bool,
    ) -> None: ...

    async def publish_all(self, events: Sequence[DomainEvent]) -> None:
        """Split-phase: sequentially hand each event to producer.send() (preserving
        per-aggregate ordering via the aggregate_id partition key), then gather acks.
        Current publish-path semantics exactly, including the aiokafka plain-Future
        handling (add_done_callback with cancelled/exception checks)."""

Task 15: Extract `KafkaConsumerLoop`; remove `get_handlers_for_event`

Files:

Interfaces:


class KafkaConsumerLoop:
    def __init__(
        self,
        config: KafkaEventBusConfig,
        connection: KafkaConnectionManager,
        serializer: EventSerializer,
        stats: KafkaEventBusStats,
        metrics: KafkaEventBusMetrics,
        retry_policy: RetryPolicy,
        handlers_for: Callable[[type[DomainEvent]], tuple[HandlerAdapter, ...]],
        resolve_event_class: Callable[[str], type[DomainEvent] | None],
        tracer: Tracer | None,
        enable_tracing: bool,
        shutdown_event: asyncio.Event,
    ) -> None: ...

    @property
    def is_consuming(self) -> bool: ...
    async def start(self) -> None: ...
    async def stop(self) -> None: ...
    def start_in_background(self) -> asyncio.Task[None]: ...

Task 16: Extract `KafkaDLQAdmin`

Files:

Interfaces:


class KafkaDLQAdmin:
    def __init__(
        self,
        config: KafkaEventBusConfig,
        connection: KafkaConnectionManager,
        serializer: EventSerializer,
        stats: KafkaEventBusStats,
    ) -> None: ...

    async def get_messages(...) -> ...: ...      # copy exact public signatures from bus.py
    async def replay_message(...) -> ...: ...
    async def get_message_count(self) -> int: ...

Task 17: Slim the Kafka facade; backfill wiring tests

Files:

Interfaces:


Task 18: Release chores β€” mutmut targets, 0.7.0, CHANGELOG, ADR 0019 renumber + ADR 0020

Files:


# ADR 0020: Broker Backend Collaborator Decomposition

## Status

Accepted (2026-07-29)

## Context

RabbitMQEventBus (~3,570-line class) and KafkaEventBus (~2,195-line class)
each mixed connection lifecycle, topology declaration, serialization,
publish paths, consume/dispatch, retry/DLQ write paths, DLQ administration,
health checks, and stats in one object. Cross-cutting methods
(health_check, _process_message, _dispatch_to_handlers) read state from
four or five responsibility clusters, and ~1,500 lines of logic were
covered only through Docker integration tests.

## Decision

Each broker backend is a package of internal, state-owning collaborators
composed by a facade that keeps the public API:

- ConnectionManager owns broker clients, connected/reconnecting flags, and
  an explicit `on_reconnect(callback)` hook; other collaborators request
  live clients at call time and never capture them at construction.
- RabbitMQTopology owns exchange/queue objects and declaration; Kafka has
  no topology collaborator (config-driven β€” the asymmetry is real).
- Publisher and Consumer own their paths; the Consumer owns the retry/DLQ
  write path and receives `handlers_for` / `resolve_event_class` callables
  so the SubscriptionRegistry stays on the facade (ADR 0013, ADR 0010).
- DLQAdmin owns get/count/replay(/purge) administration.
- Pure logic (RabbitMQ death-header introspection, serialization) lives in
  plain modules.
- Collaborators are internal: nothing new is exported from
  `eventsource.bus` or `eventsource`. Facades keep every public signature;
  the only 0.7.0 API changes are removing the already-deprecated
  `KafkaEventBus.get_handlers_for_event` and deprecating
  `record_reconnection`/`record_rebalance`.
- Shutdown/drain ordering and health-check assembly are facade-owned;
  collaborators expose stop/close primitives and health slices.

## Alternatives Considered

- **Mixin split**: files shrink but every mixin still reads shared `self`
  state β€” coupling unchanged, only harder to see. Rejected.
- **Function-module extraction**: logic moves to parameterized functions
  but all state stays on one class with 15-20 fields. Rejected.
- **Shared cross-backend contracts** (common DLQAdmin/consumer pipeline
  interfaces implemented by Redis too): deferred, not rejected β€” revisit
  once both per-backend decompositions have settled.

## Consequences

- Collaborators are unit-testable with mocked broker objects; reconnect
  and shutdown wiring have direct unit coverage instead of Docker-only.
- Delivery, retry, and error-isolation semantics (ADR 0007, ADR 0011) are
  unchanged; the conformance suite passes unmodified.
- Internal collaborator APIs may change freely between releases.

Post-plan verification (orchestrator, not a task)