Broker Backend God-Class Decomposition β€” Design

Date: 2026-07-29 Status: Approved Target version: 0.7.0 Predecessor: 2026-07-29-event-bus-contract-and-coverage-design.md (shipped as PR #68 / 0.6.0), which deliberately deferred this work.

Problem

RabbitMQEventBus (4,228-line module; the class itself ~3,570 lines) and KafkaEventBus (3,141-line module; class ~2,195 lines) are god classes. Each mixes connection lifecycle, topology declaration, serialization, publish paths, consume/dispatch, retry/DLQ write paths, DLQ administration, health checks, and stats into one object with 15-20 instance fields. Symptoms:

Decisions

# Decision Choice
1 Scope RabbitMQ and Kafka, decomposed along backend-private seams. No new cross-backend abstractions this cycle (explicitly deferred, not rejected). bus/redis.py untouched.
2 Pattern Collaborator composition: stateful collaborators own their domain; each *EventBus becomes a facade that composes and delegates. Rejected: mixin split (coupling unchanged), function-module extraction (god object survives).
3 Exposure Collaborators are internal. Nothing new is exported from bus/__init__.py or eventsource/__init__.py.
4 Compatibility Minor public breakage allowed under 0.7.0 β€” spent on exactly three Kafka items (see API Changes). Everything else keeps exact public signatures and behavior.
5 Layout Backend subpackages bus/rabbitmq/ and bus/kafka/; package __init__.py re-exports everything the old flat module exported, so all existing imports keep working verbatim.
6 Testing Full backfill: new unit tests per collaborator, plus unit backfill for facade paths currently covered only by integration tests (reconnect, shutdown ordering, health composition). Pure/logic-heavy modules join mutmut targets.

Package Layout


bus/rabbitmq/                            bus/kafka/
  __init__.py     re-exports               __init__.py     re-exports
  config.py       RabbitMQEventBusConfig   config.py       KafkaEventBusConfig
  models.py       Stats, DLQMessage,       models.py       KafkaEventBusStats,
                  QueueInfo,                               DeserializationError,
                  HealthCheckResult,                       KafkaNotAvailableError
                  RabbitMQNotAvailableError,
                  ShutdownError,
                  BatchPublishError
  connection.py   ConnectionManager        connection.py   ConnectionManager +
                                                           KafkaRebalanceListener
  topology.py     Topology                 metrics.py      KafkaEventBusMetrics +
                                                           gauge registration
  serialization.py serialize/deserialize   serialization.py EventSerializer
                  + AMQP message building                  (moves as-is)
  publisher.py    publish/batch paths      publisher.py    send/ack split-phase
  consumer.py     consume loop, dispatch,  consumer.py     consume loop, dispatch,
                  retry + DLQ write path                   retry + DLQ write path
  dlq.py          DLQAdmin (get/count/     dlq.py          DLQAdmin (get/count/
                  replay/purge)                            replay; throwaway-
                                                           consumer helper)
  death_headers.py 7 pure header functions bus.py          KafkaEventBus facade
  bus.py          RabbitMQEventBus facade

State Ownership & Wiring

Every instance field moves to exactly one owner. The only deliberately shared mutable objects are the stats dataclass and the ConnectionManager.

Public API Changes (0.7.0)

Exactly three, all Kafka:

  1. Remove KafkaEventBus.get_handlers_for_event β€” a DeprecationWarning shim since 0.6.0; this is its scheduled exit.
  2. Internalize record_reconnection / record_rebalance β€” logic moves into the connection/metrics collaborators. The facade keeps one-release deprecation shims (warn + delegate), removed in 0.8.0.
  3. KafkaRebalanceListener moves to bus/kafka/connection.py without a shim β€” it was never exported from bus/__init__.py or the top level.

Everything else preserves exact public signatures and behavior, including: bind_event_type, bind_routing_key, stats accessors, DLQ admin methods, get_queue_info, health_check, is_shutdown, start_consuming_in_background, get_topic_info, the death-header statics, and all dataclass/exception import paths.

pyproject.toml β†’ 0.7.0. CHANGELOG documents the three changes.

Testing Strategy (full backfill)

Layer 1 β€” behavior safety net (must pass unmodified): the 9-test EventBusConformanceSuite subclasses for all four backends, and every integration test that exercises only public API. Exceptions, expected to change because they pin private internals:

Layer 2 β€” collaborator unit tests in tests/unit/bus/rabbitmq/ and tests/unit/bus/kafka/ (mirroring the packages), with mocked broker objects: topology declare/redeclare idempotence; reconnect-hook firing order; retry/DLQ write-path decisions; DLQ admin paging/replay; death-header functions (pure β€” property tests where cheap); Kafka gauge registration; publisher batch strategies.

Layer 3 β€” facade-path backfill: unit tests for wiring currently proven only via Docker: reconnect re-declares topology and resumes consuming; shutdown ordering (stop β†’ drain in-flight β†’ drain background β†’ disconnect); health_check composition; stats accumulation across collaborators. All against mocked ConnectionManagers.

Mutation targets: death_headers.py, both serialization.py modules, both consumers' retry-decision logic, and β€” clearing last cycle's deferred item β€” bus/memory.py join mutmut only_mutate.

Coverage gate stays fail_under = 87; expected to rise as Docker-only logic gains unit coverage. Wall-clock perf assertions remain excluded from CI via the benchmark marker (unchanged from 0.6.0).

Risks

ADR Impact

ADR Status vs this work
0001 async-first design Stands.
0007 event bus delivery semantics Stands β€” delivery behavior untouched.
0010 uniform event bus contract Stands β€” facades still inherit BaseEventBus; registry, background semantics, drain hooks stay on the facade.
0011 handler-error isolation, no-ack Stands β€” consumers keep HandlerDispatchError/no-ack semantics bit-for-bit.
0013 handler registry composition Stands β€” this work extends its composition principle to backend internals.
0015 optional dependency extras Stands β€” guard pattern moves to package __init__.py, observable behavior identical.
0016 optional tracing no-op by default Stands β€” tracer injection preserved via constructor passing.
New: 0020 broker backend collaborator decomposition Records the facade + state-owning-collaborators pattern, internal (non-exported) seams, the reconnect hook, and rejected alternatives (mixins; function-module extraction; shared cross-backend contracts β€” deferred, not rejected).

Housekeeping folded in: docs/adrs/ currently has two 0009- files (0009-multi-instance-subscription-coordination.md, 0009-postgresql-advisory-locks.md) β€” a collision PR #73 did not cover. Both landed in the same commit, so age does not disambiguate; 0009-postgresql-advisory-locks.md renumbers to 0019 (the subscription-coordination ADR is bus-domain and keeps 0009). index.md and inbound references update; ADR bodies untouched. The new decomposition ADR then takes 0020.

Non-Goals