Skip to content

fix(#5342): storage capability matrix + arch-guard plugin boundary rules - #5348

Merged
qqeasonchen merged 1 commit into
developfrom
fix/5342-storage-arch-guard
Sep 8, 2026
Merged

fix(#5342): storage capability matrix + arch-guard plugin boundary rules#5348
qqeasonchen merged 1 commit into
developfrom
fix/5342-storage-arch-guard

Conversation

@qqeasonchen

@qqeasonchen qqeasonchen commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

What this PR does

Closes #5342 (Q6 storage capability coverage + Q7 architecture guardrails).

Q6 -- Storage capability coverage

  • New docs/storage-spi.md -- 7-capability x 3-plugin matrix, TCK contract, plugin-load-time semantics, future work. The matrix is the single source of truth for which backend implements which capability.
  • Verified: every storage plugin ships a MeshStoragePluginTCKTest (kafka / rocketmq / rocketmq5) and is included in CI by default.

Q7 -- Architecture guardrails

  • architecture-guard.yml already runs on every relevant PR (paths include eventmesh-storage-plugin/**, eventmesh-architecture-guard/**, plus the 4 other analysed modules). Confirmed by reading the workflow.
  • New ArchitectureRules.ruleStoragePluginsIsolated -- kafka plugin must not depend on rocketmq or rocketmq5 plugin packages (and vice versa).
  • New ArchitectureRules.ruleStoragePluginsDependOnlyOnApi -- storage plugins must not depend on eventmesh.runtime.. or eventmesh.connector.runtime...
  • New ArchitectureRulesTest.ruleStoragePluginsIsolated_check + ruleStoragePluginsDependOnlyOnApi_check -- FAIL mode (rule.check(classes)).
  • New FakeStorageCanary + ruleStoragePluginsIsolated_catches test -- intentional-violation proof (mirrors FakePluginCanary / ruleConnectorPluginsDependOnlyOnSpiCatchesViolations).
  • eventmesh-architecture-guard/build.gradle -- pulls in eventmesh-storage-kafka as testImplementation so the new rules have classes in scope.
  • New docs/architecture-guard.md -- canonical reference for all 12 ArchUnit rules (5 common + 4 runtime + 3 plugin SPI), with how-to-add-a-new-rule instructions.
  • New CONTRIBUTING.md -- contributor entry point, links docs/architecture-guard.md.

Acceptance checklist

Q6 plugin-load-time validation -- explicitly out of scope

The current StorageCapabilities design is runtime instanceof + compile-time TCK, not SPI-loader-time. Adding a load-time capability gate requires changing the EventMeshSPI loader to reflect over implements clauses -- future work, listed in the doc.

Verified locally

Patch-only plumbing; CI is the authority. The startup_failure on the apache/eventmesh GitHub Actions platform is a repository-level issue affecting every PR (see PR #5345 / #5346), not PR-introduced.

Co-authored-by: qqeasonchen qqeasonchen@gmail.com

This was referenced Sep 8, 2026
Closed
Merged
Closed
@qqeasonchen
fix(#5342): storage capability matrix + arch-guard plugin boundary rules
12d3916 
Closes #5342 (Q6 storage capability coverage + Q7 architecture guardrails).

Adds:
- docs/storage-spi.md: 7-capability x 3-plugin matrix, TCK contract, future work
- docs/architecture-guard.md: canonical reference for all 12 ArchUnit rules
- CONTRIBUTING.md: contributor entry point, links arch-guard doc
- ArchitectureRules.ruleStoragePluginsIsolated: kafka plugin must not depend on rocketmq*
- ArchitectureRules.ruleStoragePluginsDependOnlyOnApi: storage plugin must not depend on eventmesh.runtime.*
- ArchitectureRulesTest: 2 corresponding *_check tests + canary-catches test
- FakeStorageCanary: intentional-violation canary mirroring FakePluginCanary
- build.gradle: pull in eventmesh-storage-kafka for canary sampling

Co-authored-by: qqeasonchen <qqeasonchen@gmail.com>

Retry-CI: Retry-CI-after-outage-1788836620
@qqeasonchen
qqeasonchen force-pushed the fix/5342-storage-arch-guard branch from 3cfc948 to 12d3916 Compare September 8, 2026 03:03
qqeasonchen added a commit that referenced this pull request Sep 8, 2026
@qqeasonchen
docs(#5337): add architecture-review evidence table for #5296 sub-issues
072e8e3 
Closes #5337 (Q1 production validation evidence + Q8 closure criteria).

Links every closed sub-issue of #5296 to its PR, squash-commit hash, test
files, test command, CI run, backend/version, deployment topology, and
observed result. The 7 reliability scenarios called out in #5337 (Q1)
are answered in the "Scenarios" section with concrete PR + test refs.

Status legend distinguishes MERGED (PR landed; CI on record before the
Actions platform outage) from OPEN-CI-startup_failure (PR open; CI
blocked by a repository-level runner availability issue affecting all
workflows since 2026-09-07 16:23 UTC). 19 PRs are catalogued, of which
16 are MERGED and 3 are OPEN-CI-startup_failure (#5345 / #5346 / #5348,
all blocked on the same platform issue).

Co-authored-by: qqeasonchen <qqeasonchen@gmail.com>

Retry-CI: Retry-CI-after-outage-1788836620
@qqeasonchen
qqeasonchen merged commit c0ffb6e into develop Sep 8, 2026
1 check passed
@qqeasonchen
qqeasonchen deleted the fix/5342-storage-arch-guard branch September 8, 2026 03:11
qqeasonchen added a commit that referenced this pull request Sep 8, 2026
@qqeasonchen
docs(#5337): add architecture-review evidence table for #5296 sub-iss…
b08e591 
…ues (#5349)

Closes #5337 (Q1 production validation evidence + Q8 closure criteria).

Links every closed sub-issue of #5296 to its PR, squash-commit hash, test
files, test command, CI run, backend/version, deployment topology, and
observed result. The 7 reliability scenarios called out in #5337 (Q1)
are answered in the "Scenarios" section with concrete PR + test refs.

Status legend distinguishes MERGED (PR landed; CI on record before the
Actions platform outage) from OPEN-CI-startup_failure (PR open; CI
blocked by a repository-level runner availability issue affecting all
workflows since 2026-09-07 16:23 UTC). 19 PRs are catalogued, of which
16 are MERGED and 3 are OPEN-CI-startup_failure (#5345 / #5346 / #5348,
all blocked on the same platform issue).

Co-authored-by: qqeasonchen <qqeasonchen@gmail.com>

Retry-CI: Retry-CI-after-outage-1788836620
qqeasonchen added a commit that referenced this pull request Sep 8, 2026
@qqeasonchen
fix(#5356): fail fast on PARTITION_OWNED_PULL without shared MetaStore
4d05581 
Phase 0 enforcement of the production-HA plan (#5354): removes both
silent InMemoryMetaStore fallbacks, adds two ArchUnit guardrails so the
regression cannot return, and repairs three compile/test breaks that
landed during the 2026-09-07/08 Actions outage blind merges.

Boot changes (the #5356 core):
- UniRuntime.startPartitionOwnership: clusterMeta == null now throws
  IllegalStateException ("PARTITION_OWNED_PULL requires a shared
  MetaStore...") instead of new InMemoryMetaStore(). A misconfigured
  cluster no longer looks healthy while every instance polls every
  partition (duplicate consumption, CAS/fencing ineffective).
- EventMeshApplication: in cluster mode an unsupported meta type now
  fails fast instead of silently falling back to an isolated store.
  Single-instance mode keeps the documented in-memory store.

ArchUnit guardrails (12 -> 14 rules):
- ruleInMemoryMetaStoreOnlyFromBoot: only the boot package may depend
  on InMemoryMetaStore.
- rulePartitionOwnershipOnlyFromBootAndCluster: PartitionOwnership is
  visible only to boot / cluster / ingress / admin (read-only view).

Outage-blind-merge repairs (found by local CI run):
- EventMeshApplication was missing the DeliveryTopology import
  (#5344): eventmesh-runtime:compileJava failed outright on develop.
- FakeStorageCanary (#5348) imported a rocketmq5 class but the guard
  module did not depend on rocketmq5: added the test dependency plus
  an lz4-java capability resolution (org.lz4 vs at.yawk.lz4 both
  publish the org.lz4:lz4-java capability).
- FakeStorageCanary lived in storage.fakeplugin.. while
  ruleStoragePluginsIsolated selects storage.kafka..: the canary could
  never match, so ruleStoragePluginsIsolated_catches always failed.
  Moved the canary into the storage.kafka test package.
- ClusterSubscriptionStore (#5345) left empty topic buckets in the
  cache after the last subscriber of a topic was removed: topics()
  returned ghost topics forever (StateStoreDurabilityTest Scenario 2
  failed). Buckets are now dropped when they become empty.

Tests (all green locally, Temurin 21.0.11):
- :eventmesh-architecture-guard:test  -> BUILD SUCCESSFUL
- :eventmesh-runtime:test              -> BUILD SUCCESSFUL (~full suite)
- new: partitionOwnedPullWithoutMetaStoreFailsFast + 2 rule *_check

Refs #5356 (sub-issue of #5354).

Co-authored-by: qqeasonchen <qqeasonchen@gmail.com>
@qqeasonchen qqeasonchen mentioned this pull request Sep 8, 2026
Merged
qqeasonchen added a commit that referenced this pull request Sep 8, 2026
@qqeasonchen
fix(#5356): fail fast on PARTITION_OWNED_PULL without shared MetaStore
f374931 
Phase 0 enforcement of the production-HA plan (#5354): removes both
silent InMemoryMetaStore fallbacks, adds two ArchUnit guardrails, and
repairs the compile/test/checkstyle breaks that landed on develop via
the 2026-09-07/08 Actions-outage blind merges (all found by running
the full CI build locally).

Boot changes (the #5356 core):
- UniRuntime.startPartitionOwnership: clusterMeta == null now throws
  IllegalStateException instead of new InMemoryMetaStore().
- EventMeshApplication: unsupported meta type in cluster mode fails
  fast; single-instance mode keeps the documented in-memory store.

ArchUnit guardrails (12 -> 14 rules):
- ruleInMemoryMetaStoreOnlyFromBoot
- rulePartitionOwnershipOnlyFromBootAndCluster (boot/cluster/ingress/admin)

Outage blind-merge repairs:
- missing DeliveryTopology import (#5344): compileJava failed on develop
- guard module missing rocketmq5 dep + lz4-java capability conflict
  (org.lz4 vs at.yawk.lz4) for FakeStorageCanary (#5348)
- FakeStorageCanary in storage.fakeplugin.. never matched the
  ruleStoragePluginsIsolated that-clause (#5348): moved to
  storage.kafka test package
- ClusterSubscriptionStore left empty topic buckets after last
  subscriber removal (#5345): topics() returned ghosts; buckets now
  dropped when empty
- checkstyle (maxWarnings=0) violations in #5345/#5346 test files:
  unused imports (MetaListener, StandardCharsets, TaskStore, MetaStore,
  OffsetStore), import order (A2AMessageTransport, org.junit before
  io.cloudevents), local var names (aView/bView -> viewA/viewB),
  declaration-usage distance (reaper x2, SessionRegistry b)

Full local CI parity (Temurin 21.0.11, same tasks as .github/workflows):
- ./gradlew clean generateGrammarSource            -> BUILD SUCCESSFUL
- ./gradlew :eventmesh-architecture-guard:architectureCheck --no-daemon
                                                  -> BUILD SUCCESSFUL
- ./gradlew clean build dist jacocoTestReport --parallel --daemon
    -x spotlessJava -x generateGrammarSource -x generateDistLicense
    -x checkDeniedLicense -x :eventmesh-architecture-guard:test
                                                  -> BUILD SUCCESSFUL (511 tasks, 10m47s)
- ./gradlew installPlugin                         -> BUILD SUCCESSFUL

Refs #5356 (sub-issue of #5354).

Co-authored-by: qqeasonchen <qqeasonchen@gmail.com>
qqeasonchen added a commit that referenced this pull request Sep 8, 2026
@qqeasonchen
fix(#5356): fail fast on PARTITION_OWNED_PULL without shared MetaStor…
3dffe95 
…e + arch-guard (#5367)

Phase 0 enforcement of the production-HA plan (#5354): removes both
silent InMemoryMetaStore fallbacks, adds two ArchUnit guardrails, and
repairs the compile/test/checkstyle breaks that landed on develop via
the 2026-09-07/08 Actions-outage blind merges (all found by running
the full CI build locally).

Boot changes (the #5356 core):
- UniRuntime.startPartitionOwnership: clusterMeta == null now throws
  IllegalStateException instead of new InMemoryMetaStore().
- EventMeshApplication: unsupported meta type in cluster mode fails
  fast; single-instance mode keeps the documented in-memory store.

ArchUnit guardrails (12 -> 14 rules):
- ruleInMemoryMetaStoreOnlyFromBoot
- rulePartitionOwnershipOnlyFromBootAndCluster (boot/cluster/ingress/admin)

Outage blind-merge repairs:
- missing DeliveryTopology import (#5344): compileJava failed on develop
- guard module missing rocketmq5 dep + lz4-java capability conflict
  (org.lz4 vs at.yawk.lz4) for FakeStorageCanary (#5348)
- FakeStorageCanary in storage.fakeplugin.. never matched the
  ruleStoragePluginsIsolated that-clause (#5348): moved to
  storage.kafka test package
- ClusterSubscriptionStore left empty topic buckets after last
  subscriber removal (#5345): topics() returned ghosts; buckets now
  dropped when empty
- checkstyle (maxWarnings=0) violations in #5345/#5346 test files:
  unused imports (MetaListener, StandardCharsets, TaskStore, MetaStore,
  OffsetStore), import order (A2AMessageTransport, org.junit before
  io.cloudevents), local var names (aView/bView -> viewA/viewB),
  declaration-usage distance (reaper x2, SessionRegistry b)

Full local CI parity (Temurin 21.0.11, same tasks as .github/workflows):
- ./gradlew clean generateGrammarSource            -> BUILD SUCCESSFUL
- ./gradlew :eventmesh-architecture-guard:architectureCheck --no-daemon
                                                  -> BUILD SUCCESSFUL
- ./gradlew clean build dist jacocoTestReport --parallel --daemon
    -x spotlessJava -x generateGrammarSource -x generateDistLicense
    -x checkDeniedLicense -x :eventmesh-architecture-guard:test
                                                  -> BUILD SUCCESSFUL (511 tasks, 10m47s)
- ./gradlew installPlugin                         -> BUILD SUCCESSFUL

Refs #5356 (sub-issue of #5354).
qqeasonchen added a commit that referenced this pull request Sep 8, 2026
@qqeasonchen
fix(#5358): QuotaHandle (AutoCloseable) + gate-scoped QuotaManager ac…
b4592a1 
…cess

Phase 0 enforcement of the production-HA plan (#5354), sub-issue
#5358: gauge-style quota resources (CONNECTIONS / SUBSCRIPTIONS /
BACKLOG) leak when acquire has no guaranteed release. Production
callers today (UniHttpServer x2, A2AGatewayHttpHandler,
ConnectorScheduler) all use one-shot THROUGHPUT check() and never
release - correct for windows, but nothing structural stops the next
gauge-style call site from leaking the same way.

New API (security.gate):
- QuotaHandle implements AutoCloseable: close() releases exactly once,
  double-close is a no-op, non-releasable (THROUGHPUT) handles do not
  release (window counters self-expire)
- SecurityGate.acquire(ctx, frame) -> QuotaHandle: runs the same
  auth/ACL-first check() then returns the paired handle; throws
  QuotaExceededException (new) when exhausted, consuming no slot
- callers wrap gauge work in try-with-resources so the release runs
  on success, failure and cancellation alike

ArchUnit guardrail (15 -> 16 rules):
- ruleQuotaManagerOnlyFromGate: QuotaManager.tryAcquire/release may
  only be called from the security.gate package - any direct manager
  call outside the gate bypasses the pairing contract and fails the
  build

Tests:
- QuotaHandleTest (new, 4 cases): close-releases-exactly-once,
  try-with-resources releases on exception, THROUGHPUT handle does
  not release, exhausted acquire throws without consuming
- checkstyleMain/Test + guard tests + full CI build all green

Rebased onto develop after #5368 (rule ordering in
ArchitectureRules.java); the 16-rule set now reads
#5322(9) + #5348(2) + #5356(2) + #5357(1) + #5358(1).

Full local CI parity (Temurin 21.0.11):
- checkstyle + tests x (runtime, architecture-guard) -> BUILD SUCCESSFUL
- clean build dist jacocoTestReport (CI task set)   -> BUILD SUCCESSFUL (7m59s)

Refs #5358 (sub-issue of #5354).

Co-authored-by: qqeasonchen <qqeasonchen@gmail.com>
qqeasonchen added a commit that referenced this pull request Sep 8, 2026
@qqeasonchen
fix(#5358): QuotaHandle (AutoCloseable) + gate-scoped QuotaManager ac…
d5200c8 
…cess (#5369)

Phase 0 enforcement of the production-HA plan (#5354), sub-issue
#5358: gauge-style quota resources (CONNECTIONS / SUBSCRIPTIONS /
BACKLOG) leak when acquire has no guaranteed release. Production
callers today (UniHttpServer x2, A2AGatewayHttpHandler,
ConnectorScheduler) all use one-shot THROUGHPUT check() and never
release - correct for windows, but nothing structural stops the next
gauge-style call site from leaking the same way.

New API (security.gate):
- QuotaHandle implements AutoCloseable: close() releases exactly once,
  double-close is a no-op, non-releasable (THROUGHPUT) handles do not
  release (window counters self-expire)
- SecurityGate.acquire(ctx, frame) -> QuotaHandle: runs the same
  auth/ACL-first check() then returns the paired handle; throws
  QuotaExceededException (new) when exhausted, consuming no slot
- callers wrap gauge work in try-with-resources so the release runs
  on success, failure and cancellation alike

ArchUnit guardrail (15 -> 16 rules):
- ruleQuotaManagerOnlyFromGate: QuotaManager.tryAcquire/release may
  only be called from the security.gate package - any direct manager
  call outside the gate bypasses the pairing contract and fails the
  build

Tests:
- QuotaHandleTest (new, 4 cases): close-releases-exactly-once,
  try-with-resources releases on exception, THROUGHPUT handle does
  not release, exhausted acquire throws without consuming
- checkstyleMain/Test + guard tests + full CI build all green

Rebased onto develop after #5368 (rule ordering in
ArchitectureRules.java); the 16-rule set now reads
#5322(9) + #5348(2) + #5356(2) + #5357(1) + #5358(1).

Full local CI parity (Temurin 21.0.11):
- checkstyle + tests x (runtime, architecture-guard) -> BUILD SUCCESSFUL
- clean build dist jacocoTestReport (CI task set)   -> BUILD SUCCESSFUL (7m59s)

Refs #5358 (sub-issue of #5354).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Architecture Review] Storage capability coverage and architecture guardrail enforcement

1 participant

@qqeasonchen