See it in action
Loads on request · 1.6 MB
Recorded repository demo of the architecture-review workflow: base/head topology, semantic changes, findings, and a review decision. The displayed review is a demonstration, not evidence of a live customer's pull request.
More screenshots and results
The review-detail screen keeps the graph and decision evidence together. This manually imported example has no live pull-request merge gate.
What I wanted to solve
A Compose diff can add only a few lines and still change the architecture: a service becomes public, persistence is exposed to the host, a dependency crosses a trust boundary, or the downstream blast radius grows. I built System Synthesis to show those structural changes directly and decide whether the pull request introduced a policy violation.
What made it difficult
- The importer handles a bounded Docker Compose subset, not every feature in the Compose specification.
- Formatting, key order, revision labels, and moved lines must not look like architecture changes.
- An unrelated pull request should not fail only because the repository already has known architecture debt.
- The change under review cannot be allowed to weaken its own policy.
- Each finding and decision needs a source location, version history, and audit trail.
- A verified reviewer decision needs a linked GitHub identity and current repository access. An unconfigured deployment must not label its local decisions verified.
How it works
- 1
Bounded source adapters
The adapters import the documented part of each base and head Compose file and enforce syntax and size limits.
- 2
Canonical graphs
Services receive stable identities such as services.checkout. File and line metadata stay attached for review.
- 3
Semantic analysis
The analyzer compares graphs and calculates reachability, cycles, SCCs, articulation points, bridges, depth, blast radius, and trust crossings.
- 4
Deterministic policy
Policy compares the base and head findings, then blocks new violations using rules from the trusted base branch.
- 5
Review delivery
One analysis feeds JSON, Markdown, source-linked SARIF, pull-request comments, check status, and the browser review.
- 6
Audited decisions
With GitHub verification configured, the backend checks identity and current access, refuses self-approval by default, and stores decision and audit evidence in one transaction.
Choices I made
Stable source identities instead of line-based equality
I identify an entity by its source address, such as services.checkout. Revision and line details remain available to the reviewer, but they do not decide whether two entities are equal. A formatting-only change therefore stays formatting-only.
New findings fail; inherited findings remain visible
I compare the base and head finding sets. By default, a pull request is blocked for violations it adds; older findings stay visible without turning every later change red.
Policy comes from the trusted base revision
The workflow reads policy and secret-bearing Action code from the trusted base revision. Otherwise, the pull request being reviewed could disable its own check.
Deterministic findings before optional explanation
An LLM may help explain a result, but it cannot create, remove, or change the finding set produced by the rules.
Record importer and analyzer provenance
I store the importer and analyzer versions with each review. If either changes, an old verdict is no longer presented as current.
Verify the reviewer at decision time
On a GitHub-connected deployment, linking an account establishes identity and the decision gate checks current write access. Self-approval needs an explicit base-policy exception. If the configured verification service cannot establish access, the decision is refused; an unconfigured deployment can record only an unverified local decision.
Check who is allowed to connect a repository
A repository name is not proof of authority. Connection issuance requires administrator access tied to the caller's immutable GitHub account ID. The backend refreshes that proof after one hour and refuses ingestion when it cannot renew it. The scoped credential alone is not permanent permission.
How I tested it
- The repository-wide verify command builds every workspace and runs the unit suites used by the core, CLI, Action, backend, and frontend.
- Dedicated cases cover formatting invariance, stable identities, and base/head finding-set behaviour.
- Malformed YAML, duplicate keys, excessive aliases, and oversized inputs are rejected.
- Tests cover CLI exit codes plus the JSON, Markdown, and source-linked SARIF contracts.
- Decision tests cover current repository access, self-approval policy, collaborator pagination, unreachable GitHub, and missing App permissions.
- Six isolated consumer checks exercise the bundled Action outside the monorepo. Real-PostgreSQL tests cover synchronization, transactional decision audit, connection authority, and schema upgrades; CI fails if those database tests skip.
- The supporting modelling surface retains authorization tests and a seeded Yjs convergence test.
Known failure modes
- A Compose feature outside the documented subset stays outside the model; the importer does not guess at its meaning.
- If a configured Compose path matches no file, the check fails rather than passing an empty review.
- A stale browser decision cannot overwrite a newer revision of the review.
- A forked pull request without trusted credentials can run analysis without persisting a browser review.
- A configured GitHub verification failure blocks the decision. An unconfigured server can record an UNVERIFIED decision, not a verified approval.
- Expired or unverifiable repository-authority evidence blocks ingestion until the connection is revalidated.
What works now
- A review reports new dependencies, exposures, trust crossings, and blast-radius changes with file and line references.
- The CLI, pull-request comment, SARIF output, and browser all use the same analysis.
- Repository connections issue scoped credentials that are shown once and stored only as hashes.
- Verified decisions retain the identity and repository-access evidence used to authorize them, alongside explicit exception records.
- Existing findings remain visible but do not become new pull-request failures by default.
Collaborative architecture modelling
System Synthesis began as a multi-user graph editor. That editor now supports the main review workflow with modelling, inspection, durable history, and checked exports.
- Nested Yjs graph state supports user-local undo and is exercised by a seeded convergence test.
- Authenticated owner, editor, and viewer permissions across REST and Socket.IO mutations.
- PostgreSQL stores append-only, hash-deduplicated updates with replay and snapshot compaction.
- Redis Streams and pub/sub transport for multi-instance live distribution.
- Architecture linting and checked Compose or Terraform export work from the documented intermediate representation.
What it doesn't do yet
- The result is not a formal proof that the architecture is correct.
- System Synthesis is neither a complete Docker Compose evaluator nor an infrastructure-as-code replacement.
- Interpolation, includes, extends, profiles, and override merging are not currently resolved.
- A Kubernetes adapter exists in the core but is not yet exposed through the CLI, Action, or server.
- Browser reviews are owner-scoped, not a shared repository-wide review inbox.
- GitHub check publication is best effort: it has no durable retry outbox, and publication locking is local to one process.
- I have not measured production-scale capacity.