What I wanted to solve
A Compose diff can add only a few lines and still change the architecture: a service becomes public, persistence is exposed to the host, a dependency crosses a trust boundary, or the downstream blast radius grows. I built System Synthesis to show those structural changes directly and decide whether the pull request introduced a policy violation.
What made it difficult
- The importer handles a bounded Docker Compose subset, not every feature in the Compose specification.
- Formatting, key order, revision labels, and moved lines must not look like architecture changes.
- An unrelated pull request should not fail only because the repository already has known architecture debt.
- The change under review cannot be allowed to weaken its own policy.
- Each finding and decision needs a source location, version history, and audit trail.
- A merge decision must come from a reviewer who can be tied to a real GitHub identity and current repository access.
How it works
- 1
Bounded source adapters
The adapters import the documented part of each base and head Compose file and enforce syntax and size limits.
- 2
Canonical graphs
Services receive stable identities such as services.checkout. File and line metadata stay attached for review.
- 3
Semantic analysis
The analyzer compares graphs and calculates reachability, cycles, SCCs, articulation points, bridges, depth, blast radius, and trust crossings.
- 4
Deterministic policy
Policy compares the base and head findings, then blocks new violations using rules from the trusted base branch.
- 5
Review delivery
One analysis feeds JSON, Markdown, source-linked SARIF, pull-request comments, check status, and the browser review.
- 6
Audited decisions
The backend checks reviewer identity and repository access, refuses self-approval by default, and stores the decision history.
Choices I made
Stable source identities instead of line-based equality
I identify an entity by its source address, such as services.checkout. Revision and line details remain available to the reviewer, but they do not decide whether two entities are equal. A formatting-only change therefore stays formatting-only.
New findings fail; inherited findings remain visible
I compare the base and head finding sets. By default, a pull request is blocked for violations it adds; older findings stay visible without turning every later change red.
Policy comes from the trusted base revision
The workflow reads policy and secret-bearing Action code from the trusted base revision. Otherwise, the pull request being reviewed could disable its own check.
Deterministic findings before optional explanation
An LLM may help explain a result, but it cannot create, remove, or change the finding set produced by the rules.
Record importer and analyzer provenance
I store the importer and analyzer versions with each review. If either changes, an old verdict is no longer presented as current.
Verify the reviewer at decision time
Linking an account proves the GitHub identity; the decision gate then checks current write access to that repository. Self-approval is refused unless the trusted base policy names an explicit solo-maintainer or administrator exception.
How I tested it
- The repository-wide verify command builds every workspace and runs the unit suites used by the core, CLI, Action, backend, and frontend.
- Dedicated cases cover formatting invariance, stable identities, and base/head finding-set behaviour.
- Malformed YAML, duplicate keys, excessive aliases, and oversized inputs are rejected.
- Tests cover CLI exit codes plus the JSON, Markdown, and source-linked SARIF contracts.
- Decision tests cover current repository access, self-approval policy, collaborator pagination, unreachable GitHub, and missing App permissions.
- The supporting modelling surface retains authorization tests and a seeded Yjs convergence test.
Known failure modes
- A Compose feature outside the documented subset stays outside the model; the importer does not guess at its meaning.
- If a configured Compose path matches no file, the check fails rather than passing an empty review.
- A stale browser decision cannot overwrite a newer revision of the review.
- A forked pull request can run local analysis only when trusted secrets are unavailable.
- If GitHub access cannot be checked, the decision is recorded as unverified rather than presented as verified.
What works now
- A review reports new dependencies, exposures, trust crossings, and blast-radius changes with file and line references.
- The CLI, pull-request comment, SARIF output, and browser all use the same analysis.
- Repository connections issue scoped credentials that are shown once and stored only as hashes.
- Reviewer decisions are linked to identity and current repository access, with exceptions recorded distinctly.
- Existing findings remain visible but do not become new pull-request failures by default.
Collaborative architecture modelling
System Synthesis began as a multi-user graph editor. That editor now supports the main review workflow with modelling, inspection, durable history, and checked exports.
- Nested Yjs graph state supports user-local undo and is exercised by a seeded convergence test.
- Authenticated owner, editor, and viewer permissions across REST and Socket.IO mutations.
- PostgreSQL stores append-only, hash-deduplicated updates with replay and snapshot compaction.
- Redis Streams and pub/sub transport for multi-instance live distribution.
- Architecture linting and checked Compose or Terraform export work from the documented intermediate representation.
What it doesn't do yet
- The result is not a formal proof that the architecture is correct.
- System Synthesis is neither a complete Docker Compose evaluator nor an infrastructure-as-code replacement.
- Interpolation, includes, extends, profiles, and override merging are not currently resolved.
- A Kubernetes adapter exists in the core but is not yet exposed through the CLI, Action, or server.
- I have not measured production-scale capacity.