A line diff was the wrong unit
A pull request can change an architecture with only a few lines: expose a service, add a dependency, remove a health check, or open a path across a trust boundary. The reverse happens too. Reformatting or moving a service block can create a large diff without changing the architecture at all.
I treat the review as a comparison between two graphs. System Synthesis imports each revision, gives known entities stable identities such as services.checkout, runs graph and policy checks, and compares the resulting states. The question is what changed in the model, not where the YAML moved.
Keep the importer small
The importer understands a documented subset of Docker Compose. It does not try to evaluate every feature, extension, runtime default, or deployment environment. That boundary matters because the tool needs to tell the difference between an input it can interpret and one it cannot.
Inside that subset, canonicalization removes incidental choices. Stable service identities, normalized fields, and fixed ordering keep whitespace, key order, and line movement from creating fake architecture changes. Anything outside the model is reported rather than guessed.
base.yml -> import -> canonical base graph
head.yml -> import -> canonical head graph
semantic diff = compare(base graph, head graph)
findings = policy(head graph) - acceptedBaseline(base graph)Ask questions that need a graph
Once services and relationships are in a stable graph, I can ask structural questions. Reachability shows which components can now connect. Strongly connected components and cycle analysis expose circular dependencies. Articulation points, bridges, blast radius, and trust crossings show where one change can affect the rest of the model.
None of those algorithms can decide that a design is good. They give a reviewer a repeatable finding, its source location, and the base/head change that produced it. That is more useful than an unexplained risk score.
- Added, removed, and changed services and relationships.
- New reachability paths and trust-boundary crossings.
- New cycles or changes to strongly connected components.
- Changed articulation points, bridges, depth, and modeled blast radius.
- Rule-based policy findings linked back to their source.
Block only what the pull request adds
A repository may already contain known violations. If every pull request fails for the full historical set, people reach for blanket suppressions. System Synthesis compares the base and head findings and blocks only the relevant violations introduced by the proposed change.
Policy comes from the trusted base branch, not arbitrary pull-request code. Scoped exceptions keep their provenance and audit history. Stale-review protection also prevents an approval for one head revision from being reused after the change moves, and policy runs again at the decision boundary.
Run one analysis, then present it several ways
The analysis core feeds both a CLI and a Node 24 GitHub Action. A result can become JSON, Markdown, a pull-request comment, source-linked SARIF, or a browser review. Those are several views of one analysis, not separate rule implementations that can drift apart.
The repository-wide verification builds every workspace and runs the unit suites. In addition to graph and importer behaviour, the decision tests cover reviewer permissions, self-approval policy, collaborator pagination, an unreachable GitHub API, and missing GitHub App permissions.
The collaborative editor is still there
System Synthesis still includes the collaborative architecture editor I built with Yjs, authenticated board mutations, PostgreSQL durability, and Redis-backed transport. It remains useful for inspecting or editing a model together and keeps its own convergence and authorization tests.
It is now a supporting workflow rather than the product's main story. Pull-request architecture review comes first; collaborative modelling is another way to work with the graph.
What a graph review cannot know
The diff is only as complete as the importer and policy model. Dynamic infrastructure, runtime discovery, application-level call paths, generated configuration, and Compose behaviour outside the documented subset may never enter the graph. A true rename can also be hard to identify without enough information in the source.
Graph properties are signals, not business context. An articulation point may be intentional, a cycle may be harmless, and a tiny change may be operationally critical for reasons the model cannot see. The tool can focus a review and make findings repeatable; it cannot replace architecture, security, or operations judgement.
- Formatting invariance applies to the documented canonical model, not arbitrary infrastructure syntax.
- Rule-based findings can still encode an incomplete or unsuitable policy.
- A source link explains why a rule fired but does not prove production impact.
- Human review remains necessary for intent, business criticality, and runtime behaviour outside the model.