What I wanted to solve
A reconstruction can look convincing and still be wrong. I built SpatialForge so I could inspect every input, intermediate artifact, optimization, and rejection instead of judging the result by appearance alone.
What made it difficult
- The input contract includes calibrated frames and camera poses supplied by the dataset.
- Sparse allocation has to save memory without changing what a voxel update means.
- Reordering floating-point operations can change the final bytes even when the numerical difference looks negligible.
- Every artifact and validation report needs a traceable link back to its sensor data.
- Invalid or non-manifold output has to be rejected rather than written as a plausible-looking result.
How it works
- 1
Session format
A versioned .vgsession directory keeps RGB, depth, IMU, and supplied pose streams together.
- 2
Deterministic replay
Replay associates observations by timestamp and hashes the inputs into a digest for the scan.
- 3
Sparse block planner
The planner selects only the 8x8x8 voxel blocks reached by depth observations, plus their truncation halo.
- 4
Immutable depth context
Each depth frame is decoded once, outside the fusion hot path.
- 5
TSDF fusion
Fusion applies whole-block observation fields in a stable order. The dense and scalar implementations remain available as references.
- 6
Surface extraction
Surface extraction produces zero-crossing points. A mesh is kept only if it passes topology validation.
Choices I made
Keep a simple correctness reference
I kept the slower dense and scalar paths because they are easy to inspect. They give the sparse and vectorized implementations something trustworthy to compare against.
Preserve arithmetic order during vectorization
My first vectorized path was numerically close, but its bytes differed. I changed it to preserve expression and observation order, then made byte equality part of the test suite.
Bind artifacts to their inputs
I use SHA-256 provenance so an artifact cannot be quietly reused after its underlying session changes.
Treat explicit rejection as a valid outcome
A failed preflight or topology check is a useful result. I would rather reject the output than save a file that only looks correct.
How I tested it
- The suite runs 453 automated tests across 39 modules, with warnings treated as errors.
- Fast sparse contributions are compared byte for byte with the scalar reference.
- Held-out evaluation uses frames that contributed no voxels to the reconstructed volume.
- Tests also check that rejected operations leave the filesystem untouched.
Known failure modes
- The real-data mesher still produces non-manifold vertices, so validation rejects its output.
- Planning blocks still takes about two seconds for each 640x480 frame.
- Sparse volumes cannot yet be saved, resumed, rendered, or meshed.
- Keeping depth context in memory puts a ceiling on sequence length.
What works now
- Sparse fusion fell from about 210 seconds to 2 seconds on the documented small-room run, with output identical to the scalar path.
- The TUM freiburg1_xyz evaluation used 99 frames for fusion and held out another 98.
- Of 1,428,048 held-out depth samples, 99.8% landed inside observed voxels.
- Held-out TSDF residuals measured 9.3 mm at the median, 20.4 mm RMS, and 45.6 mm at p95.
What it doesn't do yet
- The 9.3 mm figure is a held-out TSDF residual with supplied ground-truth poses, not surveyed geometric accuracy.
- The project does not implement SLAM, visual odometry, bundle adjustment, pose estimation, or relocalization.
- It reconstructs geometry, not rooms, doors, accessibility, or other semantic map data.
- The real-data path is a working prototype rather than a complete indoor-mapping stack.