Problem
Browser speech capabilities vary sharply across browsers, privacy requirements, network conditions, and device performance. VoiceGIS separates map commands from speech engines so an application can use browser speech, on-device inference, a private Whisper server, or constrained command mode without rewriting its GIS integration.
Constraints
- Web Speech is convenient but browser support, offline behaviour, and data handling vary.
- On-device Whisper improves privacy but requires a large model download and significant CPU work.
- Leaflet and OpenLayers expose different map APIs.
- Natural phrases can be ambiguous even after transcription is correct.
- Custom application commands must coexist with built-in GIS intents.
- Parser evaluation must not be confused with end-to-end speech-recognition accuracy.
Architecture
- 1
Speech engine adapter
Normalizes browser speech, on-device Whisper, TF.js command mode, and private-server transcription.
- 2
Command parser
Maps normalized text into deterministic GIS intents, arguments, and confidence information.
- 3
Middleware chain
Lets applications inspect, block, enrich, or log a parsed command before execution.
- 4
Map adapter
Translates shared intents into Leaflet or OpenLayers operations.
- 5
History manager
Captures reversible map state for command-level undo and redo.
- 6
Offline evaluation harness
Runs a versioned command corpus in CI without microphone or network variance.
Design decisions
Adapter boundaries for both speech and maps
Speech engines vary in privacy, latency, and availability while map libraries vary in execution APIs. Keeping both behind contracts prevents either choice from spreading through command logic.
Evaluate the parser independently
A microphone benchmark mixes acoustic conditions, transcription quality, and intent parsing. The 538-case corpus isolates the deterministic parser so regressions can be reproduced exactly in CI.
Middleware before execution
Applications need domain-specific permissions, analytics, or confirmation around commands. An Express-style chain provides this without forking the library.
History at the command boundary
Voice interactions are easy to trigger accidentally. Capturing relevant map state before execution makes undo and redo part of the interaction model instead of an afterthought.
Correctness and testing
- The offline harness evaluates 538 parser cases spanning navigation, layers, markers, reset, map switching, conversational phrases, typos, and edge inputs.
- Evaluation results are stored as a reproducible project artifact and enforced through CI.
- Examples exercise both Leaflet and OpenLayers adapters.
- Custom commands, middleware, chaining, and history behaviour have dedicated tests and examples.
Failure cases
- Conversational and typo-heavy phrases remain more ambiguous than structured navigation commands.
- Browser speech may be unavailable, cloud-backed, or inconsistent across browsers.
- On-device Whisper can be too expensive for low-powered devices or slow connections.
- A correct transcript can still map to the wrong intent when commands overlap semantically.
Results
- 93.7% overall parser accuracy across 538 cases.
- 99.5% accuracy across the 387 navigation cases in the current evaluation set.
- A single library API supports Leaflet and OpenLayers through pluggable adapters.
- The package supports browser, on-device, constrained, and private-server speech profiles.
Limitations
The system is intentionally described within the boundaries supported by its implementation and evidence.
- The 93.7% figure measures parser cases, not microphone-to-action speech accuracy.
- The current TF.js engine is a limited command mode, not a purpose-trained VoiceGIS keyword-spotting model.
- The private Whisper engine is an HTTP client; hosting the transcription backend is outside the library.
- Web Speech support is strongest in Chromium-based browsers and cannot guarantee offline privacy.