Sanish Kumar
← Back to selected work

(Natural-language commands for GIS)

VoiceGIS

VoiceGIS began with spoken commands for maps. It now focuses on the step between a request and an action: resolving real layers and fields, building a plan the host can inspect, and checking permissions again before anything runs. GeoJSON, OGC services, and MCP clients use that same path.

Role
Independent open-source project
Timeline
2026
Stack
JavaScript · Node.js · GIS · OGC API Features · CQL2 · MCP · Web Speech · Whisper

See it in action

VoiceGIS map of earthquakes alongside its compile, ground, authorize, and execute stages

Loads on request · 134 KB

The repository's reference integration: a map request is compiled against a layer catalog, checked against permissions, and executed with an inspectable result. The earthquake map is the demo UI, not the indoor-routing project.

What I wanted to solve

VoiceGIS began as a voice layer for maps. The harder problem turned out to be execution: how do you turn an ambiguous request into something an application can inspect before it changes state? The current core resolves language against real layers and fields, builds typed operations, checks permissions, and records what ran.

What made it difficult

  • A command must resolve against the application's actual layers, fields, aliases, and capabilities.
  • An attribute filter cannot turn into an unchecked SQL string.
  • A stale or tampered plan has to be rejected before the first side effect.
  • A risky operation may need confirmation from the user.
  • The host application keeps ownership of its data, maps, permissions, and transcription experience.
  • Speech providers and map libraries remain optional integrations.
  • Remote services must advertise the capabilities needed for a request; a page-bounded response cannot be reported as complete.

How it works

  1. 1

    Text or speech input

    Input can come from a text box, Web Speech, Whisper, a host speech hook, or another transcription source.

  2. 2

    Data-derived catalog

    GeoJSON catalogs inspect actual properties and geometry; OGC catalogs use collection, queryable-field, and conformance metadata. Unsupported operations are left out.

  3. 3

    Catalog-grounded compiler

    The compiler resolves a command against the layers, aliases, fields, units, and actions supplied by the host.

  4. 4

    Typed plan

    The output is a serializable plan with operations, predicate ASTs, risk, permissions, required capabilities, and confirmation requirements.

  5. 5

    Policy and preflight

    Preflight checks authorization, confirmation, the catalog version, the plan schema, and adapter capabilities.

  6. 6

    Application adapter

    The host can supply its own adapter or use the shipped GeoJSON and OGC API Features adapters, whose capabilities are checked before execution.

  7. 7

    Execution receipt

    The receipt records the status of each operation and can emit lifecycle events into the host's audit trail.

  8. 8

    MCP server

    The MCP server uses either an OGC service or one or more local GeoJSON files. It derives the catalog from the selected source and grants only view/query permissions by default.

Choices I made

Separate compilation from side effects

I separated planning from execution so the application can inspect, clarify, block, or request confirmation before an adapter operation runs.

Use typed predicates instead of generated SQL

Attribute and spatial conditions become checked AST nodes. The host adapter decides how to translate them for its own environment; VoiceGIS never emits SQL.

Validate again at execution time

Compilation is not a permanent approval. I check policy and catalog grounding again immediately before the first side effect, which catches a plan that became stale or was changed after compilation.

Keep the host application in control

The project ships GeoJSON and OGC API Features adapters, and the contract leaves room for other hosts. An integration has to advertise what it can actually execute; mentioning a map library or database is not the same as shipping an adapter for it.

Keep the core headless

The compiler, policy layer, adapter contract, and executor have zero runtime dependencies. Speech and the legacy map integrations sit outside that core.

Resolve places from a host-owned gazetteer

The built-in place resolver uses a list the application controls. Unknown places return suggestions instead of making an uncontrolled geocoding request or inventing a destination.

Infer only what the source can support

The GeoJSON catalog does not force mixed values into a convenient field type. Mixed-type fields remain untyped with a warning, nested values are omitted, and data without geometry cannot advertise spatial operations. The OGC adapter similarly requires the service to declare the filtering support it uses.

How I tested it

  • Before publishing, the release guard runs tests, lint, ESM and CommonJS builds, and declaration generation.
  • Changed layer IDs, unknown predicate fields, missing capabilities, unknown schemas, and stale catalog versions are rejected before an adapter runs.
  • Serializable plans carry versions, required capabilities, permissions, and the IDs of operations that need confirmation.
  • A historical evaluation artifact keeps 538 reproducible cases for the earlier rule-based parser.
  • The Playwright suite serves fixture responses and fails any unexpected outbound request, so the browser workflow can be tested without the network.
  • An opt-in live integration check exercises the OGC adapter against a public conformant service.
  • MCP file-mode tests cover multiple layers, derived catalogs, permissions, and local query execution. GeoJSON catalog tests include mixed properties and missing geometry.

Known failure modes

  • Ambiguous language returns needs_input rather than inventing a target.
  • A forbidden operation returns blocked.
  • An adapter without a required capability fails during preflight.
  • A changed catalog or tampered plan fails the execution-time check.
  • A valid transcript can still fall outside the compiler's language grammar.
  • An OGC service without filtering support fails capability preflight instead of receiving a query it cannot honour.

What works now

  • The compiler handles navigation, layer visibility, filtering, spatial selection, counts, buffers, exports, history, and adapter switching.
  • The defaults grant only view and query permissions. A host must opt into higher-risk capabilities.
  • Every execution returns a status for each operation and can emit lifecycle events.
  • A host can add domain language through resolvers without forking the compiler.
  • The earlier parser passed 504 of 538 stored cases, or 93.7%, in its separate evaluation artifact.
  • The published package ships ESM, CommonJS, and declarations with zero runtime dependencies.
  • An MCP client can query an OGC service or local GeoJSON files through the same plan, policy, and receipt flow. Local file mode does not need a data-service network connection.

What it doesn't do yet

  • VoiceGIS Core is a controlled command compiler, not a general conversational model.
  • GeoJSON and OGC sources have built-in catalog helpers and adapters. Other data systems still need an integration that implements the contract.
  • The package does not generate SQL, invent fields, or give itself permissions.
  • The 93.7% figure measures the earlier parser corpus, not the current catalog-grounded compiler.
  • It is also not microphone-to-action speech-recognition accuracy.
  • The OGC adapter has been validated against one conformant service, and its options may change as more implementations are tested.
  • The place resolver covers a bounded set of navigation phrasings and depends on a gazetteer supplied by the host.
  • OGC queries have bounded pagination and report truncation. A partial result is not a complete dataset.
  • The zero-runtime-dependency claim applies to the core package, not the MCP server and its SDK.