What 538 GIS parser tests actually measure

This is the earlier rule-based parser's evaluation, not a score for the current catalog-grounded compiler or MCP server. I keep it because the separation between language parsing, transcription, and execution still matters.

An end-to-end voice test mixes microphone quality, room noise, accent, browser support, transcription, parsing, and map execution. When one number covers all of that, a failure does not tell me which layer broke. VoiceGIS therefore tests its rule-based parser separately from speech recognition.

Each case provides text and the structured result I expect: an intent, its arguments, and any relevant flags. The corpus runs offline without a microphone, network connection, or model download.

An overall score is useful only if I can see what produced it. I group the 538 cases by behaviour so a strong navigation result cannot hide weak typo handling or conversational language.

The harness is an ordinary command. It writes a machine-readable result and exits non-zero when an enforced threshold or expected case regresses, so CI can treat parser behaviour like any other tested interface.

The stored result is 93.7%, but the categories are uneven. Navigation is strong because it has the largest and most structured set. Conversational phrases and typo resilience are materially weaker. Those gaps are useful; the aggregate score alone hides them.

There is another trap: tuning rules only for phrases already in the file. I add cases from observed ambiguity classes and review them for duplicates, unrealistic wording, and train-on-test behaviour.

The figure is parser accuracy over a versioned text corpus. That qualifier belongs beside the number: it is not word-error rate, microphone accuracy, real-user task completion, or a guarantee that every browser can transcribe speech.