--- source_path: "tools/snsr-eval.md" canonical_url: "https://doc.sensory.com/tnl/7.8/tools/snsr-eval/" --- # snsr-eval This tool evaluates / runs TrulyNatural SDK `snsr` model files. It supports all [task types](https://doc.sensory.com/tnl/7.8/api/setting-keys/configuration.md#task-type), except wake word [enrollment](https://doc.sensory.com/tnl/7.8/api/setting-keys/values.md#enroll) which is handled by [spot-enroll](https://doc.sensory.com/tnl/7.8/tools/spot-enroll.md#spot-enroll) and [live-enroll](https://doc.sensory.com/tnl/7.8/tools/live-enroll.md#live-enroll). **Also see these related items:** [snsr-eval.c](https://doc.sensory.com/tnl/7.8/api/sample/c/snsr-eval.md#snsr-evalc) ## Usage ``` Evaluates/runs TrulyNatural SDK .snsr model files. usage: snsr-eval -t task [options] [wavefile ...] options: -a : Add tpl-vad-lvcsr to LVCSR and STT models -d directory : VAD audio output directory -f setting filename : load filename into task setting -g setting value : load string into task setting -i listFile : run evaluation on each filename in listFile -l [-l [-l]] : reduce verbosity -o out : output filename for VAD audio or listFile results -p [-p] : Enable pipeline profiling (experimental) -q setting : query a task setting -s setting=value : override a task setting -t task : specify task filename (required) -u filename : remove unused settings and save model to filename -v [-v [-v]] : increase verbosity Use a filename of - to read headerless linear 16-bit PCM little-endian audio from stdin. If you don't specify any wave files, snsr-eval uses live audio captured from the default audio device. The -d and -o options are mutually exclusive. The output directory must be writable. Audio files created by VAD segmentation are named /.wav Settings are strings used as keys to query or change task behavior. Most frequently used are operating-point for wake words and command sets, leading-silence and trailing-silence for VAD templates, partial-result-interval for LVCSR and STT, and stt-profile for STT models. Refer to the TrulyNatural SDK documentation for a complete list and descriptions of all supported settings. ``` ## Batch processing If you specify the `-i listFile` option, `snsr-eval` will evaluate the model on the filenames in `listFile`. This loads the model once and re-uses the [session](https://doc.sensory.com/tnl/7.8/api/inference.md#session) instance for each evaluation, reducing overhead. It expects one filename per line. In batch processing mode, `snsr-eval` produces output in [tab-separated value][tsv] format. Each audio file in `listFile` has a corresponding result line in the output, unless the processing the audio file results in an error. Such errors are treated as warnings and printed to `stderr`. If you don't specify an output file with `-o` output goes to `stdout` instead. Output columns are, in order: * File index, starting at `1` * Audio filename * If there is a recognition [result](https://doc.sensory.com/tnl/7.8/api/setting-keys/events.md#result): * Start alignment in ms, [begin-ms](https://doc.sensory.com/tnl/7.8/api/setting-keys/results.md#begin-ms) * End alignment in ms, [end-ms](https://doc.sensory.com/tnl/7.8/api/setting-keys/results.md#end-ms) * Recognition score, [score](https://doc.sensory.com/tnl/7.8/api/setting-keys/results.md#score) * Result hypothesis, [text](https://doc.sensory.com/tnl/7.8/api/setting-keys/results.md#text) * If there is an [nlu-intent](https://doc.sensory.com/tnl/7.8/api/setting-keys/events.md#nlu-intent): * Intent name, [nlu-intent-name](https://doc.sensory.com/tnl/7.8/api/setting-keys/results.md#nlu-intent-name) * Intent score, [nlu-intent-score](https://doc.sensory.com/tnl/7.8/api/setting-keys/results.md#nlu-intent-score) * Intent value, [nlu-intent-value](https://doc.sensory.com/tnl/7.8/api/setting-keys/results.md#nlu-intent-value) * For each NLU entity found: * Entity name, [nlu-entity-name](https://doc.sensory.com/tnl/7.8/api/setting-keys/results.md#nlu-entity-name) * Entity score, [nlu-entity-score](https://doc.sensory.com/tnl/7.8/api/setting-keys/results.md#nlu-entity-score) * Entity value, [nlu-entity-value](https://doc.sensory.com/tnl/7.8/api/setting-keys/results.md#nlu-entity-value) ## Examples Fixed-phrase. ```console % snsr-eval -t ./model/spot-hbg-enUS-1.4.0-m.snsr hbg_2.wav hbg_7.wav 1200 1905 hello blue genie 3855 4575 hello blue genie ``` Fixed-phrase on default audio capture device. ```console % snsr-eval -v -t ./model/spot-hbg-enUS-1.4.0-m.snsr.snsr Using live audio from default capture device. ^C to stop. 3180 3885 (1.00 sv) hello blue genie 9000 9720 (1.00 sv) hello blue genie ^C ``` Enrolled user-defined phrase. ```console % snsr-eval -v -t ./three-users.snsr -s sv-threshold=0\ ./data/enrollments/armadillo-1-4-c.wav ./data/enrollments/armadillo-6-0.wav\ ./data/enrollments/terminator-2-5.wav ./data/enrollments/jackalope-1-4-c.wav 435 990 (0.89 sv) armadillo-1 5940 6630 (0.99 sv) terminator-2 8100 8610 (0.32 sv) jackalope-1 ``` Lower the speaker-verification threshold. ```console % snsr-eval -v -t ./three-users.snsr -s sv-threshold=0\ ./data/enrollments/jackalope-1-5.wav ./data/enrollments/jackalope-1-5-c.wav 270 840 (0.56 sv) jackalope-1 2130 2610 (0.33 sv) jackalope-1 ``` Recognize a list of audio files. ```console % find data -name \*.wav > audio-files.txt % snsr-eval -t model/stt-enUS-automotive-medium-2.3.15-pnc.snsr -o eval.tsv -i audio-files.txt Processing file 50 of 50, 100.00% ``` [tsv]: https://en.wikipedia.org/wiki/Tab-separated_values "Tab-separated values" *[API]: Application Programming Interface *[NLU]: Natural Language Understanding model *[SDK]: Software Development Kit *[TNL]: TrulyNatural, Sensory's large-vocabulary speech recognition technology