FEBRUARY 1989 BENCHMARK TESTS

TEST MATERIAL: The February '89 Benchmark Tests are the first to
include previously unreleased material drawn from the Speaker
Dependent and Speaker Independent Evaluation Test sets.  Two sets
of test material were designated: one for Speaker Dependent systems
and another for Speaker Independent systems.  For the Speaker
Dependent tests, the test material includes 25 sentence utterances
from each of the 12 speakers in this portion of the corpus.  For
the Speaker Independent tests, the test material includes 30
sentence utterances from each of 10 speakers.  In each case, the
designated test material constitutes 300 sentence utterances. 

SPEAKER INDEPENDENT SYSTEM TRAINING: In addition to the previously
designated "standard 72-speaker training subset", an "augmented
109-speaker training set" was designated.  This augmented training
set consists of the "standard 72-speaker training set" plus 30
sentence utterances from each of 37 additional speakers from the
Speaker Independent Development Test portion of the Resource
Management Corpus, for a total of 3990 sentence utterances.  Use
of either (or both) of these training sets was at the participant's
discretion. 

LIVE TALKERS: No "live talker tests" were included in these tests. 

"HOMOPHONES CORRECT" OPTION IN SCORING: As in the June '88 tests,
substitution errors involving homophones were scored as "correct"
for the case of no imposed grammar.  

STANDARD WORD-PAIR GRAMMAR: The "word-pair" grammar, first used in
the June '88 tests, was again used as a "standard grammar".