JUNE 1988 BENCHMARK TESTS

TEST MATERIAL: The June '88 Benchmark Tests differ from previous
and subsequent tests in that only one set of test material was
released.  This set of test material was designated for use by both
Speaker-Dependent and Speaker-Independent technologies.  It was
drawn from the Speaker Dependent Development Test set, and consists
of 25 sentence utterances from each of the 12 speakers in the
Speaker Dependent portion of the Resource Management Corpus. 

SPEAKER INDEPENDENT SYSTEM TRAINING: Because some of these speakers
also appear in the Speaker Independent portions of the Corpus, it
was agreed that Benchmark Tests would be conducted using a
designated "standard" 72-speaker subset of the Speaker Independent
Training material.  This "standard 72-speaker training set"
(consisting of 40 sentence utterances from each of the 72 speakers,
for a total of 2880 sentence utterances) does not include any of
the June '88 test speakers. 

LIVE TALKERS: No "live talker tests" were included in these tests. 

"HOMOPHONES CORRECT" OPTION IN SCORING: For the first time in the
series of DARPA Benchmark Tests, substitution errors involving
homophones were scored as "correct" for the case of no imposed
grammar.  This follows the precedent established in Kai-Fu Lee's
SPHINX dissertation research. 

STANDARD WORD-PAIR GRAMMAR: For the first time in this series of
DARPA Benchmark Tests, the non-probabilistic word-pair grammar
(developed at BBN, and used in Kai-Fu Lee's SPHINX research) was
designated for standard use to complement the no-grammar condition.