OCTOBER 1989 BENCHMARK TESTS

TEST MATERIAL: The October '89 Benchmark Tests, like the February
'89 Benchmark Tests, include previously unreleased material drawn
from the Speaker Dependent and Speaker Independent Evaluation Test
sets.  One set of 300 sentence utterances was designated for the
Speaker Dependent tests (25 sentence utterances from each of the
12 speakers), and another set of 300 sentence utterances  was
designated for the Speaker Independent tests (30 sentence
utterances from each of a new set of 10 speakers). 

A new element of this set of Benchmark Tests was the designation
of a "Test/Retest Set", consisting of previously used test material
drawn from the test set used in february '89.  In addition to use
of the new test sets, participants were encouraged to process a
"Test/Retest Set" consisting of: (a) a set of 150 sentence
utterances (25 sentence utterances from each of 6 speakers) for
Speaker Dependent tests, and (b) a set of 150 sentence utterances
(30 sentence utterances from each of 5 speakers) for Speaker
Independent tests. 

SPEAKER INDEPENDENT SYSTEM TRAINING: System training conditions are
to be the same as for the February '89 tests (i.e., use of either
the 72-speaker "standard" or 109-speaker "augmented" training
sets). 

LIVE TALKERS: No "live talker tests" were included in these tests. 

"HOMOPHONES CORRECT" OPTION IN SCORING: As in the June '88 and
February '89 tests, substitution errors involving homophones were
scored as "correct" for the case of no imposed grammar. 

STANDARD WORD-PAIR GRAMMAR: The "word-pair" grammar, used in the
June '88 and February '89 tests, was again used as a "standard
grammar".