OCTOBER 1987 BENCHMARK TESTS

TEST MATERIAL: The October 1987 Benchmark Tests included use of
material from both the Speaker Dependent Development Test and
speaker Independent Development Test material.  The Speaker
Dependent test material consisted of: (1) a set of 25 utterances
per speaker for the four speakers previously identified in the
March 1987 tests (of which 10 or these sentence utterances were
"retest utterances" that had been part of the March 1987 speaker
Independent Test set), and (2) a set of 25 sentence utterances per
speaker for 4 additional speakers, for a total of 200 sentence
utterances for the October 1987 Speaker Dependent Test Set.  The
Speaker Independent test material consisted of: (1) a set of 6
speakers was identified, and sets of 10 sentence utterances per
speaker were designated from the speaker Independent Development
Test material, and (2) the set of 25 sentence utterances per
speaker for 4 speakers used in these Benchmark Tests for Speaker
Dependent systems.  The complete speaker Independent Test Set thus
consisted of 160 sentence utterances. 

SPEAKER INDEPENDENT SYSTEM TRAINING: No explicit constraint was
placed on the allowable material to be used for training Speaker
Independent systems.

LIVE TALKERS: Tests implemented at BBN and CMU included new
material from "live talkers".

"HOMOPHONES CORRECT" OPTION IN SCORING: At the time of these tests,
no special provision was made to score susbtitution errors
involving homophones as correct.
 
STANDARD WORD-PAIR GRAMMAR: At the time of these tests, no
agreement had been reached on the use of a standard word-pair
grammar.