File cat-specs.doc.910826, last modified 8/26/91

 This file currently contains two documents specifying the
contents of ATIS categorization (.cat) files.  The first,
numbered "1.0", is an augmented-BNF specification of the syntax;
the second, "2.0", is an algorithmic specification in English of
the mapping between tags and evaluation classes.  Each is
delimited by a line of hyphens.


1.0 Specification of .cat file sytax:

-------------------------------------------------------------------- 
; File cat_spec.bnf
; Categorization (.cat) File Contents Specification.
; (Comment lines start with ";")

; BASIC SYNTAX:
;
; Using standard BNF notation extended with these devices:
; "(A)" means "A optionally";
; "<A>*" means "zero or more A's";
; "<A>+" means "one or more A's".
; 
 <cat_spec> ::= <eval_class>: <characteristics>
; A .cat file specification is an evaluation class followed by
; a colon and some characteristics, e.g.
;  "X: ill-formed"
; Additional constraints between the co-occurrance of the evaluation
; class values (e.g. "X") and the characteristics are stated
; in a different format in file eval_class_proc.txt.


    <eval_class> ::= A | X | D1 | D

    <characteristics> ::= <utt_chars> <interp_id_&_chars>*
; the characteristics are a set of whole-utterance characteristics
; followed by zero or more individual interpretation i.d.'s and
; characteristics

       <utt_chars> ::= <basic_tags>* (<cd-tag>)
          <basic_tags> ::= arithmetic | bad-db | book | cancelled
              | disallowed | hopelessly-vague | ill-formed
              | multi-sentence | presupposition-failure
              | responding | testably-ambiguous | trunc-utt
              | uncooperative | unanswerable | underspecified
              | ungrammatical | wh-question | wizard-error | yes/no

; Note that each aspect of <characteristics> may be null.
; This is allowed only for "class A" utterances (and
; interpretations), so that "A:" is a valid .cat expression,
; but "X:" is not.


          <cd-tag> ::= context-dependent:<context-pointers>)
; one kind of tag is the context-dependent tag, consisting of
; the phrase "context-dependent" followed by a colon and a set
; of pointers to context, e.g. "D1: context-dependent:Q2".

<context-pointers> ::= <ptr_field> | <disjunctive_pointer_field_string>
; the context-pointers field is a disjunctive string of pointer fields

<disjunctive_pointer_field_string> ::= <ptr_field> <alternate_ptr_field>+
; the disjunctive pointer field string is a pointer field followed by
; zero or more alternate pointer fields, e.g.
; "D: context-dependent: Q2 OR Q3".

<ptr_field> ::= <basic_ptr> | <conjunctive_pointer_string>
; a pointer field is a conjunctive string of pointers

 <alternate_ptr_field> ::= OR <ptr_field>
; an alternate pointer field is "OR" followed by a pointer field

 <conjunctive_pointer_string> ::= <basic_ptr> <additional_basic_ptrs>+
; a conjunctive string of pointers is a basic pointer followed by
; zero or more additional basic pointers, e.g.
; "D: context-dependent: Q1 & Q2 OR Q3"

     <basic_ptr> ::= <ctype><UTTNO> (-<interp_id>) | :? | :X
         <ctype> ::= Q | A | Q/A 
     <additional_basic_ptrs> ::= & <basic_ptr>


<interp_id_&_chars> ::= <EOL> <interp_id>:<interp_chars>
; an interpretation i.d. plus characteristics is an interpretation
; i.d. followed by a colon followed by a set of characteristics
; for that interpretation, all on a new line, e.g.
; "A: testably-ambiguous
;    interp#1:yes/no
;    interp#2:wh-question"

<interp_id> ::= interp#<INTEGER>
<interp_chars> ::= <interp_tags>* (<cd-tag>)
<interp_tags> ::= book | disallowed | presupposition-failure
                | underspecified | wh-question | yes/no
; only a subset of the whole-utterance tags are allowed on individual
; interpretations; for instance, being ambiguous is a property of
; a whole utterance, not one particular interpretation of an utterance.

; The above formulation takes <UTTNO>, <EOL> and <INTEGER> as primitives.
; <UTTNO> is the number identifing an utterance, as used in the
; name of its .sro file; <EOL> is something that causes a new line
; to begin; and <INTEGER> is any integer.
----------------------------------------------------------------------------

2.0 Mapping Between Tags and Evaluation Classes:

----------------------------------------------------------------------------
File eval_class_proc.txt

Procedure for assigning queries to evaluation classes:

These rules are to be applied to the utterance characteristics in
the .cat file, in order as given.  The first one that applies
determines the evaluation class of the utterance.

Assign to class X ("X:") if:

    1. There are more than 6 individual interpretations.
    2. Any of these tags occur on the utterance or any of
    its interpretations:
        arithmetic
        bad-db
        book
        cancelled
        disallowed
        hopelessly-vague
        ill-formed
        presupposition-failure
        responding
        trunc-utt
        unanswerable
        uncooperative
        underspecified

Assign to class A ("A:") if the tag  "context-dependent" does not
occur on the utterance or on any of its interpretations.

Assign to class D ("D:") if there is one interpretation that is
not marked with the tag "context-dependent".

Assign to class D1 ("D1:") if 

   the context pointer of each of its interpretations specifies
   just one prior query (":Qn") or query/answer (":Q/An") as
   context, that prior query ("Qn") is the same for each
   interpretation, and each query between Qn and this
   query is tagged as "unanswerable".

otherwise assign to class D ("D:").

----------------------------------------------------------------------