NIST Open MT 2008 Evaluation (MT08) Selected References and System Translations
| Item Name: | NIST Open MT 2008 Evaluation (MT08) Selected References and System Translations |
| Author(s): | NIST Multimodal Information Group |
| LDC Catalog No.: | LDC2010T01 |
| ISBN: | 1-58563-533-2 |
| ISLRN: | 000-078-785-720-8 |
| DOI: | https://doi.org/10.35111/tzpc-v131 |
| Release Date: | January 20, 2010 |
| Member Year(s): | 2010 |
| DCMI Type(s): | Text |
| Data Source(s): | web collection, newswire |
| Project(s): | MT08, GALE |
| Application(s): | machine translation |
| Language(s): | Urdu, Mandarin Chinese, Standard Arabic, English, Chinese, Arabic |
| Language ID(s): | urd, cmn, arb, eng, zho, ara |
| License(s): |
LDC User Agreement for Non-Members |
| Online Documentation: | LDC2010T01 Documents |
| Licensing Instructions: | Subscription & Standard Members, and Non-Members |
| Citation: | NIST Multimodal Information Group. NIST Open MT 2008 Evaluation (MT08) Selected References and System Translations LDC2010T01. Web Download. Philadelphia: Linguistic Data Consortium, 2010. |
| Related Works: | View |
Introduction
This file contains documentation for NIST Open Machine Translation 2008 Evaluation (MT08) Selected Reference and System Translations, Linguistic Data Consortium (LDC) catalog number LDC2010T01 and isbn 1-58563-533-2.
NIST Open MT is an evaluation series to support research in, and help advance the state of the art of, technologies that translate text between human languages. Participants submit machine translation output of source language data to NIST (National Institute of Standards and Technology); the output is then evaluated with automatic and manual measures of quality against high quality human translations of the same source data. This program supports the growing interest in system combination approaches that generate improved translations from output of several different machine translation (MT) systems. MT system combination approaches require data sets composed of high-quality human reference translations and a variety of machine translations of the same text. The NIST Open Machine Translation 2008 Evaluation (MT08) Selected Reference and System Translations set addresses this need.
The data in this release consists of the human reference translations and corresponding machine translations for the NIST Open MT08 test sets, which consist of newswire and web data in the four MT08 language pairs -- Arabic-to-English, Chinese-to-English, English-to-Chinese (newswire only) and Urdu-to-English. Two documents per language pair and genre were removed at random from the test sets for release. For the machine translations, only output from one submission (in most cases, the participant's primary submission) per training condition (Constrained and Unconstrained training, where available) per participant is included. See section 2 of the MT08 Evaluation Plan for a description of the training conditions. The resulting data set has the following characteristics:
- Arabic-to-English: 120 documents with 1312 segments, output from 17 machine translation systems.
- Chinese-to-English: 105 documents with 1312 segments, output from 23 machine translation systems.
- English-to-Chinese: 127 documents with 1830 segments, output from 11 machine translation systems.
- Urdu-to-English: 128 documents with 1794 segments, output from 12 machine translation systems.
The data is organized and annotated in such a way that subsets for each language pair and/or data genre and/or training condition can be extracted and used separately, depending on the user's needs.
