BOLT Information Retrieval Comprehensive Training and Evaluation
| Item Name: | BOLT Information Retrieval Comprehensive Training and Evaluation |
| Author(s): | Kira Griffitt, Stephanie Strassel |
| LDC Catalog No.: | LDC2018T18 |
| ISBN: | 1-58563-855-2 |
| ISLRN: | 384-402-057-006-6 |
| DOI: | https://doi.org/10.35111/sd50-6m36 |
| Release Date: | September 17, 2018 |
| Member Year(s): | 2018 |
| DCMI Type(s): | Text |
| Data Source(s): | discussion forum |
| Project(s): | BOLT |
| Application(s): | information retrieval, machine translation |
| Language(s): | Egyptian Arabic, Mandarin Chinese, English |
| Language ID(s): | arz, cmn, eng |
| License(s): |
LDC User Agreement for Non-Members |
| Online Documentation: | LDC2018T18 Documents |
| Licensing Instructions: | Subscription & Standard Members, and Non-Members |
| Citation: | Griffitt, Kira, and Stephanie Strassel. BOLT Information Retrieval Comprehensive Training and Evaluation LDC2018T18. Web Download. Philadelphia: Linguistic Data Consortium, 2018. |
| Related Works: | View |
Introduction
BOLT Information Retrieval Comprehensive Training and Evaluation was developed by the Linguistic Data Consortium (LDC) and consists of all data produced in support of the Information Retrieval (IR) task within the DARPA Broad Operational Language Translation (BOLT) Program, including annotations, source documents and scoring software.
The BOLT program developed machine translation and information retrieval for less formal genres, focusing particularly on user-generated content. LDC supported BOLT by collecting informal data sources -- discussion forums, text messaging and chat -- in Chinese, Egyptian Arabic and English. The collected data was translated and annotated for various tasks including word alignment, treebanking, propbanking and co-reference.
The material in this release relates to the IR task, which sought to support development of systems that could: (1) take as input a natural language English query sentence; (2) return relevant responses to that query from a large corpus of informal documents in the three BOLT languages (Arabic, Chinese, and English); and (3) translate responses from non-English documents into English.
Data
BOLT Information Retrieval Comprehensive Training and Evaluation contains the pilot, dry run, and evaluation data developed for each phase of the BOLT IR task, including: (1) natural-language IR queries, system responses to queries, and manually-generated assessment judgments for system responses; (2) discussion forum source documents in Arabic, Chinese and English; (3) scoring software for each evaluation phase; and (4) experimental data developed in Phase 2.
Source data is presented as a series of zip archives containing xml files. Queries and responses data are presented as XML as well. Judgments are included as tab delimited files.
Acknowledgement
This material is based upon work supported by the Defense Advanced Research Projects Agency (DARPA) under Contract No. HR0011-11-C-0145. The content does not necessarily reflect the position or the policy of the Government, and no official endorsement should be inferred.
Samples
Please view the following samples:
Updates
None at this time.