TAC KBP English Surprise Slot Filling -- Comprehensive Training and Evaluation Data 2010

Item Name: TAC KBP English Surprise Slot Filling -- Comprehensive Training and Evaluation Data 2010
Author(s): Joe Ellis, Jeremy Getman, Stephanie Strassel
LDC Catalog No.: LDC2021T06
ISBN: 1-58563-957-5
ISLRN: 834-183-247-193-7
DOI: https://doi.org/ 10.35111/nqry-b234
Release Date: February 15, 2021
Member Year(s): 2021
DCMI Type(s): Text
Data Source(s): newswire, web collection
Project(s): TAC
Application(s): information extraction, knowledge base population, knowledge representation, relation extraction
Language(s): English
Language ID(s): eng
License(s): LDC User Agreement for Non-Members
Online Documentation: LDC2021T06 Documents
Licensing Instructions: Subscription & Standard Members, and Non-Members
Citation: Ellis, Joe, Jeremy Getman, and Stephanie Strassel. TAC KBP English Surprise Slot Filling -- Comprehensive Training and Evaluation Data 2010 LDC2021T06. Web Download. Philadelphia: Linguistic Data Consortium, 2021.
Related Works: View


TAC KBP English Surprise Slot Filling -- Comprehensive Training and Evaluation Data 2010 was developed by the Linguistic Data Consortium and contains training and evaluation data produced in support of the 2010 TAC KBP Surprise Slot Filling track, the only year in which the track was run.

Text Analysis Conference (TAC) is a series of workshops organized by the National Institute of Standards and Technology (NIST). TAC was developed to encourage research in natural language processing and related applications by providing a large test collection, common evaluation procedures, and a forum for researchers to share their results. Through its various evaluations, the Knowledge Base Population (KBP) track of TAC encouraged the development of systems to match entities mentioned in natural texts with those appearing in a knowledge base, to extract novel information about entities from a document collection, and to add it to a new or existing knowledge base.

The regular English Slot Filling track involved mining information about entities from text using a specified set of "slots", or attributes. The goal of the Surprise Slot Filling task was to support the development of information extraction systems that could rapidly adapt to new types of relations and events. Surprise Slot Filling participants were given four new slot types -- "diseases", "awards-won" and "charity-supported" for persons, and "products" for organizations -- along with annotation guidelines and training data. They were instructed to develop their systems and to run them on the source collection in four days. More information about the TAC KBP Surprise Slot Filling track and other TAC KBP evaluations can be found on the NIST TAC website.


This data in this release includes queries, the "manual runs" (human-produced responses to the queries), and the final round of assessment results.

The corresponding source document collections cover English newswire, broadcast material, and web text . These documents are included in TAC KBP Comprehensive English Source Corpora 2009-2014 (LDC2018T03). The corresponding Knowledge Base (KB) for much of the data - a 2008 snapshot of Wikipedia - is contained in TAC KBP Reference Knowledge Base (LDC2014T16) .

All text data is encoded as UTF-8.


This material is based on research sponsored by Air Force Research Laboratory and Defense Advance Research Projects Agency under agreement number FA8750-13-2-0045. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright notation thereon. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of Air Force Research Laboratory and Defense Advanced Research Projects Agency or the U.S. Government.


Please view the following samples:


None at this time.

Available Media

View Fees

Login for the applicable fee