Semantic Textual Similarity (STS) 2013 Machine Translation

Item Name: Semantic Textual Similarity (STS) 2013 Machine Translation
Author(s): Eneko Agirre, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, Weiwei Guo
LDC Catalog No.: LDC2013T18
ISBN: 1-58563-656-8
ISLRN: 857-492-590-583-5
Release Date: September 16, 2013
Member Year(s): 2013
DCMI Type(s): Text
Data Source(s): newswire, web collection
Project(s): NIST MT, GALE
Application(s): machine translation
Language(s): English
Language ID(s): eng
License(s): LDC User Agreement for Non-Members
Online Documentation: LDC2013T18 Documents
Licensing Instructions: Subscription & Standard Members, and Non-Members
Citation: Agirre, Eneko, et al. Semantic Textual Similarity (STS) 2013 Machine Translation LDC2013T18. Web Download. Philadelphia: Linguistic Data Consortium, 2013.

Introduction

Semantic Textual Similarity (STS) 2013 Machine Translation was developed as part of the STS 2013 Shared Task which was held in conjunction with *SEM 2013, the second joint conference on lexical and computational semantics organized by the ACL (Association of Computational Linguistics) interest groups SIGLEX and SIGSEM. It is comprised of one text file containing 750 English sentence pairs translated from the Arabic and Chinese newswire and web data sources.

The goal of the Semantic Textual Similarity (STS) task was to create a unified framework for the evaluation of semantic textual similarity modules and to characterize their impact on natural language processing (NLP) applications. STS measures the degree of semantic equivalence. The STS task was proposed as an attempt at creating a unified framework that allows for an extrinsic evaluation of multiple semantic components that otherwise have historically tended to be evaluated independently and without characterization of impact on NLP applications. More information is available at the STS 2013 Shared Task homepage.

Data

The source data is Arabic and Chinese newswire and web data collected by LDC that was translated and used in the DARPA GALE (Global Autonomous Language Exploitation) program and in several NIST Open Machine Translation evaluations. Of the 750 sentence pairs, 150 pairs are from the GALE Phase 5 collection and 600 pairs are from NIST 2008-2012 Open Machine Translation (OpenMT) Progress Test Sets (LDC2013T07).

The data was built to identify semantic textual similarity between two short text passages. The corpus is comprised of two tab delimited sentences per line. The first sentence is a translation and the second sentence is a post-edited translation. Post-editing is a process to improve machine translation with a minimum of manual labor. The gold standard similarity values and other STS datasets can be obtained from the STS homepage, linked above.

Samples

Please view this text sample.

Updates

None at this time.

Available Media

View Fees





Login for the applicable fee