Semantic Textual Similarity (STS) 2013 Machine Translation
| Item Name: | Semantic Textual Similarity (STS) 2013 Machine Translation |
| Author(s): | Eneko Agirre, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, Weiwei Guo |
| LDC Catalog No.: | LDC2013T18 |
| ISBN: | 1-58563-656-8 |
| ISLRN: | 857-492-590-583-5 |
| DOI: | https://doi.org/10.35111/cy4d-7c39 |
| Release Date: | September 16, 2013 |
| Member Year(s): | 2013 |
| DCMI Type(s): | Text |
| Data Source(s): | newswire, web collection |
| Project(s): | NIST MT, GALE |
| Application(s): | machine translation |
| Language(s): | English |
| Language ID(s): | eng |
| License(s): |
LDC User Agreement for Non-Members |
| Online Documentation: | LDC2013T18 Documents |
| Licensing Instructions: | Subscription & Standard Members, and Non-Members |
| Citation: | Agirre, Eneko, et al. Semantic Textual Similarity (STS) 2013 Machine Translation LDC2013T18. Web Download. Philadelphia: Linguistic Data Consortium, 2013. |
| Related Works: | View |
Introduction
Semantic Textual Similarity (STS) 2013 Machine Translation was developed as part of the STS 2013 Shared Task which was held in conjunction with *SEM 2013, the second joint conference on lexical and computational semantics organized by the ACL (Association of Computational Linguistics) interest groups SIGLEX and SIGSEM. It is comprised of one text file containing 750 English sentence pairs translated from the Arabic and Chinese newswire and web data sources.
The goal of the Semantic Textual Similarity (STS) task was to create a unified framework for the evaluation of semantic textual similarity modules and to characterize their impact on natural language processing (NLP) applications. STS measures the degree of semantic equivalence. The STS task was proposed as an attempt at creating a unified framework that allows for an extrinsic evaluation of multiple semantic components that otherwise have historically tended to be evaluated independently and without characterization of impact on NLP applications. More information is available at the STS 2013 Shared Task homepage.
Data
The source data is Arabic and Chinese newswire and web data collected by LDC that was translated and used in the DARPA GALE (Global Autonomous Language Exploitation) program and in several NIST Open Machine Translation evaluations. Of the 750 sentence pairs, 150 pairs are from the GALE Phase 5 collection and 600 pairs are from NIST 2008-2012 Open Machine Translation (OpenMT) Progress Test Sets (LDC2013T07).
The data was built to identify semantic textual similarity between two short text passages. The corpus is comprised of two tab delimited sentences per line. The first sentence is a translation and the second sentence is a post-edited translation. Post-editing is a process to improve machine translation with a minimum of manual labor. The gold standard similarity values and other STS datasets can be obtained from the STS homepage, linked above.
Samples
Please view this text sample.
Updates
None at this time.