Spanish Gigaword Third Edition
| Item Name: | Spanish Gigaword Third Edition |
| Author(s): | Ângelo Mendonça, Daniel Jaquette, David Graff, Denise DiPersio |
| LDC Catalog No.: | LDC2011T12 |
| ISBN: | 1-58563-596-0 |
| ISLRN: | 595-627-966-073-3 |
| DOI: | https://doi.org/10.35111/v6z1-cb91 |
| Release Date: | October 21, 2011 |
| Member Year(s): | 2011 |
| DCMI Type(s): | Text |
| Data Source(s): | newswire |
| Application(s): | natural language processing, language modeling, information retrieval |
| Language(s): | Spanish |
| Language ID(s): | spa |
| License(s): |
LDC User Agreement for Non-Members |
| Online Documentation: | LDC2011T12 Documents |
| Licensing Instructions: | Subscription & Standard Members, and Non-Members |
| Citation: | Mendonça, Ângelo, et al. Spanish Gigaword Third Edition LDC2011T12. Web Download. Philadelphia: Linguistic Data Consortium, 2011. |
| Related Works: | View |
Introduction
Spanish Gigaword Third Edition is a comprehensive archive of Spanish newswire text data acquired by the Linguistic Data Consortium. It includes all of the content of the second edition (LDC2009T21) and adds data collected from January 1, 2009 through December 31, 2010.
The three distinct international sources of Spanish newswire in this edition, and the time spans of collection covered for each, are as follows:
- Agence France-Presse, Spanish (afp_spa) May 1994 - Dec 2010
- Associated Press, Spanish (apw_spa) Nov 1993 - Dec 2010
- Xinhua News Agency, Spanish (xin_spa) Sep 2001 - Dec 2010
The seven-letter codes in the parentheses above include the three-character source name abbreviations and the three-character language code (spa) separated by an underscore (_) character. The three-letter language code conforms to LDCs internal convention based on the ISO 639-3 standard.
Data
All text data are presented in SGML/XML form, using a very simple, minimal markup structure all text consists of printable ASCII, whitespace, and printable code points in the Latin1 Supplement character table, as defined by both ISO-8859-1 and the Unicode Standard (ISO 10646) for the accented characters used in Spanish. The Supplement/accented characters are rendered using UTF-8 encoding.
For all of the documents in this corpus, a rudimentary (and _approximate_) categorization of DOC units into four distinct types has been applied. The classification is indicated by the type=string attribute that is included in each opening DOC tag. The four types are:
- story : This is by far the most frequent type, and it represents the most typical newswire item: a coherent report on a particular topic or event, consisting of paragraphs and full sentences.
- multi : This type of DOC contains a series of unrelated blurbs, each of which briefly describes a particular topic or event this is typically applied to DOCs that contain summaries of todays news, news briefs in ... (some general area like finance or sports), and so on.
- advis : (short for advisory) These are DOCs which the news service addresses to news editors -- they are not intended for publication to the end users (the populations who read the news). This type contains formulaic, repetitive content (contact phone numbers, etc).
- other : This represents DOCs that clearly do not fall into any of the above types -- in general, items of this type are intended for broad circulation (they are not advisories), they may be topically coherent (unlike multi type DOCS), and they typically do not contain paragraphs or sentences (they arent really stories) these are things like lists of sports scores, stock prices, temperatures around the world, and so on.
Sample
Please view this sample.
Updates
An update to Spanish Gigaword Third Edition was issued to fix an issue with 26 consecutive months of data files from Xinhua Spanish: xin_spa_200601 through xin_spa_200802 i.e. all files from 2006 and 2007, plus the first two files from 2008. The problem was that all letters with diacritic marks had been omitted in the text data for that portion of the collection. For example, the word año was presented as ao (minus the n-with-tilde character), aspiracion appeared as aspiracin, and similarly for all accented characters (UTF-8 letters outside the ASCII range). All copies of Spanish Gigaword Third Edition ordered after February 2013 will have this update included. More information is included in the readme associated with this update.