English Gigaword Fourth Edition
| Item Name: | English Gigaword Fourth Edition |
| Author(s): | Robert Parker, David Graff, Junbo Kong, Ke Chen, Kazuaki Maeda |
| LDC Catalog No.: | LDC2009T13 |
| ISBN: | 1-58563-515-4 |
| ISLRN: | 874-388-164-791-7 |
| DOI: | https://doi.org/10.35111/y4px-6y07 |
| Release Date: | May 22, 2009 |
| Member Year(s): | 2009 |
| DCMI Type(s): | Text |
| Data Source(s): | newswire |
| Project(s): | GALE |
| Application(s): | natural language processing, language modeling, information retrieval |
| Language(s): | English |
| Language ID(s): | eng |
| License(s): |
LDC User Agreement for Non-Members |
| Online Documentation: | LDC2009T13 Documents |
| Licensing Instructions: | Subscription & Standard Members, and Non-Members |
| Citation: | Parker, Robert, et al. English Gigaword Fourth Edition LDC2009T13. Web Download. Philadelphia: Linguistic Data Consortium, 2009. |
| Related Works: | View |
Introduction
This file contains documentation for English Gigaword Fourth Edition, Linguistic Data Consortium (LDC) catalog number LDC2009T13 and isbn 1-58563-515-4.
English Gigaword, now being released in its fourth edition, is a comprehensive archive of newswire text data that has been acquired over several years by the LDC at the University of Pennsylvania. The fourth edition includes all of the contents in English Gigawaord Third Edition (LDC2007T07) plus new data covering the 24-month period of January 2007 through December 2008.
The six distinct international sources of English newswire included in this edition are the following:
- Agence France-Presse, English Service (afp_eng)
- Associated Press Worldstream, English Service (apw_eng)
- Central News Agency of Taiwan, English Service (cna_eng)
- Los Angeles Times/Washington Post Newswire Service (ltw_eng)
- New York Times Newswire Service (nyt_eng)
- Xinhua News Agency, English Service (xin_eng)
New in the Fourth Edition
- Articles with significant Spanish language content have now been identified and documented.
- Markup has been simplified and made consistent throughout the corpus.
- Information structure has been simplified.
- Character entities have been simplified.
Samples
For an example of the data in this corpus, please review this sample file.
Sponsorship
This work was supported in part by the Defense Advanced Research Projects Agency, GALE Program Grant No. HR0011-06-1-0003. The content of this publication does not necessarily reflect the position or the policy of the Government, and no official endorsement should be inferred.