2011 NIST Language Recognition Evaluation Test Set

Item Name: 2011 NIST Language Recognition Evaluation Test Set
Author(s): Craig Greenberg, Alvin Martin, David Graff, Kevin Walker, Karen Jones, Stephanie Strassel
LDC Catalog No.: LDC2018S06
ISBN: 1-58563-846-3
ISLRN: 766-428-831-656-3
Release Date: August 15, 2018
Member Year(s): 2018
DCMI Type(s): Sound
Sample Type: pcm
Sample Rate: 8000
Data Source(s): telephone conversations, broadcast news
Project(s): NIST LRE
Application(s): language identification
Language(s): Mesopotamian Arabic, South Levantine Arabic, North Levantine Arabic, Moroccan Arabic, Bengali, Czech, Dari, English, Persian, Hindi, Lao, Mandarin Chinese, Panjabi, Pushto, Polish, Russian, Slovak, Spanish, Tamil, Thai, Turkish, Ukrainian, Urdu, Standard Arabic
Language ID(s): acm, ajp, apc, ary, ben, ces, prs, eng, fas, hin, lao, cmn, pan, pus, pol, rus, slk, spa, tam, tha, tur, ukr, urd, arb
License(s): LDC User Agreement for Non-Members
Online Documentation: LDC2018S06 Documents
Licensing Instructions: Subscription & Standard Members, and Non-Members
Citation: Greenberg, Craig, et al. 2011 NIST Language Recognition Evaluation Test Set LDC2018S06. Web Download. Philadelphia: Linguistic Data Consortium, 2018.

Introduction

2011 NIST Language Recognition Evaluation Test Set contains selected training data and the evaluation test set for the 2011 NIST Language Recognition Evaluation. It consists of approximately 204 hours of conversational telephone speech and broadcast audio collected by the Linguistic Data Consortium (LDC) in the following 24 languages and dialects: Arabic (Iraqi), Arabic (Levantine), Arabic (Maghrebi), Arabic (Standard), Bengali, Czech, Dari, English (American), English (Indian), Farsi, Hindi, Lao, Mandarin, Punjabi, Pashto, Polish, Russian, Slovak, Spanish, Tamil, Thai, Turkish, Ukrainian and Urdu.

The goal of the NIST (National Institute of Standards and Technology) Language Recognition Evaluation (LRE) is to establish the baseline of current performance capability for language recognition of conversational telephone speech and to lay the groundwork for further research efforts in the field. NIST conducted language recognition evaluations in 1996, 2003, 2005, 2007, and 2009. The 2011 evaluation emphasized the language pair condition and involved both conversational telephone speech (CTS) and broadcast narrow-band speech (BNBS). Further information regarding this evaluation can be found in the evaluation plan which is also included in the documentation for this release.

LDC released the prior LREs as:

  • 2003 NIST Language Recognition Evaluation (LDC2006S31)
  • 2005 NIST Language Recognition Evaluation (LDC2008S05)
  • 2007 NIST Language Recognition Evaluation Test Set (LDC2009S04)
  • 2007 NIST Language Recognition Evaluation Supplemental Training Set (LDC2009S05)
  • 2009 NIST Language Recognition Evaluation Test Set (LDC2014S06)

Data

This release includes training data for nine language varieties that had not been represented in prior LRE cycles -- Arabic (Iraqi), Arabic (Levantine), Arabic (Maghrebi), Arabic (Standard), Czech, Lao, Punjabi, Polish and Slovak -- contained in 893 audited segments of roughly 30 seconds duration and in 400 full-length CTS recordings. The evaluation test set comprises a total of 29,511 audio files, all manually audited at LDC for language and divided equally into three different test conditions according to the nominal amount of speech content per segment.

Data was collected by LDC between 2009 and 2011. The CTS data was obtained using a "claque" collection model in which speakers (claques) called friends or relatives in their social network for a 10-minute conversation in the claque's native language, such that each call would involve a unique callee. Participants were free to speak on topics of their own choosing. All calls were routed through a telephone collection system at LDC which stored the raw mu-law sample stream into separate audio files for each call side. Auditing and selection were applied to the callee side of every call and to the caller (claque) side in at most one call made by each claque. Contiguous regions containing between 25 and 35 seconds of speech were identified by signal analysis and extracted for manual audit. In some cases, shorter segments were also selected for audit.

Broadcast audio was recorded via capture of satellite-receiver MPEG streams or analog audio receivers digitizing at 16 KHz. Platforms for data capture were located at LDC and in Tunisia and India. Recordings were analyzed to extract contiguous segments of narrow-band speech of at least 33 seconds duration; longer segments were trimmed to a maximum length of 35 seconds for audit.

All audited segments for training and test are presented as 8-KHz, 16-bit PCM, single-channel audio files with NIST SPHERE headers. The full-length CTS data is the same, except that it consists of two channels.

Samples

Please listen to this Urdu sample, Pashto sample, and English sample.

Updates

None at this time.

Available Media

View Fees





Login for the applicable fee