oai:www.ldc.upenn.edu:LDC2004T11
| |
OLAC Record oai:www.ldc.upenn.edu:LDC2004T11 |
| Metadata | ||
| Title: | Arabic Treebank: Part 3 v 1.0 | |
| Access Rights: | Licensing Instructions for Subscription & Standard Members, and Non-Members: http://www.ldc.upenn.edu/language-resources/data/obtaining | |
| Bibliographic Citation: | Maamouri, Mohamed, et al. Arabic Treebank: Part 3 v 1.0 LDC2004T11. Web Download. Philadelphia: Linguistic Data Consortium, 2004 | |
| Contributor: | Maamouri, Mohamed | |
| Bies, Ann | ||
| Buckwalter, Tim | ||
| Jin, Hubert | ||
| Date (W3CDTF): | 2004 | |
| Date Issued (W3CDTF): | 2004-05-21 | |
| Description: | *Introduction* Arabic Treebank: Part 3 v 1.0 was developed by the Linguistic Data Consortium (LDC) and contains approximately 300,000 Arabic word tokens with part of speech (POS) annotation that includes complete vocalization with case endings, lemma IDs, and more specific POS tags for verbs and particles. The goal of the Arabic Treebank project is to support the development of data-driven approaches to natural language processing (NLP), human language technologies, automatic content extraction (topic extraction and/or grammar extraction), cross-lingual information retrieval, information detection, and other forms of linguistic research on Modern Standard Arabic in general. LDC was sponsored to develop an Arabic POS and Treebank of 1 million words. This corpus is part three of that project. Treebanks are language resources that provide annotations of natural languages at various levels of structure: at the word level, the phrase level, and the sentence level. Treebanks have become crucially important for the development of both data-driven and general linguistic research. This corpus is designed for those who study and use languages either professionally or academically, and who need text corpora in their work. The Penn Arabic Treebank, which started in November 2001 as part of the DARPA TIDES project, is particularly suitable for language developers, computational linguists, and computer scientists who are interested in various aspects of NLP. It started in the fall of 2001 with the objective of annotating a large Arabic machine-readable text corpus manually and automatically. *Data* The following table gives a breakdown of the data contained in the entire Arabic Treebank project, with discrepancies between versions for Parts 1 and 3. The fields include source, number of stories, total number of tokens, number of tokens after clitic separation, and number of Arabic word tokens after punctuation, numbers, and latin strings have been taken out. The totals given at the bottom are calculated from the latest versions where discrepencies exist, and do not include tokens after clitic separation since that number is missing from Part 4. Part Source Stories Total Tokens Tokens After Clitic Separation Arabic Word Tokens 1 (V 2.0) Agence France Presse 734 140,265 168,123 N/A 1 (V 3.0 and 4.1) Agence France Presse 734 145,386 166,068 123,795 2 Ummah Press 501 144,199 169,319 125,709 3 (V 1.0 and 2.0) An Nahar News Agency 600 340,281 400,213 293,035 3 (V 3.2) An Nahar News Agency 599 339,710 402,291 292,554 4 Assabah 397 161,915 N/A 146,491 Totals 2,231 791,210 688,549 For this corpus, the 600 An Nahar News Agency stories were taken from Arabic Gigaword (LDC2003T12). Tim Buckwalter's lexicon and morphological analyzer was used to generate a candidate list of POS tags for each word. (Please note that some words do not exist in this lexicon.) The POS task is just to select the correct POS tag. Of the 293,035 Arabic-only word tokens, 290,842 (99.25%) were provided with an acceptable morphological analysis and POS tag by the morphological parser, and 2,193 (0.75%) were items that the morphological parser failed to analyze correctly. This corpus has two subsequent versions. They are: * Arabic Treebank: Part 3 (full corpus) v 2.0 (MPG + Syntactic Analysis) * Arabic Treebank: Part 3 v 3.2 (LDC2010T08) *Samples* For examples of the data in this corpus, please view the following samples: * Source sample (SGM) * POS sample (XML) * POS sample (TXT) *Updates* None at this time. | |
| Extent: | Corpus size: 471358 KB | |
| Identifier: | LDC2004T11 | |
| https://catalog.ldc.upenn.edu/LDC2004T11 | ||
| ISBN: 1-58563-298-8 | ||
| ISLRN: 026-183-445-670-4 | ||
| DOI: 10.35111/jf6e-hm83 | ||
| Language: | Standard Arabic | |
| Language (ISO639): | arb | |
| License: | LDC User Agreement for Non-Members: https://catalog.ldc.upenn.edu/license/ldc-non-members-agreement.pdf | |
| Medium: | Distribution: Web Download | |
| Publisher: | Linguistic Data Consortium | |
| Publisher (URI): | https://www.ldc.upenn.edu | |
| Relation (URI): | https://catalog.ldc.upenn.edu/docs/LDC2004T11 | |
| Rights Holder: | Portions © 2002 An Nahar, © 2003, 2004 Trustees of the University of Pennsylvania | |
| Type (DCMI): | Text | |
| Type (OLAC): | primary_text | |
OLAC Info |
||
| Archive: | The LDC Corpus Catalog | |
| Description: | http://www.language-archives.org/archive/www.ldc.upenn.edu | |
| GetRecord: | OAI-PMH request for OLAC format | |
| GetRecord: | Pre-generated XML file | |
OAI Info |
||
| OaiIdentifier: | oai:www.ldc.upenn.edu:LDC2004T11 | |
| DateStamp: | 2021-08-02 | |
| GetRecord: | OAI-PMH request for simple DC format | |
Search Info | ||
| Citation: | Maamouri, Mohamed; Bies, Ann; Buckwalter, Tim; Jin, Hubert. 2004. Linguistic Data Consortium. | |
| Terms: | area_Asia country_SA dcmi_Text iso639_arb olac_primary_text | |