arTenTen: a new, vast corpus for Arabic
| Authors | |
|---|---|
| Year of publication | 2013 |
| Type | Article in Proceedings |
| Conference | Proceedings of WACL’2 Second Workshop on Arabic Corpus Linguistics |
| MU Faculty or unit | |
| Citation | |
| Web | |
| Keywords | Arabic corpus; Arabic Corpus Linguistics; MADA; Arabic Gigaword; Modern Standard Arabic |
| Description | We present arTenTen, a web crawled corpus of Arabic, gathered in 2012, and a member of the TenTen Corpus Family (Jakubíček et al 2013). arTenTen comprises 5.8 billion words. It has been carefully cleaned, including duplicate removal, using the JusText and Onion tools (Pomikalek 2011). We are currently (May 2013) in the process of tokenising, lemmatising and part-of-speech tagging arTenTen with the leading MADA tool version 3.2 (Habash and Rambow 2005; Habash et al. 2009). Once arTenTen is fully encoded, we will compare it with Arabic Gigaword and an earlier web-crawled corpus (Sharoff 2006). We also plan to explore arTenTen’s composition in relation to Modern Standard Arabic and the dialects, using, amongst other things, Buckwalter and Parkinson’s Frequency Dictionary (2011) and the keywords method presented in (Kilgarriff 2012). |
| Related projects: |