Reconsidering Language Identification for Written Language Resources

Hughes, B. and Baldwin, T. and Bird, S. G. and Nicholson, J. and MacKinlay, A. (2006) Reconsidering Language Identification for Written Language Resources. In Proceedings 5th International Conference on Language Resources and Evaluation (LREC2006), pages pp. 485-488, Genoa, Italy.

Abstract

The task of identifying the language in which a given document (ranging from a sentence to thousands of pages) is written has been relatively well studied over several decades. Automated approaches to written language identification are used widely throughout research and industrial contextx, over both oral and written source materials. Despite this widespread acceptance, a review of previous research in written language identification reveals a number of questions which remain open and ripe for further investigation.