On Language Models for Creoles

Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review

Standard

On Language Models for Creoles. / Lent, Heather; Bugliarello, Emanuele; De Lhoneux, Miryam; Qiu, Chen; Søgaard, Anders.

Proceedings of the 25th Conference on Computational Natural Language Learning. Association for Computational Linguistics, 2021. p. 58-71.

Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review

Harvard

Lent, H, Bugliarello, E, De Lhoneux, M, Qiu, C & Søgaard, A 2021, On Language Models for Creoles. in Proceedings of the 25th Conference on Computational Natural Language Learning. Association for Computational Linguistics, pp. 58-71, 25th Conference on Computational Natural Language Learning, Online, 01/11/2021. https://doi.org/10.18653/v1/2021.conll-1.5

APA

Lent, H., Bugliarello, E., De Lhoneux, M., Qiu, C., & Søgaard, A. (2021). On Language Models for Creoles. In Proceedings of the 25th Conference on Computational Natural Language Learning (pp. 58-71). Association for Computational Linguistics. https://doi.org/10.18653/v1/2021.conll-1.5

Vancouver

Lent H, Bugliarello E, De Lhoneux M, Qiu C, Søgaard A. On Language Models for Creoles. In Proceedings of the 25th Conference on Computational Natural Language Learning. Association for Computational Linguistics. 2021. p. 58-71 https://doi.org/10.18653/v1/2021.conll-1.5

Author

Lent, Heather ; Bugliarello, Emanuele ; De Lhoneux, Miryam ; Qiu, Chen ; Søgaard, Anders. / On Language Models for Creoles. Proceedings of the 25th Conference on Computational Natural Language Learning. Association for Computational Linguistics, 2021. pp. 58-71

Bibtex

@inproceedings{905f459c74954667a18040121c0052a1,
title = "On Language Models for Creoles",
abstract = "Creole languages such as Nigerian Pidgin English and Haitian Creole are under-resourced and largely ignored in the NLP literature. Creoles typically result from the fusion of a foreign language with multiple local languages, and what grammatical and lexical features are transferred to the creole is a complex process. While creoles are generally stable, the prominence of some features may be much stronger with certain demographics or in some linguistic situations. This paper makes several contributions: We collect existing corpora and release models for Haitian Creole, Nigerian Pidgin English, and Singaporean Colloquial English. We evaluate these models on intrinsic and extrinsic tasks. Motivated by the above literature, we compare standard language models with distributionally robust ones and find that, somewhat surprisingly, the standard language models are superior to the distributionally robust ones. We investigate whether this is an effect of over-parameterization or relative distributional stability, and find that the difference persists in the absence of over-parameterization, and that drift is limited, confirming the relative stability of creole languages.",
author = "Heather Lent and Emanuele Bugliarello and {De Lhoneux}, Miryam and Chen Qiu and Anders S{\o}gaard",
year = "2021",
doi = "10.18653/v1/2021.conll-1.5",
language = "English",
pages = "58--71",
booktitle = "Proceedings of the 25th Conference on Computational Natural Language Learning",
publisher = "Association for Computational Linguistics",
note = "25th Conference on Computational Natural Language Learning ; Conference date: 01-11-2021 Through 01-11-2021",

}

RIS

TY - GEN

T1 - On Language Models for Creoles

AU - Lent, Heather

AU - Bugliarello, Emanuele

AU - De Lhoneux, Miryam

AU - Qiu, Chen

AU - Søgaard, Anders

PY - 2021

Y1 - 2021

N2 - Creole languages such as Nigerian Pidgin English and Haitian Creole are under-resourced and largely ignored in the NLP literature. Creoles typically result from the fusion of a foreign language with multiple local languages, and what grammatical and lexical features are transferred to the creole is a complex process. While creoles are generally stable, the prominence of some features may be much stronger with certain demographics or in some linguistic situations. This paper makes several contributions: We collect existing corpora and release models for Haitian Creole, Nigerian Pidgin English, and Singaporean Colloquial English. We evaluate these models on intrinsic and extrinsic tasks. Motivated by the above literature, we compare standard language models with distributionally robust ones and find that, somewhat surprisingly, the standard language models are superior to the distributionally robust ones. We investigate whether this is an effect of over-parameterization or relative distributional stability, and find that the difference persists in the absence of over-parameterization, and that drift is limited, confirming the relative stability of creole languages.

AB - Creole languages such as Nigerian Pidgin English and Haitian Creole are under-resourced and largely ignored in the NLP literature. Creoles typically result from the fusion of a foreign language with multiple local languages, and what grammatical and lexical features are transferred to the creole is a complex process. While creoles are generally stable, the prominence of some features may be much stronger with certain demographics or in some linguistic situations. This paper makes several contributions: We collect existing corpora and release models for Haitian Creole, Nigerian Pidgin English, and Singaporean Colloquial English. We evaluate these models on intrinsic and extrinsic tasks. Motivated by the above literature, we compare standard language models with distributionally robust ones and find that, somewhat surprisingly, the standard language models are superior to the distributionally robust ones. We investigate whether this is an effect of over-parameterization or relative distributional stability, and find that the difference persists in the absence of over-parameterization, and that drift is limited, confirming the relative stability of creole languages.

U2 - 10.18653/v1/2021.conll-1.5

DO - 10.18653/v1/2021.conll-1.5

M3 - Article in proceedings

SP - 58

EP - 71

BT - Proceedings of the 25th Conference on Computational Natural Language Learning

PB - Association for Computational Linguistics

T2 - 25th Conference on Computational Natural Language Learning

Y2 - 1 November 2021 through 1 November 2021

ER -

ID: 299208351