Ancestor-to-Creole Transfer is Not a Walk in the Park

Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review

Standard

Ancestor-to-Creole Transfer is Not a Walk in the Park. / Lent, Heather; Bugliarello, Emanuele; Søgaard, Anders.

Proceedings of the Third Workshop on Insights from Negative Results in NLP. Association for Computational Linguistics (ACL), 2022. p. 68-74.

Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review

Harvard

Lent, H, Bugliarello, E & Søgaard, A 2022, Ancestor-to-Creole Transfer is Not a Walk in the Park. in Proceedings of the Third Workshop on Insights from Negative Results in NLP. Association for Computational Linguistics (ACL), pp. 68-74, Third Workshop on Insights from Negative Results in NLP, Dublin, Ireland, 01/05/2022. https://doi.org/10.18653/v1/2022.insights-1.9

APA

Lent, H., Bugliarello, E., & Søgaard, A. (2022). Ancestor-to-Creole Transfer is Not a Walk in the Park. In Proceedings of the Third Workshop on Insights from Negative Results in NLP (pp. 68-74). Association for Computational Linguistics (ACL). https://doi.org/10.18653/v1/2022.insights-1.9

Vancouver

Lent H, Bugliarello E, Søgaard A. Ancestor-to-Creole Transfer is Not a Walk in the Park. In Proceedings of the Third Workshop on Insights from Negative Results in NLP. Association for Computational Linguistics (ACL). 2022. p. 68-74 https://doi.org/10.18653/v1/2022.insights-1.9

Author

Lent, Heather ; Bugliarello, Emanuele ; Søgaard, Anders. / Ancestor-to-Creole Transfer is Not a Walk in the Park. Proceedings of the Third Workshop on Insights from Negative Results in NLP. Association for Computational Linguistics (ACL), 2022. pp. 68-74

Bibtex

@inproceedings{23b874e28d8e437ab1002a82c600a0d1,
title = "Ancestor-to-Creole Transfer is Not a Walk in the Park",
abstract = "We aim to learn language models for Creole languages for which large volumes of data are not readily available, and therefore explore the potential transfer from ancestor languages (the {\textquoteleft}Ancestry Transfer Hypothesis{\textquoteright}). We find that standard transfer methods do not facilitate ancestry transfer. Surprisingly, different from other non-Creole languages, a very distinct two-phase pattern emerges for Creoles: As our training losses plateau, and language models begin to overfit on their source languages, perplexity on the Creoles drop. We explore if this compression phase can lead to practically useful language models (the {\textquoteleft}Ancestry Bottleneck Hypothesis{\textquoteright}), but also falsify this. Moreover, we show that Creoles even exhibit this two-phase pattern even when training on random, unrelated languages. Thus Creoles seem to be typological outliers and we speculate whether there is a link between the two observations.",
author = "Heather Lent and Emanuele Bugliarello and Anders S{\o}gaard",
year = "2022",
doi = "10.18653/v1/2022.insights-1.9",
language = "English",
pages = "68--74",
booktitle = "Proceedings of the Third Workshop on Insights from Negative Results in NLP",
publisher = "Association for Computational Linguistics (ACL)",
address = "United States",
note = "Third Workshop on Insights from Negative Results in NLP ; Conference date: 01-05-2022 Through 01-05-2022",

}

RIS

TY - GEN

T1 - Ancestor-to-Creole Transfer is Not a Walk in the Park

AU - Lent, Heather

AU - Bugliarello, Emanuele

AU - Søgaard, Anders

PY - 2022

Y1 - 2022

N2 - We aim to learn language models for Creole languages for which large volumes of data are not readily available, and therefore explore the potential transfer from ancestor languages (the ‘Ancestry Transfer Hypothesis’). We find that standard transfer methods do not facilitate ancestry transfer. Surprisingly, different from other non-Creole languages, a very distinct two-phase pattern emerges for Creoles: As our training losses plateau, and language models begin to overfit on their source languages, perplexity on the Creoles drop. We explore if this compression phase can lead to practically useful language models (the ‘Ancestry Bottleneck Hypothesis’), but also falsify this. Moreover, we show that Creoles even exhibit this two-phase pattern even when training on random, unrelated languages. Thus Creoles seem to be typological outliers and we speculate whether there is a link between the two observations.

AB - We aim to learn language models for Creole languages for which large volumes of data are not readily available, and therefore explore the potential transfer from ancestor languages (the ‘Ancestry Transfer Hypothesis’). We find that standard transfer methods do not facilitate ancestry transfer. Surprisingly, different from other non-Creole languages, a very distinct two-phase pattern emerges for Creoles: As our training losses plateau, and language models begin to overfit on their source languages, perplexity on the Creoles drop. We explore if this compression phase can lead to practically useful language models (the ‘Ancestry Bottleneck Hypothesis’), but also falsify this. Moreover, we show that Creoles even exhibit this two-phase pattern even when training on random, unrelated languages. Thus Creoles seem to be typological outliers and we speculate whether there is a link between the two observations.

U2 - 10.18653/v1/2022.insights-1.9

DO - 10.18653/v1/2022.insights-1.9

M3 - Article in proceedings

SP - 68

EP - 74

BT - Proceedings of the Third Workshop on Insights from Negative Results in NLP

PB - Association for Computational Linguistics (ACL)

T2 - Third Workshop on Insights from Negative Results in NLP

Y2 - 1 May 2022 through 1 May 2022

ER -

ID: 340703243