🚀 Exciting News! We're thrilled to introduce Nigeria's first pretrained "foundational" language model, SabiYarn-125M, with 125 million parameters! 🎉
Try it out here:
🔗 [AI Playground](https://t.co/8NZbJKhzcE)
🔗 [Hugging Face Download](https://t.co/8SeGEuGCjZ)
Pretrained on approximately 29GB of cleaned data with a multi-task learning objective using causal language modeling, SabiYarn-125M performs in:
- 🌐 Text Generation
- 🌍 Translation (including inter-language translation)
- 😊 Sentiment Classification
- 📚 Topic Classification
- 📰 Text Summarization
- 🖊 Headline Generation
- Question Generation (majorly in english)
- 🔤 Text Diacritization
It performs (especially in the Nigerian languages) on although slightly worse:
- 📜 Simple Instruction Prompting
- Basic Question answering
- 🧹 Text Cleaning
Model did not perform on:
- Coding and Basic Reasoning tasks
Supporting multiple Nigerian languages: English, Yoruba, Hausa, Pidgin, Igbo, Efik, Urhobo, Fulah, and Fulfulde.
With a maximum token length of 1024! Model has not been finetuned in anyway.
Model Evaluation (mostly human): Ongoing to enhance capabilities further.
Contributors: @Jeffreypaul_ , @damilojohn, @DavidOkpare, @JOloganj,
Model name by @nuelhex0
With this work, we aim to collaborate with key stakeholders and others to build more intelligent, robust, and culturally aware AI models and systems for the African AI community and Nigeria as a whole. Together, we can enhance communication across all Nigerian dialects.
@awarritech@NaijaNLP@bosuntijani@OfficialSamAyo@davlanade@NITDANigeria@uniccongroup@MasaKhaneNLP@chuksekwuemee@DataFestAfrica@googleafrica@GoogleAI @neuralNet314 @3MTTNigeria@abacaj@janleike@ilyasut
Please check it out, retweet, like, and comment to help us reach more people! Let's put #SabiYarn to the test!
#SabiYarn #AI #NLP #Nigeria #Tech #Innovation
Special thanks to @karpathy for NanoGPT and the @huggingface transformers library for enabling advanced generation capabilities.