#133 · Primary category: Foundation Models

cabrita

Finetuning InstructLLaMA with portuguese data

Project last updated:06/06/23

GitHub Stars

558

Forks

69

Contributors

2

License

Apache-2.0

Why we included this project

Anyone working on Portuguese-language NLP will find this a useful, honest case study of how a small team adapted a general-purpose instruction model to one language. The repo walks through the whole pipeline behind Cabrita, a Portuguese instruction-tuned LLaMA-7B: translating the Stanford Alpaca dataset into Portuguese, finetuning with LoRA/PEFT on a single A100 in about an hour, and checking the results. The notebooks let you reproduce the training and try the released LoRA adapter yourself, which helps if you are planning a similar low-budget finetuning run for another language or domain. The authors are upfront about the tradeoffs, like relying on machine translation for the training data and keeping the model research-only, so you get a realistic sense of what limited compute can achieve rather than a polished demo.

Articles for this project

No articles for this project yet.

To suggest a topic or contribute an article, contact us.

Related projects in this category