#133 · Primary category: Foundation Models
cabrita
Finetuning InstructLLaMA with portuguese data
Project last updated:06/06/23
GitHub Stars
558
Forks
69
Contributors
2
License
Apache-2.0
Why we included this project
Anyone working on Portuguese-language NLP will find this a useful, honest case study of how a small team adapted a general-purpose instruction model to one language. The repo walks through the whole pipeline behind Cabrita, a Portuguese instruction-tuned LLaMA-7B: translating the Stanford Alpaca dataset into Portuguese, finetuning with LoRA/PEFT on a single A100 in about an hour, and checking the results. The notebooks let you reproduce the training and try the released LoRA adapter yourself, which helps if you are planning a similar low-budget finetuning run for another language or domain. The authors are upfront about the tradeoffs, like relying on machine translation for the training data and keeping the model research-only, so you get a realistic sense of what limited compute can achieve rather than a polished demo.
Articles for this project
No articles for this project yet.
To suggest a topic or contribute an article, contact us.
Related projects in this category
transformers
🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
CLIP
CLIP (Contrastive Language-Image Pretraining), Predict the most relevant text snippet given an image
MiniCPM-V
A Pocket-Sized MLLM for Ultra-Efficient Image and Video Understanding on Your Phone
generative-models
Generative Models by Stability AI
unilm
Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities