#104 · Primary category： Foundation Models
unit-minions
AI Development Efficiency: Hands-on LoRA Training, covering Llama (Alpaca LoRA) and ChatGLM (ChatGLM Tuning) LoRA. Training tasks: user story generation, test code generation, code assistance, text-to-SQL, text-to-code, etc.
Project last updated：01/03/24
GitHub Stars
1.1K
Forks
124
Contributors
3
License
Other
Why we included this project
unit-minions records how one small team fine-tuned LLaMA and ChatGLM with LoRA for software-development work, and it is worth reading if you want similar results without leaning on a hosted API. The notebooks walk through building instruction datasets and running Alpaca-style and ChatGLM tuning runs for tasks like splitting work into user stories, generating tests, and turning text into SQL, and the trained LoRA adapters ship alongside the training code. The most useful part is the data preparation: the authors break software workflows into small steps and show the exact prompts and formats that made each model behave. Treat it as a research log and a reference for reproducing similar fine-tunes on your own data rather than as something to deploy directly.
Articles for this project
No articles for this project yet.
To suggest a topic or contribute an article, contact us.
Related projects in this category
transformers
🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
CLIP
CLIP (Contrastive Language-Image Pretraining), Predict the most relevant text snippet given an image
MiniCPM-V
A Pocket-Sized MLLM for Ultra-Efficient Image and Video Understanding on Your Phone
generative-models
Generative Models by Stability AI
unilm
Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities