#85 · Primary category： Foundation Models
magicoder
[ICML'24] Magicoder: Empowering Code Generation with OSS-Instruct
Project last updated：11/01/24
GitHub Stars
2.1K
Forks
170
Contributors
4
License
MIT
Why we included this project
Magicoder is a UIUC research project that argues you can train a competitive code model without distilling from proprietary teachers. Its OSS-Instruct method builds instruction data by rewriting real open-source snippets, which sidesteps the synthetic-data bias common in LLM-generated datasets. The released 6.7B and 7B checkpoints outperform gpt-3.5-turbo-1106 and Gemini Ultra on HumanEval, and the repo documents the full pipeline from data construction to evaluation, so the results are reasonably reproducible. Because the approach has since influenced follow-ups like Google's CodeGemma, it is a useful reference for teams weighing open data-driven training against closed-model distillation.
Articles for this project
No articles for this project yet.
To suggest a topic or contribute an article, contact us.
Related projects in this category
transformers
🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
CLIP
CLIP (Contrastive Language-Image Pretraining), Predict the most relevant text snippet given an image
MiniCPM-V
A Pocket-Sized MLLM for Ultra-Efficient Image and Video Understanding on Your Phone
generative-models
Generative Models by Stability AI
unilm
Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities