#73 · Primary category： Foundation Models
DeepSeek-V2
DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
Project last updated：09/25/24
GitHub Stars
5.0K
Forks
550
Contributors
7
License
MIT
Why we included this project
If you're shopping for open-weight models you can actually self-host, DeepSeek-V2 is worth a close look. The repo ships weights for both the full 236B-parameter model and a 16B Lite variant, with base and chat checkpoints, so you can match the size to your hardware. Its two headline ideas, Multi-head Latent Attention and a sparse MoE layout that only activates a fraction of parameters per token, directly cut inference cost and KV-cache memory, which is what determines how many GPUs you'll need. The README's benchmark tables across English, Chinese, math, and coding, plus a vLLM integration for serving, make it a practical baseline when you're comparing open models for a real deployment.
Articles for this project
No articles for this project yet.
To suggest a topic or contribute an article, contact us.
Related projects in this category
transformers
🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
CLIP
CLIP (Contrastive Language-Image Pretraining), Predict the most relevant text snippet given an image
MiniCPM-V
A Pocket-Sized MLLM for Ultra-Efficient Image and Video Understanding on Your Phone
generative-models
Generative Models by Stability AI
unilm
Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities