#79 · Primary category: Foundation Models
Skywork
Skywork series models are pre-trained on 3.2TB of high-quality multilingual (mainly Chinese and English) and code data. We have open-sourced the model, training data, evaluation data, evaluation methods, etc.
Project last updated:03/07/25
GitHub Stars
1.5K
Forks
148
Contributors
7
License
Other
Why we included this project
Skywork is a bilingual Chinese-English model family that open-sources not just the weights but the training data, evaluation data, and evaluation methods, so a small team can study how a 13B-scale model is actually built rather than treat it as a black box. The repo includes a base, chat, math, and multimodal variant, plus quantized checkpoints sized for consumer GPUs. The documentation covers a 3.2-trillion-token pretraining recipe and the benchmarks used, which makes it feasible to reproduce comparisons against other 13B models. Teams working primarily in Chinese or wanting a lightweight local deployment are the most obvious audience. One thing to check first: the project uses a custom community license, not a standard open-source one, so read the terms before commercial use.
Articles for this project
No articles for this project yet.
To suggest a topic or contribute an article, contact us.
Related projects in this category
transformers
🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
CLIP
CLIP (Contrastive Language-Image Pretraining), Predict the most relevant text snippet given an image
MiniCPM-V
A Pocket-Sized MLLM for Ultra-Efficient Image and Video Understanding on Your Phone
generative-models
Generative Models by Stability AI
unilm
Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities