#47 · Primary category： Foundation Models
PhoBERT
PhoBERT: Pre-trained language models for Vietnamese (EMNLP-2020 Findings)
Project last updated：08/04/26
GitHub Stars
807
Forks
113
Contributors
1
License
MIT
Why we included this project
PhoBERT gives Vietnamese NLP projects a strong starting point without the cost of pretraining your own model. The base and large encoders follow the RoBERTa recipe and were trained on roughly 20GB of Wikipedia and news text, which at the time set new state-of-the-art results on part-of-speech tagging, dependency parsing, named-entity recognition, and natural language inference. Both sizes load through the Hugging Face transformers library, so fine-tuning takes only a few lines of code. The one thing to plan around is the tokenizer: it expects word-segmented input, so most people pair it with VnCoreNLP's segmenter before feeding in raw text. That makes it a practical fit for Vietnamese classifiers, NER systems, and retrieval embeddings.
Articles for this project
No articles for this project yet.
To suggest a topic or contribute an article, contact us.
Related projects in this category
transformers
🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
CLIP
CLIP (Contrastive Language-Image Pretraining), Predict the most relevant text snippet given an image
MiniCPM-V
A Pocket-Sized MLLM for Ultra-Efficient Image and Video Understanding on Your Phone
generative-models
Generative Models by Stability AI
unilm
Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities