#58 · Primary category： NLP Tools & Text Processing
budoux
A compact, standalone machine learning tool for line breaking that supports multiple languages and HTML inputs.
Project last updated：08/28/26
GitHub Stars
1.8K
Forks
45
Contributors
16
License
Apache-2.0
Why we included this project
BudouX tackles a fiddly problem that shows up whenever you build an interface in a language without spaces: deciding where a line may break so wrapped text stays readable instead of splitting mid-word. Its model is small, around 15 KB with the weights, and runs standalone, so you can use it client-side without routing text through a cloud segmenter. Pretrained models cover Japanese, Simplified and Traditional Chinese, and Thai, and the training script lets you build models for other languages. Bindings exist for Python, JavaScript, and Java, and the parser accepts HTML directly, meaning you can process marked-up content without stripping tags first. Teams doing CJK or Thai layout will find it a lighter, more natural alternative to character-width heuristics.
Articles for this project
No articles for this project yet.
To suggest a topic or contribute an article, contact us.
Related projects in this category
flair
A very simple framework for state-of-the-art Natural Language Processing (NLP)
compromise
modest natural-language processing
tokenizers
💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
CoreNLP
CoreNLP: A Java suite of core NLP tools for tokenization, sentence segmentation, NER, parsing, coreference, sentiment analysis, etc.
Chinese-Word-Vectors
100+ Chinese Word Vectors 上百种预训练中文词向量