ekphrasis

Why we included this project

Social media text is messy in ways standard tokenizers tend to mangle: misspellings, glued-together hashtags, emoticons. Ekphrasis is a small library built to handle exactly that kind of input before you pass it to a model. It understands complex emoticons, emojis, dates, and other unstructured expressions in Twitter and Facebook content, and it can split hashtags into their constituent words, normalize informal spellings, and correct misspelled words. The segmentation and spell correction run on word statistics built from English Wikipedia and a corpus of 330 million tweets, and you can generate your own statistics from a domain-specific corpus if the built-in ones do not fit your data. It is a focused utility rather than a full NLP framework, so it fits into a preprocessing pipeline without much setup. One honest caveat: the project is no longer actively developed, so weigh that if you need ongoing maintenance, though the core functionality remains stable and usable.