#777 · Primary category: Education & Research

nlp-in-practice

gensim machine-learning natural-language-processing nlp text-classification text-mining tf-idf word2vec

Starter code to solve real world text data problems. Includes: Gensim Word2Vec, phrase embeddings, Text Classification with Logistic Regression, word count with pyspark, simple text preprocessing, pre-trained embeddings and more.

Project last updated:12/02/20

GitHub Stars

1.2K

Forks

785

Contributors

4

License

Other

Why we included this project

This is one of the more practical NLP starter repos out there: a set of small notebooks that cover the text tasks data teams actually run into, like cleaning raw text, pulling keywords out with TF-IDF, training a logistic regression news classifier, and getting Word2Vec to behave. Each example comes with a short tutorial article and its dataset, so you can follow along end to end instead of hunting for inputs. Beginners get clear answers to the fiddly details, such as how TFIDFTransformer differs from TFIDFVectorizer and when to pick HashingVectorizer over CountVectorizer. Because every task is self-contained, the repo works best as a recipe book to lift working patterns from, not as something you deploy. It also holds up as a study companion for the classic Gensim and scikit-learn text pipeline, the one most modern LLM tooling grew out of.

Articles for this project

No articles for this project yet.

To suggest a topic or contribute an article, contact us.

Related projects in this category