BERT-pytorch

Why we included this project

This codebase is a PyTorch implementation of Google's 2018 BERT, written with clarity as the priority. The modules are small and heavily annotated, so you can follow exactly how masked language modeling and next-sentence prediction are implemented, which makes it a good bridge from theory to code. It also includes command-line tools for building a vocabulary from your own corpus and pre-training a model, so you can run the whole training loop on a small dataset to see it in action. The project is explicitly unfinished and unverified, so treat it as a learning and reference resource rather than production-ready software.