albert_pytorch

Why we included this project

This PyTorch implementation of ALBERT keeps the model architecture and the fine-tuning plumbing together, so you don't have to pull in a heavyweight framework just to run a lite BERT variant. It includes a script that converts the official TensorFlow checkpoints into PyTorch weights, covering the base through xxlarge sizes, and the GLUE task scripts (SST-2 among them) give you a working starting point for sentence and sentence-pair classification. The README also records the dev-set scores the author reproduced on several GLUE tasks, which helps you gauge what to expect. If you want to see how ALBERT's parameter sharing and embedding factorization are actually implemented, or need a compact reference for fine-tuning, the code is readable and easy to adapt.