mt-dnn

Why we included this project

MT-DNN is a research codebase from Microsoft that shows how a single neural network can be trained on several natural language understanding tasks at once. Built on PyTorch and the Transformers library, it shares one backbone across text classification, named-entity recognition, reading comprehension, and ranking-style objectives within a single training run. Alongside the core model, the repo packages techniques from the team's papers, including adversarial training for pre-training and fine-tuning, knowledge distillation, and f-divergence based regularizers, which help when robustness matters as much as raw accuracy. It includes GLUE and SQuAD training scripts, so it works as a practical starting point for reproducing or adapting these experiments to your own data. Expect a research toolkit you read and modify, not a finished product.