ESFT

Why we included this project

Full fine-tuning of a Mixture-of-Experts model is wasteful because most experts sit idle for any given task. ESFT works around that by scoring each expert against your evaluation data, keeping only the task-relevant ones trainable, and generating a sparse adapter config so you update just a fraction of the parameters. The repo includes the training code, the expert-scoring and config-generation scripts, and evaluation utilities, so you can run the whole pipeline on your own MoE model and dataset instead of just reading about it. It's a research codebase tied to an EMNLP paper, so expect to adapt scripts and configs rather than use a polished library. If you're fine-tuning sparse MoE models and want to cut memory and storage without losing task performance, this is a concrete reference implementation worth studying.