#138 · Primary category: MLOps & Evaluation

can-i-finetune-this

bitsandbytes fine-tuning gpu hugging-face llm lora memory-estimation peft pytorch qlora transformers vram

Estimate whether a Hugging Face model fits and fine-tunes on your local GPU.

Project last updated:07/23/26

GitHub Stars

792

Forks

107

Contributors

1

License

MIT

Why we included this project

Anyone who has downloaded a 14 GB model only to watch a 12 GB card run out of memory on the first training step will recognize the problem this CLI solves. It estimates up front whether a given Hugging Face model can be fine-tuned on your specific GPU, and it breaks the VRAM budget into parts that static estimators usually skip, like the logits/cross-entropy chain and the fp32 upcast of embeddings under QLoRA. When the answer is yes, it can also find a workable batch size, sequence length, LoRA rank, and quantization, then hand you a ready-to-run PEFT + TRL training script for that configuration. The benchmark and calibrate commands let you check those estimates against real measurements on your own machine instead of trusting a guess. For solo developers and small teams fine-tuning open-weight LLMs on consumer hardware, that turns a frustrating trial-and-error loop into a quick decision.

Articles for this project

No articles for this project yet.

To suggest a topic or contribute an article, contact us.

Related projects in this category