llmtools

Why we included this project

Finetuning a large language model usually means renting a cluster of GPUs, but llmtools targets the opposite situation: a single consumer card and not much memory to spare. It implements ModuLoRA, a Cornell research algorithm that combines low-precision LoRA finetuning with modular quantizers, so you can train at 2-bit, 3-bit, or 4-bit precision instead of full weights. The same code path also loads quantized weights and runs generation, and the API follows Hugging Face conventions closely enough that anyone comfortable with transformers can get going without relearning much. It is a research project rather than a polished product, so expect a focused scope and some rough edges, but for teams exploring how far low-resource finetuning can go, it is one of the few concrete implementations of this idea.