optimate

Why we included this project

Nebuly built these model-optimization libraries for its own use and later released them to the community. The standout is Speedster, which tests different acceleration techniques against your model and hardware so you can cut inference costs and latency without tuning each step by hand. The other two pieces handle adjacent problems: Nos helps you get more work out of your Kubernetes GPU cluster through dynamic partitioning and elastic quotas, and ChatLLaMA lowers the cost of fine-tuning and RLHF alignment. Just know the project is in a legacy phase now, with no updates or official support, so it is best treated as reference code worth stealing ideas from rather than something to build new systems on.