zeta

Why we included this project

Zeta is a PyTorch library that packages the building blocks researchers and engineers reach for when assembling modern transformer-based models. If you are tired of copying the same attention and feedforward code between projects, it is worth a look. It bundles a wide range of attention variants, mixture-of-experts routing, quantization layers like BitLinear, and complete encoder/decoder and vision-transformer structures, all exposed as drop-in modules you can compose into your own architectures. The included examples show how to wire these pieces together, including a full multi-modal vision-language model built from the library's components. Teams prototyping new model designs or teaching themselves how these pieces fit together will find it a convenient reference and starting point, though you should treat it as a component toolkit rather than a turnkey application.