Transformers.jl

Why we included this project

Julia users who want transformer models have usually had to leave the language and call into Python. Transformers.jl changes that. It implements the transformer architecture on top of Flux.jl and includes a HuggingFace module that can load pretrained checkpoints such as bert-base-uncased and run inference through a compact API. The text encoder handles the fiddly preprocessing, including special tokens, truncation, padding, and one-hot encoding, so you are not hand-rolling tokenizers. That makes it a practical fit for Julia researchers and engineers who need BERT-style encoders inside a larger Flux pipeline, whether for embeddings, classification, or feature extraction. It is a library rather than a turnkey service, so expect to write Julia code and read the docs, but for teams already invested in the Julia ecosystem it removes the usual friction of bridging to Python for transformer work.