#128 · Primary category: Foundation Models

Lemur

code-generation language-model machine-learning natural-language-processing nlp text-reasoning

[ICLR 2024] Lemur: Open Foundation Models for Language Agents

Project last updated:10/28/23

GitHub Stars

557

Forks

35

Contributors

7

License

Apache-2.0

Why we included this project

Lemur is a 70B open-weight model that pairs natural-language reasoning with solid coding ability, so it works as a single backbone for teams building tool-using agents rather than a chatbot or a ready-made agent framework. You load the base and instruction-tuned checkpoints through the standard transformers pipeline and wire them into your own workflows. The repo documents the two-stage training recipe, continued pretraining of Llama-2-70B on a code-heavy corpus followed by instruction tuning on text and code examples, which makes it a useful reference if you are studying how to balance language and coding skills in one model. The weights are Apache-licensed and served with vLLM, so self-hosting for evaluation or fine-tuning is straightforward. Teams looking for open alternatives to commercial models for agent-style tasks will find the checkpoints and the evaluation scripts for benchmarks such as MMLU, HumanEval, and WebArena ready to use.

Articles for this project

No articles for this project yet.

To suggest a topic or contribute an article, contact us.

Related projects in this category