MLOps & Evaluation

Experiment tracking, deployment, monitoring, and model evaluation.

187 projects

See methodology for ranking rules; order uses public GitHub metrics within this scenario.

61–80 of 187

Rank Project Stars Forks
61 pipelines

Machine Learning Pipelines for Kubeflow

4.2K 2.1K
62 harbor

Framework for evaluating and improving agents

4.8K 1.7K
63 AugLy

A data augmentations library for audio, image, text, and video.

5.1K 311
64 Kiln

Build, Evaluate, and Optimize AI Systems. Includes evals, RAG, agents, fine-tuning, synthetic data generation, dataset management, MCP, and more.

5.0K 375
65 VLMEvalKit

Open-source evaluation toolkit of large multi-modality models (LMMs), support 220+ LMMs, 80+ benchmarks

4.4K 756
66 tpot

A Python Automated Machine Learning tool that optimizes machine learning pipelines using genetic programming.

10.1K 1.6K
67 transformerlab-app

The open source research environment for AI researchers to seamlessly train, evaluate, and scale models from local hardware to GPU clusters.

5.2K 547
68 lmms-eval

One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks

4.4K 647
69 MedicalGPT

MedicalGPT: Training Your Own Medical GPT Model with ChatGPT Training Pipeline.

5.8K 790
70 onnxsim

Simplify your onnx model

4.4K 432
71 cube-studio

Open-source cloud-native AI platform for end-to-end MLOps, covering deep learning and large model training, inference, pipeline orchestration, and resource management.

5.1K 881
72 alignment-handbook

Robust recipes to align language models with human and AI preferences

5.7K 491
73 SwanLab

⚡️SwanLab - an open-source, modern-design AI training tracking and visualization tool. Supports Cloud / Self-hosted use. Integrated with PyTorch / Transformers / verl / LLaMA Factory / ms-swift / Ultralytics / MMEngine / Keras etc.

4.2K 217
74 lightning-hydra-template

PyTorch Lightning + Hydra. A very user-friendly template for ML experimentation. ⚡🔥⚡

5.3K 754
75 mteb

MTEB: State-of-the-art evaluation of embeddings across languages and modalities

3.4K 681
76 autotrain-advanced

🤗 AutoTrain Advanced

4.6K 627
77 deepo

Setup and customize deep learning environment in seconds.

6.3K 736
78 polyaxon

AI Infra / AI Orchestration / AI Control Plane

3.7K 331
79 maestro

Maestro: Netflix’s Workflow Orchestrator

3.8K 311
80 trulens

Evaluation and Tracking for LLM Experiments and AI Agents

3.5K 334