docarray

Why we included this project

Multimodal data rarely arrives as a single clean tensor. In practice it is text alongside images, audio, embeddings, and metadata that all have to stay attached from training through to search. DocArray gives you Pydantic-based typed structures that hold those heterogeneous fields, let you declare tensor shapes, and batch them into vectors ready for model training. To move or persist that data, it serializes to JSON over HTTP or Protobuf over gRPC, and it ships ready-made connectors to Weaviate, Qdrant, Elasticsearch, Redis, and Mongo Atlas, so the same objects you define can be indexed and pulled back with nearest-neighbor search. That makes it a practical fit for teams building semantic-search or RAG-style pipelines who want one consistent schema across storage backends instead of hand-writing converters. It integrates natively with NumPy, PyTorch, TensorFlow, and JAX, so it drops into existing deep-learning workflows without imposing a new framework.