spark-vllm-docker

Why we included this project

Most vLLM guides assume a single GPU server, so the DGX Spark owner who wants to spread a model across two or three machines is usually left to wire the cluster together themselves. This repo fills that gap with Docker images and launch scripts that run vLLM on one Spark or several linked by InfiniBand or RoCE, using either Ray or vLLM's native PyTorch distributed mode. The awkward parts are already handled: passwordless SSH between nodes, copying a downloaded model to every machine so each one doesn't fetch it again, fastsafetensors and InstantTensor loading, and pinned container images that pair a model with a tested build. Build flags let you choose precompiled wheels or a full source rebuild, which matters if you want to track vLLM's main branch. It's a community project rather than an official NVIDIA offering, so occasional breakage is the price of getting current vLLM features on this hardware.