TheAIBlog.
GPU economics, inference, multi-node training and bare-metal infrastructure — notes from the team running whole-node B300.
Kimi K3 (2.8T) and six other open models on a single 8×B300 node — full vLLM numbers
We ran seven current open models — from Gemma 4 31B to Kimi K3's 2.78T parameters — through out-of-box vLLM on one 8× B300 node. TTFT, throughput, what didn't launch, and what surprised us.
The economics of large-model inference on GPUs
Why inference cost is set by memory bandwidth and the KV cache, not raw FLOPs — and how batching, quantization and VRAM headroom decide your cost per token.