[r/ML] [D] - 1M tokens/second serving Qwen 3.5 27B on B200 GPUs, benchmark results and findings

Summary

Researchers achieved an impressive 1.1 million tokens/second serving Qwen 3.5 27B (FP8) on 96 B200 GPUs using vLLM, demonstrating significant advancements in large language model inference speed. A key finding was that data parallelism (DP=8) nearly quadrupled throughput over tensor parallelism for this model size on B200s, highlighting optimal parallelization strategies. The setup also showed high scaling efficiency across multiple nodes, pushing the boundaries of high-throughput LLM serving.

Continue Reading

Explore related coverage about community news and adjacent AI developments: [r/ML] [D] MYTHOS-INVERSION STRUCTURAL AUDIT, [r/LocalLLaMA] karpathy / autoresearch, [r/ML] [R] Agentic AI and Occupational Displacement: A Multi-Regional Task Exposure Analysis (236 occupations, 5 US metros), [r/ML] Building behavioural response models of public figures using Brain scan data (Predict their next move using psychological modelling) [P].

Related Articles

• [r/ML] [D] MYTHOS-INVERSION STRUCTURAL AUDIT

  March 29, 2026

• [r/LocalLLaMA] karpathy / autoresearch

  March 10, 2026

• [r/ML] [R] Agentic AI and Occupational Displacement: A Multi-Regional Task Exposure Analysis (236 occupations, 5 US metros)

  April 7, 2026

• [r/ML] Building behavioural response models of public figures using Brain scan data (Predict their next move using psychological modelling) [P]

  April 5, 2026