Which Routing Does Each MoE Model Use? DeepSeek, Qwen, Mixtral, Llama 4, gpt-oss Compared

Why it matters

A survey of ten openly available Mixture-of-Experts models shows that while they all rely on token-choice routing, they diverge sharply on nearly every implementation detail. The comparison draws on Hugging Face configuration files, model cards, and technical reports, covering DeepSeek, Qwen, Meta, OpenAI, Moonshot, and Zhipu systems. Expert counts range from Mixtral's small setup to far larger pools in DeepSeek-V4-Pro and Kimi K2, and the number of experts consulted per token varies widely as well. Some models retain an always-on shared expert; others, including Mixtral, gpt-oss-120b, and the original Qwen3, do not. DeepSeek-V3 established a sigmoid scoring function with bias-based balancing that Kimi K2 directly inherits, while DeepSeek-V4-Pro moves to a different scoring approach and adds fixed hash routing in early layers. A notable gap is transparency: Llama 4 Maverick and gpt-oss-120b leave gating or balancing details unpublished. The piece argues these routing choices matter practically, since they determine memory footprint and communication costs when deploying such models.