apple-silicon streaming-format speculative-decoding openai-compatibility release-notes

Qwen3.5 faster on Apple, OpenAI streaming parity

MLX engine now auto-enables speculative decoding via MTP head; streaming format aligns with OpenAI's wire protocol for easier client migration.

Summary

Speculative decoding on Apple Silicon reduces inference latency for local deployments. Matching OpenAI's streaming format eliminates format translation logic when switching providers or running local alternatives.

Why it matters

Implementation verdict

Replaces manual speculative decoding setup for Qwen3.5 on MLX; requires upgrading to v0.32.6-rc0. Streaming change is transparent if you parse chunks generically, but verify finish_reason and usage handling in your streaming consumer. Image generation users must stay on 0.32.5 until feature returns. Worth upgrading for Apple GPU users and OpenAI client compatibility; treat RC as release-candidate-ready.

Sources

1.Qwen3.5 is faster on Apple GPUs: the MLX engine now uses the model's MTP head for speculative decoding automatically
2./v1/chat/completions streaming now matches OpenAI's wire format: role only on the first chunk, finish_reason on its own chunk, and usage in a separate chunk with stream_options.include_usage
3.Experimental image generation has been temporarily removed. Continue using 0.32.5 for image generation support

Dev Signal

Get briefs like this in your inbox — free, every weekday.

100+ sources compressed into one 4-minute read. Ranked, cited, implementation-ready.

Read the full issue →All briefs