Doubled Qwen3.6-27B decode speed on an Apple M5 Pro, from 14.6 to 29.4 tokens per second, with Metal GPU kernels and speculative decoding in llama.cpp. Every change is measured with a frozen A/B harness and logged, failures included.
C++ · Metal · llama.cpp · GPU kernels · Code · Kernels
Runs DeepSeek V4 Flash on Apple silicon machines without enough memory to hold it, by streaming mixture-of-experts weights from SSD through a slot-bank cache. Fork of the DwarfStar 4 engine.
C · Metal · Apple Neural Engine · MoE · Code