Developer reports near-Strata MoE inference on a 32GB Mac with a Transformers prototype
Tarek Ziadé reports a Transformers prototype for running the 180B-parameter Qwen3.8-Flash-Next language model on a 32GB Mac. It streams the mixture-of-experts model's selected weights from storage through a bounded cache, alongside model-specific memory handling. His final tests approach Strata's generation speed, but he still finds it too slow …
SourcesHugging Face Blog
Read development