merve

releasing my local AI slide deck covers from prefill vs decode, MoE vs dense, VRAM vs unified memory, quantization to speculative decoding, everything with llama.cpp feel free to reuse with attribution

if you aren't using https://llama.app to run models locally then I have nothing to say to you 🙂‍↔️

you don't need large GPUs to run most decision models, this makes them a perfect match for llama.cpp give the blog a read on how to get them up and running! team will support more and more models in upcoming days

October will be a releaseful month @huggingface

reason why I use my local setup (Qwen-3.8 + Hermes) as personal agent and sometimes a pen + paper for my to-dos

if you want to get every last bit of performance out of your local setup, you need to know a bit about inference 🥵 but we've got you covered, shipping conceptual guides 🔥 here's the first one about prefill, decode, KV cache what you should optimize for 🙌🏻

Le Chonko will be open-sourced 🥵 https://huggingface.co/mistralai/Mistral-Large-4.0-1T05-A52B

"I prefilled but I couldn't decode" me, 2026, when I forgot what I was going to say to @multimodalart for a sec

Meta is releasing Muse Spark Open as the most-capable US open-weight model.. 🥁 and it's an agent focused MoE that takes all modalities 🔥 Welcoming the model to Hub with day-0 support in transformers and llama.cpp 🤗 https://huggingface.co/blog/meta-muse-spark-open

personal take: as much as I'm techno optimist, I genuinely dislike it when people on X dunk on regular folk for criticizing AI a company has told everyone they will lose their jobs, many companies did layoffs, people have families. on top of saying all the layoffs are necessary so AI proves econ value until it cures the cancer or saves planet but they did anything but those so far you might be making a ton from AI, many don't. some people also found a ton of purpose from their work which is gone. I'm not saying we should console people but maybe also don't dunk on a frustrated mass? the field is already detested and this is not helping

new Llama.cpp release ships with (multimodal!) Jev-like models support, performance upgrade for Metal and more! 🔥 super simple: llama serve -hf ggml-org/Clef-Flash-GGUF browse all the decision models here https://huggingface.co/models?apps=llama.cpp&other=decision-model&sort=trending we also polished Llama App website & docs https://llama.app 🌟

"I run my models locally" this is how it looks like in 2026 you get model recommendations and zero excuses not to run models locally