Unsloth AI

Qwen-Image-2.1 can now run locally on 12GB VRAM with Unsloth GGUFs! 🖼️ The 7B model performs on par with Nano Banana 2.0. For higher quality, you can also run Dynamic FP8 on just 6GB of VRAM via offloading. GGUF: https://huggingface.co/unsloth/Qwen-Image-2.1-GGUF Guide: https://unsloth.ai/docs/models/qwen-image-2.1

We made GLM-5.3-Flash run 3.3x faster locally! Local GGUF inference is now 1.6–3.4× faster with optimized decoding and bonus multi-token prediction. Run 3-bit on 128GB setups via Unsloth Desktop or llama.cpp. Guide: https://unsloth.ai/docs/models/glm-5.3-flash#faster-inference-and-mtp-support GGUF: https://huggingface.co/unsloth/GLM-5.3-Flash-GGUF

You can now run Laya Decision models locally on just 4GB RAM! 🔥 Works on CPU, Mac, Windows, Linux and GPU setups. Serve Laya through a Jev-compatible API via Unsloth Desktop. GitHub: https://github.com/unslothai/unsloth Guide: https://unsloth.ai/docs/models/decision-laya

Qwen3.8-Flash can now run 1.7× faster locally with MTP!⚡️ GGUFs can reach 170 tokens/s on a RTX PRO 6000. MTP enables Qwen3.8-Flash-Next ~1.3–1.7× faster inference with no accuracy change. GGUFs: https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF Guide: https://unsloth.ai/docs/models/qwen3.8-next

Qwen3.8-27B Unsloth GGUF is now the #1 most-liked GGUF of all time! The model hit 10M downloads and 3.7K likes in just 24 days on Hugging Face - all thanks to you. 🤗🦥 GGUF: https://huggingface.co/unsloth/Qwen3.8-27B-GGUF Guide: https://unsloth.ai/docs/models/qwen3.8

You can now run Unsloth GGUFs locally in one-click via Hermes! ✨ Qwen3.8-27B, Qwen3.8-Flash, DeepSeek-V4-Flash and more are all supported.

You can now train and run 500+ models locally with our Unsloth Docker image! 🐳 Use our new GUI or notebooks workflow. No setup required. Works on NVIDIA and AMD. Guide: https://unsloth.ai/docs/get-started/install/docker GitHub: https://github.com/unslothai/unsloth

Unsloth has surpassed 500M model downloads on Hugging Face! 🦥🤗 Qwen3.8-27B GGUF is already Unsloth’s #1 most-downloaded model ever. Thanks for all your support!

We’re releasing new DeepSeek-V4.1-Flash-1017 GGUFs, ranking highest across benchmarks. Dynamic V3.1 leads across nearly all benchmarks: GPQA Diamond, AIME25, LiveCodeBench v6, MMLU-Pro, Aider & more. Run on 64GB+ RAM. GGUF: https://huggingface.co/unsloth/DeepSeek-V4.1-Flash-1017-GGUF Blog: https://unsloth.ai/docs/models/deepseek-v4.1-flash

DeepSeek V4.1-Flash-1017 can now run locally! 🐳 We made 1-bit retain ~88.2% accuracy while shrinking size from 508GB to 250GB (−52%). Run on a 128GB+ RAM device. V4.1 Flash 1017 is the strongest open model to date Guide: https://unsloth.ai/docs/models/deepseek-v4.1-flash GGUF: https://huggingface.co/unsloth/DeepSeek-V4.1-Flash-1017-GGUF

Meta releases Muse Spark Open, a new SOTA open model. Run on a Mac Studio via Dynamic 2-bit Unsloth GGUFs. Muse-Spark-Open-816B-A42B supports text, images, video and audio input with 1M context. GGUF: https://huggingface.co/unsloth/Muse-Spark-Open-816B-A42B-GGUF Guide: https://unsloth.ai/docs/models/muse-spark-open

Windows now has sandboxing! Microsoft released an open-source repo, mxc, for sandboxed code execution. We collaborated with Windows to add mxc OS level sandboxing to Unsloth which adds just <100 ms of overhead. GitHub: https://github.com/unslothai/unsloth Guide: https://unsloth.ai/docs/new/studio/sandboxing-in-unsloth