Introduction
The latest release of the Hugging Face Transformers library, version 5.15.0, brings a host of new multimodal models, important breaking changes, and a wealth of bug fixes and optimizations across attention mechanisms, vision processing, generation, caching, kernels, quantization, audio, parallelization, tokenization, and more. This release is particularly notable for the addition of several state-of-the-art models like Meta’s Muse Glimmer and SKT’s A.X series, as well as foundational enhancements that improve performance and maintainability for self-hosted deployments.
New Model Additions
Meta’s Muse Glimmer is a dense 30B parameter multimodal model released under Apache 2.0, designed for agentic use cases. It consists of a 2B ViT-style perception encoder and a 28B text decoder, making it suitable for privacy-aware local applications such as coding, document analysis, and personal assistants. Alongside this, the release introduces support for IBM’s GraniteMoeSWA and GraniteSWA models, which leverage sparse weight averaging techniques for enhanced efficiency. Additionally, SKT’s A.X-K1 and A.X-K2 models are now integrated, offering new capabilities in vision-language tasks, and Cosmos3 Edge, a model optimized for edge deployments, has been added.
Breaking Changes
Several breaking changes require user attention. Kernels for linear attention models (like Mamba and GDN) are now opt-in rather than mandatory, meaning users who previously relied on automatic kernel selection must enable them explicitly. The cache cropping API now only accepts negative values (relative offsets) instead of absolute sizes, so direct callers need to adjust their code. The T5 model family now supports multiple attention backends, including SDPA, and the default attention implementation may differ; users depending on the previous eager path should set attn_implementation="eager" to avoid issues. Finally, some private helper functions (e.g., _is_url, _build_image_tokens) have been removed from multimodal processors; any downstream library importing these directly must update accordingly.
Attention and Vision Improvements
Attention mechanisms saw critical fixes, including corrections to Multi-Head Latent Attention (MLA) cache compression, optimizations for Flash Attention max sequence length computation in vision models, and bug fixes for CTRL flex-attention and SDPA prefill with position bias. Modernized conv-only model handling and clearer heterogeneous attention configuration for Gemma 4 improve maintainability. On the vision front, image preprocessing for vision-language models has been significantly accelerated by eliminating redundant tensor copies, and the Flash Attention variable-length path is now more efficient by precomputing maximum sequence lengths once per forward pass. Several dtype and initialization bugs were squashed in models like Kosmos2, Phi4Multimodal, and Hunyuan-VL.
Generation and Caching Enhancements
Generation capabilities now include batched audio generation for Qwen2.5/3-Omni, and speculative decoding now works with sliding window cache layers. Memory overhead from static cache persistence across generate() calls has been eliminated. Multiple model-specific generation crashes were fixed, including those in KyutaiSpeechToText, MusicgenForCausalLM, and assisted decoding for EncoderDecoder cache and OlmoHybrid models. Cache-related bugs, such as a missing layer-type mapping in NemotronH and improper handling of recurrent-layer padding masks during chunked prefill, have been resolved.
Kernels and Quantization
The kernels integration is moving toward a required dependency for transformers[torch], with improved robustness through layer-based function handling and fixes for kernel downloads on read-only filesystems. Quantization support expanded with FP8 kernels for compressed-tensors models, fixes for FP8 module normalization, and a multi-device MXFP4 dequantization race condition fix. Tests for GPTQ and MXFP4 now cover Intel XPU devices.
Audio, Parallelization, Tokenization, and More
Audio models gained batched generation support and various fixes for dtype mismatches and missing backend guards. FSDP parallelization now covers 94 ForCausalLM classes with auto-generated plans, end-to-end tests, and a dedicated CI job. Tokenization adds native support for Mistral’s “tekken” tokenizer format and fixes a CodeLlama whitespace and a ReDoS vulnerability. The serve chat parsing has been unified to simplify new model support, and a multitude of other bug fixes and improvements, from spelling corrections to training loop enhancements, round out the release.
Community Contributions
This release owes much to the significant contributions of community members. Notable among them are ydshieh, who tirelessly improved CI pipelines and kernel download robustness; kaixuanliu, who ensured broad device compatibility and fixed test suites for XPU and other hardware; vasqu, who overhauled kernel integration and added new model architectures; kmswin1, who championed the A.X model family; remi-or, who patched attention and vision bugs; juliendenize, who delivered the tekken tokenizer support; IMvision12, who enabled batched audio generation; jiqing-feng, who integrated SDPA for T5 and squashed generation bugs; and many others whose efforts have made this release more stable and feature-rich. For a full list of changes, see the official changelog.
