assisted generation

Inkling Model Integration Fixes This patch addresses several hiccups that surfaced after integrating the Inkling model. One fix resolves a problem with scaled dot-product attention (SDPA) during prefill when a position bias is used and no padding is applied with the StaticCache. Another important correction ensures that assisted generation works properly for models that rely […]