You Could Have Come Up with Kimi Delta Attention

你本可以提出Kimi Delta注意力机制

HN 186 分 · 62 条评论 · 作者 AnhTho_FR · 来源 blog.doubleword.ai · HN 讨论

【摘要】

The article uses bra-ket notation to explain the derivation of linear attention variants, specifically tracing the evolution from standard attention to DeltaNet and Kimi Delta Attention. It aims to clarify how these complex models can be understood through simple assertions about hidden state updates.

⋯ 继续阅读请登录会员 ⋯