Javascript is not enabled. This site can still works but it'll be more interactive when javascript is enabled.
loading...
Hacker News
new
|
past
|
comments
|
ask
|
show
|
jobs
|
submit
janalsncm
7 hours ago
|
on: Attention Is Not What You Need (2025)
In Qwen 3.5, 75% of the layers are linear attention. I think the theory is that linear attention is usually enough, but some regular attention layers help to handle more sophisticated long-range relationships.
reply
hujun236
7 hours ago
|
parent
[flagged]
reply