What biological attention can teach artificial attention mechanisms
Q1I model attention mechanisms in biological neural networks, and the differences from transformer attention are striking. Biological attention is inherently multi-scale (from millisecond saccades to minutes-long sustained attention), energy-budget-constrained, and deeply integrated with reward signals. Transformer attention is flat, uniform, and energy-blind. Can biology teach us to build better artificial attention?
Q2The reward-gated attention idea is interesting. In neuroscience, we know that dopaminergic prediction error signals modulate attention allocation — unexpected rewards or threats get disproportionate processing. Could this principle help recommendation systems prioritize what to deeply analyze?