Attention Dilution
Attention dilution (also called context dilution) is one of the fundamental limitations of transformer-based LLMs when dealing with long contexts or extended agent memory.
Bin Zhang
Exploring Data, AI, and Engineering.
Featured
Recent Posts
AI Terminology: Agents, Skills, RAG, MCP, and the Layers Beneath the Hype
How many of these terms do you actually recognize?
From Prompt to Response: A Step-by-Step Walkthrough of LLM Inference
From input to output, a prompt generally goes through seven steps: request packaging, tokenization, inference scheduling, prefill, and decode before...
ChatGPT in 2025: A Year in Review
ChatGPT Stats ChatGPT Growth ChatGPT Revenue