♎
Limited AI
Ctrlk
  • Machine Learning
  • Deep Learning
    • Summary v2
    • Basic Neural Network
    • Basic CNN
    • Advance CNN
    • Basic RNN
    • Advance RNN
    • Attention Mechanisms and Transformers
      • Queries, Keys, and Values
      • Attention is all you need
        • Attention and Kernel
        • Attention Scoring Functions
        • The Bahdanau Attention Mechanism
        • Multi-Head Attention
        • Self-Attention
        • Attention的实现
      • The Transformer Architecture
      • Large-Scaling Pretraning with Transformers
      • Transformers for Vision
      • Transformer for Multiomodal
    • NLP Pretraining
  • GenAI
  • Statistics and Optimization
  • Machine Learning System Design
  • Responsible AI
  • Extra Research
Powered by GitBook
On this page
  1. Deep Learning
  2. Attention Mechanisms and Transformers

Attention is all you need

Attention and KernelAttention Scoring FunctionsThe Bahdanau Attention MechanismMulti-Head AttentionSelf-AttentionAttention的实现
PreviousQueries, Keys, and ValuesNextAttention and Kernel