♎
Limited AI
search
⌘Ctrlk
♎
Limited AI
  • Machine Learning
  • Deep Learning
    • Summary v2
    • Basic Neural Network
    • Basic CNN
    • Advance CNN
    • Basic RNN
    • Advance RNN
    • Attention Mechanisms and Transformers
      • Queries, Keys, and Values
      • Attention is all you need
        • Attention and Kernel
        • Attention Scoring Functions
        • The Bahdanau Attention Mechanism
        • Multi-Head Attention
        • Self-Attention
        • Attention的实现
      • The Transformer Architecture
      • Large-Scaling Pretraning with Transformers
      • Transformers for Vision
      • Transformer for Multiomodal
    • NLP Pretraining
  • GenAI
  • Statistics and Optimization
  • Machine Learning System Design
  • Responsible AI
  • Extra Research
gitbookPowered by GitBook
block-quoteOn this pagechevron-down
  1. Deep Learningchevron-right
  2. Attention Mechanisms and Transformers

Attention is all you need

Attention and Kernelchevron-rightAttention Scoring Functionschevron-rightThe Bahdanau Attention Mechanismchevron-rightMulti-Head Attentionchevron-rightSelf-Attentionchevron-rightAttention的实现chevron-right
PreviousQueries, Keys, and Valueschevron-leftNextAttention and Kernelchevron-right