Deep Learning Visuals

Over 200 figures and diagrams of the most popular deep learning architectures and layers FREE TO USE in your blog posts, slides, presentations, or papers.

Home | Activation Functions | Architectures | Assorted | Attention | Batch Norm | BERT | Classification | Convolutions | Decoder | Dropout | ELMo | Encoder | FFN | Gradient Descent | Initializations and Clipping | Layer Norm | Optimizers and Schedulers | Patch Embeddings | Positional Encoding | RNNs | Seq2Seq | Transformers


Attention

Shield: CC BY 4.0

These images were originally published in the book “Deep Learning with PyTorch Step-by-Step: A Beginner’s Guide”.

They are also available at the book’s official repository: https://github.com/dvgodoy/PyTorchStepByStep.

Index

** CLICK ON THE IMAGES FOR FULL SIZE **

Papers

Attention

Scaled Dot Product

Source: Chapter 9

Source: Chapter 9

Source: Chapter 9

Source: Chapter 9

Source: Chapter 9

Alignment Scores

Source: Chapter 9

Context Vector

Source: Chapter 9

Source: Chapter 9

Attention Scores

Encoder

Source: Chapter 9

Source: Chapter 9

Decoder

Source: Chapter 9

Source: Chapter 9

Source: Chapter 9

Masked

Source: Chapter 9

Cross-Attention

Source: Chapter 9

Source: Chapter 9

Encoder+Decoder+Cross-Attention

Source: Chapter 9

Translation Example

Source: Chapter 9

Source: Chapter 9

Source: Chapter 9

Source: Chapter 9

Source: Chapter 9

Source: Chapter 9

Multi-Headed Attention

Source: Chapter 9

Wide Attention

Source: Chapter 9

Narrow Attention

Source: Chapter 10

Source: Chapter 10

Source: Chapter 10

Self-Attention

Self-Attention Encoder

Source: Chapter 9

Source: Chapter 9

Source: Chapter 9

Source: Chapter 10

Self-Attention Decoder

Source: Chapter 9

Source: Chapter 9

Source: Chapter 10

Encoder+Decoder

Source: Chapter 9

This work is licensed under a Creative Commons Attribution 4.0 International License.

CC BY 4.0