BatchNorm

Shield:

These images were originally published in the book “Deep Learning with PyTorch Step-by-Step: A Beginner’s Guide”.

They are also available at the book’s official repository: https://github.com/dvgodoy/PyTorchStepByStep.

Index

• Back
• Distribution
  • Before
  • After
• Comparison
  • BatchNorm vs Initialization Scheme
  • BatchNorm vs LayerNorm

** CLICK ON THE IMAGES FOR FULL SIZE **

Papers

• LayerNorm: Layer Normalization by Lei Ba, J. et al. (2016)

Distribution

Before BatchNorm

Source: Chapter 7

After BatchNorm

Source: Chapter 7

Comparison

BatchNorm vs Initialization Scheme

Source: Chapter Extra

BatchNorm vs LayerNorm

Source: Chapter 10

This work is licensed under a Creative Commons Attribution 4.0 International License.