https://docs.google.com/presentation/d/1MRaB0376556pfngUAtC3Q5FNQ3tQu1aV/edit?usp=sharing&ouid=107956359237789538519&rtpof=true&sd=true

Code examples:

Colab autoencoder (TensorFlow)

pytorch code examples: https://uvadlc-notebooks.readthedocs.io/en/latest/tutorial_notebooks/tutorial9/AE_CIFAR10.html

pytorch lightning code examples:

https://lightning.ai/docs/pytorch/stable/notebooks/course_UvA-DL/08-deep-autoencoders.html

Followings are from wikipedia

Formulation

${\displaystyle \phi :{\mathcal {X}}\rightarrow {\mathcal {F}}}$

${\displaystyle \psi :{\mathcal {F}}\rightarrow {\mathcal {X}}}$

${\displaystyle \phi ,\psi ={\underset {\phi ,\psi }{\operatorname {arg\,min} }}\,\|X-(\psi \circ \phi )X\|^{2}}$

Simple form

${\displaystyle {\mathcal {L}}(\mathbf {x} ,\mathbf {x'} )=\|\mathbf {x} -\mathbf {x'} \|^{2}=\|\mathbf {x} -\sigma '(\mathbf {W'} (\sigma (\mathbf {Wx} +\mathbf {b} ))+\mathbf {b'} )\|^{2}}$

Variation

Denoising Autoencoder

Denoising autoencoders (DAE) try to achieve a good representation by changing the reconstruction criterion.

Two assumptions are inherent to this approach: