Week 5: Fun Implementations

    less than 1 minute read    

Week 5

After reading about wonderful neural net architectures for working with sequences of texts, I decided to try some implementations, complementing my theoretical understanding with a practical understanding. I decide to give TensorFlow a try since I had already had experience with the lovely PyTorch. Let me start you off with my RNN:


You can find my colab notebook for this implementation here!. Out of curiosity, I wanted to see the TensorBoard visualization, so let me share 2-unit wide visualization with you!

Vanilla RNN Graph


Here is my google colab notebook for this implementaiton. I also visualized a 2-unit wide version of my implementation out of curiosity. Take a look!

LSTM Graph1

LSTM Graph2

LSTM TensorFlow Implimentaiton

For a sanity check, I utilized TensorFlow’s implementation on the same task as my own implementation( Penn Tree Bank). Thankfully, I had identical loss functions :)You can see the colab notebook here. Here’s the visualization:

LSTM Premade Graph