After reading about wonderful neural net architectures for working with sequences of texts, I decided to try some implementations, complementing my theoretical understanding with a practical understanding. I decide to give TensorFlow a try since I had already had experience with the lovely PyTorch. Let me start you off with my RNN:
You can find my colab notebook for this implementation here!. Out of curiosity, I wanted to see the TensorBoard visualization, so let me share 2-unit wide visualization with you!
Here is my google colab notebook for this implementaiton. I also visualized a 2-unit wide version of my implementation out of curiosity. Take a look!
LSTM TensorFlow Implimentaiton
For a sanity check, I utilized TensorFlow’s implementation on the same task as my own implementation( Penn Tree Bank). Thankfully, I had identical loss functions :)You can see the colab notebook here. Here’s the visualization: