Week 5: Fun Implementations

    less than 1 minute read    

Week 5

After reading about wonderful neural net architectures for working with sequences of texts, I decided to try some implementations, complementing my theoretical understanding with a practical understanding. I decide to give TensorFlow a try since I had already had experience with the lovely PyTorch. Let me start you off with my RNN:

RNN

You can find my colab notebook for this implementation here!. Out of curiosity, I wanted to see the TensorBoard visualization, so let me share 2-unit wide visualization with you!

Vanilla RNN Graph

LSTM

Here is my google colab notebook for this implementaiton. I also visualized a 2-unit wide version of my implementation out of curiosity. Take a look!

LSTM Graph1

LSTM Graph2

LSTM TensorFlow Implimentaiton

For a sanity check, I utilized TensorFlow’s implementation on the same task as my own implementation( Penn Tree Bank). Thankfully, I had identical loss functions :)You can see the colab notebook here. Here’s the visualization:

LSTM Premade Graph

Updated:

You May Also Enjoy

Final Project: Transformer Knowledge Distillation

    7 minute read    

After learning about the Transformer and it’s incredible capacity to work with language, I was I excited to integrate it into a tool for people to use. I ha...

Week 4: Fun Architectures

    7 minute read    

Week 4 I’ll be sharing some wonderful neural network architectures for working with sequences of text! Here they are: Vanilla Recurrent Neural Net (RN...

Week 3: Refined Focus

    less than 1 minute read    

Week 3 I have enjoyed an wonderful breadth of knowledge the past two weeks, and now it’s time to refine my plan and focus! To accelerate my progression, I ...

Week 2: Getting A Groove

    4 minute read    

Week 2 After kicking this week off, it took some time but I feel that I’ve got a groove I can get with to get things done! I was exposed to a lot of new kn...