https://utsavdarlami.github.io/blogs/categories/deep-learning/Recent content in Deep Learning on felladogHugo -- gohugo.ioenSun, 03 Mar 2024 21:33:07 +0545https://utsavdarlami.github.io/blogs/notes/2021-05-26--11-31-31z--neural_network/Wed, 26 May 2021 17:16:00 +0545https://utsavdarlami.github.io/blogs/notes/2021-05-26--11-31-31z--neural_network/References : Done Reading: Sebastian Raschka Perceptron Lecture Slide Reading: Tom Mitchell Lectures slides of chapter 4 Tom Mitchell, Machine Learning Chapter 4 To Read: https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/ https://zzsza.github.io/data/2018/05/13/cs231n-backpropagation-and-neural-networks/ Questions : Biological Motivation Mimicing human brain Figure 1: biological neuron vs aritifical neuron from this post properties that match with the human brain Many neuron like threshold switching units Many weighted interconnections among units Highly parallel, distributed process Emphasis on tuning weights automatically Representations The ANNs can graphs be with many types of structures: acyclic or cyclic directed or undirected The backprop algorithm assumes ANN to have structure that corresponds to a directed graph, and possibly containing cyclees.https://utsavdarlami.github.io/blogs/notes/2021-03-06--02-21-33z--batch_normalization/Sat, 06 Mar 2021 08:06:00 +0545https://utsavdarlami.github.io/blogs/notes/2021-03-06--02-21-33z--batch_normalization/Acknowledgement : d2l.ai batch normalization L11.2 How BatchNorm Works - Sebastian Raschka https://twitter.com/karpathy/status/1013245864570073090 To read: Why is batch norm becoming so unpopular Questions : What is Normalization? Normalization A popular and effective technique that constantly accelerates the convergence of deep network. Normalizes hidden layer inputs. Helps with exploding/vanishing gradient problems. Can increase the training stability and convergence rate. Can be understood as additional(normalization) layers(with additional parameters). How is it done Batch normalization is applied to individual layers (optionally, to all of them) and works as follows:https://utsavdarlami.github.io/blogs/notes/20210213135051-recurrent_neural_networks/Tue, 02 Mar 2021 12:40:00 +0545https://utsavdarlami.github.io/blogs/notes/20210213135051-recurrent_neural_networks/ Acknowledgement https://d2l.ai/chapter%5Frecurrent-neural-networks/index.html https://github.com/fastai/fastbook/blob/master/12%5Fnlp%5Fdive.ipynb https://stanford.edu/~shervine/teaching/cs-230/cheatsheet-recurrent-neural-networks https://colah.github.io/posts/2015-08-Understanding-LSTMs/ [Mainly LSTM] https://deeplearning.cs.cmu.edu/S20/document/recitation/recitation-7.pdf http://cs231n.stanford.edu/slides/2017/cs231n%5F2017%5Flecture10.pdf https://www.youtube.com/watch?v=6niqTuYFZLQ https://github.com/fastai/fastbook/blob/master/12%5Fnlp%5Fdive.ipynb [ fast ai nlp dive rnn archi] http://ethen8181.github.io/machine-learning/deep%5Flearning/rnn/1%5Fpytorch%5Frnn.html#Recurrent-Neural-Network-(RNN) [Main reference] https://medium.com/ecovisioneth/building-deep-multi-layer-recurrent-neural-networks-with-star-cell-2f01acdb73a7 [Multi Layer] https://towardsdatascience.com/pytorch-basics-how-to-train-your-neural-net-intro-to-rnn-cb6ebc594677 https://www.jeremyjordan.me/introduction-to-recurrent-neural-networks/ https://towardsdatascience.com/pytorch-basics-how-to-train-your-neural-net-intro-to-rnn-cb6ebc594677 Sebastian Raschka Character Generation using lstm cell pytorch Recurrent neural networks (RNNs) are designed to better handle sequential information. (text and stocks) RNNs introduce state variables(hidden state) to store past information, together with the current inputs, to determine the current outputs. Figure 1: rnn vs nn Figure 2: a folded rnn Figure 3: a unfolded rnn Internal State of RNN Figure 4: a state diagram, Pytorch Implementation Summary