Feng's Notes Isn't coding fun?
Posts with the tag Algorithm:

Backward propagation of Neural Network explained

Backpropagation is the foundation of the deep neural network. Usually, we consider it to be kind of ‘dark magic’ we are not able to understand. However, it should not be the black box which we stay away. In this article, I will try to explain backpropagation as well as the whole neural network step by step in the original mathematical way.

Stochastic gradient descent

Stochastic Gradient decent is one of the fundamental algorithm in deep learning. It is used when we perform optimization of the cost function.Suppose the function is $ f(x) $


Basics of words embedding

Why embedding

Natural language processing systems traditionally treat words as discrete atomic symbols, and this may lead to some obstacles in word preprocessing:

What is Attention and how to use


Attention or Bahdanau Attention is getting more and more interest in Neural Machine Translation(NMT) and other sequence prediction research, in this article I will briefly introduce what is Attention mechanism, why important it is and how do we use it(in Tensorflow)