Show Menu

Neural Nets Cheat Sheet by


An Artificial Neural Network (ANN) is an inform­ation processing paradigm that is inspired by the way biological nervous systems, such as the brain, process inform­ation.
The key element of this paradigm is the novel structure of the inform­ation processing system. It is composed of a large number of highly interc­onn­ected processing elements (neurones) working in unison to solve specific problems.
ANNs, like people, learn by example. An ANN is configured for a specific applic­ation, such as pattern recogn­ition or data classi­fic­ation, through a learning process. Learning in biological systems involves adjust­ments to the synaptic connec­tions that exist between the neurones. This is true of ANNs as well.


Adaptive learning: An ability to learn how to do tasks based on the data given for training or initial experi­ence.
Self-O­rga­nis­ation: An ANN can create its own organi­sation or repres­ent­ation of the inform­ation it receives during learning time.
Real Time Operation: ANN comput­ations may be carried out in parallel, and special hardware devices are being designed and manufa­ctured which take advantage of this capabi­lity.

Neural Model

Neural Net


Learning Process

Activation Function
Each neuron has an activation function that defines the output of the neuron
The activation function is used to introduce non-li­nearity in the modeling capabi­lities of the network
occurs when the network is exposed to the training data and these cross the entire neural network for their predic­tions (labels) to be calcul­ated.
loss function
to estimate the loss (or error) and to compare and measure how good/bad our prediction result was in relation to the correct result (remember that we are in a supervised learning enviro­nment and we have the label that tells us the expected value). Ideally, we want our cost to be zero
as the model is being trained, the weights of the interc­onn­ections of the neurons will gradually be adjusted until good predic­tions are obtained.
Once the loss has been calcul­ated, this inform­ation is propagated backwards.

Visual Scheme

Learning process of one perceptron



No comments yet. Add yours below!

Add a Comment

Your Comment

Please enter your name.

    Please enter your email address

      Please enter your Comment.

          Related Cheat Sheets

          NTLK Language Processing Python Cheat Sheet
          Clean ML code and PEP8 guidelines Cheat Sheet