An Overview on Multilayer Perceptron (MLP)

Understanding Single-layer ANN

Perceptron rule and Adaline rule were used to train a single-layer neural network.

Weights are updated based on a unit function in perceptron rule or on a linear function in Adaline Rule.

History of Multi-layer ANN

Deep Learning deals with training multi-layer artificial neural networks, also called Deep Neural Networks. After Rosenblatt perceptron was developed in the 1950s, there was a lack of interest in neural networks until 1986, when Dr.Hinton and his colleagues developed the backpropagation algorithm to train a multilayer neural network. Today it is a hot topic with many leading firms like Google, Facebook, and Microsoft which invest heavily in applications using deep neural networks.

Multi-layer ANN

A fully connected multi-layer neural network is called a Multilayer Perceptron (MLP).

It has 3 layers including one hidden layer. If it has more than 1 hidden layer, it is called a deep ANN. An MLP is a typical example of a feedforward artificial neural network. In this figure, the ith activation unit in the lth layer is denoted as ai(l).

The number of layers and the number of neurons are referred to as hyperparameters of a neural network, and these need tuning. Cross-validation techniques must be used to find ideal values for these.

The weight adjustment training is done via backpropagation. Deeper neural networks are better at processing data. However, deeper layers can lead to vanishing gradient problems. Special algorithms are required to solve this issue.

Notations

In the representation below:

• ai(in) refers to the ith value in the input layer
• ai(h) refers to the ith unit in the hidden layer
• ai(out) refers to the ith unit in the output layer
• ao(in) is simply the bias unit and is equal to 1; it will have the corresponding weight w0
• The weight coefficient from layer l to layer l+1 is represented by wk,j(l)

A simplified view of the multilayer is presented here. This image shows a fully connected three-layer neural network with 3 input neurons and 3 output neurons. A bias term is added to the input vector.

Your AI/ML Career is Just Around The Corner!

AI Engineer Master's ProgramExplore Program

Forward Propagation

In the following topics, let us look at  the forward propagation in detail.

MLP Learning Procedure

The MLP learning procedure is as follows:

• Starting with the input layer, propagate data forward to the output layer. This step is the forward propagation.
• Based on the output, calculate the error (the difference between the predicted and known outcome). The error needs to be minimized.
• Backpropagate the error. Find its derivative with respect to each weight in the network, and update the model.

Repeat the three steps given above over multiple epochs to learn ideal weights.

Finally, the output is taken via a threshold function to obtain the predicted class labels.

Forward Propagation in MLP

In the first step, calculate the activation unit al(h) of the hidden layer.

Activation unit is the result of applying an activation function φ to the z value. It must be differentiable to be able to learn weights using gradient descent. The activation function φ is often the sigmoid (logistic) function.

It allows nonlinearity needed to solve complex problems like image processing.

Sigmoid Curve

The sigmoid curve is an S-shaped curve.

Activation of Hidden Layer

The activation of the hidden layer is represented as:

z(h) = a(in) W(h)

a(h) =

For the output layer:

Z(out) = A(h) W(out)

A(out) =

Choose the Right Program

Supercharge your career in AI and ML with Simplilearn's comprehensive courses. Gain the skills and knowledge to transform industries and unleash your true potential. Enroll now and unlock limitless possibilities!

Program Name	AI Engineer	Post Graduate Program In Artificial Intelligence	Post Graduate Program In Artificial Intelligence
Geo	All Geos	All Geos	IN/ROW
University	Simplilearn	Purdue	Caltech
Course Duration	11 Months	11 Months	11 Months
Coding Experience Required	Basic	Basic	No
Skills You Will Learn	10+ skills including data structure, data manipulation, NumPy, Scikit-Learn, Tableau and more.	16+ skills including chatbots, NLP, Python, Keras and more.	8+ skills including Supervised & Unsupervised Learning Deep Learning Data Visualization, and more.
Additional Benefits	Get access to exclusive Hackathons, Masterclasses and Ask-Me-Anything sessions by IBM Applied learning via 3 Capstone and 12 Industry-relevant Projects	Purdue Alumni Association Membership Free IIMJobs Pro-Membership of 6 months Resume Building Assistance	Upto 14 CEU Credits Caltech CTME Circle Membership
Cost	$$	$$$$	$$$$
	Explore Program	Explore Program	Explore Program

Conclusion

New age technologies like AI, machine learning and deep learning are proliferating at a rapid pace. And if you wish to secure your job, mastering these new technologies is going to be a must.

This tutorial covered everything about multilayer artificial neural networks. However, if you wish to master AI and machine learning, Post Graduate Program In Artificial Intelligence, in partnership with IBM, must be your next stop. Together with Caltech’s top faculty masterclasses and Simplilearn’s online bootcamp, become an AI and machine learning pro like never before!