Multi-layer ANN

Multi-layer ANN

A fully connected multi-layer neural network is called a Multilayer Perceptron (MLP).

It has 3 layers including one hidden layer. If it has more than 1 hidden layer, it is called a deep ANN. An MLP is a typical example of a feedforward artificial neural network. In this figure, the ith activation unit in the lth layer is denoted as ai(l).

The number of layers and the number of neurons are referred to as hyperparameters of a neural network, and these need tuning. Cross-validation techniques must be used to find ideal values for these.

The weight adjustment training is done via backpropagation. Deeper neural networks are better at processing data. However, deeper layers can lead to vanishing gradient problems. Special algorithms are required to solve this issue.

Notations

In the representation below:

• ai(in) refers to the ith value in the input layer
• ai(h) refers to the ith unit in the hidden layer
• ai(out) refers to the ith unit in the output layer
• ao(in) is simply the bias unit and is equal to 1; it will have the corresponding weight w0
• The weight coefficient from layer l to layer l+1 is represented by wk,j(l)

A simplified view of the multilayer is presented here. This image shows a fully connected three-layer neural network with 3 input neurons and 3 output neurons. A bias term is added to the input vector.

Forward Propagation

In the following topics, let us look at  the forward propagation in detail.

MLP Learning Procedure

The MLP learning procedure is as follows:

• Starting with the input layer, propagate data forward to the output layer. This step is the forward propagation.
• Based on the output, calculate the error (the difference between the predicted and known outcome). The error needs to be minimized.
• Backpropagate the error. Find its derivative with respect to each weight in the network, and update the model.

Repeat the three steps given above over multiple epochs to learn ideal weights.

Finally, the output is taken via a threshold function to obtain the predicted class labels.

Forward Propagation in MLP

In the first step, calculate the activation unit al(h) of the hidden layer.

Activation unit is the result of applying an activation function φ to the z value. It must be differentiable to be able to learn weights using gradient descent. The activation function φ is often the sigmoid (logistic) function.

It allows nonlinearity needed to solve complex problems like image processing.

Sigmoid Curve

The sigmoid curve is an S-shaped curve.

Ques 1 Describe Artificial Network (ANN) with different layers.

Answer

Artificial Neural Network as the name suggests is a copy of a natural neural network present in  a human mind.

This is an image of a natural neural network which gives basic idea for the formation of artificial neural networks consisting of 3 layers ,i.e. Input Layer ,Hidden Layer,Output Layer.

Definition of ANN:

A decision function (𝜙(𝑧)) that takes a linear combination of certain input values, x, and a corresponding weight vector, w, where z is the so-called net input. 

Artificial Neural Network is able to learn from the data and provide responses in the form of predictions or classifications.It learns from the example data sets.One can enhance existing data analysis techniques owing to their advanced predictive capabilities.

It comprises of 3 layers:

1. Input Layer(The input layer is the first layer of an ANN that receives the input information)
2. Hidden Layer(They are in the middle of neural networks,there can be 1 or more hidden layers which perform computations on input data.)
3. Output Layer(It is the last layer of the neural network this layer helps to determine output through computations performed by hidden layers.)

Applications:

1. Image recognition
2. Speech recognition
3. Machine translation
4. Medical diagnosis

 

Ques 2 What are the advantages and disadvantages of Artificial Neural Network?

Answer

Advantages of Artificial Neural Networks ( ANN)

• Storing information on the entire network
• Ability to work with incomplete knowledge
• Having fault tolerance
• Having a distributed memory
• Gradual corruption

Disadvantages of Artificial Neural Networks (ANN)

• Hardware dependence
• Unexplained behavior of the network
• Determination of proper network structure 
• Difficulty of showing the problem to the network 
• The duration of the network is unknown

 

Ques 3 What are the characteristics of the Artificial Neural Network?

Answer

Characteristics of Artificial Neural Networks:

• Any Artificial Neural Network, irrespective of the style and logic of implementation, has a few basic characteristics. 
• An Artificial Neural Network consists of a large number of  “neuron” like processing elements.
• The connections between the elements provide a distributed representation of data.

 

 

Ques 4 Explain the application areas of artificial neural network.

Answer

The two important areas where ANNs have a huge potential of applications are Speech and Image Processing.

Applications in Speech

• Vowel Classification
• Recognition of vowel-consonant segments

Applications in Image Processing

• Recognition of Symbols (used in Olympics)
• Recognition of handwriting
• Segmentation of image