Rectified Linear Unit (ReLU)

 

Rectified Linear Unit (ReLU)

 

In a neural network, the activation function is answerable for converting the added weighted input from the knot into the activation of the knot or product for that input.

 

The corrected direct relu activation function or ReLU for little is a piecewise direct function that will product the input directly if it's positive, else, it'll product zero. It has come the misprision activation function for numerous types of neural networks because a model that uses it's effortless to train and frequently achieves better version.

 

Before I spade into the details of activation functions, let us rapidly come through the conception of neural networks and how they create. A neural network is a veritably significant machine literacy medium which principally mimics how a natural brain learns.

 

The brain receives the boost from the outside world, does the processing on the input, and also generates the product. As the task gets elaborate, many neurons form a complex network, reaching information among themselves.

An Artificial Neural Network tries to mimic a alike conduct. The network you see below is a neural network made of connected neurons. Each neuron is characterized by its weight, bias and activation function.

 

 

Activation Function

 

We understand that using an activation function introduces an fresh step at each subcaste during the forward propagation. Now the question is – if the activation function increases the difficulty so much, can we do without an activation function?

.

Imagine a neural network without the activation functions. In that case, every neuron will only be performing a direct changeover on the inputs using the weights and impulses. Although direct changeovers make the neural network simpler, but this network would be less important and won't be competent to learn the complex patterns from the data.

 

A neural network without an activation function is basically just a direct retrogression model.