Ever gazed at the marvels of artificial intelligence and wondered, "How on Earth do machines learn?" Well today we are unraveling this mystery and taking our first step into the fascinating world of neural networks .In this puzzle, we're starting with that corner piece – the perceptron. While the world of neural networks is deep and intricate, our journey today is all about simplicity. So, get ready to learn the basics of neural networks as we create our very own perceptron.
Firstly, What is a Perceptron ?
Perceptron is a single-layer neural network used for supervised learning of various binary classifiers. Think of it as a digital decision-maker – a building block that helps machines make sense of information.
In technical terms, a perceptron takes multiple binary inputs, assigns weights to each input, sums them up, and then applies an activation function to produce an output.
Imagine you're deciding whether to go for a walk. Your decision might be influenced by various factors like the weather, your mood, and the time of day. Each of these factors can be represented as inputs to a perceptron. The perceptron assigns weights to each factor based on its importance, calculates the sum, and finally, decides whether it's a good day for a stroll. Hence, it is a digital decision-maker that learns to make predictions based on the information it receives.
Problem Set
Let’s assume we have the given problem set and our goal is to now train our machine in such a way so that it can predict the output of any given set of inputs.
| Input 1 | Input 2 | Input 3 | Outputs | |
| Example1 | 0 | 0 | 1 | 0 |
| Example2 | 1 | 1 | 1 | 1 |
| Example3 | 1 | 0 | 1 | 1 |
| Example4 | 0 | 1 | 1 | 0 |
Test | 1 | 0 | 0 | ? |
Step 1: Importing Libraries.
import numpy as np
import random
NumPy is a library widely used for numerical operations in Python. Random will be used further to generate random values for weight.
Step 2: Defining normalization function.
#normalization function (sigmoid)
def sigmoid(x):
return 1/(1+np.exp(-x))
The sigmoid function takes any number and transforms it into a value between 0 and 1. Mathematically, it's defined as:
sigmoid(x)=1/(1+e−x1)
In simpler terms, it turns any input into a probability-like output, making it a perfect fit for our digital brain's decision-making process.
def sigmoid_derivative(x):
return x*(1-x)
The derivative of the sigmoid function is used in neural networks during backpropagation. Backpropagation is a process that allows neural networks to improve their performance over time by learning from their mistakes. Mathematically, it's expressed as:
sigmoid_derivative(x)=x⋅(1−x)
Picture it as a slope indicator on our decision-maker's scale. A steeper slope(high confidence) means our network is pretty certain, while a gentler slope(low confidence) hints at some uncertainty. During training, this helps us adjust our model's internal settings, bringing it closer to making accurate predictions.
In short, the sigmoid function is the go-to translator of inputs into a language our neural network understands, and the sigmoid derivative is the navigator that ensures our network learns from its experiences.
Step 3: Input and Output Datasets
training_inputs = np.array([[0,0,0],
[1,1,1],
[1,0,1],
[0,1,1]
])
Here, a 2D NumPy array training_inputs is defined, representing the input features for the neural network. Each row corresponds to a set of input values.
training_outputs = np.array([[0,1,1,0]]).T
This is a 2D array training_outputs representing the expected output values corresponding to the input patterns defined earlier. The .T transposes the array to make it a column vector.
Step 4: Initializing synaptic weights.
np.random.seed(1)
synaptic_weights = 2*np.random.random((3,1))-1
np.random.seed(1) sets the seed for the random number generator in NumPy. The seed is a starting point for generating random numbers. By setting the seed to a specific value (in this case, 1), we ensure that every time we run the program, we get the same sequence of random numbers. This is useful for reproducibility so that we can replicate the same random initialization each time while running the code. This can be crucial for debugging, testing, and comparing different runs of our neural network.
The np.random.random((3, 1)) generates a 3x1 matrix of random numbers between 0 and 1. Multiplying by 2 scales these values to be between 0 and 2. Subtracting 1 then shifts the range to be between -1 and 1. Therefore now our range for the weights is set between -1 to 1.
Step 5: Training Iterations:
for iteration in range(10000):
input_layer = training_inputs
outputs = sigmoid(np.dot(input_layer,synaptic_weights))
error = training_outputs- outputs
adjustments = error* sigmoid_derivative(outputs)
synaptic_weights+= np.dot(input_layer.T, adjustments)
Lets breakdown this code snippet step by step :
For Loop (Training Iterations):
- **for iteration in range(10000):**This is the number of times the neural network will adjust its weights to learn from the provided training data.
Forward Pass:
A forward pass is the process of calculating the values of the output layers from the input data. It involves traversing through all neurons from the first to the last layer.
input_layer = training_inputs: The input layer is set to the training inputs.
- outputs = sigmoid([np.dot](file:///C:\Users\HP\AppData\Local\Temp\msohtmlclip1\01\clip_filelist.xml)(input_layer, synaptic_weights)): The neural network performs a forward pass by calculating the dot product of the input layer and the synaptic weights(xi*wi), passing the result through the sigmoid activation function. This gives the predicted outputs of the neural network.
Error Calculation:
- error = training_outputs - outputs: The error is calculated by subtracting the predicted outputs from the actual training outputs. This gives us a measure of how far off the predictions are from the true values.
Backpropagation:
It is the method of taking the error rate of a forward propagation and feeding this loss backward through the neural network layers to fine-tune the weights.
adjustments = error * sigmoid_derivative(outputs): Here, we initiate backpropagation. The error is multiplied by the derivative of the sigmoid function applied to the outputs. This represents how much the weights need to be adjusted to reduce the error.
- synaptic_weights += [np.dot](file:///C:\Users\HP\AppData\Local\Temp\msohtmlclip1\01\clip_filelist.xml)(input_layer.T, adjustments): The weights are updated by adding the dot product of the transpose of the input layer and the adjustment values. In this step the weights are adjusted to minimize the error.
Now we can use any sample data to predict the output generated by our perceptron.
In a nutshell, we’ve explored the details of neural networks, managing tasks like forward passes, error calculations, and backpropagation. Through 10,000 iterations, our single-layer neural network improved its predictive abilities.
Congratulations, by following all these steps you have officially built your first ever single layer neural network from scratch. To get the entire code, you can visit Github Repository. This was the first step in understanding artificial intelligence, opening doors for various possibilities in the realm of neural networks. The adventure doesn't end here; it's merely the start of an exciting journey into the world of neural networks. Happy Coding!
This post was written by Archisha Dhyani