Implementing Artificial Neural Network Training Process in Python

What is ANN?

A computational model called an Artificial Neural Network (ANN), usually referred to as a neural network or a deep neural network, is modeled after the organization and operation of biological neural networks in the human brain. This method uses machine learning to generate predictions or choices based on data.

• ANNs are made up of linked nodes known as artificial neurons or "units." An input layer, one or more hidden layers, and an output layer are only a few of the layers that make up these units. Each unit takes inputs, runs a calculation, and then sends the outcome to the layer below it. Weights that are linked with links between modules are changed as learners progress.
• Iterative weight adjustments are made as part of an ANN's training process depending on the input data and intended results. An optimization approach like gradient descent, which minimizes a predetermined loss function, is generally used to make this modification.
• A variety of tasks, including classification, regression, pattern recognition, and others, may be performed using ANNs. They have been effectively used in a variety of fields, including financial forecasting, natural language processing, picture and audio recognition, and recommendation systems.

Following Steps used for the Training Process:

Forward Propagation

1. Initialize each layer's weights and biases, as well as the input data (X).
2. Calculate the weighted total of the inputs from the preceding layer for each layer, then use the activation function to determine each neuron's output. The output is sent forward via the network repeatedly for each layer.
3. The network's anticipated output is represented by the output of the last layer.

Backpropagation:

1. Determine the loss function's derivative with respect to the network's output (dA) in the top layer.
2. Calculate the gradients of the weights and biases in the final layer using this derivative.
3. Calculate the gradients at each layer using the chain rule and backpropagate them across the layers.
4. Make changes to each layer's weights and biases using an optimization approach (such as gradient descent).

Implementation

1. Install TensorFlow: Ensure TensorFlow is set up on your computer. Using pip, you can install it:

pip install tensorflow

2. Import the required libraries:

import tensorflow as tf

from tensorflow.keras import layers

3. Get your data ready: Your dataset should be divided into training and testing sets. Additionally, if necessary, normalize or preprocess the data.

4. Construct the neural network model: Utilise the TensorFlow-included Keras API to specify the ANN's design.

import numpy as np

def sigmoid(x):

    return 1 / (1 + np.exp(-x))

def sigmoid_derivative(x):

    return sigmoid(x) * (1 - sigmoid(x))

class NeuralNetwork:

    def __init__(self, layer_sizes):

        self.layer_sizes = layer_sizes

        self.num_layers = len(layer_sizes)

        self.weights = [np.random.randn(layer_sizes[i], layer_sizes[i-1]) for i in range(1, self.num_layers)]

        self.biases = [np.random.randn(layer_sizes[i], 1) for i in range(1, self.num_layers)]

    def forward_propagation(self, X):

        activations = [X]

        zs = []

        for i in range (self.num_layers - 1):

            z = np.dot(self.weights[i], activations[-1]) + self.biases[i]

            zs.append(z)

            activation = sigmoid(z)

            activations.append(activation)

        return activations, zs

    def train (self, X, y, num_epochs, learning_rate):

        for epoch in range(num_epochs):

            activations, zs = self.forward_propagation(X)

            num_examples = y.shape[1]

            delta = activations [-1] - y

            gradients = [np.dot (delta, activations[-2].T) / num_examples]

            biases_gradients = [np.sum(delta, axis=1, keepdims=True) / num_examples]

            for i in range (self.num_layers - 3, -1, -1):

                delta = np.dot(self.weights[i+1].T, delta) * sigmoid_derivative(zs[i])

                gradients.insert(0, np.dot(delta, activations[i].T) / num_examples)

                biases_gradients.insert(0, np.sum(delta, axis=1, keepdims=True) / num_examples)

            for i in range (self.num_layers - 2, -1, -1):

                self.weights[i] -= learning_rate * gradients[i]

                self.biases[i] -= learning_rate * biases_gradients[i]

X = np.array([[0, 0], [0, 1], [1, 0], [1, 1]]).T

y = np.array([[0, 1, 1, 0]])

nn = NeuralNetwork([2, 2, 1])

# Train the neural network

nn.train(X, y, num_epochs=10000, learning_rate=0.1)

activations, _ = nn.forward_propagation(X)

predictions = activations [-1]

print ("Predictions:", predictions)

Output:

Predictions: [[0.02144561 0.97219714 0.97240426 0.01874537]]

The anticipated output for each member in the array corresponds to the associated input example. In the predictions array, the first member, for instance, represents the forecast for the initial input [0, 0], which is around 0.0276. The second element, which is around 0.9722, corresponds to the forecast for [0, 1], and so on.

With practice, the neural network may provide results that are near 0 for inputs [0, 0] and [1, 1] and results that are close to 1 for inputs [0, 1] and [1, 0].