neural_network.cpp File Reference

Implementation of [Multilayer Perceptron] (https://en.wikipedia.org/wiki/Multilayer_perceptron). More...

#include <algorithm>
#include <cassert>
#include <chrono>
#include <cmath>
#include <fstream>
#include <iostream>
#include <sstream>
#include <string>
#include <valarray>
#include <vector>
#include "vector_ops.hpp"

Include dependency graph for neural_network.cpp:

Classes
class	machine_learning::neural_network::layers::DenseLayer
class	machine_learning::neural_network::NeuralNetwork

Namespaces
namespace	machine_learning
	A* search algorithm
namespace	neural_network
	Neural Network or Multilayer Perceptron.
namespace	activations
	Various activation functions used in Neural network.
namespace	util_functions
	Various utility functions used in Neural network.
namespace	layers
	This namespace contains layers used in MLP.

Functions
double	machine_learning::neural_network::activations::sigmoid (const double &x)
double	machine_learning::neural_network::activations::dsigmoid (const double &x)
double	machine_learning::neural_network::activations::relu (const double &x)
double	machine_learning::neural_network::activations::drelu (const double &x)
double	machine_learning::neural_network::activations::tanh (const double &x)
double	machine_learning::neural_network::activations::dtanh (const double &x)
double	machine_learning::neural_network::util_functions::square (const double &x)
double	machine_learning::neural_network::util_functions::identity_function (const double &x)
static void	test ()
int	main ()
	Main function.

Detailed Description

Implementation of [Multilayer Perceptron] (https://en.wikipedia.org/wiki/Multilayer_perceptron).

Author

Deep Raval

A multilayer perceptron (MLP) is a class of feedforward artificial neural network (ANN). The term MLP is used ambiguously, sometimes loosely to any feedforward ANN, sometimes strictly to refer to networks composed of multiple layers of perceptrons (with threshold activation). Multilayer perceptrons are sometimes colloquially referred to as "vanilla" neural networks, especially when they have a single hidden layer.

An MLP consists of at least three layers of nodes: an input layer, a hidden layer and an output layer. Except for the input nodes, each node is a neuron that uses a nonlinear activation function. MLP utilizes a supervised learning technique called backpropagation for training. Its multiple layers and non-linear activation distinguish MLP from a linear perceptron. It can distinguish data that is not linearly separable.

See Backpropagation for training algorithm.

Note

This implementation uses mini-batch gradient descent as optimizer and MSE as loss function. Bias is also not included.

Function Documentation

drelu()

double machine_learning::neural_network::activations::drelu

(

const double &

x

)

Derivative of relu function

Parameters

X

Value

Returns

derivative of relu(x)

{ return x >= 0.0 ? 1.0 : 0.0; }

Here is the call graph for this function:

dsigmoid()

double machine_learning::neural_network::activations::dsigmoid

(

const double &

x

)

Derivative of sigmoid function

Parameters

X

Value

Returns

Returns derivative of sigmoid(x)

{ return x * (1 - x); }

Here is the call graph for this function:

dtanh()

double machine_learning::neural_network::activations::dtanh

(

const double &

x

)

Derivative of Sigmoid function

Parameters

X

Value

Returns

Returns derivative of tanh(x)

{ return 1 - x * x; }

Here is the call graph for this function:

identity_function()

double machine_learning::neural_network::util_functions::identity_function

(

const double &

x

)

Identity function

Parameters

X

Value

Returns

Returns x

{ return x; }

Here is the call graph for this function:

main()

int main

(

void

)

Main function.

Returns

0 on exit

           {
    // Testing
    test();
    return 0;
}

Here is the call graph for this function:

relu()

double machine_learning::neural_network::activations::relu

(

const double &

x

)

Relu function

Parameters

X

Value

Returns

relu(x)

{ return std::max(0.0, x); }

Here is the call graph for this function:

sigmoid()

double machine_learning::neural_network::activations::sigmoid

(

const double &

x

)

Sigmoid function

Parameters

X

Value

Returns

Returns sigmoid(x)

{ return 1.0 / (1.0 + std::exp(-x)); }

Here is the call graph for this function:

square()

double machine_learning::neural_network::util_functions::square

(

const double &

x

)

Square function

Parameters

X

Value

Returns

Returns x * x

{ return x * x; }

Here is the call graph for this function:

tanh()

double machine_learning::neural_network::activations::tanh

(

const double &

x

)

Tanh function

Parameters

X

Value

Returns

Returns tanh(x)

{ return 2 / (1 + std::exp(-2 * x)) - 1; }

Here is the call graph for this function:

test()

static void test ( )

static

Function to test neural network

Returns

none

                   {
    // Creating network with 3 layers for "iris.csv"
    machine_learning::neural_network::NeuralNetwork myNN =
        machine_learning::neural_network::NeuralNetwork({
            {4, "none"},  // First layer with 3 neurons and "none" as activation
            {6,
             "relu"},  // Second layer with 6 neurons and "relu" as activation
            {3, "sigmoid"}  // Third layer with 3 neurons and "sigmoid" as
                            // activation
        });
    // Printing summary of model
    myNN.summary();
    // Training Model
    myNN.fit_from_csv("iris.csv", true, 100, 0.3, false, 2, 32, true);
    // Testing predictions of model
    assert(machine_learning::argmax(
               myNN.single_predict({{5, 3.4, 1.6, 0.4}})) == 0);
    assert(machine_learning::argmax(
               myNN.single_predict({{6.4, 2.9, 4.3, 1.3}})) == 1);
    assert(machine_learning::argmax(
               myNN.single_predict({{6.2, 3.4, 5.4, 2.3}})) == 2);
    return;
}

Here is the call graph for this function: