neural_network.cpp File Reference

Implementation of [Multilayer Perceptron] (https://en.wikipedia.org/wiki/Multilayer_perceptron). More...

#include <algorithm>
#include <cassert>
#include <chrono>
#include <cmath>
#include <fstream>
#include <iostream>
#include <sstream>
#include <string>
#include <valarray>
#include <vector>
#include "vector_ops.hpp"

Include dependency graph for neural_network.cpp:

Go to the source code of this file.

Classes
class	machine_learning::neural_network::layers::DenseLayer
class	machine_learning::neural_network::NeuralNetwork

Namespaces
namespace	machine_learning
	A* search algorithm
namespace	neural_network
	Neural Network or Multilayer Perceptron.
namespace	activations
	Various activation functions used in Neural network.
namespace	util_functions
	Various utility functions used in Neural network.
namespace	layers
	This namespace contains layers used in MLP.

Functions
double	machine_learning::neural_network::activations::sigmoid (const double &x)
double	machine_learning::neural_network::activations::dsigmoid (const double &x)
double	machine_learning::neural_network::activations::relu (const double &x)
double	machine_learning::neural_network::activations::drelu (const double &x)
double	machine_learning::neural_network::activations::tanh (const double &x)
double	machine_learning::neural_network::activations::dtanh (const double &x)
double	machine_learning::neural_network::util_functions::square (const double &x)
double	machine_learning::neural_network::util_functions::identity_function (const double &x)
static void	test ()
int	main ()
	Main function.

Detailed Description

Implementation of [Multilayer Perceptron] (https://en.wikipedia.org/wiki/Multilayer_perceptron).

Author

Deep Raval

A multilayer perceptron (MLP) is a class of feedforward artificial neural network (ANN). The term MLP is used ambiguously, sometimes loosely to any feedforward ANN, sometimes strictly to refer to networks composed of multiple layers of perceptrons (with threshold activation). Multilayer perceptrons are sometimes colloquially referred to as "vanilla" neural networks, especially when they have a single hidden layer.

An MLP consists of at least three layers of nodes: an input layer, a hidden layer and an output layer. Except for the input nodes, each node is a neuron that uses a nonlinear activation function. MLP utilizes a supervised learning technique called backpropagation for training. Its multiple layers and non-linear activation distinguish MLP from a linear perceptron. It can distinguish data that is not linearly separable.

See Backpropagation for training algorithm.

Note

This implementation uses mini-batch gradient descent as optimizer and MSE as loss function. Bias is also not included.

Definition in file neural_network.cpp.

Function Documentation

drelu()

double machine_learning::neural_network::activations::drelu

(

const double &

x

)

Derivative of relu function

Parameters

X

Value

Returns

derivative of relu(x)

Definition at line 81 of file neural_network.cpp.

{ return x >= 0.0 ? 1.0 : 0.0; }

dsigmoid()

double machine_learning::neural_network::activations::dsigmoid

(

const double &

x

)

Derivative of sigmoid function

Parameters

X

Value

Returns

Returns derivative of sigmoid(x)

Definition at line 67 of file neural_network.cpp.

{ return x * (1 - x); }

dtanh()

double machine_learning::neural_network::activations::dtanh

(

const double &

x

)

Derivative of Sigmoid function

Parameters

X

Value

Returns

Returns derivative of tanh(x)

Definition at line 95 of file neural_network.cpp.

{ return 1 - x * x; }

identity_function()

double machine_learning::neural_network::util_functions::identity_function

(

const double &

x

)

Identity function

Parameters

X

Value

Returns

Returns x

Definition at line 112 of file neural_network.cpp.

{ return x; }

main()

int main

(

void

)

Main function.

Returns

0 on exit

Definition at line 833 of file neural_network.cpp.

           {
    // Testing
    test();
    return 0;
}

relu()

double machine_learning::neural_network::activations::relu

(

const double &

x

)

Relu function

Parameters

X

Value

Returns

relu(x)

Definition at line 74 of file neural_network.cpp.

{ return std::max(0.0, x); }

sigmoid()

double machine_learning::neural_network::activations::sigmoid

(

const double &

x

)

Sigmoid function

Parameters

X

Value

Returns

Returns sigmoid(x)

Definition at line 60 of file neural_network.cpp.

{ return 1.0 / (1.0 + std::exp(-x)); }

square()

double machine_learning::neural_network::util_functions::square

(

const double &

x

)

Square function

Parameters

X

Value

Returns

Returns x * x

Definition at line 106 of file neural_network.cpp.

{ return x * x; }

tanh()

double machine_learning::neural_network::activations::tanh

(

const double &

x

)

Tanh function

Parameters

X

Value

Returns

Returns tanh(x)

Definition at line 88 of file neural_network.cpp.

{ return 2 / (1 + std::exp(-2 * x)) - 1; }

test()

static void test ( )

static

Function to test neural network

Returns

none

Definition at line 805 of file neural_network.cpp.

                   {
    // Creating network with 3 layers for "iris.csv"
    machine_learning::neural_network::NeuralNetwork myNN =
        machine_learning::neural_network::NeuralNetwork({
            {4, "none"},  // First layer with 3 neurons and "none" as activation
            {6,
             "relu"},  // Second layer with 6 neurons and "relu" as activation
            {3, "sigmoid"}  // Third layer with 3 neurons and "sigmoid" as
                            // activation
        });
    // Printing summary of model
    myNN.summary();
    // Training Model
    myNN.fit_from_csv("iris.csv", true, 100, 0.3, false, 2, 32, true);
    // Testing predictions of model
    assert(machine_learning::argmax(
               myNN.single_predict({{5, 3.4, 1.6, 0.4}})) == 0);
    assert(machine_learning::argmax(
               myNN.single_predict({{6.4, 2.9, 4.3, 1.3}})) == 1);
    assert(machine_learning::argmax(
               myNN.single_predict({{6.2, 3.4, 5.4, 2.3}})) == 2);
    return;
}