machine_learning.multilayer_perceptron_classifier

Attributes

rng

Classes

Dataloader	DataLoader class for handling dataset, including data shuffling,
MLP	A custom MLP class for implementing a simple multi-layer perceptron with

Module Contents

class machine_learning.multilayer_perceptron_classifier.Dataloader(features: list[list[float]], labels: list[int])

DataLoader class for handling dataset, including data shuffling, one-hot encoding, and train-test splitting.

Example usage: >>> X = [[0.0, 0.0], [1.0, 1.0], [1.0, 0.0], [0.0, 1.0]] >>> y = [0, 1, 0, 0] >>> loader = Dataloader(X, y) >>> len(loader.get_train_test_data()) # Returns train and test data 4 >>> loader.one_hot_encode([0, 1, 0], 2) # Returns one-hot encoded labels array([[0.99, 0. ],

[0. , 0.99], [0.99, 0. ]])

>>> loader.get_inout_dim()
(2, 3)
>>> loader.one_hot_encode([0, 2], 3)
array([[0.99, 0.  , 0.  ],
       [0.  , 0.  , 0.99]])

get_inout_dim() → tuple[int, int]

get_train_test_data() → tuple[numpy.ndarray, list[numpy.ndarray], numpy.ndarray, list[numpy.ndarray]]

Splits the data into training and testing sets. Here, we manually split the data.

Returns:

A tuple containing: - Train data - Train labels - Test data - Test labels

static one_hot_encode(labels: list[int], num_classes: int) → numpy.ndarray

Perform one-hot encoding for the given labels.

Args:

labels: List of integer labels. num_classes: Total number of classes for encoding.

Returns:

A numpy array representing one-hot encoded labels.

shuffle_data(paired_data: list[tuple[numpy.ndarray, int]]) → list[tuple[numpy.ndarray, int]]

Shuffles the data randomly.

Args:

paired_data: List of tuples containing data and corresponding labels.

Returns:

A shuffled list of data-label pairs.

X

class_weights

y

class machine_learning.multilayer_perceptron_classifier.MLP(dataloader: Dataloader, epoch: int, learning_rate: float, gamma: float = 1.0, hidden_dim: int = 2)

A custom MLP class for implementing a simple multi-layer perceptron with forward propagation, backpropagation.

Attributes:

learning_rate (float): Learning rate for gradient descent. gamma (float): Parameter to control learning rate adjustment. epoch (int): Number of epochs for training. hidden_dim (int): Dimension of the hidden layer. batch_size (int): Number of samples per mini-batch. train_loss (List[float]): List to store training loss for each fold. train_accuracy (List[float]): List to store training accuracy for each fold. test_loss (List[float]): List to store test loss for each fold. test_accuracy (List[float]): List to store test accuracy for each fold. dataloader (Dataloader): DataLoader object for handling training data. inter_variable (dict): Dictionary to store intermediate variables for backpropagation. weights1_list (List[Tuple[np.ndarray, np.ndarray]]): List of weights for each fold.

Methods:

get_inout_dim:obtain input dimension and output dimension. relu: Apply the ReLU activation function. relu_derivative: Compute the derivative of the ReLU function. forward: Perform a forward pass through the network. back_prop: Perform backpropagation to compute gradients. update_weights: Update the weights using gradients. update_learning_rate: Adjust the learning rate based on test accuracy. accuracy: Compute accuracy of the model. loss: Compute weighted MSE loss. train: Train the MLP over multiple folds with early stopping.

static accuracy(label: numpy.ndarray, y_hat: numpy.ndarray) → float

Computes the accuracy of predictions by comparing predicted and true labels.

Args:

label: True labels, shape (batch_size, num_classes). y_hat: Predicted outputs, shape (batch_size, num_classes).

Returns:

Accuracy as a float between 0 and 1.

Examples:

>>> mlp = MLP(None, 1, 0.01)
>>> label = np.array([[1, 0], [0, 1], [1, 0]])
>>> y_hat = np.array([[0.9, 0.1], [0.2, 0.8], [0.6, 0.4]])
>>> mlp.accuracy(label, y_hat)
np.float64(1.0)

back_prop(input_data: numpy.ndarray, true_labels: numpy.ndarray, w2: numpy.ndarray) → tuple[numpy.ndarray, numpy.ndarray]

Performs backpropagation to compute gradients for the weights.

Args:

input_data: Input data, shape (batch_size, input_dim). true_labels: True labels, shape (batch_size, output_dim). w2: Weight matrix for hidden to output layer, shape (hidden_dim, output_dim).

Returns:

Tuple of gradients (grad_w1, grad_w2) for the weight matrices.

Examples:

>>> mlp = MLP(None, 1, 0.1, hidden_dim=2)
>>> x = np.array([[1.0, 2.0, 1.0]])  # batch_size=1, input_dim=2 + bias
>>> y = np.array([[0.0, 1.0]])  # batch_size=1, output_dim=2
>>> w1 = np.array([[0.1, 0.2], [0.3, 0.4], [0.5, 0.6]])
>>> w2 = np.array([[0.7, 0.8], [0.9, 1.0]])  # (hidden_dim=2, output_dim=2)
>>> _ = mlp.forward(x, w1, w2)  # Run forward to set inter_variable
>>> grad_w1, grad_w2 = mlp.back_prop(x, y, w2)
>>> grad_w1.shape
(3, 2)
>>> grad_w2.shape
(2, 2)

forward(input_data: numpy.ndarray, w1: numpy.ndarray, w2: numpy.ndarray, no_gradient: bool = False) → numpy.ndarray

Performs a forward pass through the neural network with one hidden layer.

Args:

input_data: Input data, shape (batch_size, input_dim). w1: Weight matrix for input to hidden layer, shape (input_dim + 1, hidden_dim). w2: Weight matrix for hidden to output layer, shape (hidden_dim, output_dim). no_gradient: If True, returns output without storing intermediates.

Returns:

Output of the network after forward pass, shape (batch_size, output_dim).

Examples:

>>> mlp = MLP(None, 1, 0.1, hidden_dim=2)
>>> x = np.array([[1.0, 2.0, 1.0]])  # batch_size=1, input_dim=2 + bias
>>> w1 = np.array([[0.1, 0.2], [0.3, 0.4], [0.5, 0.6]])
>>> w2 = np.array([[0.7, 0.8], [0.9, 1.0]])
>>> output = mlp.forward(x, w1, w2)
>>> output.shape
(1, 2)

get_acc_loss() → tuple[list[float], list[float]]

Returns the recorded test accuracy and test loss.

Returns:

Tuple of (test_accuracy, test_loss) lists.

Examples:

>>> mlp = MLP(None, 1, 0.1)
>>> mlp.test_accuracy = [0.8, 0.9]
>>> mlp.test_loss = [0.1, 0.05]
>>> acc, loss = mlp.get_acc_loss()
>>> acc
[0.8, 0.9]
>>> loss
[0.1, 0.05]

get_inout_dim() → tuple[int, int]

obtain input dimension and output dimension.

Returns:

Tuple of weights (input_dim, output_dim) for the network.

>>> X = [[0.0, 0.0], [1.0, 1.0], [1.0, 0.0], [0.0, 1.0]]
>>> y = [0, 1, 0, 0]
>>> loader = Dataloader(X, y)
>>> mlp = MLP(loader, 10, 0.1)
>>> mlp.get_inout_dim()
(2, 3)

initialize() → tuple[numpy.ndarray, numpy.ndarray]

Initialize weights using He initialization.

Returns:

Tuple of weights (w1, w2) for the network.

>>> X = [[0.0, 0.0], [1.0, 1.0], [1.0, 0.0], [0.0, 1.0]]
>>> y = [0, 1, 0, 0]
>>> loader = Dataloader(X, y)
>>> mlp = MLP(loader, 10, 0.1)
>>> w1, w2 = mlp.initialize()
>>> w1.shape
(3, 2)
>>> w2.shape
(2, 3)

static loss(output: numpy.ndarray, label: numpy.ndarray) → float

Computes the mean squared error loss between predictions and true labels.

Args:

output: Predicted outputs, shape (batch_size, num_classes). label: True labels, shape (batch_size, num_classes).

Returns:

Mean squared error loss as a float.

Examples:

>>> mlp = MLP(None, 1, 0.1)
>>> output = np.array([[0.9, 0.1], [0.2, 0.8]])
>>> label = np.array([[1.0, 0.0], [0.0, 1.0]])
>>> round(mlp.loss(output, label), 3)
np.float64(0.025)

relu(input_array: numpy.ndarray) → numpy.ndarray

Apply the ReLU activation function element-wise.

Parameters:

input_array – Input array.

Returns:

Output array after applying ReLU.

>>> mlp = MLP(None, 1, 0.1)
>>> mlp.relu(np.array([[-1, 2], [3, -4]]))
array([[0, 2],
       [3, 0]])

relu_derivative(input_array: numpy.ndarray) → numpy.ndarray

Compute the derivative of the ReLU function.

Parameters:

input_array – Input array.

Returns:

Derivative of ReLU function element-wise.

>>> mlp = MLP(None, 1, 0.01)
>>> mlp.relu_derivative(np.array([[-1, 2], [3, -4]]))
array([[0., 1.],
       [1., 0.]])

train() → None

Trains the MLP model using the provided dataloader for multiple folds and epochs.

Saves the best model parameters for each fold and records accuracy/loss.

Examples:

>>> X = [[0.0, 0.0], [1.0, 1.0], [1.0, 0.0], [0.0, 1.0]]
>>> y = [0, 1, 0, 0]
>>> loader = Dataloader(X, y)
>>> mlp = MLP(loader, epoch=2, learning_rate=0.1, hidden_dim=2)
>>> mlp.train()
Test accuracy: ...

update_learning_rate(learning_rate: float) → float

Updates the learning rate by applying the decay factor gamma.

Args:

learning_rate: Current learning rate.

Returns:

Updated learning rate.

Examples:

>>> mlp = MLP(None, 1, 0.1, gamma=0.9)
>>> round(mlp.update_learning_rate(0.1), 2)
0.09

update_weights(w1: numpy.ndarray, w2: numpy.ndarray, grad_w1: numpy.ndarray, grad_w2: numpy.ndarray, learning_rate: float) → tuple[numpy.ndarray, numpy.ndarray]

Updates the weight matrices using the computed gradients and learning rate.

Args:

w1: Weight matrix for input to hidden layer, shape (input_dim + 1, hidden_dim). w2: Weight matrix for hidden to output layer, shape (hidden_dim, output_dim). grad_w1: Gradient for w1, shape (input_dim + 1, hidden_dim). grad_w2: Gradient for w2, shape (hidden_dim, output_dim). learning_rate: Learning rate for weight updates.

Returns:

Updated weight matrices (w1, w2).

Examples:

>>> mlp = MLP(None, 1, 0.1)
>>> w1 = np.array([[0.1, 0.2], [0.3, 0.4], [0.5, 0.6]])
>>> w2 = np.array([[0.7, 0.8], [0.9, 1.0]])
>>> grad_w1 = np.array([[0.1, 0.2], [0.3, 0.4], [0.5, 0.6]])
>>> grad_w2 = np.array([[0.7, 0.8], [0.9, 1.0]])
>>> lr = 0.1
>>> new_w1, new_w2 = mlp.update_weights(w1, w2, grad_w1, grad_w2, lr)
>>> new_w1==np.array([[0.09, 0.18], [0.27, 0.36], [0.45, 0.54]])
array([[ True,  True],
       [ True,  True],
       [ True,  True]])
>>> new_w2==np.array([[0.63, 0.72], [0.81, 0.90]])
array([[ True,  True],
       [ True,  True]])

dataloader

epoch

gamma = 1.0

hidden_dim = 2

inter_variable: dict[str, numpy.ndarray]

learning_rate

test_accuracy: list[float] = []

test_loss: list[float] = []

train_accuracy: list[float] = []

train_loss: list[float] = []

weights1_list: list[numpy.ndarray] = []

machine_learning.multilayer_perceptron_classifier.rng