neural_network.optimizers.adam_optimizer

Adam Optimizer

Implements Adam (Adaptive Moment Estimation) for neural network training using NumPy. Adam combines momentum and adaptive learning rates using first and second moment estimates.

Reference: https://arxiv.org/abs/1412.6980 Author: Adhithya Laxman Ravi Shankar Geetha Date: 2025.10.21

Attributes

optimizer

Classes

Adam	Adam optimizer.

Module Contents

class neural_network.optimizers.adam_optimizer.Adam(learning_rate: float = 0.001, beta1: float = 0.9, beta2: float = 0.999, epsilon: float = 1e-08)

Adam optimizer.

Combines momentum and RMSProp:

m = beta1 * m + (1 - beta1) * gradient v = beta2 * v + (1 - beta2) * gradient^2 m_hat = m / (1 - beta1^t) v_hat = v / (1 - beta2^t) param = param - learning_rate * m_hat / (sqrt(v_hat) + epsilon)

update(param_id: int, params: numpy.ndarray, gradients: numpy.ndarray) → numpy.ndarray

Update parameters using Adam.

Args:

param_id (int): Unique identifier for parameter group. params (np.ndarray): Current parameters. gradients (np.ndarray): Gradients of parameters.

Returns:

np.ndarray: Updated parameters.

>>> optimizer = Adam(learning_rate=0.1)
>>> params = np.array([1.0, 2.0])
>>> grads = np.array([0.1, 0.2])
>>> updated = optimizer.update(0, params, grads)
>>> updated.shape
(2,)

beta1 = 0.9

beta2 = 0.999

epsilon = 1e-08

learning_rate = 0.001

m: dict[int, numpy.ndarray]

t: dict[int, int]

v: dict[int, numpy.ndarray]

neural_network.optimizers.adam_optimizer.optimizer