neural_network.optimizers.momentum_sgd

Momentum SGD Optimizer

Implements SGD with momentum for neural network training using NumPy. Momentum helps accelerate gradients in the relevant direction and dampens oscillations.

Reference: https://en.wikipedia.org/wiki/Stochastic_gradient_descent#Momentum Author: Adhithya Laxman Ravi Shankar Geetha Github: https://github.com/Adhithya-Laxman Date: 2025.10.22

Attributes

optimizer

Classes

MomentumSGD

SGD with momentum optimizer.

Module Contents

class neural_network.optimizers.momentum_sgd.MomentumSGD(learning_rate: float = 0.01, momentum: float = 0.9)

SGD with momentum optimizer.

Updates parameters using momentum:

velocity = momentum * velocity - learning_rate * gradient param = param + velocity

update(param_id: int, params: numpy.ndarray, gradients: numpy.ndarray) → numpy.ndarray

Update parameters using momentum.

Args:

param_id (int): Unique identifier for parameter group. params (np.ndarray): Current parameters. gradients (np.ndarray): Gradients of parameters.

Returns:

np.ndarray: Updated parameters.

>>> optimizer = MomentumSGD(learning_rate=0.1, momentum=0.9)
>>> params = np.array([1.0, 2.0])
>>> grads = np.array([0.1, 0.2])
>>> updated = optimizer.update(0, params, grads)
>>> updated.shape
(2,)

learning_rate = 0.01

momentum = 0.9

velocity: dict[int, numpy.ndarray]

neural_network.optimizers.momentum_sgd.optimizer