neural_network.optimizers.nesterov_accelerated_sgd

Nesterov Accelerated Gradient (NAG) Optimizer

Implements Nesterov momentum for neural network training using NumPy. NAG looks ahead and computes gradients at the anticipated position.

Reference: https://cs231n.github.io/neural-networks-3/#sgd Author: Adhithya Laxman Ravi Shankar Geetha Date: 2025.10.21

Attributes

optimizer

Classes

NesterovAcceleratedGradient

Nesterov Accelerated Gradient (NAG) optimizer.

Module Contents

class neural_network.optimizers.nesterov_accelerated_sgd.NesterovAcceleratedGradient(learning_rate: float = 0.01, momentum: float = 0.9)

Nesterov Accelerated Gradient (NAG) optimizer.

Updates parameters using Nesterov momentum:

velocity = momentum * velocity - learning_rate * gradient_at_lookahead param = param + velocity

update(param_id: int, params: numpy.ndarray, gradients: numpy.ndarray) → numpy.ndarray

Update parameters using NAG.

Args:

param_id (int): Unique identifier for parameter group. params (np.ndarray): Current parameters. gradients (np.ndarray): Gradients at lookahead position.

Returns:

np.ndarray: Updated parameters.

>>> optimizer = NesterovAcceleratedGradient(learning_rate=0.1, momentum=0.9)
>>> params = np.array([1.0, 2.0])
>>> grads = np.array([0.1, 0.2])
>>> updated = optimizer.update(0, params, grads)
>>> updated.shape
(2,)

learning_rate = 0.01

momentum = 0.9

velocity: dict[int, numpy.ndarray]

neural_network.optimizers.nesterov_accelerated_sgd.optimizer