neural_network.optimizers.adagrad

Adagrad Optimizer

Implements Adagrad (Adaptive Gradient) for neural network training using NumPy. Adagrad adapts the learning rate for each parameter based on historical gradients.

Reference: https://en.wikipedia.org/wiki/Stochastic_gradient_descent#AdaGrad Author: Adhithya Laxman Ravi Shankar Geetha Date: 2025.10.22

Attributes

optimizer

Classes

Adagrad

Adagrad optimizer.

Module Contents

class neural_network.optimizers.adagrad.Adagrad(learning_rate: float = 0.01, epsilon: float = 1e-08)

Adagrad optimizer.

Adapts learning rate individually for each parameter:

accumulated_grad += gradient^2 param = param - (learning_rate / sqrt(accumulated_grad + epsilon)) * gradient

update(param_id: int, params: numpy.ndarray, gradients: numpy.ndarray) → numpy.ndarray

Update parameters using Adagrad.

Args:

param_id (int): Unique identifier for parameter group. params (np.ndarray): Current parameters. gradients (np.ndarray): Gradients of parameters.

Returns:

np.ndarray: Updated parameters.

>>> optimizer = Adagrad(learning_rate=0.1)
>>> params = np.array([1.0, 2.0])
>>> grads = np.array([0.1, 0.2])
>>> updated = optimizer.update(0, params, grads)
>>> updated.shape
(2,)

accumulated_grad: dict[int, numpy.ndarray]

epsilon = 1e-08

learning_rate = 0.01

neural_network.optimizers.adagrad.optimizer