machine_learning.rmsprop
========================

.. py:module:: machine_learning.rmsprop

.. autoapi-nested-parse::

   RMSprop (Root Mean Square Propagation) optimizer implementation.

   RMSprop is an adaptive learning rate optimizer that maintains a moving
   average of squared gradients to normalize the gradient. It was proposed
   by Geoffrey Hinton in his Coursera course on Neural Networks.

   Key idea: Instead of using a fixed learning rate, RMSprop adapts the
   learning rate for each parameter by dividing by a running average of
   recent gradient magnitudes.

   Update rules:
       v(t) = rho * v(t-1) + (1 - rho) * gradient^2
       param = param - (learning_rate / sqrt(v(t) + epsilon)) * gradient

   Where:
       v(t)          = moving average of squared gradients
       rho           = decay factor (typically 0.9)
       learning_rate = step size
       epsilon       = small value to avoid division by zero

   Reference: https://en.wikipedia.org/wiki/Stochastic_gradient_descent#RMSProp

   >>> rmsprop([0.0], [0.1], 0.01)  # doctest: +ELLIPSIS
   [-0.031...]
   >>> rmsprop([1.0, -1.0], [0.5, -0.5], 0.01)  # doctest: +ELLIPSIS
   [0.968..., -0.968...]



Attributes
----------

.. autoapisummary::

   machine_learning.rmsprop.param


Functions
---------

.. autoapisummary::

   machine_learning.rmsprop.rmsprop


Module Contents
---------------

.. py:function:: rmsprop(params: list[float], gradients: list[float], learning_rate: float, rho: float = 0.9, epsilon: float = 1e-08, moving_avg: list[float] | None = None) -> list[float]

   Perform one step of the RMSprop optimization algorithm.

   :param params: Current parameter values to be updated.
   :param gradients: Gradients of the loss with respect to each parameter.
   :param learning_rate: Step size for the update (must be positive).
   :param rho: Decay factor for the moving average (default 0.9).
   :param epsilon: Small constant to avoid division by zero (default 1e-8).
   :param moving_avg: Running average of squared gradients. Updated in
                      place each call. Initialized to zeros if not provided.
   :return: Updated parameter values after one RMSprop step.

   :raises ValueError: If params and gradients have different lengths.
   :raises ValueError: If learning_rate, rho, or epsilon are out of range.

   >>> rmsprop([0.0], [0.0], 0.01)
   [0.0]
   >>> rmsprop([1.0], [0.0], 0.01)
   [1.0]
   >>> len(rmsprop([1.0, 2.0, 3.0], [0.1, 0.2, 0.3], 0.01)) == 3
   True
   >>> rmsprop([1.0], [0.5], learning_rate=0.01)  # doctest: +ELLIPSIS
   [0.968...]


.. py:data:: param
   :value: [5.0]