Derivation of the Gradient Descent Rule
How to calculate the direction of steepest descent along the error surface?
The direction of steepest can be found by computing the derivative of E with respect to each component of the vector wvector . This vector derivative is called the gradient of E with respect to wvector , written as
The
gradient specifies the direction of steepest increase of E, the training rule
for gradient descent is
- Here η is a positive constant called the learning rate, which determines the step size in the gradient descent search.
- The negative sign is present because we want to move the weight vector in the direction that decreases E.
![]()
Calculate the gradient at each step. The vector of 𝜕𝐸/𝜕𝑤𝑖 derivatives that form the gradient can
be obtained by differentiating E from Equation (2), as
Labels: Machine Learning, Machine Learning Unit-3



Discussion & Queries (<$I18NNumComments$>):
<$CommentPager$>
-
<$I18NAtCommentTimeWithPermalink$>, <$I18NCommentAuthorSaid$>
-
<$CommentPager$>
<$BlogCommentBody$>
<$BlogCommentDeleteIcon$>