---

-   References :
    -   Done Reading:
        -   [Sebastian Raschka Gradient Descent Lecture Slide](https://github.com/rasbt/stat453-deep-learning-ss20/blob/master/L05-grad-descent/L05%5Fgrad-descent%5F%5Fslides.pdf)
    -   To Read:
        -   <https://www.wikiwand.com/en/Gradient>
-   Questions :

---

In vector calculus, the gradient is a multi-variable generalization of the derivative.
Whereas the ordinary derivative of a function of a single variable is a scalar-valued function, the gradient of a function of several variables is a vector-valued function

Simply, derivative of multivariable function

The gradient (or gradient vector field) of a scalar function f(x1, x2, x3, …, xn) is denoted ∇f or ∇→f where ∇ (nabla) denotes the vector differential operator, del.

\\( \nabla f(p) = { \begin {bmatrix} {\frac {\partial f}{\partial x\_{1}}}(p) \\\ \vdots \\\ {\frac {\partial f}{\partial x\_{n}}}(p) \end{bmatrix}} \\)


### Example {#example}

\\(f(x, y) = x^{2}y + y\\),

\\( \nabla f(x, y)
= {\begin{bmatrix}{\frac {\partial f}{\partial x}}(x^{2}y + y)\\\ \\\ {\frac {\partial f}{\partial y}}(x^{2}y + y)\end{bmatrix}}
= {\begin{bmatrix}(2xy) \\\ \\\ (x^{2} + 1)\end{bmatrix}} \\)


## Gradients and The Multivariable Chain Rule {#gradients-and-the-multivariable-chain-rule}

{{< figure src="/ox-hugo/2021-06-03_08-54-12_screenshot.png" caption="Figure 1: A example" width="400" height="350" target="/blogs" >}}

{{< figure src="/ox-hugo/2021-06-03_08-56-26_screenshot.png" caption="Figure 2: A example" width="700" height="300" target="/blogs" >}}

-   Result
    -   \\(  \frac{d}{dx}[f(g(x), h(x))]\ = [2gh.3] + [(g^2 + 1).2x] = 2xg^2 + 6gh + 2x \\)

{{< figure src="/ox-hugo/2021-06-03_09-10-34_screenshot.png" caption="Figure 3: In vector form" width="650" height="500" target="/blogs" >}}


## The Jacobian Matrix {#the-jacobian-matrix}

{{< figure src="/ox-hugo/2021-06-03_09-13-45_screenshot.png" caption="Figure 4: Jacobian Matrix" width="500" height="400" target="/blogs" >}}