# Gradient rules

Just leaving some notes to differentiate expressions with the \( \nabla \) operator to compute gradients of various functions.

### Usual operations

Sum rule, Product rule, sum rule, division rule, scalar rule (\(f\) and \(g\) both scalar functions \(g: \mathbb R^n \rightarrow \mathbb R\), \(f: \mathbb R^n \rightarrow \mathbb R\)):

$$ \begin{array}{lcl} \nabla [ f + g ] & = &\nabla f + \nabla g \\ \nabla [ f . g ] & = & \nabla f . g + f . \nabla g \\ \nabla \left [ \frac{f}{g} \right ] & = &\frac{\nabla f . g - f . \nabla g}{g^2} \\ \nabla [ \alpha . f ] & = & \alpha . \nabla f \end{array}$$

### Gradient of the norm

$$\nabla [ \| \vec {x} \| ] = \frac{ \vec {x}}{ \| \vec {x} \|} $$

$$\nabla [ \| \vec {x} \|^2 ] = 2 . \vec {x} $$

### Gradient of a matrix

With \( M \) a \( n \times n \) matrix:

$$\nabla [ M \vec {x} ] = M $$

Likewise for rigid transformations (rotations \( M \in \mathbb R^{3 \times 3}\) and translations \(\vec t \in \mathbb R^3\)):

$$\nabla [ M \vec {x} + \vec t ] = M $$

### Chain rules

With \(s: \mathbb R \rightarrow \mathbb R\) univariate and \(f: \mathbb R^n \rightarrow \mathbb R\) multivariate real valued the operation boils down to a uniform scale of the gradient:

$$ \nabla \left [ s( f(\vec {x}) ) \right ] = s'( f(\vec {x}) ) \nabla f(\vec {x}) $$

With \(m: \mathbb R^n \rightarrow \mathbb R^n\) deformation map and \(f: \mathbb R^n \rightarrow \mathbb R\) multivariate scalar function the operation boils down to transform the gradient with a matrix:

$$ \nabla \left [ f( m(\vec {x}) ) \right ] = \mathbf{J}\left [ m(\vec {x}) \right ]^\mathsf{T} \nabla f(m(\vec {x}))$$

Where \( \mathbf{J}\left [ m(\vec {x}) \right ]^\mathsf{T} \) denotes the transpose of the \(n \times n \) Jacobian matrix.

### Related Jacobian rules

Jacobian of a linear function is the identity matrix \( I \) times the slope \( a \):

$$ J[a \vec {x} + c] = a.I $$

### Related chain rule

A univariate differentiation can lead to the use of \( \nabla \): with \(f: \mathbb R^n \rightarrow \mathbb R\) multivariable scalar function and \(p: \mathbb R \rightarrow \mathbb R^n\) a parametric function:

$$ f'(g(x)) = \nabla f(g(x))^T . \vec {g'(x)} $$

In short, we do the dot product between the gradient of \( f \) and the speed of \( g \)

Some links:

Multivariate / multivariable chain rule

No comments