# Differentiation¶

## Differentiation from first principles¶

### Derivatives of functions of one variable¶

A derivative of a single-variable function describes the rate of change of a function with respect to its input.

\frac{dy}{dx}=\lim\limits_{\Delta x \to 0} \frac{f(x+\Delta x)-f(x)}{\Delta x}

The definition of the derivative (above). Note that not all functions are differentiable.

The derivative of a function $f(x)$ can be denoted in different ways.

• $\frac{df(x)}{dx}$ – this is known as Leibniz notation. The $d$ indicates that $dy$ and $dx$ are called infinitesimals. $dx$ means a very small piece of $x$ and $dy$ means the same, but a small piece of $y$ instead of a small piece of $x$.
• $f'(x)$ – this is sometimes called Newton's notation. I prefer Leibniz's notation than Newton's notation. Newton's notation becomes particularly confusing when doing integrals (used for calculating the area under the curve) and partial derivatives (derivatives for functions of more than one variable). Newton's notation is useful for the calculation of Taylor/Maclaurin series where Leibniz's notation is less helpful.

### Higher order derivatives¶

Just as the derivative of a function can be taken, one can take the derivative of the derivative. This can be denoted using both Leibniz's and Newton's notation. The derivative of the derivative is called the second derivative. The derivative of the second derivative is known as the third derivative (and so on). $$\frac{d2y}{dx2}$$ $$f''(x)$$ This example, using Newton's notation can also be denoted using a number (see below). The $f{^(2)}(x)$ refers to the second derivative of $f(x)$ not $f(x)^2$ (the output of $f(x)$ squared). $f^{(0)}$ (the zeroth derivative of $f$) is simply the original function.
$$f^{(2)} (x)$$

### Derivatives of functions of more than one variable¶

The rate of change of a function of more than one variable when only one the inputs is changed and all the others are kept constant. $$\frac{\partial y}{\partial x_1}=\lim\limits_{\Delta x \to 0} \frac{f(x_1+\Delta x, x_2, ..., x_n)-f(x_1, x_2, ..., x_n)}{\Delta x}$$

## Calculating derivatives symbolically¶

There are a number of rules which are useful in calculating the value of derivatives of functions which can be written down as formulae.

### The product rule¶

If there are two functions $g(x)$ and $h(x)$ which both depend on $x$, then the derivative of $g(x)h(x)$ (which can be denoted $\frac{d(g(x)h(x))}{dx}$) is simply $\frac{dg}{dx}h(x) + g(x)\frac{dh}{dx}$.

#### Application of the binomial theorem to differentiating products¶

\frac{d}{dx}(f(x)g(x)) = \sum_{k=0}^{n} \binom{n}{k} f^{(n)}(x) g^{(n-k)}(x)

### The quotient rule¶

This content is not yet ready.

### The chain rule¶

#### Single-variable functions¶

Some functions can be rewritten as the 'chaining' of two different functions together. For instance, the function $y(x) = (x-2)^3$ can be rewritten as $y(u(x))$ where $y(x) = x^3$ and $u(x) = x-2$. The calculation of derivatives of these sorts of functions can easily be done with the use of the chain rule.

\frac{dy}{dx} = \frac{dy}{du}\frac{du}{dx}

The above equation gives the chain rule for a function of one variable. I interpret the chain rule to mean the effect that a small change in $u$ has upon the value of $y$ - $\frac{dy}{du}$ – multiplied by the effect that the change in $x$ has upon $u$ – $\frac{du}{dx}$. $u$ is, in effect, a substitution.

## Calculating derivatives numerically.¶

Please see the computer science section for details on numeric computation of derivatives.