Basics of Calculus

Approximations

We can approximate the value at any point of the curve by using polynomials of increasing orders.

Linear Approximation

Consider a point $x=a$ on the graph at which we wish to approximate the value of the function using a line centered around some $x=a$. We center at $a$ because the approximation is fundamentally about local behavior, and locality is measured relative to the point you care about.

The equation of this approximation line would be $L(x)=m(x-a)+c$. Around some really small $ϵ-$neighbourhood around $x=a$ we can say that the curve of the graph and the line coincide, thus $f(x)=L(x)$.

By plugging in $x=a$ we can get $c=f(a)$. Due to this we can even say that,

$$\begin{array}{rlrl}& & f(x)& =m(x-a)+c\\ & \Rightarrow & m& =\frac{f(x)-f(a)}{x-a}\\ & \Rightarrow & \underset{x\to a}{\lim }m& =\underset{x\to a}{\lim }\frac{f(x)-f(a)}{x-a}\\ & \Rightarrow & m& =f^{\prime }(a)\end{array}$$

Using this information we can finalize it all by doing,

$$\begin{array}{rlrl}& & L(x)& =m(x-a)+c\\ & \Rightarrow & f(x)& =f(a)+f^{\prime }(a)(x-a)\end{array}$$

If we consider a fixed point $x$ instead of $a$ and we wish to find the value of any point some $△x$ distance away from $x$, we can rewrite the above equations as

$$f(x+△x)=f(x)+f^{\prime }(x)△x$$

This is the First-Order Approximation of a function, also called as the Linear Approximation.

Quadratic Approximation

Similar to how we had a linear centered around $x=a$ for linear approximation, we can also have a parabola centered around $x=a$ for a second order approximation of the function. This parabola can be represented as,

$$p(x)=A(x-a{)}^2+B(x-a)+C$$

Again, by plugging in $x=a$ we get that $C=p(a)$. As in some $ϵ-$neighbourhood around $a$ the parabola and the curve of the graph would coincide, we can say that $p(a)=f(a)$.

We can get $B$ by doing,

$$\begin{array}{rlrl}& & f(x)& =A(x-a{)}^2+B(x-a)+C\\ & \Rightarrow & f(x)& =A(x-a{)}^2+B(x-a)+f(a)\\ & \Rightarrow & \frac{f(x)-f(a)}{x-a}& =A(x-a)+B(x-a)\\ & \Rightarrow & \underset{x\to a}{\lim }A(x-a)+B(x-a)& =\underset{x\to a}{\lim }\frac{f(x)-f(a)}{x-a}\\ & \Rightarrow & B& =f^{\prime }(a)\end{array}\text{\hspace{1em}(1)}$$

We can get $A$ by taking the derivative of $(1)$ and solving for $A$.

$$\begin{array}{rlrl}& & f(x)& =A(x-a{)}^2+B(x-a)+C\\ & \Rightarrow & f^{\prime }(x)& =2A(x-a)+B\\ & \Rightarrow & f^{\prime }(x)& =2A(x-a)+f^{\prime }(a)\\ & \Rightarrow & A& =\frac{f^{\prime }(x)-f^{\prime }(a)}{2(x-a)}\\ & \Rightarrow & \underset{x\to a}{\lim }A& =\frac{1}{2}\underset{x\to a}{\lim }\frac{f^{\prime }(x)-f^{\prime }(a)}{(x-a)}\\ & \Rightarrow & A& =\frac{1}{2}{f}^{\prime \prime }(a)\end{array}\text{\hspace{1em}(1)}$$

By putting it all together we get,

$$f(x)=f(a)+f^{\prime }(a)(x-a)+\frac{1}{2}{f}^{\prime \prime }(a)(x-a{)}^2$$

or if we write this in the terms of $△x$, we get

$$f(x+△x)=f(x)+f^{\prime }(x)△x+\frac{1}{2}{f}^{\prime \prime }(x)△x^2$$

This same method can then be extended to higher degrees to get higher order approximations of a function. That’s the Taylor Series.

Taylor Series

The Taylor series provides a higher order approximation for a function’s value around some neighbourhood of any point $x$.

$$\begin{array}{rl}f(x+△x)& =f(x)+f^{\prime }(x)△x+\frac{1}{2!}{f}^{\prime \prime }(x)△x^2+\frac{1}{3!}{f}^{\prime \prime \prime }(x)△x^3+\dots \\ & =\sum _{i=0}\frac{1}{i!}{f}^{(i)}(x)△x^i\end{array}$$

Cauchy Schwarz Inequality

Directional Derivative

Direction of steepest descent is $-\nabla f(x^{\ast })$. This can be shown by either using the Cauchy Schwarz Inequality or Taylor Series.