ELI5: Explain Like I'm 5

Proofs involving ordinary least squares

When we have some data, we want to find a line that best fits the data points. This line is called a regression line. Ordinary least squares is a method of finding this regression line that minimizes the sum of the squares of the differences between the actual data points and the predicted values on the regression line.

So, let's say we have some data about the heights of children based on their ages. We want to find a line that fits these data points.

First, we plot the data points on a graph with age on the x-axis and height on the y-axis. Now we need to find the equation of the line that best fits these points.

To do this, we use the ordinary least squares method. We start by choosing a line and calculating the sum of the squares of the differences between the actual data points and the predicted values on the line. We repeat this for many different lines until we find the line that gives us the smallest sum of square differences (also called the residual sum of squares or RSS).

Once we've found this best fitting line, we can use it to make predictions about the heights of children at different ages. For example, if we know the age of a child, we can use the equation of the line to predict their height.

So, in summary, ordinary least squares is a method of finding the line that best fits a set of data points by minimizing the sum of the squares of the differences between the actual data points and the predicted values on the line.