What is Statistical learning?

“ Statistical learning theory is a framework for machine learning drawing from the fields of statistics and functional analysis. Statistical learning theory deals with the problem of finding a predictive function based on data.” — Wikipedia

The above mentioned is a formal definition for statistical learning, provided in the Wikipedia page. We will have an example to help you understand clearly what is statistical learning.

Consider you are hired by a company to increase the profit earned by their company. You are given the data set sales, income, expenditure of their products all over the globe. From the given set of data, if you are able to find out any relationship between the given data, that can increase influence the profit earned by the company , in order to analyze the relationships within data you have to use statistical learning.

Let Y be the output or label, X be the input or features predictors. Then an expression can be written as

The expression above gives a relationship between the input X and output Y, defines as f. The function f is fixed over data, ε refers to the error occurred in deriving Y from X. ε is a zero mean error term that is completely independent of X.

The important thing is to estimate the function f, such that it can model Y with more accuracy using X. Collection of algorithms that we use to estimate f is known as Statistical learning. There are many algorithms, that are used to estimate f, they will be covered in later.