Regression vs ANOVA
Regression and ANOVA (Analysis of Variance) are two methods in the statistical theory to analyze the behavior of one variable compared to another. In regression, it is often the variation of dependent variable based on independent variable while, in ANOVA, it is the variation of the attributes of two samples from two populations.
More about Regression
Regression is a statistical method used to draw the relation between two variables. Often when data are collected there might be variables which are dependent on others. The exact relation between those variables can only be established by regression methods. Determining this relationship helps to understand and predict the behavior of one variable to the other.
The most common application of the regression analysis is to estimate the value of the dependent variable for a given value or range of values of the dependent variables. For example, using regression we can establish the relation between the commodity price and consumption based on the data collected from a random sample. Regression analysis will produce a regression function of the data set, which is a mathematical model that best fits the available data. This can easily be represented by a scatter plot. Graphically regression is equivalent to finding the best fitting curve for the give data set. The function of the curve is the regression function. Using the mathematical model, the usage of a commodity can be predicted for a given price.
Therefore, the regression analysis is widely used in predicting and forecasting. It is also used to establish relationships in experimental data, in the fields of physics, chemistry, and many natural sciences and engineering disciplines. If the relationship or the regression function is a linear function, then the process is known as a linear regression. In the scatter plot, it can be represented as a straight line. If the function is not a linear combination of the parameters, then the regression is non-linear.
More about ANOVA (Analysis of Variance)
ANOVA does not involve the analysis of a relation between two or more variables explicitly. Rather it checks whether two or more samples from different populations have the same mean. For example, consider the test results of an exam held for a grade in the school. Even though the tests are different, performance may be alike from class to class. One method of verifying this is by comparing the means of every class. ANOVA or ANalysis Of Variance allows this hypothesis to be tested. At the basics, ANOVA can be considered as an extension of the t-test, where the means of the two samples drawn from two populations are compared.
Fundamental idea of ANOVA is to consider the variation within the sample and variation between the samples. The variation within the sample can be attributed to the randomness, whereas the variation among samples can be attributed to both randomness and other external factors. Analysis of variance is based on three models; fixed effects model, random effects model, and mixed effects model.
What is the difference between Regression and ANOVA?
• ANOVA is the analysis of variation between two or more samples while regression is the analysis of a relation between two or more variables.
• ANOVA theory is applied using three basic models (fixed effects model, random effects model, and mixed effects model) while regression is applied using two models (linear regression model and multiple regression model).
• ANOVA and Regression are both two versions of the General Linear Model (GLM). ANOVA is based on categorical predictor variables, while regression is based on quantitative predictor variables.
• Regression is the more flexible technique, and it is used in forecasting and predicting while ANOVA is used to compare the equality of two or more populations.