TN Online TestSamacheer Kalvi practice

Regression Analysis - Study Notes

Share this chapter: Telegram

Chapter Summary

Regression analysis finds the mathematical functional relationship between a dependent (response) variable and one or more independent variables (regressors or predictors). Correlation describes the type and intensity of a linear relationship but gives no function for prediction; regression supplies it and is widely used for prediction.

Background

Types of Regression

TypeMeaning
Simple linear regressionLinear relationship with one independent variable: \( Y = a + bX + e \)
Multiple linear regressionLinear relationship with two or more independent variables: \( Y = a + b_1X_1 + b_2X_2 + \dots + b_tX_t + e \)
Non-linear (curvilinear) regressionThe relationship is not a straight line

In \( Y = a + bX + e \), \( X \) is the independent variable, \( Y \) the dependent variable, \( a \) the intercept, \( b \) the slope and \( e \) the error term. Multiple and non-linear regression are outside the syllabus and are given only for knowledge.

Uses of Regression

Why Two Regression Lines?

When \( X \) and \( Y \) are interchangeable, we get the regression line of \( Y \) on \( X \) and that of \( X \) on \( Y \). Both are valid, but one must choose the one suited to the situation. If \( X \) only causes \( Y \), there is only one line, of \( Y \) on \( X \).

Assumptions

Method of Least Squares

The line of best fit minimises the error in estimating \( Y \) for any \( X \) in its range. The residual is \( e_i = y_i - \hat{y}_i \). The method chooses \( a \) and \( b \) so that

Setting the partial derivatives of \( E(a,b) \) to zero gives the normal equations:

Their solution is \( \hat{a} = \bar{y} - \hat{b}\bar{x} \) and \( \hat{b} = \frac{Cov(X,Y)}{V(X)} = r_{XY}\frac{SD(Y)}{SD(X)} \).

Regression Coefficients

Y on XX on Y
Coefficient\( b_{YX} = r_{XY}\frac{SD(Y)}{SD(X)} \)\( b_{XY} = r_{XY}\frac{SD(X)}{SD(Y)} \)
From sums\( \frac{n\sum xy - \sum x \sum y}{n\sum x^2 - (\sum x)^2} \)\( \frac{n\sum xy - \sum x \sum y}{n\sum y^2 - (\sum y)^2} \)
Slope-point form\( \hat{Y} - \bar{y} = b_{YX}(x - \bar{x}) \)\( \hat{X} - \bar{x} = b_{XY}(y - \bar{y}) \)

The slope \( b_{YX} \) is the change in the mean of \( Y \) for every unit change in \( X \). When the means are decimals, deviations from assumed values \( u = x - A \), \( v = y - B \) can be used, since the coefficients do not change with origin.

Properties of Regression Coefficients

  1. \( r_{XY} = \sqrt{b_{XY} \times b_{YX}} \): correlation is the geometric mean of the coefficients.
  2. Both coefficients have the same sign.
  3. If one coefficient is greater than unity, the other is less than unity.
  4. \( r_{XY} \) has the same sign as the coefficients.
  5. \( \frac{b_{XY} + b_{YX}}{2} \ge r_{XY} \).
  6. They are independent of change of origin but not of scale.

Properties of Regression Equations

Correlation versus Regression

CorrelationRegression
Shows only the nature and extent of linear relationshipStudies the impact of the independent variable on the dependent variable; used for prediction
Either variable can be x or yIndependent and dependent variables must be chosen with care
Symmetric: \( r_{XY} = r_{YX} \)Not symmetric: \( b_{XY} \) and \( b_{YX} \) differ in meaning

Worked Example (book Example 5.7)

Yield: mean 10, SD 8; rainfall: mean 8, SD 2; \( r = 0.5 \). \( b_{YX} = 0.5 \times \frac{8}{2} = 2 \), so at rainfall 9, \( \hat{Y} = 10 + 2(9 - 8) = 12 \) kg per unit area.

Solved MCQs → Practice test →

More for this chapter

Book Back Questions19 textbook MCQs · solved Additional MCQs15 extra MCQs · solved Practice TestInteractive · instant score Book Back TestTest yourself on the textbook set Additional MCQ TestTest yourself on the extra set Formula SheetAll key formulas

More chapters in Statistics

View all
1 Tests of Significance - Basic Concepts and Large Sample Tests 2 Tests Based on Sampling Distributions - I 3 Tests Based on Sampling Distributions - II 4 Correlation Analysis 6 Index Numbers 7 Time Series and Forecasting 8 Vital Statistics and Official Statistics