Correlation Analysis - Study Notes
Lesson summary
Correlation analysis studies two variables at the same time to find whether they are linearly related and how strongly. If movements in one variable are accompanied by movements in the other, the two are said to be correlated. Karl Pearson developed the product moment coefficient, Charles Spearman developed rank correlation in 1904, and Udny Yule gave the coefficient of association.
4.1 Definition of correlation
- A.M. Tuttle: "An analysis of the co-variation of two or more variables is usually called correlation."
- Ya-kun-chou: "The attempts to determine the degree of relationship between variables."
- Croxton and Cowden: when the relationship is quantitative, the tool for discovering and measuring it is correlation.
- The measure of correlation is the correlation coefficient (correlation index). It is an absolute measure.
- Uses: it investigates the type and strength of the relationship between two variables.
4.2 Types of correlation
| Type | Meaning | Symbol |
|---|---|---|
| Simple (linear) | Between two given variables | \( r_{xy} \) |
| Partial | Between two variables after removing the effect of others | \( r_{xy.z} \) |
| Multiple | Between a group of variables and a variable outside the group | \( R_{y.(xz)} \) |
Five types of simple correlation
| Type | Behaviour | Example |
|---|---|---|
| Positive (direct) | Both variables move in the same direction | Income and savings; marks in Maths and Statistics |
| Negative (inverse) | Variables move in opposite directions | Price and demand; unemployment and purchasing power |
| Uncorrelated | No linear association, \( r = 0 \) | Age and health care (non-linear) |
| Perfect positive | x and y increase or decrease proportionately | \( r = +1 \) |
| Perfect negative | x increases and y decreases proportionately | \( r = -1 \) |
Uncorrelated does not imply independence: there may be a non-linear relationship. The method of measuring correlation depends on the scale: ratio (Pearson), ordinal (Spearman) or nominal (Yule; chi-square for larger tables).
4.3 Scatter diagram
- One variable on the X-axis, the other on the Y-axis; the pattern of points shows the type of correlation.
- Band rising from lower left to upper right: positive, \( 0 < r < 1 \). Band falling from upper left to lower right: negative, \( -1 < r < 0 \).
- Points spread over the plane: uncorrelated, \( r = 0 \). All points on a rising line: \( r = +1 \); on a falling line: \( r = -1 \).
- Merits: simple and non-mathematical, not influenced by extreme items, the first step in studying a relationship, gives a rough idea at a glance.
- Demerits: shows only the direction, not the exact strength; no mathematical formula is involved.
4.4 Karl Pearson's correlation coefficient
- Covariance: \( \text{cov}(X,Y) = E[(X - E(X))(Y - E(Y))] = E(XY) - E(X)E(Y) \).
- \( r = \dfrac{\text{cov}(X,Y)}{\sqrt{V(X)V(Y)}} \), also called the product moment correlation coefficient.
- Computing form: \( r = \dfrac{n\sum xy - \sum x \sum y}{\sqrt{n\sum x^2 - (\sum x)^2}\sqrt{n\sum y^2 - (\sum y)^2}} \).
- Properties: \( r_{xy} = r_{yx} \); free from units of measurement; unaffected by change of origin and scale (short-cut method with \( u = \frac{x - A}{c} \), \( v = \frac{y - B}{d} \)).
- Interpretation: \( -1 \le r \le 1 \); \( |r| \ge 0.7 \) is called high. If X and Y are independent, \( r = 0 \), but not conversely.
- Example 4.1: heights of fathers and sons give \( r = 0.603 \) by both the direct and short-cut methods. Example 4.2: marks in two tests give \( r = 0.95 \).
Limitations
- Outliers strongly influence r; they may be dropped for a meaningful conclusion.
- Correlation does not imply a causal relationship.
- \( r = 0 \) means no linear relationship, but a curvilinear one may exist (age and health care).
- Spurious correlation: an association drawn from r that may not exist in reality ("spurious" is Latin for false).
4.5 Spearman's rank correlation coefficient
- Used for ordinal data, denoted \( \rho \): \( \rho = 1 - \dfrac{6\sum D_i^2}{n(n^2 - 1)} \), where \( D_i = R_{1i} - R_{2i} \).
- Example 4.3: \( \sum D^2 = 60 \), \( n = 10 \), \( \rho = 0.636 \) (strong agreement). Example 4.4: \( \rho = -1 \). Example 4.5: \( \rho = -0.6071 \).
| Value of \( \rho \) | Interpretation |
|---|---|
| 0.01 to 0.19 | Very weak agreement |
| 0.20 to 0.39 | Weak agreement |
| 0.40 to 0.59 | Moderate agreement |
| 0.60 to 0.79 | Strong agreement |
| 0.80 to 1.0 | Very strong agreement |
Negative values read the same way with "disagreement".
Repeated ranks
- Tied items get the average of the ranks they would receive: two at the 8th place get 8.5 each and the next rank is 10; three at the 8th place get 9 each and the next is 11.
- \( \rho = 1 - \dfrac{6\left[\sum D_i^2 + \frac{m_1^3 - m_1}{12} + \frac{m_2^3 - m_2}{12} + \dots\right]}{n(n^2 - 1)} \). Example 4.6 gives \( \rho = 0 \).
4.6 Yule's coefficient of association
- For two attributes A and B (with complements \( \alpha \), \( \beta \)) in a \( 2 \times 2 \) table: \( Q = \dfrac{(AB)(\alpha\beta) - (A\beta)(\alpha B)}{(AB)(\alpha\beta) + (A\beta)(\alpha B)} \).
- Q ranges from -1 to +1: negative values show inverse association, positive values direct association.
- Example 4.7: coaching and success, \( Q = 0.561 \), a positive association.
- Consistency: build the \( 2 \times 2 \) table; if any cell frequency is negative the data are inconsistent (Example 4.8: \( (\alpha\beta) = -20 \)).
More for this chapter
Book Back Questions19 textbook MCQs · solved
Additional MCQs15 extra MCQs · solved
Practice TestInteractive · instant score
Book Back TestTest yourself on the textbook set
Additional MCQ TestTest yourself on the extra set
Formula SheetAll key formulas