Consider the simple linear regression model Y =B0 + B1x + e suppose that the analyst wants to use z = x – x as the regress or variable.
(a) Using the data in Exercise 11-12, construct one scatter plot of the (xi, yi) points and then another of the (zi = xi – x, y) points. Use the two plots to intuitively explain how the two models, Y = B0 + Bix + E and, are related.
(b) Find the least squares estimates of B0 and BI and in the model Y = B0 + BIz + E. How do they relate to the least squares estimates and B0 and B1?
The Correct Answer and Explanation is:
To address this question properly, we’ll break it into the two parts: (a) visual and intuitive explanation, and (b) algebraic derivation and relationship of coefficients.
Let’s first clarify the two models:
- Original regression model:
Y=β0+β1x+εY = \beta_0 + \beta_1 x + \varepsilon - Centered variable model (where z=x−xˉz = x – \bar{x}):
Y=β0∗+β1∗z+εY = \beta_0^* + \beta_1^* z + \varepsilon
(a) Scatter Plots and Intuitive Explanation
The variable z=x−xˉz = x – \bar{x} is simply a centered version of xx, meaning we shift the x-values so that their mean is zero. This transformation does not change the shape or slope of the scatter plot; it just repositions the data horizontally.
- The scatter plot of (xi,yi)(x_i, y_i) shows the original data and its linear relationship.
- The scatter plot of (zi,yi)=(xi−xˉ,yi)(z_i, y_i) = (x_i – \bar{x}, y_i) is identical in shape but shifted so the x-axis is centered around zero.
Intuition: Since we’re just shifting the x-axis, the slope β1\beta_1 of the line doesn’t change. However, because the x-values now average to zero, the intercept of the new model β0∗\beta_0^* is the value of YY when x=xˉx = \bar{x}. This simplifies interpretation and calculations.
(b) Least Squares Estimates and Relationship
From the original model:
- β^1=∑(xi−xˉ)(yi−yˉ)∑(xi−xˉ)2\hat{\beta}_1 = \frac{\sum (x_i – \bar{x})(y_i – \bar{y})}{\sum (x_i – \bar{x})^2}
- β^0=yˉ−β^1xˉ\hat{\beta}_0 = \bar{y} – \hat{\beta}_1 \bar{x}
In the transformed model with zi=xi−xˉz_i = x_i – \bar{x}:
- β^1∗=β^1\hat{\beta}_1^* = \hat{\beta}_1 (the slope remains the same)
- β^0∗=yˉ\hat{\beta}_0^* = \bar{y} (since the mean of zz is 0)
Explanation
In simple linear regression, centering the independent variable xx by subtracting its mean (z=x−xˉz = x – \bar{x}) simplifies the regression model without altering its predictive power. This transformation merely redefines the horizontal axis to be centered at the average of xx, which enhances interpretability.
The slope of the regression line, β^1\hat{\beta}_1, represents the rate of change in YY for a unit change in xx. This rate remains unchanged in the centered model because the relationship between xx and yy is linear and invariant under shifting. Therefore, β^1∗=β^1\hat{\beta}_1^* = \hat{\beta}_1.
However, the intercept does change. In the original model, β^0\hat{\beta}_0 is the predicted value of YY when x=0x = 0, which may lie outside the data range, making interpretation less meaningful. In the centered model, the intercept β^0∗\hat{\beta}_0^* equals the mean of YY, yˉ\bar{y}, because z=0z = 0 when x=xˉx = \bar{x}. This simplifies the model and interpretation: the intercept now represents the average response at the average xx.
In practical terms, centering helps reduce multicollinearity in multiple regression and can improve numerical stability. It doesn’t change predictions or goodness-of-fit; it only repositions the model’s baseline to be more meaningful and interpretable.
Thus, the relationship is:
- β^1∗=β^1\hat{\beta}_1^* = \hat{\beta}_1
- β^0∗=β^0+β^1xˉ=yˉ\hat{\beta}_0^* = \hat{\beta}_0 + \hat{\beta}_1 \bar{x} = \bar{y}
This shows that centering leads to a more intuitive intercept while preserving the slope.
