Linear Regression Formulas

The Model Equation & Error Metrics (SSE, MSE, RMSE)
$$ y = mx + b $$
Linear Regression Equation
What it is: The basic formula for a straight line used to make predictions.
  • \(x\): Input
  • \(y\): Output (Prediction)
  • \(m\): Slope (often called \(w\) or "weight" in ML)
  • \(b\): Bias / Intercept
$$ SSE = \sum_{i=1}^{n} (y_i - \hat{y}_i)^2 $$
SSE (Sum of Squared Errors)

What it is: The sum of the squares of all errors. The fundamental metric.

Why it's useful:

  • Mathematically ideal for optimization (graph is a smooth convex bowl).
  • Heavily "punishes" large errors (error squared grows very fast).

Downside: Value grows with the size of the dataset.

$$ MSE = \frac{SSE}{n} = \frac{1}{n} \sum_{i=1}^{n} (y_i - \hat{y}_i)^2 $$
MSE (Mean Squared Error)

What it is: The average value of the squared error.

Why it's useful:

  • Independent of the number of data points (\(n\)).
  • Allows comparison of model quality across datasets of different sizes.

Downside: Units are "squared dollars" or "squared meters", which are hard to interpret.

$$ RMSE = \sqrt{MSE} = \sqrt{\frac{1}{n} \sum_{i=1}^{n} (y_i - \hat{y}_i)^2} $$
RMSE (Root Mean Squared Error)

What it is: The square root of the mean squared error.

Why it's useful:

  • Returns dimensionality to original units (e.g., dollars).
  • Easy to explain to business: "On average, our model is off by X dollars".
  • The de facto standard for evaluating accuracy in real tasks.