statsmodels: OLS Regression, Formulas & Statistical Output

statsmodels emphasizes statistical INFERENCE -- coefficients, standard errors, p-values, confidence intervals, diagnostic tests -- distinct from scikit-learn's prediction/ML-pipeline orientation. Complementary, not competing: scikit-learn for the broader ML workflow, statsmodels when a stakeholder needs to understand WHY a model predicts what it does.

Array-Based API

import statsmodels.api as sm

# GOTCHA: unlike the formula API, this does NOT add an intercept
# automatically -- fits a model forced through the origin unless
# you explicitly add one
X = sm.add_constant(X)  # adds a column of 1s for the intercept

model = sm.OLS(y, X)
results = model.fit()
print(results.summary())  # rich statistical report, not just coefficients

results.params       # fitted coefficients
results.pvalues       # per-coefficient significance
results.conf_int()    # 95% confidence intervals
results.rsquared       # goodness of fit
predictions = results.predict(X_new)

Formula API (R-Style)

import statsmodels.formula.api as smf

# Reads naturally, references DataFrame columns directly, adds an
# intercept automatically, handles categorical encoding via C()
results = smf.ols('sales ~ price + C(region)', data=df).fit()

# Interaction term: does the effect of ad_spend on sales depend on season?
# Shorthand for ad_spend + season + ad_spend:season
results = smf.ols('sales ~ ad_spend * season', data=df).fit()

# Logistic regression via GLM -- generalizes beyond continuous outcomes
import statsmodels.api as sm
logit = sm.GLM(y_binary, X, family=sm.families.Binomial()).fit()

Interpreting the Output Correctly

• Low p-value = evidence against 'this predictor has zero true effect' -- NOT proof of causation, and doesn't establish practical importance.

• A 95% confidence interval describes the long-run behavior of the interval-construction PROCEDURE across repeated sampling -- not a direct 95% probability the true value is in THIS specific interval.

• Statistical significance and practical importance are separate questions -- always check both.