Modeling Pass/Fail Outcomes with Minitab's Binary Logistic Regression

The problem: predicting a binary outcome

Engineers often need to predict whether a part will pass or fail based on process settings such as temperature, pressure, and cycle time. The response is binary (Pass = 1, Fail = 0), so ordinary linear regression is inappropriate. Minitab’s Binary Logistic Regression fits a logistic model using maximum‑likelihood estimation, provides Wald tests for each predictor, and automatically produces an ROC curve with the area under the curve (AUC) for threshold‑independent discrimination assessment.

Fitting the model in Minitab

1. Open the worksheet containing the response column (e.g., PassFail) and predictor columns (e.g., Temp, Pressure, CycleTime).
2. Choose Stat > Regression > Binary Logistic Regression.
3. In the dialog, set Response to PassFail and Predictors to the three continuous variables.
4. Click Options and check ROC curve and Store deviance residuals, leverages, and predicted probabilities.
5. Under Model Selection you may enable Backward elimination with a removal α = 0.10 or use AIC; be aware this can over‑fit small data sets (see Trade‑offs).
6. Press OK to run the analysis.

Equivalent session commands (for reproducibility) are:

MTB > BLogistic 'PassFail' = 'Temp' 'Pressure' 'CycleTime';
SUBC>   Options ROC;
SUBC>   Storage Residuals Leverages Fits;
SUBC>   Selection Backward 0.10.

Diagnosing fit and discrimination

Coefficient table (illustrative)

Predictor	Coef	SE Coef	Wald χ²	P‑value	VIF
Constant	-12.34	3.21	14.78	0.0001	—
Temp	0.045	0.012	14.06	0.0002	1.3
Pressure	-0.018	0.009	4.00	0.0455	1.1
CycleTime	0.0021	0.0015	1.96	0.161	1.4

All VIF values are well below 5, indicating no problematic multicollinearity. The Wald test shows Temp and Pressure are significant at α = 0.05, while CycleTime is not.

ROC curve and AUC

Minitab displays the ROC curve and reports an AUC of 0.87 (illustrative). An AUC above 0.8 generally indicates good discrimination, but remember that AUC can be inflated when the positive class (Pass) is rare in the population. If the production pass rate is 5 %, a high AUC may still correspond to a low positive predictive value.

Residual diagnostics

Using the stored deviance residuals and leverages, create a scatterplot of residuals versus predicted probabilities (Graph > Scatterplot). Look for points with large absolute residuals (> 2) and high leverage (> 2 × average leverage). Those observations may be influential and warrant investigation.

Trade‑offs and limitations

• Backward elimination simplifies the model but can capitalize on random noise, especially with fewer than 10 events per predictor. Prefer subject‑matter driven variable selection or penalized methods (e.g., Firth’s penalized likelihood) when data are sparse.
• Complete or quasi‑complete separation (e.g., a predictor perfectly separates Pass/Fail) causes the maximum‑likelihood algorithm to diverge. Minitab will warn and may fail to converge; the remedy is to use Firth’s penalized likelihood (available via the Penalized option in newer releases) or to collapse categories.
• Class imbalance inflates AUC. Complement AUC with calibration plots (observed vs. predicted probabilities) and, if needed, decision‑curve analysis to choose a probability threshold that matches business costs.
• Independence assumption – correlated observations (repeated measures on the same machine) violate the model and inflate Type I error. Use a generalized estimating equations (GEE) or mixed‑effects logistic model instead.

Actionable next steps

1. Run the binary logistic regression with the Cross‑Validation option (leave‑one‑out) to obtain an optimism‑adjusted AUC. Compare the cross‑validated AUC to the apparent AUC; a large drop signals over‑fitting.
2. Save deviance residuals and leverages (Storage dialog) and plot them against fitted probabilities. Flag any observation with |residual| > 2 and leverage > 2 × mean leverage for review.
3. If separation warnings appear, re‑run the analysis with Penalized likelihood (Firth) enabled, or add a small ridge penalty.
4. Validate the final model on a hold‑out sample (or a later production lot). Compute the empirical AUC by ranking observed outcomes against predicted probabilities and verify it matches Minitab’s reported value within sampling error.
5. Document the chosen probability threshold, the expected false‑positive/false‑negative rates, and the monitoring plan for model drift.

By following these steps you get a statistically sound, diagnostically checked logistic model that can be deployed for real‑time pass/fail prediction while keeping an eye on the common pitfalls of separation, multicollinearity, and class imbalance.