Predicting Student Performance with Principal Component Analysis (PCA)

Abstract

Predicting student academic success from big, high-dimensional institutional datasets is hampered by substantial multicollinearity among behavioral, engagement, and continuous-assessment factors, which can weaken the stability and interpretability of downstream classifiers. This work studies Principal Component Analysis (PCA) as a dimensionality-reduction technique for multi-class student performance prediction on the dataset (15,000 students, one universities, six academic years). After removing cumulative-grade-point-average (CGPA)-derived features to prevent target leakage, 38 leakage-aware predictors were standardized and one-hot encoded into a 78-dimensional feature space. PCA was applied to this space, and 46 principle components were determined to jointly explain 95% of total variance — a 41.0% reduction in dimensionality. Seven classifiers (Naive Bayes, Logistic Regression, Decision Tree, k-Nearest Neighbors, Support Vector Machine, Random Forest, and XGBoost) were trained and evaluated, under an identical held-out test protocol, on both the original 78-dimensional feature space and the 46-dimensional PCA-reduced representation. The PCA-reduced Logistic Regression model produced the best overall result (87.07% accuracy, 87.55% macro F1-score, macro ROC-AUC = 0.981), barely beating its own non-reduced counterpart. More significantly, Naive Bayes improved from 77.13% to 83.63% accuracy following PCA — a 6.50-percentage-point gain attributable to PCA's de-correlation of the transformed feature space, which better meets Naive Bayes conditional-independence premise. A similar, smaller gain was observed for the Decision Tree. Loading analysis and a two-dimensional PCA projection further demonstrate that the first main component is dominated by continuous-assessment and engagement indicators (Project_Score, Attendance_Pct, Assignment_Score) and visually distinguishes the four performance classes along a single axis. These findings show that PCA provides a viable and understandable method for reducing the dimensionality of the data in the prediction of student performance with comparable or better accuracy at reduced dimensions, with the largest improvements found in classifiers whose assumptions are vulnerable to feature correlation..