Assessing the accuracy of ChatGPT in college admission predictions

Generative artificial intelligence (AI) tools are increasingly utilized in higher education, particularly in college admissions. This study assesses OpenAI’s ChatGPT in predicting admission outcomes, comparing it to CollegeVine’s “chancing” engine. Using 64 Common App submissions from three volunteers applying to various colleges, we asked ChatGPT to predict admission probabilities and decisions based on their profiles. ChatGPT achieved 84.38% prediction accuracy, matching CollegeVine, and had a slightly better Brier Score (0.1164 vs. 0.1186), indicating better probability calibration. These results suggest that generative AI can perform similarly to dedicated admissions prediction models. However, findings are limited to a small sample size, highlighting the need for further research. We discuss the responsible integration of AI in college advising, stressing transparency, fairness, and ethical considerations.