Michael Leon Chrzan

Data Scientist bridging practice, policy, and advanced analytics through machine learning, causal inference, and NLP to drive equity and evidence-based improvement in education and beyond.

I am beginning to develop a line of work on measurement: how to build and validate useful measures from text at scale. These projects explore LLM-generated pairwise comparisons as an alternative to direct ratings, applying the approach to both item difficulty and emotional affect.

Consensus without Accuracy: Investigating LLM’s Recovery of Item Difficulty Using Paired Comparisons

Can large language models recover item difficulty from pairwise comparisons? This study uses Bradley-Terry models to turn LLM comparisons into difficulty estimates, then evaluates how closely the resulting rankings align with 1PL item response theory parameters. The analysis spans five Item Response Warehouse datasets and four LLMs, examining the potential for scalable, AI-assisted item calibration.

Read the paper in the ACL Anthology

Citation: Chrzan, M. L., & Domingue, B. (2026). Consensus without Accuracy: Investigating LLM’s Recovery of Item Difficulty Using Paired Comparisons. Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Works in Progress, 24-36.

EmoPair: A New Paradigm for Measuring Emotional Affect

EmoPair updates the EmoBank text corpus by replacing direct scalar emotion ratings with LLM-driven pairwise comparisons. The comparisons are converted into continuous Valence-Arousal-Dominance (VAD) scores using a Bradley-Terry model. The study reports improved human inter-rater reliability for Arousal and Dominance, and finds that models trained on EmoPair outperform EmoBank baselines on those two dimensions. The resulting dataset offers a scalable source of supervision for research on emotional affect in text.

Read the paper on OpenReview