Date Thesis Awarded
5-2024
Access Type
Honors Thesis -- Open Access
Degree Name
Bachelors of Arts (BA)
Department
Computer Science
Advisor
Denys Poshyvanyk
Committee Members
Pradeep Kumar
GuanNan Wang
Abstract
I introduce HaskellEval, a Haskell evaluation benchmark for Large Language Models. HaskellEval’s curation leverages a novel synthetic generation framework, streamlining the process of dataset curation by minimizing manual intervention. The core of this research is an extensive analysis of the trustworthiness of synthetic generations, ensuring accuracy, realism, and diversity. Additional, I provide a comprehensive evaluation of existing open-source models on HaskellEval.
Recommended Citation
Chen, Andrew, "Evaluating Large Language Model Performance on Haskell" (2024). Undergraduate Honors Theses. William & Mary. Paper 2186.
https://scholarworks.wm.edu/honorstheses/2186
