Loading...
Evaluating Large Language Model Performance on Haskell
Chen, Andrew
Chen, Andrew
Abstract
I introduce HaskellEval, a Haskell evaluation benchmark for Large Language Models. HaskellEval’s curation leverages a novel synthetic generation framework, streamlining the process of dataset curation by minimizing manual intervention. The core of this research is an extensive analysis of the trustworthiness of synthetic generations, ensuring accuracy, realism, and diversity. Additional, I provide a comprehensive evaluation of existing open-source models on HaskellEval.
Description
Date
2024-05-01
Journal Title
Journal ISSN
Volume Title
Publisher
Collections
Download Dataset
Rights Holder
Usage License
Embargo
Research Projects
Organizational Units
Journal Issue
Keywords
Citation
Advisor
Poshyvanyk, Denys
Kumar, Pradeep
Wang, GuanNan
Kumar, Pradeep
Wang, GuanNan
Department
Computer Science