Machine Learning-Based Automated Grading and Feedback Tools for Programming: A Meta-Analysis

Research output: Chapter in Book/Report/Conference proceedingConference paperpeer-review

12 Downloads (Pure)


Research into automated grading has increased as Computer Science courses grow.
Dynamic and static approaches are typically used to implement these graders, the most common implementation being unit testing to grade correctness.
This paper expands upon an ongoing systematic literature review to provide an in-depth analysis of how machine learning (ML) has been used to grade and give feedback on programming assignments.
We conducted a backward snowball search using the ML papers from an ongoing systematic review and selected 27 papers that met our inclusion criteria.
After selecting our papers, we analysed the skills graded, the preprocessing steps, the ML implementation, and the models’ evaluations.

We find that most the models are implemented using neural network-based approaches, with most implementing some form of recurrent neural network (RNN), including Long Short-Term Memory, and encoder/decoder with attention mechanisms.
Some graders implement traditional ML approaches, typically focused on clustering.
Most ML-based automated grading, not many use ML to evaluate maintainability, readability, and documentation, but focus on grading correctness, a problem that dynamic and static analysis techniques, such as unit testing, rule-based program repair, and comparison to models or approved solutions, have mostly resolved.
However, some ML-based tools, including those for assessing graphical output, have evaluated the correctness of assignments that conventional implementations cannot.
Original languageEnglish
Title of host publicationProceedings of the 28th ACM Conference on Innovation and Technology in Computer Science Education
PublisherAssociation for Computing Machinery, Inc
Publication statusAccepted/In press - 6 Mar 2023


  • Machine Learning (ML)
  • Automated Grading
  • Computer science education
  • Meta-analyses


Dive into the research topics of 'Machine Learning-Based Automated Grading and Feedback Tools for Programming: A Meta-Analysis'. Together they form a unique fingerprint.

Cite this