Automating Visualization Quality Assessment: a Case Study in Higher Education

Nicolas Holliman

doi:10.2312/cgvc.20211316

Automating Visualization Quality Assessment: a Case Study in Higher Education

Nicolas Holliman

Informatics

Research output: Contribution to conference types › Paper › peer-review

33 Downloads (Pure)

Abstract

We present a case study in the use of machine+human mixed intelligence for visualization quality assessment, applying automated visualization quality metrics to support the human assessment of data visualizations produced as coursework by students taking higher education courses. A set of image informatics algorithms including edge congestion, visual saliency and colour analysis generate machine analysis of student visualizations. The insight from the image informatics outputs has proved helpful for the marker in assessing the work and is also provided to the students as part of a written report on their work. Student and external reviewer comments suggest that the addition of the image informatics outputs to the standard feedback document was a positive step. We review the ethical challenges of working with assessment data and of automating assessment processes.

Original language	English
Pages	49-57
Number of pages	9
DOIs	https://doi.org/10.2312/cgvc.20211316
Publication status	Published - 8 Sept 2021
Event	Computer Graphics and Visual Computing - University of Lincoln/Online, Lincoln, United Kingdom Duration: 8 Sept 2021 → 10 Sept 2021 https://cgvc.org.uk/CGVC2021/

Conference

Conference	Computer Graphics and Visual Computing
Abbreviated title	CGCV
Country/Territory	United Kingdom
City	Lincoln
Period	8/09/2021 → 10/09/2021
Internet address	https://cgvc.org.uk/CGVC2021/

Access to Document

10.2312/cgvc.20211316Licence: Unspecified

NSH_paper1000_CGVC2021_CRC_v2Accepted author manuscript, 4.65 MBLicence: CC BY

Cite this

@conference{9cef9f8b89e241e3af4df41b99d73e2e,

title = "Automating Visualization Quality Assessment: a Case Study in Higher Education",

abstract = "We present a case study in the use of machine+human mixed intelligence for visualization quality assessment, applying automated visualization quality metrics to support the human assessment of data visualizations produced as coursework by students taking higher education courses. A set of image informatics algorithms including edge congestion, visual saliency and colour analysis generate machine analysis of student visualizations. The insight from the image informatics outputs has proved helpful for the marker in assessing the work and is also provided to the students as part of a written report on their work. Student and external reviewer comments suggest that the addition of the image informatics outputs to the standard feedback document was a positive step. We review the ethical challenges of working with assessment data and of automating assessment processes.",

author = "Nicolas Holliman",

note = "CGVC Best Paper Award 2021, The Ken Brodlie prize.; Computer Graphics and Visual Computing, CGCV ; Conference date: 08-09-2021 Through 10-09-2021",

year = "2021",

month = sep,

day = "8",

doi = "10.2312/cgvc.20211316",

language = "English",

pages = "49--57",

url = "https://cgvc.org.uk/CGVC2021/",

}

TY - CONF

T1 - Automating Visualization Quality Assessment: a Case Study in Higher Education

AU - Holliman, Nicolas

N1 - CGVC Best Paper Award 2021, The Ken Brodlie prize.

PY - 2021/9/8

Y1 - 2021/9/8

N2 - We present a case study in the use of machine+human mixed intelligence for visualization quality assessment, applying automated visualization quality metrics to support the human assessment of data visualizations produced as coursework by students taking higher education courses. A set of image informatics algorithms including edge congestion, visual saliency and colour analysis generate machine analysis of student visualizations. The insight from the image informatics outputs has proved helpful for the marker in assessing the work and is also provided to the students as part of a written report on their work. Student and external reviewer comments suggest that the addition of the image informatics outputs to the standard feedback document was a positive step. We review the ethical challenges of working with assessment data and of automating assessment processes.

AB - We present a case study in the use of machine+human mixed intelligence for visualization quality assessment, applying automated visualization quality metrics to support the human assessment of data visualizations produced as coursework by students taking higher education courses. A set of image informatics algorithms including edge congestion, visual saliency and colour analysis generate machine analysis of student visualizations. The insight from the image informatics outputs has proved helpful for the marker in assessing the work and is also provided to the students as part of a written report on their work. Student and external reviewer comments suggest that the addition of the image informatics outputs to the standard feedback document was a positive step. We review the ethical challenges of working with assessment data and of automating assessment processes.

U2 - 10.2312/cgvc.20211316

DO - 10.2312/cgvc.20211316

M3 - Paper

SP - 49

EP - 57

T2 - Computer Graphics and Visual Computing

Y2 - 8 September 2021 through 10 September 2021

ER -

Automating Visualization Quality Assessment: a Case Study in Higher Education

Abstract

Conference

Access to Document

Fingerprint

Cite this