TY - CHAP
T1 - The Effectiveness of Different Assessment Techniques being used to Measure the Impact of Haptic Technologies on Dental Undergraduate Students' 3-D Perceptions, Motor and Clinical Skills
AU - Cox, Margaret Josephine
AU - San Diego, Jonathan
AU - Newton, Jonathan Timothy
AU - Woolford, Mark John
AU - Quinn, Barry Francis Arthur
AU - Dunne, Stephen
AU - Millar, Brian James
AU - Shahriari-Rad, Arash
PY - 2010/9
Y1 - 2010/9
N2 - 0723
The effectiveness of different assessment techniques being used to measure the impact of haptic technologies on Dental undergraduate students’ 3-D perceptions, motor and clinical skills.
Margaret Cox, Jonathan P. San Diego, Tim Newton, Mark Woolford, Barry Quinn, Stephen Dunne, Brian Millar, Arash Shariari-Rad
The Dental Institute, King's College London, London, United Kingdom
Background
The growth of IT in higher education (HE) has led to an expanse of technology enhanced learning
(TEL) courses at university level, many of these using IT-based resources such as on-line Virtual
Learning Environments (VLEs), computer simulations and video-conferencing (McGill and Hobbs,
2007). There is also a widening use of IT to aid in the assessment of students’ learning in higher
education although much of this is based on online multiple choice questions and focuses on
summative assessment at the end of a particular course (Cox et al., 2008). However, although there
is significant research into the benefits of formative assessment (Webb and Cox, 2007) much of this is
with children in schools and there is relatively little evidence of the strengths and weaknesses of
different formative and summative methods used in higher education. Formative assessment is
essentially regarded as a qualitative exercise providing a feedback process to the learner, engaging
the learner and teacher in acquiring a common understanding of what has been understood and
helping the teacher to modify their teaching to take account of what has been learnt (Black and
William, 1998). Using traditional methods, formative assessment in HE includes providing feedback to
individual students during a clinical session amongst the students themselves and between tutor and
student, challenging students with questions. The assessment research described in this paper is part
of the hapTEl project (http://www.haptel.kcl.ac.uk), funded by the ESRC/EPSRC’s Teaching and
Learning Research Programme’s Technology Enhanced Learning Phase (see http://www.tlrp.org/tel/).
When students are using haptic work-stations then they will also receive feedback from the system
itself, not only through the tactile experience but through the on-screen images which show them how
well they have achieved the clinical task. There is therefore a need to research assessment methods
to determine how effective different assessment techniques are and how these compare with the
traditional formative and summative assessments used in most Dental Undergraduate courses.
Research Questions
The hapTEL project has a range of research goals which relate to the technical, curriculum and
evaluation strands, all of which contribute to the work of the project, however the specific focus of the
TEL assessment work reported in this paper is to address the following research questions:
What are the strengths and weaknesses of psychological spatial reasoning and manipulation skills
tests being used to measure the impact of TEL on students’ skills and performance?
How do such tests compare with the traditional formative and summative assessments of students’
clinical skills as applied to undergraduate dental students?
Methods
The focused study described in this paper involves 70 Year 2 (total =140) university undergraduate
dental students who were tested for a range of 3-D perceptions and spatial reasoning skills at the end
of their first university year (June 2009). These involved a series of spatial reasoning, attitude and
gross and fine motor skills tests recommended by the British Psychological Society. This cohort of
students did not subsequently use the project’s haptics technologies but are being assessed (March,
2010) for their clinical skills following two terms of teaching in the clinical skills laboratory. In order to
assess the strengths and weaknesses of these pre-tests which are also being used to assess the
impact of TEL on the Year 1 students, we shall be comparing the Year 2 results with the traditional
skills test results. This will enable us to determine the strengths and weaknesses of such tests to
measure the impact of ICT haptic environments on students’ learning.
In addition to the results described above, the following data is now being collected and compiled:
• A taxonomy of the traditional assessment methods and techniques used in the undergraduate
dental curriculum;
• A record of the formative assessment techniques used by tutors in assessing the clinical skills
learnt by the students;
Frame
The analytical framework is based on creating data sets of the pre-test results for each of the 70
students to compare with their clinical skills traditional assessment results. Correlation statistics will be
used to identify and examine the interrelationships between the students’ abilities as revealed by the
spatial reasoning pre-tests and any changes as assessed through post clinical skills tests.
Research findings
The analysis of the pre and post-experience test results will show whether standard psychological
tests can be used to predict subsequent clinical skills acquisition or whether such skills can be taught
through traditional laboratory work. If the results show a positive correlation with the skills acquired by
the students through their clinical laboratory work over the last 6 months, then this will verify that such
spatial reasoning tests can be used to predict clinical skills acquisition. The results will provide
evidence about the strengths and weaknesses of such measures for use with other TEL research
studies.
Acknowledgements: Administrator: Clarabella Gray; Strand 1 team: Harwin, Barrow, Tse, Elson;
additional Strands 2 and 3 team members: Hindmarsh, Green, Reynolds, Wilson, Hyland, Robinson,
Banerjee, Harper, Banerji. See http://www.haptel.kcl.ac.uk. for further information.
References
Black P, & William D. (1998) Assessment and classroom learning. Assessment in Education; 5: 7-71.
Cox, M.J., Schleyer, T., Johnson, L.A., Eaton, K. A, Reynolds, P.A. (2008) Making a Mark-Taking
Assessment to Technology. British Dental Journal. Volume 205 No. 1. July 12. 33-39
McGill, T.J., & Hobbs, V.J (2007) How students and instructors using a virtual learning environment
perceive the fit between technology and the task. Journal of Computer Assisted Learning 24:3. 191-
202
Webb, M. & Cox, M.J. (2007) ICT Inside the Black Box"; May 2007; ISBN 978 0 7087 1763 9; 24pp
Granada Learning. London.
AB - 0723
The effectiveness of different assessment techniques being used to measure the impact of haptic technologies on Dental undergraduate students’ 3-D perceptions, motor and clinical skills.
Margaret Cox, Jonathan P. San Diego, Tim Newton, Mark Woolford, Barry Quinn, Stephen Dunne, Brian Millar, Arash Shariari-Rad
The Dental Institute, King's College London, London, United Kingdom
Background
The growth of IT in higher education (HE) has led to an expanse of technology enhanced learning
(TEL) courses at university level, many of these using IT-based resources such as on-line Virtual
Learning Environments (VLEs), computer simulations and video-conferencing (McGill and Hobbs,
2007). There is also a widening use of IT to aid in the assessment of students’ learning in higher
education although much of this is based on online multiple choice questions and focuses on
summative assessment at the end of a particular course (Cox et al., 2008). However, although there
is significant research into the benefits of formative assessment (Webb and Cox, 2007) much of this is
with children in schools and there is relatively little evidence of the strengths and weaknesses of
different formative and summative methods used in higher education. Formative assessment is
essentially regarded as a qualitative exercise providing a feedback process to the learner, engaging
the learner and teacher in acquiring a common understanding of what has been understood and
helping the teacher to modify their teaching to take account of what has been learnt (Black and
William, 1998). Using traditional methods, formative assessment in HE includes providing feedback to
individual students during a clinical session amongst the students themselves and between tutor and
student, challenging students with questions. The assessment research described in this paper is part
of the hapTEl project (http://www.haptel.kcl.ac.uk), funded by the ESRC/EPSRC’s Teaching and
Learning Research Programme’s Technology Enhanced Learning Phase (see http://www.tlrp.org/tel/).
When students are using haptic work-stations then they will also receive feedback from the system
itself, not only through the tactile experience but through the on-screen images which show them how
well they have achieved the clinical task. There is therefore a need to research assessment methods
to determine how effective different assessment techniques are and how these compare with the
traditional formative and summative assessments used in most Dental Undergraduate courses.
Research Questions
The hapTEL project has a range of research goals which relate to the technical, curriculum and
evaluation strands, all of which contribute to the work of the project, however the specific focus of the
TEL assessment work reported in this paper is to address the following research questions:
What are the strengths and weaknesses of psychological spatial reasoning and manipulation skills
tests being used to measure the impact of TEL on students’ skills and performance?
How do such tests compare with the traditional formative and summative assessments of students’
clinical skills as applied to undergraduate dental students?
Methods
The focused study described in this paper involves 70 Year 2 (total =140) university undergraduate
dental students who were tested for a range of 3-D perceptions and spatial reasoning skills at the end
of their first university year (June 2009). These involved a series of spatial reasoning, attitude and
gross and fine motor skills tests recommended by the British Psychological Society. This cohort of
students did not subsequently use the project’s haptics technologies but are being assessed (March,
2010) for their clinical skills following two terms of teaching in the clinical skills laboratory. In order to
assess the strengths and weaknesses of these pre-tests which are also being used to assess the
impact of TEL on the Year 1 students, we shall be comparing the Year 2 results with the traditional
skills test results. This will enable us to determine the strengths and weaknesses of such tests to
measure the impact of ICT haptic environments on students’ learning.
In addition to the results described above, the following data is now being collected and compiled:
• A taxonomy of the traditional assessment methods and techniques used in the undergraduate
dental curriculum;
• A record of the formative assessment techniques used by tutors in assessing the clinical skills
learnt by the students;
Frame
The analytical framework is based on creating data sets of the pre-test results for each of the 70
students to compare with their clinical skills traditional assessment results. Correlation statistics will be
used to identify and examine the interrelationships between the students’ abilities as revealed by the
spatial reasoning pre-tests and any changes as assessed through post clinical skills tests.
Research findings
The analysis of the pre and post-experience test results will show whether standard psychological
tests can be used to predict subsequent clinical skills acquisition or whether such skills can be taught
through traditional laboratory work. If the results show a positive correlation with the skills acquired by
the students through their clinical laboratory work over the last 6 months, then this will verify that such
spatial reasoning tests can be used to predict clinical skills acquisition. The results will provide
evidence about the strengths and weaknesses of such measures for use with other TEL research
studies.
Acknowledgements: Administrator: Clarabella Gray; Strand 1 team: Harwin, Barrow, Tse, Elson;
additional Strands 2 and 3 team members: Hindmarsh, Green, Reynolds, Wilson, Hyland, Robinson,
Banerjee, Harper, Banerji. See http://www.haptel.kcl.ac.uk. for further information.
References
Black P, & William D. (1998) Assessment and classroom learning. Assessment in Education; 5: 7-71.
Cox, M.J., Schleyer, T., Johnson, L.A., Eaton, K. A, Reynolds, P.A. (2008) Making a Mark-Taking
Assessment to Technology. British Dental Journal. Volume 205 No. 1. July 12. 33-39
McGill, T.J., & Hobbs, V.J (2007) How students and instructors using a virtual learning environment
perceive the fit between technology and the task. Journal of Computer Assisted Learning 24:3. 191-
202
Webb, M. & Cox, M.J. (2007) ICT Inside the Black Box"; May 2007; ISBN 978 0 7087 1763 9; 24pp
Granada Learning. London.
UR - https://www.researchgate.net/publication/268447105_The_Effectiveness_of_Different_Assessment_Techniques_being_Used_to_Measure_the_Impact_of_Haptic_Technologies_on_Dental_Undergraduate_Students%27_3-D_Perceptions_Motor_and_Clinical_Skills
M3 - Conference paper
BT - British Educational Research Association, At Warwick University, U.K.
ER -