Alan Dix > 2008

Identifying students’ mathematical skills from a multiple-choice diagnostic test using an iterative technique to minimise false positives

Steve Manning 1 and Alan Dix2

1. Computing and Electronic Technology Department, University of Bolton, Bolton, UK
2. Computing Department, Infolab21, Lancaster University, Lancaster, UK

< Steve on the Web > < Alan on the Web >

To appear in Computers and Education see pre-publication online version at ScienceDirect

Abstract

There is anecdotal evidence that a significant number of students studying computing related courses at degree level have difficulty with sub-GCE mathematics.  Testing of students’ skills is often performed using diagnostic tests and a number of computer-based diagnostic tests exist, which work, essentially, by testing one specific diagnostic skill at a time.

This paper proposes using a multiple-choice computer-based diagnostic test where each question has a number of diagnostic skills associated with it in order to allow more flexible questions.  A simple measure of a diagnostic skill’s competency could be obtained by calculating the number of questions answered correctly, divided by the total number of questions, associated with that skill.  However, because a question may have many skills associated with it, if a question is wrong, then each skill is deemed to be problematic, even though there may not be a problem with all skills. 

A technique has been developed that refines the initial skill competencies and iteratively re-calculates the skills based on all other competencies.  Pilots of the new diagnostic test with first year computing students indicate that particular mathematical problems exist for many students, and suggest that the iterative algorithm produces a more precise indication of competencies than a simple competence measure approach.

Keywords: intelligent tutoring systems, post-secondary education, interactive learning environments, applications in subject areas

References

  1. Appleby, J. (1999) Introduction to DIAGNOSYS. Available from http://www.staff.ncl.ac.uk/john.appleby/diagpage/descrip.htm  [accessed May 2004]

  2. Appleby, J., Samuels, P., Treasure-Jones, T. (1997) ‘Diagnosys - A knowledge-based diagnostic test of basic mathematical skills’ Computers & Education Vol. 28 No. 2 pp 113-131

  3. Artigue, M., Assude, T., Grugeon, G. and Lenfant, A. (2001), ‘Teaching and learning algebra: approaching complexity trough complementary perspectives’, in Chick, H., Stacey, K., Vincent, J. and Vincent, J. (eds.), The Future of the Teaching and Learning of Algebra, Proceedings of 12 the ICMI Study Conference, The University of Melbourne, Australia, December 9-14, 2001.

  4. Bland, M. (2000) An introduction to medical statistics 3rd ed. UK: Oxford University Press

  5. Bostock, L., Smith, E., Chandler, F. S., Shepherd, A. (1996) GCSE Intermediate Mathematics: A Full Course Nelson Thornes

  6. Child, D. (1990) Psychology and the Teacher 4th ed.  Kent: Cassell

  7. Curzon, L. B. (1997) Teaching in Further Education: An outline of principles and practice. 5th ed. London: Cassell

  8. Delozanne, E., Prévit, D., Grugeon, B., Jacoboni, P.  (2003) Supporting teachers when diagnosing their students in algebra, in É. Delozanne, K. Stacey (eds.), Workshop Advanced Technologies for Mathematics Education, Proceedings of Artificial Intelligence in Education, Sydney, July 2003, IOS Press, Amsterdam, 461-470.

  9. Devlin, K. (2001) ‘The real reason why software engineers need math.’ Communications of the ACM Vol. 44, No. 10

  10. Dix, A., Finlay, J., Abowd, G., Beale, R. (2003) Human-Computer Interaction 3rd ed.  Prentice Hall

  11. Galloway, C. A. (2001) Vygotsky’s learning theory, in M. Orey (ed.), Emerging perspectives on learning, teaching and technology  http://www.coe.uga.edu/epltt [accessed January 2004]

  12. Hair, J.F., Anderson, R.E., Tatham, R.L. (1998) Multivariate Data Analysis 5th ed. New Jersey: Prentice Hall

  13. Harper, W. H. (1991) Statistics (The M&E Handbook Series) 6th ed. London: Prentice Hall

  14. Hawkins, J. (1993) Oxford mini-dictionary 3rd ed  UK: Oxford University Press

  15. Linn, S. (2004) ‘A new conceptual approach to teaching the interpretation of clinical tests’ Journal of Statistics Education Volume 12, Number 3

  16. Luger, G. F. (2005) Artificial Intelligence: Structures and strategies for complex problem solving 5th ed. Addison-Wesley: New York

  17. Minsky, M. L. (1961) Steps toward Artificial Intelligence.  Proceedings of the Institute of Radio Engineers, 49:8-30. Reprinted in E. A. Feigenbaum and J. Feldman (eds.), Computers and Thought, New York: McGraw-Hill, 1963

  18. Mitchell, T. M. (1997) Machine Learning  Singapore: McGraw-Hill International

  19. Russell, S. and Norvig, P. (2003) Artificial intelligence: a modern approach 2nd ed. US: Pearson Education Inc.

  20. Sutton, R. S., Barto, A. G. (1998) Reinforcement Learning: an introduction London: The MIT Press

  21. Tanner, M. A. (1996) Tools for statistical Inference: methods for the exploration of posterior distribution and likelihood functions 3rd ed.  Springer: New York

  22. Tariq, V.N., (2003) Diagnosis of Mathematical Skills among Bioscience Entrants.  In Diagnostic Testing for Mathematics, LTSN MathsTEAM

  23. Todd, K. J., (2001) Historical study of Correlation between A-Level Grades and Subsequent Performance.  In Diagnostic Testing for Mathematics, LTSN MathsTEAM
Full reference:

S. Manning and A. Dix (2008).
Identifying students’ mathematical skills from a multiple-choice diagnostic test using an iterative technique to minimise false positives. Computers & Education. 51 pp. 1154–1171
http://www.hcibook.com/alan/papers/
CE2008-Diagnostic/
online pre-publication: doi:10.1016/j.compedu.2007.10.010

 

http://www.hcibook.com/alan/papers/CE2008-Diagnostic/

Alan Dix 13/1/2008