Redesigning Online Maths for Social Sciences Assessments in the GenAI Age
Abstract
In the fast-changing educational world, Generative AI (GenAI) has brought about big changes, especially in online formative and summative assessments. Universities are concerned about the unethical GenAI use, compromising academic integrity. This study proposes and evaluates strategies to design questions in Maths for Social Sciences education that are challenging for ChatGPT-3.5 to solve. Drawing on Bloom’s Taxonomy, a trend analysis of academic performances and focus group discussions, it proposes a transformed approach to assessment design: the SHARP (Strategic, Holistic, Adaptive, Reflective, Process) assessment cycle. This framework is iterative and integrates real-time feedback to ensure inclusivity and transparency, stemming from a Reflect-Rewrite-Retest-Review redesign approach, focusing on higher-order cognitive questions. A quantitative analysis between 2020 and 2024 reveals a significant increase in higher-order level questions (e.g. from 29% to 84% in a test) and a significant but not drastic drop in academic performance. The effectiveness of ChatGPT-challenging designs is corroborated by focus group discussions, highlighting the need for a balance between student accessibility and academic rigour. This study contributes to the literature by providing unique empirical evidence on the validity of the strategies and offering actionable steps for educators, policymakers and institutions to maintain academic integrity in Maths for Social Sciences education.
References
Anderson, L. W., & Krathwohl, D. R. (Eds.). (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. Longman.
Bain, J. (2010). Integrating student voice: Assessment for empowerment. Practitioner Research in Higher Education, 4(1), 14–29.
Bitzenbauer, P. (2023). ChatGPT in physics education: A pilot study on easy-to-implement activities. Contemporary Educational Technology, 15(3). https://doi.org/10.30935/cedtech/13176
Bloom, B. S., Engelhart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. R. (1956). Taxonomy of educational objectives: The classification of educational goals. Handbook I: Cognitive domain. Longmans, Green.
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa
Brown, G. T. L. (2022). The past, present and future of educational assessment: A transdisciplinary perspective. Frontiers in Education, 7. https://doi.org/10.3389/feduc.2022.1060633
Bryman, A. (2008). Of methods and methodology. Qualitative Research in Organizations and Management, 3(2), 159–168. https://doi.org/10.1108/17465640810900568
Clarke, O., Chan, W. Y. D., Bukuru, S., Logan, J., & Wong, R. (2023). Assessing knowledge of and attitudes towards plagiarism and ability to recognize plagiaristic writing among university students in Rwanda. Higher Education, 85(2), 247–263. https://doi.org/10.1007/S10734-022-00830-Y
Clements, D. H., & Battista, M. T. (1990). Constructivist learning and teaching. Arithmetic Teacher, 38(1), 34–35. https://doi.org/10.5951/AT.38.1.0034
Digital Education Council. (2024). Global AI student survey 2024. https://www.digitaleducationcouncil.com/post/digital-education-council-global-ai-student-survey-2024
Dunne, S., & Lee, D. (2022). Designing assessment for neurodiverse students. Liverpool John Moores University.
Eke, D. O. (2023). ChatGPT and the rise of generative AI: Threat to academic integrity? Journal of Responsible Technology, 13, 100060. https://doi.org/10.1016/j.jrt.2023.100060
Else, H. (2023). Abstracts written by ChatGPT fool scientists. Nature, 613(7944), 423. https://doi.org/10.1038/d41586-023-00056-7
Farrelly, T., & Baker, N. (2023). Generative artificial intelligence: Implications and considerations for higher education practice. Education Sciences, 13(11). https://doi.org/10.3390/educsci13111109
Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive-developmental inquiry. American Psychologist, 34, 906–911.
Gikandi, J. W., Morrow, D., & Davis, N. E. (2011). Online formative assessment in higher education: A review of the literature. Computers & Education, 57(4), 2333–2351. https://doi.org/10.1016/j.compedu.2011.06.004
Henderson, M., et al. (2022). Online exams: Exploring student experience and integrity behaviours as we return to campus. ASCILITE Publications. https://doi.org/10.14742/apubs.2022.95
Hersh, W., & Hollis, K. F. (2024). Results and implications for generative AI in a large introductory biomedical course. npj Digital Medicine, 7, 247. https://doi.org/10.1038/s41746-024-01251-0
Holden, O. L., Norris, M. E., & Kuhlmeier, V. A. (2021). Academic integrity in online assessment: A research review. Frontiers in Education, 6. https://doi.org/10.3389/feduc.2021.639814
Lancaster, T., & Cotarlan, C. (2021). Contract cheating by STEM students. International Journal for Educational Integrity, 17(1). https://doi.org/10.1007/s40979-021-00070-0
Liu, D., & Bridgeman, A. (2023). How can I update assessments to deal with ChatGPT? University of Sydney.
Lumivero. (2023). NVivo [Software].
Lye, C. Y., & Lim, L. (2024). Generative AI in tertiary education. Education Sciences, 14(6). https://doi.org/10.3390/educsci14060569
Nikolic, S., Daniel, S., Haque, M. E., Belikov, O., Rizwan, M., Glassey, J., Ryan, M., & Grundy, J. (2023). ChatGPT versus engineering education assessment. European Journal of Engineering Education, 48(4), 559–614. https://doi.org/10.1080/03043797.2023.2213169
Paul, R., & Elder, L. (2013). Critical thinking. Pearson.
Phillips, A. J., Briggs, J. C., & Jensen, J. L. (2019). Beyond Bloom’s taxonomy. Journal of Psychological Research, 1(1), 24–32. https://doi.org/10.30564/jpr.v1i01.421
Piaget, J. (1976). To understand is to invent. Penguin.
Rasul, T., et al. (2023). The role of ChatGPT in higher education. Journal of Applied Learning and Teaching, 6(1), 41–56. https://doi.org/10.37074/jalt.2023.6.1.29
Reardon, S. F., et al. (2018). Test item format and gender achievement gaps. Educational Researcher, 47(5), 284–294. https://doi.org/10.3102/0013189X18762105
Reedy, A., et al. (2021). Academic integrity in online exams. International Journal for Educational Integrity, 17(1), 9. https://doi.org/10.1007/s40979-021-00075-9
Roe, J., Perkins, M., & Ruelle, D. (2024). AI use in assessment. arXiv. https://doi.org/10.48550/arXiv.2406.15808
Salinas-Navarro, D. E., et al. (2024). Generative AI in experiential learning. Education Sciences, 14(1). https://doi.org/10.3390/educsci14010083
Sallam, M., et al. (2023). ChatGPT applications in health education. Narra J, 3(1). https://doi.org/10.52225/narra.v3i1.103
Seo, K., et al. (2021). AI and learner-instructor interaction. International Journal of Educational Technology in Higher Education, 18. https://doi.org/10.1186/s41239-021-00292-9
Silverman, D. (2016). Qualitative research. SAGE.
Soobedar de Villeneuve, Z. (2025). Online maths for social sciences assessment design. Advance HE.
St-Onge, C., et al. (2022). COVID-19 and e-assessment. British Journal of Educational Technology, 53(2), 349–366. https://doi.org/10.1111/bjet.13169
Stamov Roßnagel, C., Lo Baido, K., & Fitzallen, N. (2021). Constructive alignment. PLOS ONE, 16(8). https://doi.org/10.1371/journal.pone.0253949
Su, J., & Yang, W. (2023). ChatGPT framework in education. ECNU Review of Education, 6(3), 355–366. https://doi.org/10.1177/20965311231168423
Sweller, J., Ayres, P., & Kalyuga, S. (2011). Cognitive load theory. Springer.
Tan, T. F., et al. (2023). LLMs in ophthalmology. Ophthalmology Science, 3(4), 100394. https://doi.org/10.1016/j.xops.2023.100394
UCL Assessment Working Group. (2020). Designing effective online assessment. UCL.
Van Dis, E. A. M., et al. (2023). ChatGPT: Five priorities for research. Nature, 614(7947), 224–226. https://doi.org/10.1038/d41586-023-00288-7
Vellanki, S., Mond, S., & Khan, Z. (2023). Academic integrity in online assessment. TESL-EJ, 26(4). https://doi.org/10.55593/ej.26104a7
Vygotsky, L. S. (1980). Mind in society. Harvard University Press.
Weimer, M. (2018). Multiple-choice tests. Faculty Focus.
The Copyright Transfer Form to ASERS Publishing (The Publisher)
This form refers to the manuscript, which an author(s) was accepted for publication and was signed by all the authors.
The undersigned Author(s) of the above-mentioned Paper here transfer any and all copyright-rights in and to The Paper to The Publisher. The Author(s) warrants that The Paper is based on their original work and that the undersigned has the power and authority to make and execute this assignment. It is the author's responsibility to obtain written permission to quote material that has been previously published in any form. The Publisher recognizes the retained rights noted below and grants to the above authors and employers for whom the work performed royalty-free permission to reuse their materials below. Authors may reuse all or portions of the above Paper in other works, excepting the publication of the paper in the same form. Authors may reproduce or authorize others to reproduce the above Paper for the Author's personal use or for internal company use, provided that the source and The Publisher copyright notice are mentioned, that the copies are not used in any way that implies The Publisher endorsement of a product or service of an employer, and that the copies are not offered for sale as such. Authors are permitted to grant third party requests for reprinting, republishing or other types of reuse. The Authors may make limited distribution of all or portions of the above Paper prior to publication if they inform The Publisher of the nature and extent of such limited distribution prior there to. Authors retain all proprietary rights in any process, procedure, or article of manufacture described in The Paper. This agreement becomes null and void if and only if the above paper is not accepted and published by The Publisher, or is with drawn by the author(s) before acceptance by the Publisher.