Ten ways to make online exams work in STEM
Online exams allow students to search the internet for answers and even collude – but with creative design and some key considerations, they can still be a valuable tool for assessing understanding, write David Smith and David Coates
September 5th 2022
The assessment process is central to all degree programs, and during the COVID-19 pandemic, online examinations replaced many of the traditional invigilated examinations as the primary means of assessment. The approach has been welcomed by many institutions, even after the end of restrictions, thanks to perceived benefits around cost, flexibility and student preference.
The very obvious problem with these unsupervised exams is students’ free access to books and the internet, which means it’s not the student’s knowledge that’s being tested, but their ability to search for an answer. While this is a useful skill, there is still a need to demonstrate knowledge and understanding that does not depend on looking up fundamental principles and applications. Online assessments are also often unsupervised and collusion is a major issue.
A careful consideration of this very different assessment environment is required if online exams are to be used as a replacement for the traditional written and invigilated (or proctored) examination. The RSB Accreditation Committee recently commissioned work to evaluate good practice and support academics in creating assessments that challenge students to apply their knowledge and think critically.
The key message from the report is simple: ‘Authentic’ assessments require students to apply what they have learned to a new situation and determine what information and skills are relevant and how they should be used (See 'Internet proofing questions', below). How students perceive online exams has also been well studied and valuable lessons learned around the student experience of this form of assessment. The guide below aims to support those responsible for assessment in the design and delivery of inclusive online exams through evidence-informed design principles and considerations. 1 Question design: A variety of approaches can be used to create questions where answer can’t simply be found through a search on the internet. See box, below, for examples.
2 Clarity: Keep it simple, avoid multiple sections and large amounts of choice. Use clear and concise language.
3 Time and Length: Reading and typing online are slower than writing by hand, so ensure the time needed to complete the exam is achievable. Give indicative word counts.
4 Accessible: Be considerate of accessibility. Not all students have a stable internet connection, nor the ability to scan or print. In some cases, students may not have access to a suitable work environment and alternative provisions should be considered.
5 Inclusive: Removing barriers that prevent interaction with, or access to, websites, digital tools and technologies by people with disabilities.
6 Achievable: Weaker students will struggle with higher-level cognitive skills, so a balance is required in the question or the paper.
7 Expectation: Ensure students understand what is expected of them in the examination. The assessment should be a test of their knowledge and understanding, not a quiz on how to do an online exam, so formative (practice) papers are essential training.
8 Cognitive Level: Assume that students have full access to the internet and design your questions accordingly. Test the breath of cognitive ability across the module or course programme.
9 Preparations: High-level cognitive skills need formative assessment support and should be incorporated into module design. Students should be allowed to experience the technology platform used for the assessment before the test. Ethical behaviour in online examinations should be discussed.
10 Integrity: The exact assessment rules will depend on the conditions of the assessment, and behaviour in online examinations should be discussed. In all cases it should be made clear to the students what the expectations are around internet use and collaborative working to ensure students are aware of what is allowed.
Internet-proofing questions In the creation of the RSB report on online assessment, members of the HUBS Bioscience Educators Network provided case study questions to gain a multi-faceted understanding of the approaches taken when writing exam questions. The predominant solution was to employ an authentic assessment design in which creative and realistic learning experiences are used to test skills and knowledge. In each case, the solutions could not be directly looked up and required some form of processing. Approaches include: • Understanding questions are commonly used at first year of study. Students must draw on knowledge and use information, rules, and procedures in concrete situations. (See example below.) “The peptide below has deliberate errors added in. Use your knowledge of amino acids and peptides to identify the errors.”
• Case studies drawn from real-world situations are presented to the students, and questions are centred on a deeper explanation of the situation, appraising the information or choosing what course of action could be taken.
• Production-based questions, where knowledge or understanding is used to create a new or reworked artefact, for example, a figure, abstract or lay description. e.g “Produce a figure and use it to illustrate the signalling pathway that allows light to be perceived by the vertebrate eye.”
• Experimental design approaches measure critical or creative thinking. Students are asked to synthesise information into an explanation justifying a decision or course of action.
