How to make your assessment remote in 2020/21
Most education and assessment will still be online in the current academic year. This page contains advice on how to switch your assessment to remote in 2020/21. Follow the steps in the reading list below.
Table of contents and links to related pages
- Read the eight guidelines for remote assessment
- Follow the decision tree: How to switch to remote assessment?
Choose your remote assessment type
Educational how-to guides per assessment type:
- Written exams:
- Oral exams:
Choose your remote assessment tools
And find out how to use the selected tool:
- Choose your fraud prevention and detection measures
- Follow the procedure for switching your assessment to remote
0. Read the eight guidelines for remote assessment
The guiding principle for developing quality remote assessment is that during assessments, we should enable our students to demonstrate how well they master the learning objectives. This requires constructive alignment and fairness. The eight guidelines in the table below ensure that.
Table 1: Eight guidelines for developing good quality remote assessment
The underlying assessment quality requirements are validity and reliability
The assessment should cover the learning objectives, and nothing more.
- If one learning objective cannot be assessed properly in the remote situation, dispensation may be given if that objective is tested elsewhere in the programme. However, this has to be approved by the Board of Examiners.
Validity matches with the concept of constructive alignment (Biggs, 1996, see Figure 1). This implies that the assessment of your course should match a) the learning objectives and b) the learning activities during the course. As a result, students should be able to practice with what they will be assessed on.
The grade should reliably represent how well a student masters the learning objectives.
For a reliable grade instructions in assignments and questions should be clear for students (e.g. they should know the required level of detail of the answer in order to get a full score).
- Grading should have enough precision. This implies that large questions that are automatically graded as ‘correct’ (all points) or ‘incorrect’ (no points) should be prevented or mitigated, for example by having students explain their calculation method and granting partial points for partially correct answers.
- Assessors should receive clear grading instructions that should ensure objective assessment, minimize bias and align the grade with the learning objectives and assignment instructions.
- While assessing, grading, or determining fraud, the lecturer should strive to be unbiased. If possible, grading should be anonymous and oral authenticity checks should be based on an unbiased sample of students.
Take sufficient fraud prevention measures to reduce the risk, while minimizing the downsides of these measures (see the other requirements): our students should still be able to demonstrate how well they master the learning objectives, without being hindered by the fraud prevention measures.
Take away the need to commit fraud
Be available during the assessment to resolve possible unclarities in questions/instructions, and for technical issues. You could use email, telephone, Brightspace Discussions (requires refreshing the screen to be able to see changes, both for students and you), or conference / chat tools. Ask help from TA’s or colleagues if needed. This will reduce the need to commit fraud.
Students should be able to practice with the assessment setting, questions and tool(s), ample time before the assessment day. Do a practice exam 1-2 weeks before the real exam. Goal: to test the procedure as well as students’ devices and apps. Furthermore, it will familiarize students (and you) with the procedure, which will diminish stress and improve performance.
- Study delay should be prevented.
- Check for conflicts with deadlines and assessment slots in other courses (5).
- The assessment should be feasible for the lecturers, examiners and assessors (during assessment construction, during the actual assessment and during grading), (5).
This is based on the underlying assessment quality requirement of inclusiveness.
- Some students are entitled to extra time (10 extra minutes per hour), as is stated in the TER (article 25). You will need to adjust the settings of the assessment tool.
Click for step-by-step guides on how to organize extra time in Brightspace Quizzes and Brightspace Assignments, using ‘Special Access’. Detailed instructions for quizzes can be found here and for assignments here.
In case of a Möbius exam, Digital Exams will take care of the extra time settings. When starting the exam, students will see two options which they can select: the exam with the regular duration and one with extra time.
- Students with special needs should have enough time to request and discuss special accommodations. Adjustment can involve form, timing, aids permitted during testing, and location. In case your students are entitled to other types of adjustments, like audio support, it might be useful to create separate exams. Special needs provisions are given by the Board of Examiners. Students can apply for these provisions via the student counsellors. The Boards of Examiners (via the study counsellors) should inform you which provisions need to be available for which student. Lecturers themselves are not allowed to grant these provisions.
- Use the approved tools from the TU Delft tool list that are compliant with privacy rules and regulations. In case this is not possible, students have the possibility to refuse to use the tool, and other measures should be taken by the lecturer. However, this implies extra work for the lecturer, so this should be prevented.
- In case recordings are made, this should be considered privacy sensitive data, and the recordings should be kept in a safe place and deleted as soon as their existence is not necessary anymore. Click here for guidelines for security and privacy of (recordings of) oral exams.
- Students need to be informed as soon as possible of any changes in assessment and procedure compared to the study guide description.
- Communicate details of the exam procedure. That way, they know what to expect and can focus on the content.
- It should be clear on what criteria students will be assessed in assignments, and what level of detail is expected in exams and assignments.
- The grading of the assessment and weighing of assessment parts (or criteria) should be clear before, during, and after the assessment.
- One way of creating clear criteria for assignments and projects is a rubric. For more information on rubrics, go to: Rubrics to Grade Assignments
1. How to switch to remote assessment?
Use the following scheme to determine how large the changes are that you need to change. Click on the image below for a pdf with links. It answers the following questions:
- Do I need a (partially) closed-book assessment?
- If so, how do I set up a closed-book assessment?
- In general, do I need to change the set-up of my assessment (or can I use the remote version of the on-campus assessment)?
In case your conclusion is that you need to make large changes that need approval, read the section below.
2. Choose your remote assessment types
Here is a rough schedule that can help you to determine your remote assessment type. The general advice is to keep the changes small, by preferably switching to the remote version of the original assessment.
Figure 2: Frequently used changes of assessment type. For *written exams, there are multiple possibilities, see running text.
The schedule above demonstrates frequently chosen changes from on-campus assessment type to remote assessment type. A combination of remote assessment types is also possible. In that case, strive for a spread of deadlines and exams for your course and the other courses that students take.
Remote written exams can be administered in two ways:
- as downloadable assignments: During the exam part’s time slot, students download the exam and upload their answers before the deadline, for example 90 minutes later. Exams can be split into 2-4 parts with each their own time slot. The answers can, for example, be typed word-documents or pdfs, or scanned handwritten answers in pdf-format. Uploading and converting documents can take up to 10 minutes per test. Uploading videos is also possible (both for the student and the lecturer).
For a how-to in Brightspace Assignments, see here.
- as a (partly) automatically graded digital exam: you and your team can either manually grade longer questions, which allows for partial grading (giving a student for example 2 out of 5 points for partially correct answers), or use automatically graded questions with short answers. In the latter case, you would have to review the incorrect answers for unforeseen answers which are nevertheless correct.
For a how-to in Möbius, see here.
For a how-to in Brightspace Quizzes, see here.
ANS Delft or Grasple is available in some faculties. For a comparison between the tools mentioned above, please click here.
Larger projects and assignments can have multiple deliverables and assessment items, like reports, presentations (either live in YouSeeU or MS Teams session, or by a submitting a recording in Brightspace Assignments), code, or observed skills. Handing in content in Brightspace Assignments with plagiarism check is frequently part of the solution, as well as a (group) presentation via a conference tool. Make sure that you provide students with sufficient supervision and support.
Depending on your selected remote assessment type (see above), you can choose an assessment tool and fraud prevention measures. Open the next chapter to read more on the different assessment tools.
3. Choose your remote assessment tools
The available tools can be divided into two solutions:
- Downloading the exam and having students upload the results (in Brightspace Assignments).
- Using an assessment tool in which students fill in the answers per question, and which are (potentially) partially automatically graded (other tools).
The latter option gives more possibilities for automatic grading and giving students unique exams. The first is requires less effort from your side, because you can use a similar workflow as for an on-campus exam.
An extensive comparison table of the available tools can be found here.
For oral exams, you can use the following tools:
- YouSeeU: integrated in all Brightspace courses. Waiting room and break-out rooms are available to have TAs welcome students and help them test and setup their connection (handy in case of short exams or oral checks and large student numbers).
- Skype4Business: no recording available on Macs, no waiting room. Students cannot login.
- MS Teams: Break-out rooms have become available. Students login using their netID.
- Brightspace assignments: students upload their assignment or report individually or as a group (Brightspace allows for group assignments), before the deadline. The uploaded work is typically a report (pdf or .docx), code, drawing or a recording of a presentation (movie). You can find more information on how to create good assignments and how to create rubrics.
- Brightspace assignments can also be used formatively, to give students larger assignments or projects and have them upload intermediate products for you to provide feedback.
- For peer feedback on intermediate products, use FeedbackFruits Peer review within Brightspace. Contact Brightspace Support to enable this in your Brightspace course.
5. Follow the procedure for changing your assessment to remote in 2020-2021
Depending on the faculty, lecturers adapt the assessment description to the remote situation in the study guide either:
- Before the start of the academic year: Lecturers described both on-campus and remote assessment in the study guide (which is an elaboration of the Teaching and Education Regulations, TER) before the start of the academic year.
- Shortly before the start of the semester / quartile: In some faculties, lecturers can propose changes to the study guide shortly before the start of the semester of quartile, to adjust the assessment (and education method) to the current COVID-19 measures. The proposed changes in the study guide may need to be approved by the dean, after consent of the Faculty Student Council and advice of the Board of Studies.
However, if you need to change your assessment later and need to deviate from the study guide text due to unforeseen circumstances (for example an unforeseen extremely low/high number of students, or a sudden change in COVID-19 measures), you need to ask permission from your Board of Examiners, depending on their rules and guidelines.
Contact your programme coordinator or programme director for information about the procedures in your faculty.
Request permission for online proctoring
Online proctoring during digital exams as anti-fraud measure is only allowed as a last resort. Be aware that you need to ask permission to use online proctoring as an anti-fraud measure, usually from the Board of Examiners and sometimes from the faculty. Contact your programme coordinator or programme director for information about the procedures in your faculty.
The main criteria that are used to assess your assessment plans for the course are listed in the accordions below:
In case of suspicion of large-scale fraud, the Board of Examiners can declare all announced grades invalid, if it is impossible to determine who committed fraud and who did not. Any kind of participation in large-scale fraud is considered a case of serious fraud. In case of serious fraud, ‘the Board of Examiners is entitled to propose to the Executive Board that the student’s enrolment on the degree programme be permanently terminated’. Click here for more information on fraud and consequences.
In case the reliability of the grade is not sufficient to express the results as a grade (1-10), the Board of Examiners can allow the examiner to change the grade to a pass/fail decision (fail-pass). Variations on pass/fail are not allowed.
Pass/fails do not count for the students’ average grade (GPA).
If a student passes a course with a 6.0, and wishes to retake the exam with a pass/fail exam, the grade will not be replaced by a ‘pass’, because the (new) Rules & Guidelines of the Boards of Examiners state that when having received both a grade and a pass/fail after a retake, any pass grade is considered to be higher than a ‘pass’ and will be kept
Students with special needs can be entitled to extra time (10 extra minutes per hour), as is stated in the TER (article 25). Other adjustments can involve form, aids permitted during testing and location.
There are two practical ways to organize extra time:
- Give ‘Special Access’ to groups of students with disabilities. Set their end-time of the test 10 minutes later per available 60 minutes. Detailed instructions for Brightspace Assignments can be found here and for Brightspace Quizzes here.
- You could also create two identical exams that differ only in duration, split the students into a normal-time group and an extra-time group, and make each quiz available to either of the groups. Make sure they can only access the applicable relevant exam.
- Incorporated the tips from Q3 lecturers in this page, fraud prevention page and the step-by-step guides on how to set up examinations in Brightspace Assignments and Brightspace Quizzes.
- Merged guidelines from Boards of Examiners into general assessment guidelines. Added the guiding principle that in assessments, students should be able to demonstrate how well they master the course to the guidelines for assessment. Focused the guidelines on this guiding principle. Improved the key elements for good quality remote assessment.
- Changed order of page (start with the guidelines)
- Improved overview on content and subpages at start of page.
- Adapted the procedure to new Rules and Guidelines and Teaching and Education Regulations 20-21.
The following guidelines were added and changes made:
- The subsite How to create fraud resistant exams was improved. Overlap with this page was reduced and overlapping parts were moved to the fraud resistant exam page.
- Added links to the Online Proctored Examinations Regulation. Updated content on online proctoring here and at the fraud resistant exam page to match the content of the exam.
- Updated links, changed minor parts of the page, mostly textual.
- Added a link to a recording of an introductory webinar on MapleTA by Kasja de Jong.
- Added two step-by-step manuals (under construction) on how to create exams in Brightspace Quizzes and Brightspace Assignments.
- Guidelines on the following topics were processed on this page:
- Do’s, don’ts and tips based on experience of teachers with remote assessment in Q3
- Postponement of assessment is no longer an option: deleted text that discussed the options to postpone assessments.
Guidelines on the following topics were processed on this page:
- Update Taskforce Assessment (6 April 2020) on studying with a disability, security and privacy guidelines for oral exams, and complementary oral checks.
- Advice on ‘poor man’s proctoring’ (30 March 2020)
The following manuals were added:
The previous guidelines on the Brightspace support website are quite strict, because Boards of Examiners were initially quite reluctant for remote assessment. Given the current situation, they are more open to remote assessment, if the quality of assessment is assured. Part of the quality of assessment is to minimize the risk of fraud. Therefore, we have adapted the homepage to reflect more lenient guidelines for remote assessment.
The first version of this page was published on 24 March 2020