A novel approach to mathematics examination design and marking

David Easdown (University of Sydney, Australia), Ruth Corran (American University of Paris, France), Brad Roberts (University of Sydney)

Presentation at the 11th Southern Hemisphere Conference on the Teaching and Learning of Undergraduate Mathematics and Statistics in Gramado, Brazil, November 2017, “Brazil Delta 2017’

The new approach to marking evolved over 3-4 years, prompted by policy changes regarding assessment.

Old method of assigning grades: norm-referenced: use professional judgement to decide what a passing grade was, map to the students in the class. Policy said a maximum of 5% can achieve the top bracket grade (over 85%).

Intermediate method: Instead of grades, assessment classifies students’ test performance as ‘masterful’, ‘excellent’, ‘credible’ or ‘routine.’ Less greyness around borderlines compared to previous norm-referenced scale.

Policy change meant the raw mark achieved on an assessment would be their score. Tests need to be developed so that a student who deserved to pass would score 50% of the marks on the test, but only those who deserved a distinction would achieve the marks.

The SOLO taxonomy (Biggs and Collis, 1982) was used to guide the design of the exam, 5 phases.

From http://www.johnbiggs.com.au/academic/solo-taxonomy/

Also the idea of liminal space, passing through the portal of a threshold concept (Meyer and  Land, https://www.jstor.org/stable/25068074?seq=1#page_scan_tab_contents), once you have passed the portal you are transformed.

AA and A – extended abstract phase

BB and B – relational etc.

Mark by reading an answer, draw a vertical line next to the section, mark AA if no areas of weakness, A if one weakness (mark with a cross on the vertical line), BB for 2 weaknesses, B for 3. Top down approach for grading work showing relational and extended abstract thinking and marking with vertical lines, not ticks.

For disjointed work where the logic of the answer has broken down, use a bottom up approach. Mark in line with a tick where there is a correct point. F if only one correct point, E if 2 correct points, D for 3, … maximum of CCC.

You have to change the nature of the exam questions. Fewer very tough questions when raw marks count as final as there is no scope for professional judgement to adjust marks.

** Danger when lecturers set very difficult questions and don’t have an idea of what students can do because they get marking relief and tutoring relief. Students change from year to year, interacting with students helps to understand their current level of ability.