May 25, 2026 by Doug McCurry from BooBook Education

Workload, Quality information and Computer scoring of MCQ and SAQ

I have long been opposed to the old forms of computer scoring of writing, but the advent of AI in the form of Large Language Models means that the usefulness of computer scoring of writing is now, for me at least, an open question. But I will leave that issue for another blog.

The computer based scoring of multiple choice questions (MCQ) is readily available and easily used by teachers at no cost. As I have argued in an earlier blog, Google Forms can be very simple to use for MCQ testing and it gives automatic scoring, reporting and data analysis of student performance.

https://draft.blogger.com/blog/post/edit/787694387229307544/6972716941595211449?hl=en-GB

It is currently claimed by researchers (and vendors of course) that AI scoring of short answer questions (SAQ) can also now be done very reliably with AI.

But why would you bother?

The amount of time spent in correction is a serious issue for most teachers, and for English teachers in particular. One of the reasons assessment is such a big challenge for English teachers is that they usually and sometimes exclusively use extended writing for assessment tasks.

There are a range of good reasons why extended writing is the most common assessment method used by English teachers, but there are good reasons for English teachers making some use of SAQ and MCQ for formative and summative assessments.

Reading skills, in particular, are amenable to the closed and semi-close methods of MCQ and SAQ testing. There are also other kinds of learning and assessment tasks for VCE English that can be undertaken in MCQ and SAQ.

I argued in a blog called Learning from testing: testing for learning that closed and semi-closed assessment tasks can be structured learning activities of an explicit kind. A good MCQ or SAQ question that can be scored unambiguously or a good SAQ with a more or less specific answer gives an opportunity for deconstructing an aspect of senior English skills for teachers and learners. But I will come back to this substantive reason for English teachers exploring MCQ and SAQ in another blog.

Some ways you might use MCQ and SAQ in VCE English

With a collection of 20 essay topics for a Section A text, you could write one of more MCQ about each essay topic that would show the student and you if they understand and can appropriately interpret an essay question. Such an exercise would give you a direct insight into the extent to which students understood set essay topics. A student does not have to write an essay (and you don’t have to read what they have written) for you to get information about their understanding of what essay topics require. The discussion of such test questions is a focussed way of exploring with students what is involved in understanding and interpreting essay topics. Similarly writing essay plans rather than essays (with a limit of 50 words?) is a good and cost-effective way of working on essay writing skills.

For Section B you could ask students closed or semi-closed questions about what would be appropriate sub-themes of a Framework. You could ask about the relationship between a set title and stimulus and the sub-themes of a Framework, and you could ask about the appropriate links of a set stimulus to a sub-theme of the Framework.

For Section C you could ask MCQ about argument and language concepts and metalanguage relevant to the exam analysis task, such as the following.

• Why are some questions called ‘rhetorical’ and others are not?
• Which of the questions in the text is a ‘rhetorical question’?
• How is the question at the end of paragraph two used in the text?

You could also use MCQ to test comprehension and interpretation of Section C type texts, and such questions can be focussed at a very specific level. There are many ways in which closed and semi-closed questions can be used to teach, learn and assess the skills of English as a study, and such questions would deal with aspects of the skills that underpin senior English.

The great advantage of MCQ is that they can be automatically scored and that the scores can be transferred to and analysed in a database. Any test using a number of questions is in a sense standardised and offers a simplified basis for comparisons of student performance. These activities (and the data they produce) can be conveniently used to make more or less objective (and convenient) comparisons of different groups (Was there a difference in the scores for the different classes?) in different years (Were the scores for this year better than for the group last year?).

It is not uncommon for MCQ testing to be disdained, and there are certainly a lot of weak MCQ tests around, but standardised testing using MCQ is a powerful if old technology. It is not unreasonable to see SAQ testing as more valid than MCQ because students need to produce rather than identify a good or correct answer. Other things being equal, this is a reasonable comment on the comparative validity of most MCQ and SAQ, but it is not always the case that SAQ are more valid than MCQ. There are weak SAQ that are worse (because unfocussed or vague) than good MCQ, and really good MCQ are more sharply focussed and subtle than can be paralleled in good SAQ. And it is useful to know (and sometime surprising) which students can deal with really subtle MCQ questions.

I am regularly reminded how much English teachers are concerned about workload and particularly the amount of time spent in reading and responding to student work. I would like to help teachers use technology based assessments of student learning to relieve some of the pressure of current formative assessments and to add some different information to their overall assessment of their students’ learning.

I am interested in encouraging English teachers to use MCQ and SAQ assessment methods if they can be validly used to ease the assessment burden on English teachers to some extent. More use could be made of SAQ testing if, as with MCQ, there are a valid and convenient ways of computer scoring of student responses.

These considerations have led me to look for a way of cost-effectively using computer based scoring of SAQ on topics of interest to English teachers. After a number of disappointments and dead ends, I have found that I can easily and cost-effectively use Google Forms to score MCQ (and record SAQ answers) and use ChatGPT to score SAQ in two simple stages. (See the question to do so with ChatGPT I am using below.)

I am currently trying to gather some student performance data to assess with the teachers of those students the validity and reliability of ChatGPT scoring of SAQ. The methods of computer-based assessment I am exploring would be useful, of course, to English teachers at all year levels. I am currently using the following question to explore with ChatGPT analysis of the output of Google Quiz items.

A question to ChatGPT for the scoring MCQ and SAQ output from a Google quiz

Dear ChatGPT.

The attached spreadsheet contains in the first sheet the name of a student and the responses to 12 multiple choice questions (columns D to M) and 10 short answer questions (columns P to Y).

The first set of responses (row 2) is the correct multiple choice answers and the model short answer responses.

Score each set of responses and produce a table with multiple choice total and short answer total for each set of responses.

Score 1 point for each multiple choice question and score each short answers with partial credit on a scale of 0 to 3 on the following criteria.

Criteria for short answer question assessment

• the understanding of the question as shown by the relevance of the response
• the substance and accuracy of the response to the question and the text

Note that students may uses words other than those in the model SAQ answer and the scoring task is to be judged on whether the student’s response is equivalent to or similar in meaning to the model answer. The correctness or completeness of the writing is not to be assessed as such.