They are just 12-15 minutes long, but oral exams are crucial in determining a Leaving Cert student’s performance in a language subject.
Oral components account for between 25 and 40 per cent of overall marks for the final exam, depending on the language.
In normal years, an external examiner visits a school to carry out oral interviews with candidates and marks their performance.
This year was different. Due to Covid-19 restrictions, teachers were appointed by schools to carry out the interviews, which were recorded and sent to the State Examinations Commission (SEC). These recordings were later marked by independent examiners appointed by the SEC.
Given the fact that the format of this year’s exams – a choice of exams or accredited grades – was only agreed in mid-February, it was a race against time to organise the orals which began on March 26th.
The SEC says it provided comprehensive guidance to school managers on the arrangements that would apply to these oral tests, followed by instructions for oral interviewers on the detailed format.
Several language teachers who spoke to The Irish Times, however, say this subject-specific guideline arrived just three days before the oral exams got underway.
In addition, there was no training provided for teachers conducting the interviews who – in many cases – had no experience of examining an oral test for State exams.
The instructions for oral language interviewers – issued on March 23rd – make it clear that close adherence to the guidelines is compulsory to ensure fair and accurate results for the candidates.
“You are required to give each candidate an equal opportunity to demonstrate their proficiency in oral communications. This will be achieved by careful and consistent adherence to these instructions.”
However, when examiners began to mark recordings of these interviews, some found that teachers, in many cases, had not correctly followed instructions on how to conduct oral exams or ask enough mandatory questions. In some cases, key sections of the oral interviews were omitted entirely.
As a result, they said they had no choice but to mark students down, resulting in significant numbers of candidates losing marks.
The extent to which different subjects may be affected by these omissions varies, say teachers and examiners, because the structure of oral exams varies by subject.
French - with a heavy emphasis on general conversation - may be less affected, say some, while languages such as as German - where the oral is more structured and has more elements - is more likely to be affected.
The German oral exam, for example, consists of three parts: general conversation; project or picture sequence; and role plays.
The detailed marking scheme given to examiners divides these components into sub-sections which require a minimum number of questions.
‘Consistently neglected’
The role-play section alone, for example, requires the interviewer to ensure the candidate is given an opportunity to attempt five tasks.
One examiner – speaking on condition of anonymity – has filed a protected disclosure to the SEC due to the extent of their concerns.
The examiner marked a total of 78 orals and said a significant number of these were not conducted in accordance with the instructions for oral language interviewers.
In many, the examiner said, the interviewer “consistently neglected” to adequately address the key sections, ask enough required questions or follow certain instructions.
“This neglect on behalf of the interviewer meant that many candidates lost marks through no fault of their own,” the examiner said.
This examiner said a majority of 64 candidates who were interviewed by the same teacher lost marks because key questions were not asked.
In some cases, the examiner said, strong candidates were penalised in a way that did “not adequately reflect the candidate’s level of achievement”.
In another example, the interviewer - or teacher - failed to follow the guidelines to ask recommended questions during a role play section.
“The audio recording shows that the candidate clearly struggled due to this failure . . . In my opinion, the candidate was clearly capable of full marks in this section if the correct questions had been asked.”
When the examiner raised the problem with their advising examiner, they were told they could only mark the content heard on the audio recording.
“I was given no assurance that the candidates would not lose out in cases where the interviewers failed to correctly carry out the interviews according to the very clear guidelines,” the examiner said.
The examiner said they were told at a marking conference in May that interviewers - or teachers - could not be judged as they had “facilitated the process with no training” and the advice they received was to “completely ignore the interviewers”.
One examiner said they and others received an email from an advising examiner instructing them not to identify any problems with the work of the interviewers in their official feedback reports.
‘Unprecedented year’
“As we all know, this has been an unprecedented year, so she has asked that we just write about the work of marking candidates and not mention anything about the interviewers at all!” the email from the advising examiner states.
“In other words, please don’t write about whether interviews were too long or too short; whether too many or too few questions were asked or whether role plays were well-examined or not. Just focus on the standard of the candidates, their good and weak points, etc . . . Also, please do not mention any schools by name or any candidates by name or number.”
As a result, the examiner said, students will not know whether concerns or shortcomings were evident in the way their orals were conducted.
Under the appeals process, a candidate may seek a copy of the oral interview. However, they will only be provided with a recording of their own voice and will not be provided with the questions that were asked in the interview.
The examiner who has flagged concerns in a protected disclosure says: “Such a direction contradicts the standards of openness and accountability which are supposed to be at the core of the State Examinations Commission.”
In response to questions, the SEC said it was reliant on locally appointed teachers and school management ensuring the tests were conducted in compliance with the instructions.
“Given that this process is ongoing and that candidates who sat these oral tests are now undertaking their final written examinations, the SEC believes it would be inappropriate and unfair on candidates to offer any further comment at this time other than to confirm its commitment to ensuring that candidates are treated fairly and equitably . . .”
The mission statement of the SEC is to provide a “high-quality, candidate-centred State examinations service” and that its process for dealing with errors states that it is its aspiration to “preside over a system that is completely error-free”.
However, the anonymous examiner says the shortcomings in the orals this year are set to remain hidden from view and that examining staff have, in effect, been told to “ignore errors, to the disadvantage of the student”.
“To fail to draw attention to core failings which clearly disadvantage examination candidates and threaten the standards of fairness, openness and accountability which are supposed to be central to the SEC would be an abdication of my duty,” the examiner said.
“I cannot remain silent about a process wherein examination candidates are disadvantaged for no fault of their own.”
