A dispute between academics and judges over suggestions in a research paper that High Court judges use Wikipedia when preparing judgments has been reignited by the publication of an amended paper.
High Court judges, when responding to the original paper published last summer, insisted their judgments are based on legal submissions, not internet searches.
Following the release in recent weeks of an amended version of the paper, Mr Justice Richard Humphreys told The Irish Times that the “extensive” changes suggest the core conclusion in the original paper was “problematical”.
The changes to the original could be seen as “involving at least some degree of implicit acceptance of the central criticism of the original paper, which is that legal submissions, not judges’ alleged internet researches, primarily drive citations in judgments”, he said.
Christmas digestifs: buckle up for the strong stuff once dinner is done
Western indifference to Israel’s thirst for war defines a grotesque year of hypocrisy
Why do so many news sites look so boringly similar? Because they have to play by Google and Meta’s rules
Christmas dinner for under €35? We went shopping to see what the grocery shop really costs
The president of the High Court, Mr Justice David Barniville, said: “It is perhaps a little unfortunate that so much publicity was given to the claims in the original paper, and perhaps equal attention should be given to the fact of the changed wording.”
However, the lead author of the research, Brian Flanagan, Associate Professor at Maynooth University School of Law, said: “Our conclusion is unchanged: the best explanation of our results is that judges or their assistants are using Wikipedia.”
The research was carried out by academics at Maynooth University and two US third level institutions.
The researchers arranged for 150 new Wikipedia legal articles on Irish Supreme Court decisions to be written by law students. Half were randomly chosen to be uploaded where they could be used by legal personnel, including judges, and the rest were kept offline and treated as the control group.
Our conclusion is unchanged: the best explanation of our results is that judges or their assistants are using Wikipedia
The researchers looked at two measures – whether the cases on Wikipedia were more likely to be cited as precedents by subsequent judicial decisions, and whether the argumentation in court judgments echoed the linguistic content of the new Wikipedia pages.
Their key finding was that getting a Wikipedia article increased a case’s citations by more than 20 per cent. The increase was bigger for citations by the High Court and mostly absent for citations by the appellate courts.
The original paper, Trial by Internet: A Randomized Field Experiment on Wikipedia’s Influence on Judges’ Legal Reasoning, was criticised by several High Court judges, and by some of their judicial assistants, as not reflecting reality.
“Judgments are written based on the legal authorities and submissions put before the court, they are not based on a Google search or Wikipedia,” said one judge, speaking on condition of anonymity. “The last thing a judge will do is just go into Google to get the answer to the case.”
Asked by The Irish Times about the amended paper, Mr Justice Humphreys said: “The change in wording in the conclusion section from ‘judges do rely on Wikipedia’ to ‘legal professionals do rely on Wikipedia’ would give the impression to a reasonable reader that the central conclusion of the original paper was problematical.”
“The change could be seen as involving at least some degree of implicit acceptance of the central criticism of the original paper, which is that legal submissions, not judges’ alleged internet researches, primarily drive citations in judgments.
“This is particularly so where there are no claims that the authors had access to new empirical data regarding parties’ submissions which could account for this change.
The last thing a judge will do is just go into Google to get the answer to the case
“Without at all questioning the honesty of the authors, any claim that gives the impression that there is no change in the conclusion lacks transparency and credibility. The extensive changes to the original paper, including but not limited to the change to the conclusion section, speak for themselves. However, the new weaker wording can only be considered as a first step towards facing up to the serious problems with this paper.”
A more detailed response to the “fundamental flaws” in the paper will be issued in due course, he added.
Mr Flanagan referred to a note in the amended paper that states: “Yet judges, and their clerks, are human and are confronted with overfilled dockets that demand expediency, and so they might be tempted to use Wikipedia. Our findings show that they do.”
He said the authors have “posted a preprint of a companion paper including more detail on the methodology used” and addressing “additional mechanisms that may be at play, such as lawyers using Wikipedia”.
Some changes made to the wording of the original paper “reflect this more expansive analysis”, he said.
“But the experimental evidence that High Court judgments are being shaped by Wikipedia, both in citations and prose, remains strong and robust,” Mr Flanagan concluded, adding that he was happy to discuss the research with the judiciary.
