Reliability of Wikipedia
The reliability of Wikipedia and its volunteer-driven and community-regulated [|editing model], particularly its English-language edition, has been questioned and tested. Wikipedia is written and edited by volunteer editors who generate online content with the editorial oversight of other volunteer editors via community-generated policies and guidelines. The reliability of the project has been tested statistically through comparative review, analysis of the historical patterns, and strengths and weaknesses inherent in its editing process. The online encyclopedia has been criticized for its factual unreliability, principally regarding its content, presentation, and editorial processes. Studies and surveys attempting to gauge the reliability of Wikipedia have mixed results. Wikipedia's reliability was frequently criticized in the 2000s but has been improved; its English-language edition has been generally praised in the late 2010s and early 2020s.
Select assessments of its reliability have examined how quickly vandalism—content perceived by editors to constitute false or misleading information—is removed. Two years after the project was started, in 2003, an IBM study found that "vandalism is usually repaired extremely quickly—so quickly that most users will never see its effects". The inclusion of false or fabricated content has, at times, lasted for years on Wikipedia due to its volunteer editorship. Its editing model facilitates multiple systemic biases, namely selection bias, inclusion bias, participation bias, and group-think bias. The majority of the encyclopedia is written by male editors, leading to a gender bias in coverage, and the make up of the editing community has prompted concerns about racial bias, spin bias, corporate bias, and national bias, among others. An ideological bias on Wikipedia has also been identified on both conscious and subconscious levels. A series of studies from Harvard Business School in 2012 and 2014 found Wikipedia "significantly more biased" than Encyclopædia Britannica but attributed the finding more to the length of the online encyclopedia as opposed to slanted editing.
Instances of non-neutral or conflict-of-interest editing and the use of Wikipedia for "revenge editing" has attracted attention to false, biased, or defamatory content in articles, especially biographies of living people. Articles on less technical subjects, such as the social sciences, humanities, and culture, have been known to deal with [|misinformation cycles], cognitive biases, [|coverage discrepancies], and editor disputes. The online encyclopedia does not guarantee the validity of its information. Nevertheless, it is regarded as a valuable "starting point" for researchers seeking to examine the listed references, citations, and sources. Academics suggest reviewing reliable sources when assessing the quality of articles.
Its coverage of medical and scientific articles such as pathology, toxicology, oncology, pharmaceuticals, and psychiatry were compared to professional and peer-reviewed sources in a 2005 Nature study. A year later Encyclopædia Britannica disputed the Nature study, whose authors, in turn, replied with a further rebuttal. Concerns regarding readability and the overuse of technical language were raised in studies published by the American Society of Clinical Oncology, Psychological Medicine, and European Journal of Gastroenterology and Hepatology. The Simple English Wikipedia serves as a simplified version of articles to make complex articles more accessible to the layperson on a given topic in Basic English. Wikipedia's popularity, mass readership, and free accessibility has led the encyclopedia to command a substantial second-hand cognitive authority across the world.
Wikipedia editing model
Wikipedia allows anonymous editing; contributors are not required to provide any identification or an email address. A 2009 study of Dartmouth College in the English Wikipedia noted that, contrary to usual social expectations, anonymous editors were some of Wikipedia's most productive contributors of valid content. The Dartmouth study was criticized by John Timmer of Ars Technica for its methodological shortcomings.Wikipedia trusts the same community to self-regulate and become more proficient at quality control. Wikipedia has harnessed the work of millions of people to produce the world's largest knowledge-based site along with software to support it, resulting in more than nineteen million articles written, across more than 280 different language versions, in fewer than twelve years. For this reason, there has been considerable interest in the project both academically and from diverse fields such as information technology, business, project management, knowledge acquisition, software programming, other collaborative projects and sociology, to explore whether the Wikipedia model can produce quality results, what collaboration in this way can reveal about people and whether the scale of involvement can overcome the obstacles of individual limitations and poor editorship which would otherwise arise.
Wikipedia's degree of truthfulness extends from its technology, policies, and editor culture. Edit histories are publicly visible. Footnotes show the origins of claims. Editors remove unverifiable claims and overrule claims not phrased in a neutral point of view. Wikipedia editors also tend towards self-examination and acknowledge Wikipedia's flaws. Its open model permits article-tampering including short-lived jokes and longer hoaxes. Some editors dedicate as much time to trolling as others do improving the encyclopedia. The English Wikipedia's editor pool, roughly 40,000 active editors who make five edits monthly, largely skews male and white, leading to gender- and race-based systemic biases in coverage. Variations in coverage mean that Wikipedia can be both, as online communities professor Amy S. Bruckman put it, "the most accurate form of information ever created by humans" on the whole while short articles can be "total garbage".
Academics view Wikipedia as representing a "consensus truth" in which readers can check reality in an age of contested facts. For example, when facts surrounding the COVID-19 pandemic rapidly changed or were debated, editors removed claims that did not adhere to the "verifiability" and "NPOV" guidelines.
Fact-checking
Fact-checking of Wikipedia is the process through which Wikipedia editors perform fact-checking of content published in Wikipedia, while fact-checking using Wikipedia is the use of Wikipedia for fact-checking other publications. The broader topic of fact checking in the context of Wikipedia also includes the cultural discussion of the place of Wikipedia in fact-checking. Major platforms including YouTube and Facebook use Wikipedia's content to confirm the accuracy of information in their own media collections. Seeking public trust is a major part of Wikipedia's publication philosophy.Wikipedia has grown beyond a simple encyclopedia to become what The New York Times called a "factual netting that holds the digital world together". Common questions asked of search engines are answered using knowledge ingested from Wikipedia, and often credit or link to Wikipedia as their source. Wikipedia is likely the most important single source used to train generative artificial intelligence models, such as ChatGPT, for which Wikipedia is valued as a well-curated data set with highly structured formatting. The accuracy of AI models depend on the quality of their training data, but these models are also fundamentally unable to cite their original source for their knowledge, thus AI users use Wikipedia knowledge without knowing that Wikipedia is its source. AI users also receive results that intertwine facts originating from Wikipedia with fictional data points, lowering the quality of information absent a real-time fact-check during information retrieval.
Assessments
Criteria for evaluating reliability
The reliability of Wikipedia articles can be measured by the following criteria:- Accuracy of information provided within articles
- Appropriateness of the images provided with the article
- Appropriateness of the style and focus of the articles
- Susceptibility to, and exclusion and removal of, false information
- Comprehensiveness, scope and coverage within articles and in the range of articles
- Identification of reputable third-party sources as citations
- Verifiability of statements by respected sources
- Stability of the articles
- Susceptibility to editorial and systemic bias
- Quality of writing
Comparative studies
On October 24, 2005, the British newspaper, The Guardian, published a story entitled "Can you trust Wikipedia?" in which a panel of experts were asked to review seven entries related to their fields, giving each article reviewed a number designation from 0 to 10. Most of these reviewed articles received marks between 5 and 8. The most common critiques were poor prose, or ease-of-reading issues, omissions or inaccuracies, often small but including key omissions in some articles, and poor balance, with less important areas being given more attention and vice versa. The most common praises were factually sound and correct, no glaring inaccuracies, and much useful information, including well-selected links, making it possible to "access much information quickly".In December 2005, the journal Nature published results of an attempted blind study seeking reviewer evaluations of the accuracy of a small subset of articles from Wikipedia and Encyclopædia Britannica. The non-peer-reviewed study was based on Natures selection of 42 articles on scientific topics, including biographies of well-known scientists. Factual errors, omissions or misleading statements found in the sampled articles was 162 for Wikipedia and 123 for Britannica. For serious errors, such as misinterpretations of important concepts, 4 were found in Wikipedia, and 4 in Britannica. The study concluded that "Wikipedia comes close to Britannica in terms of the accuracy of its science entries", although Wikipedia's articles were often "poorly structured".
Encyclopædia Britannica expressed concerns, leading Nature to release further documentation of its survey method. Based on this additional information, Encyclopædia Britannica denied the validity of the Nature study, stating that it was "fatally flawed". Among Britannicas criticisms were that excerpts rather than the full texts of some of their articles were used, that some of the extracts were compilations that included articles written for the youth version, that Nature did not check the factual assertions of its reviewers, and that many points the reviewers labeled as errors were differences of editorial opinion. Britannica further stated that "While the heading proclaimed that 'Wikipedia comes close to Britannica in terms of the accuracy of its science entries,' the numbers buried deep in the body of the article said precisely the opposite: Wikipedia in fact had a third more inaccuracies than Britannica. " Nature acknowledged the compiled nature of some of the Britannica extracts, but denied that this invalidated the conclusions of the study. Encyclopædia Britannica also argued that a breakdown of the errors indicated that the mistakes in Wikipedia were more often the inclusion of incorrect facts, while the mistakes in Britannica were "errors of omission", making "Britannica far more accurate than Wikipedia, according to the figures". Nature has since rejected the Britannica response, stating that any errors on the part of its reviewers were not biased in favor of either encyclopedia, that in some cases it used excerpts of articles from both encyclopedias, and that Britannica did not share particular concerns with Nature before publishing its "open letter" rebuttal.
The point-for-point disagreement between these two parties that addressed the compilation/text excerpting and very small sample size issues—argued to bias the outcome in favor of Wikipedia, versus a comprehensive, full article, large sample size study favoring the quality-controlled format of Britannica—have been echoed in online discussions, including of articles citing the Nature study, e.g., where a "flawed study design" for manual selection of articles/article portions, the lack of study "statistical power" in its comparing 40 articles from over 100,000 Britannica and over 1 million English Wikipedia articles, and the absence of any study statistical analyses has also been noted. Science communicator Jonathan Jarry said in 2024 that the study was historically important, and had been cited in almost every science paper on Wikipedia's reliability since then, but that research of this kind will only provide a "snapshot" and quickly become unreliable.
In June 2006, Roy Rosenzweig, a professor specializing in American history, published a comparison of the Wikipedia biographies of 25 Americans to the corresponding biographies found on Encarta and American National Biography Online. He wrote that Wikipedia is "surprisingly accurate in reporting names, dates, and events in U.S. history" and described some of the errors as "widely held but inaccurate beliefs". However, he stated that Wikipedia often fails to distinguish important from trivial details, and does not provide the best references. He also complained about Wikipedia's lack of "persuasive analysis and interpretations, and clear and engaging prose".
A web-based survey conducted from December 2005 to May 2006 by Larry Press, a professor of Information Systems at California State University at Dominguez Hills, assessed the "accuracy and completeness of Wikipedia articles". Fifty people accepted an invitation to assess an article. Of the fifty, seventy-six percent agreed or strongly agreed that the Wikipedia article was accurate, and forty-six percent agreed or strongly agreed that it was complete. Eighteen people compared the article they reviewed to the article on the same topic in the Encyclopædia Britannica. Opinions on accuracy were almost equal between the two encyclopedias, and eleven of the eighteen found Wikipedia somewhat or substantially more complete, compared to seven of the eighteen for Britannica. The survey did not attempt a random selection of the participants, and it is not clear how the participants were invited.
The German computing magazine c't performed a comparison of Brockhaus Multimedial, Microsoft Encarta, and the German Wikipedia in October 2004: Experts evaluated 66 articles in various fields. In overall score, Wikipedia was rated 3.6 out of 5 points. A second test by c't in February 2007 used 150 search terms, of which 56 were closely evaluated, to compare four digital encyclopedias: Bertelsmann Enzyklopädie 2007, Brockhaus Multimedial premium 2007, Encarta 2007 Enzyklopädie and Wikipedia. It concluded: "We did not find more errors in the texts of the free encyclopedia than in those of its commercial competitors."
Viewing Wikipedia as fitting the economists' definition of a perfectly competitive marketplace of ideas, George Bragues, examined Wikipedia's articles on seven top Western philosophers: Aristotle, Plato, Immanuel Kant, René Descartes, Georg Wilhelm Friedrich Hegel, Thomas Aquinas, and John Locke. Wikipedia's articles were compared to a consensus list of themes culled from four reference works in philosophy. Bragues found that, on average, Wikipedia's articles only covered 52% of consensus themes. No errors were found, though there were significant omissions.
PC Pro magazine asked experts to compare four articles in their scientific fields between Wikipedia, Britannica and Encarta. In each case Wikipedia was described as "largely sound", "well handled", "performs well", "good for the bare facts" and "broadly accurate". One article had "a marked deterioration towards the end" while another had "clearer and more elegant" writing, a third was assessed as less well written but better detailed than its competitors, and a fourth was "of more benefit to the serious student than its Encarta or Britannica equivalents". No serious errors were noted in Wikipedia articles, whereas serious errors were noted in one Encarta and one Britannica article.
In October 2007, the Australian magazine PC Authority published a feature article on the accuracy of Wikipedia. The article compared Wikipedia's content to other popular online encyclopedias, namely Britannica and Encarta. The magazine asked experts to evaluate articles pertaining to their field. A total of four articles were reviewed by three experts. Wikipedia was comparable to the other encyclopedias, topping the chemistry category.
In December 2007, German magazine Stern published the results of a comparison between the German Wikipedia and the online version of the 15-volume edition of Brockhaus Enzyklopädie. The test was commissioned to a research institute, whose analysts assessed 50 articles from each encyclopedia on four criteria, and judged Wikipedia articles to be more accurate on the average. Wikipedia's coverage was also found to be more complete and up to date; however, Brockhaus was judged to be more clearly written, while several Wikipedia articles were criticized as being too complicated for non-experts, and many as too lengthy.
In its April 2008 issue, British computing magazine PC Plus compared the English Wikipedia with the DVD editions of World Book Encyclopedia and Encyclopædia Britannica, assessing for each the coverage of a series of random subjects. It concluded, "The quality of content is good in all three cases" and advised Wikipedia users "Be aware that erroneous edits do occur, and check anything that seems outlandish with a second source. But the vast majority of Wikipedia is filled with valuable and accurate information."
A 2008 paper in Reference Services Review compared nine Wikipedia entries on historical topics to their counterparts in Encyclopædia Britannica, The Dictionary of American History and American National Biography Online. The paper found that Wikipedia's entries had an overall accuracy rate of 80 percent, whereas the other encyclopedias had an accuracy rate of 95 to 96 percent.
A 2010 study assessed the extent to which Wikipedia pages about the history of countries conformed to the site's policy of verifiability. It found that, in contradiction of this policy, many claims in these articles were not supported by citations, and that many of those that were, sourced to popular media and government websites rather than to academic journal articles.
In April 2011, a study was published by Adam Brown of Brigham Young University in the journal PS Political Science & Politics which examined "thousands of Wikipedia articles about candidates, elections, and officeholders". The study found that while the information in these articles tended to be accurate, the articles examined contained many errors of omission.
A 2012 study co-authored by Shane Greenstein examined a decade of Wikipedia articles on United States politics and found that the more contributors there were to a given article, the more neutral it tended to be, in line with a narrow interpretation of Linus's law.
Reavley et al. compared the quality of articles on select mental health topics on Wikipedia with corresponding articles in Encyclopædia Britannica and a psychiatry textbook. They asked experts to rate article content with regard to accuracy, up-to-dateness, breadth of coverage, referencing and readability. Wikipedia scored highest on all criteria except readability, and the authors concluded that Wikipedia is as good as or better than Britannica and a standard textbook.
A 2014 perspective piece in the New England Journal of Medicine examined Wikipedia pages about 22 prescription drugs to determine if they had been updated to include the most recent FDA safety warnings. It found that 41% of these pages were updated within two weeks after the warning, 23% were updated more than two weeks later, and the remaining 36% had not been updated to include the warning as of more than 1 year later as of January 2014.
A 2014 study in the Journal of the American Pharmacists Association examined 19 Wikipedia articles about herbal supplements, and concluded that all of these articles contained information about their "therapeutic uses and adverse effects", but also concluded that "several lacked information on drug interactions, pregnancy, and contraindications". The study's authors therefore recommended that patients not rely solely on Wikipedia as a source for information about the herbal supplements in question.
Another study published in 2014 in PLOS ONE found that Wikipedia's information about pharmacology was 99.7% accurate when compared to a pharmacology textbook, and that the completeness of such information on Wikipedia was 83.8%. The study also determined that completeness of these Wikipedia articles was lowest in the category "pharmacokinetics" and highest in the category "indication". The authors concluded that "Wikipedia is an accurate and comprehensive source of drug-related information for undergraduate medical education".