Research transparency
Research transparency is a major aspect of scientific research. It covers a variety of scientific principles and practices: reproducibility, data and code sharing, citation standards or verifiability.
The definitions and norms of research transparency significantly differ depending on the disciplines and fields of research. Due to the lack of consistent terminology, research transparency has frequently been defined negatively by addressing non-transparent usages.
After 2010, recurrent issues of research methodology have been increasingly acknowledged as structural crisis, that involve deep changes at all stages of the research process. Transparency has become a key value of the open science movement, which evolved from an initial focus on publishing to encompass a large diversity of research outputs. New common standards for research transparency, like the TOP Guidelines, aims to build and strengthen open research culture across disciplines and epistemic cultures.
Definitions
Confused terminologies
There is no widespread consensus on the definition of research transparency.Differences between disciplines and epistemic cultures has largely contributed to different acceptions. The reproduction of past research has been a leading source of dissent. In an experimental setting, reproduction relies on the same set-up and apparatus, while replication only requires the use of the same methodology. Conversely, computational disciplines use reversed definitions of the term replicability and reproducibility. Alternative taxonomies have proposed to make do entirely with the ambiguity of reproducibility/replicability/repeatability. Goodman, Fanelli and Ioannidis recommended instead a distinction between method reproducibility and result reproducibility.
Core institutional actors continue to disagree on the meaning and usage of key concepts. In 2019, the National Academies of Science of the United States retained the experimental definition of replication and reproduction, which remains "at odds with the more flexible way they are used by major organizations". The Association for Computing Machinery opted in 2016, for the computational definition and added also an intermediary notion of repeatability, where a different team of research use exactly the same measurement system and procedure.
Debate over research transparency has also created new convergences between different disciplines and academic circles. In the Problem of science, Rufus Barker Bausell argues that all disciplines, including the social sciences, currently face similar issues to medicine and physical sciences: "The problem, which has come to be known as the reproducibility crisis, affects almost all of science, not one or two individual disciplines."
Negative definitions
Due to lack of consistent terminology over research transparency, scientists, policy-makers and other major stake-holders have increasingly rely on negative definitions: what are the practices and forms that harm or disrupt any common ideal of research transparency.The taxonomy of scientific misconducts has been gradually expanded since the 1980s. The concept of questionable research practices was first incepted in a 1992 report of the Committee on Science, Engineering, and Public Policy as a way to address potentially non-intentional research failures. Questionable research practices uncover a large grey area of problematic practices, which are frequently associated to deficiencies in research transparency. In 2016, a study identified as much as 34 questionable research practices or "degree of freedom", that can occur at all the steps of the project.
Surveys of disciplinary practices have shown large differences in the admissibility and spread of questionable research practices. While data fabrication and, to a lesser extent, rounding of statistical indicators like the p value are largely rejected, the non-publication of negative results or the adjonctions of supplementary data are not identified as major issues.
In 2009, a meta-analysis of 18 surveys estimated that less than 2% of scientists "admitted to have fabricated, falsified or modified data or results at least once". Real prevalence may be under-estimated due to self-reporting: regarding "the behaviour of colleagues admission rates were 14.12%". Questionable research practices are more widespread as more than one third of the respondents admit to have done it once. A large 2021 survey of 6,813 respondents in the Netherlands found significantly higher estimate, with 4% of the respondents engaging in data fabrication and more than half of the respondents engaging in questionable research practices. Higher rates can be either attributed to a deterioration of ethic norms or to "the increased awareness of research integrity in recent years".
A new dimension of open science?
Transparency has been increasingly acknowledged as an important component of open science. Until the 2010s, definitions of open science have been mostly focused on technical access and enhanced participation and collaboration between academics and non-academics. In 2016, Liz Lyon identified transparency as a "third dimension" of open science, due to the fact that "the concept of transparency and the associated term 'reproducibility', have become increasingly important in the current interdisciplinary research environment." According to Kevin Elliott, the open science movement "encompasses a number of different initiatives aimed at somewhat different forms of transparency."First drafted in 2014, the TOP guidelines have significantly contributed to bring transparency on the agenda of the open science movements. They aim to promote an "open research culture" and implement "strong incentives to be more transparent". They rely on eight standards, with different levels of compliance. While the standards are modular, they also aim to articulate a consistent ethos of science as "they also complement each other, in that commitment to one standard may facilitate adoption of others.".
This open science framework of transparency has been in turn coopted by leading contributors and institutions on the topic of research transparency. After 2015, contributions from science historians underlined that there have been no significant deterioration of research quality, as past experiments and research design were not significantly better conceived and the rate of false or partially false has likely remained approximately constant for the last decades. Consequently, proponents of research transparency have come to embrace more explicitly the discourse of open science: the culture of scientific transparency becomes a new ideal to achieve rather than a fundamental principle to re-establish.
The concept of transparency has contributed to create convergences between open science and other open movements in different areas such as open data or open government. In 2015, the OECD describe transparency as a common "rationale for open science and open data".
History
Discourse and practices of research transparency (before 1945)
Transparency has been a fundamental criterion of experimental research for centuries. Successful replications have become an integral part of the institutional discourse of natural sciences in the 17th century. An early scientific society of Florence the Accademia del Cimento adopted in 1657 the motto provando e riprovando as a call for "repeated performances of experimental trials" A key member of the Accademia, the naturalist Francesco Redi described extensively of the forms and benefits of procedural experimentation, that made it possible to check for random effects, the soundness of the experiment design, or causal relationships through repeated trials Replication and the open documentation of scientific experiments has become a key component of the diffusion of scientific knowledge in society: once they attained a satisfying rate of success, experiments could be performed in a variety of social spaces such as courts, marketplaces or learned salon.Although transparency has been early on acknowledged as a key component of science, it was not defined consistently. Most concept associated today with research transparency have arisen as terms of the art with no clear and widespread definitions. The concept of reproducibility appeared in an article on the "Methods of illuminations" first published in 1902: one of the methods examined was deemed limited regarding "reproducibility and constancy" In 2019, the National Academies underlined that the distinction between reproduction, repetition and replication has remained largely unclear and unharmonized across disciplines: "What one group means by one word, the other group means by the other word. These terms — and others, such as repeatability — have long been used in relation to the general concept of one experiment or study confirming the results of another."
Beyond this lack of formalization, there was a significant drift between the institutional and disciplinary discourse on research transparency and the reality of research work, that has persisted till the 21st century. Due to the high cost of the apparatus and the lack of incentives, most experiences were not reproduced by contemporary researchers: even a committed proponent of experimentalism like Robert Doyle had to devolve to a form of virtual experimentalism, by describing in detail a research design that has only been run once For Friedrich Steinle, the gap between the postulated virtue of transparency and the material conditions of science has never been solved: "The rare cases in which replication actually is attempted are those that either are central for theory development or promise broad attention due to major economical perspectives. Despite the formal ideal of replicability, we do not live in a culture of replication."