Open Science Initiatives promote Diversity, Social Justice, and Sustainability

by Jon Grahe

As I follow the recent social media ruckus centered on replication science questioning motives and methods, it becomes clear that the open science discussion needs to consider the point made by the title of this blog; maybe repeatedly. For readers who weren’t following this, this blog by a political scientist and another post from the SPSP Blog might be of interest. I invite you to join me in evaluating this argument as the discussion progresses. I contend that “Open Science Initiatives promote Diversity, Social Justice, and Sustainability.” Replication science and registered reports are two Open Science Initiatives and by extension should also promote these ideals. If this is not true, I will abandon this revolution and go back to the status quo. However, I am confident that when considering all the evidence, you will agree with me that these idealistic principles benefit from openness generally and by open science specifically.

Before suggesting specific mechanisms by which this occurs, I will briefly note that the definitions of Open Science, Diversity, Social Justice, and Sustainability that are listed on Wikipedia are sufficient for this discussion since Wikipedia itself is an Open Science initiative. Also, I would like to convey the challenge of advancing each of these simultaneously. My own institution, Pacific Lutheran University (PLU), in our recent long range plan, PLU2020, highlighted the importance of uplifting each of these at our own institution as introduced on page 11, “As we discern our commitments for the future, we reaffirm the ongoing commitments to diversity, sustainability, and justice that already shape our contemporary identity, and we resolve to integrate these values ever more intentionally into our mission and institution.” This is easier said than done because at times the goals of these ideals sometimes conflict. For instance, the environmental costs of feeding billions of people and heating their homes are enormous. Sometimes valuing diversity (such as scholarships targeted for people of color) seems unjust because resources are being assigned unevenly. These tensions can be described with many examples across numerous goals in all three dimensions and highlight the need to make balanced decisions.

PLU has not yet resolved this challenge in uplifting all three simultaneously, but I hope that we succeed as we continue the vigorous discussion. Why each is important should be considered from is a Venn diagram on the sustainability Wikipedia page showing sustainable development as intersections between three pillars of sustainable development, social (people), economic, and environmental because even sustainability itself represents competing interests. Diversity and Social Justice are both core aspects of the social dimension, where uplifting diversity highlights the importance of distinct ideas and cultures and helps us understand why people and their varied ideas, in addition to oceans and forests are important resources of our planet. The ideals of social justice aim to provide mechanisms such that all members of our diverse population receive and contribute our fair share of these resources. Because resources are limited and society complex and flawed, these ideals are often more aspirational rather than practical. However, the basic premise of uplifting all three is that we are better when valuing diversity, providing social justice, and sustainably using the planet’s resources (people, animals, plants, and rocks). Below I provide examples for how OSIs promote each of these principles while illustrating why each is important to science.

Open Science Initiatives promote Diversity

All Voices Are Important. While diversity is often confused with labels that describe cultural, ethnic, and economic differences, it is much broader as well because it includes diversity of ideas. As humans, our ideas can easily cross these demographic categories, and as individuals our ideas are the most personal and diverse concepts on the planet. There are many reasons to value diversity, and scientific diversity encompasses both the complexity of research questions (between and within disciplines) as well as complexity of approaches to one research question. Science advances as these disparate approaches center on the most correct answer available at the time. Here I note three ways that OSIs promote Diversity. The first is the capacity for everyone’s voice to be heard. The Open Science Framework and similar platforms allow all researchers to document a study and then to share findings publicly. Therefore, if someone has the best new idea or finding, the data can be available without going through a review process. However because not all research is valuable, quality filters are still necessary. Open Access journals provide opportunity for any idea to be published, but only articles and studies that peers deem worthy will be replicated and advanced. The status quo does not encourage diversity because methods and ideas must meet the approval of people in power (editors and reviewers) before they are considered as part of larger scientific discourse.

Open Data Can Be Re-Evaluated by Anyone. Open science principles recommend full disclosure of materials and data when possible. While there are some exceptions why researchers may withhold data to protect participants, the vast majority of our published work should share anonymous data so that “data detectives” can re-evaluate it to challenge claims or make their own assertions. The Status Quo in psychology suggests that data from published work should be made available upon request from qualified peers published work (see APA ethical guidelines; Article 8.14a, see page 13). However, in practice, this does not seem to be the case as demonstrated by Wickerts, Bakker, and Molenaar (2011). In the recent academic discussion about the validity of the Johnson, Cheung, and Donnellan (2014) pre-registered replication of Schnall, Benton, & Harvey (2008), five different researchers (see links below) have shared reanalyses of the data (to date) to examine the assertion that a ceiling effect caused the findings. Because Open Science requires open data, this discussion was possible:

Yoel Inbar Chris Fraley Tal Yarkoni Felix Schonbrodt Uri Simonsohn

Different Publication Models Provide Empirical Scientific Tests. Valuing diversity means that we can agree to disagree. It is not surprising that the status quo has not embraced Open Science without reservation. A core characteristic of having power is the desire to maintain that power. Change is expensive and frightening. The Open Science paradigm shift will take time. During that time we can take up the challenge posited by Brent Roberts in his blog suggesting an empirical test of the difference between findings determined by editorial decisions from status quo publications versus Pre-Registered reports such as those published in the recent Social Psychology special issue. While his suggestion for a premiere journal to accept only pre-registered reports for a year might be too challenging to embrace, I imagine that some data detective is already planning to test effect sizes of pre-registered vs. status quo publications for a future research project. It would be interesting to randomly assign authors to condition, but that would not be fair. Maybe a within-subject design would be more appropriate and we could counterbalance the order.

Open Science Initiatives promote Social Justice

Underprivileged Need More Voice. When arguing that OSIs promote Diverse opinions because everyone’s scientific contributions should be considered, we should remember that people who currently have the least access to premiere publication outlets are often underprivileged. From a systems perspective, we know that is it very challenging for bright young people to escape from the economic and cultural limitations of their upbringing. Access to higher education is limited to those who can afford it, and not all institutions are similarly resourceful. Subscription costs to access premiere research journals can be excessive. Further, the costs of completing expensive research are similarly daunting. Therefore, OSIs promote social justice in three ways. First, providing access to published research means that scientists can ask better questions. Second, Meta science projects provide access to researchers with limited means to participate in bigger science. Third, OSIs can offer outcomes so that underprivileged non-scientists can better consume and utilize scientific research (e.g., patients with conditions targeted in the research). The Reproducibility Project, Many Labs (1, 2, and 3) projects, and the Collaborative Replications and Education Project offer diverse approaches for a diverse group of researchers. While resources to complete research are still required, individual contributions to larger projects enable researchers to participate in bigger science.

Social Justice Requires Freedom of Speech. A core principle of freedom is free speech through protections of the press and individuals. This can be eroded through government censorship, and limited speech is experienced by too many people on the planet. However, status quo research practices also limit freedom of speech. Publication bias is often discussed as a major cause of poor scientific practices. This bias favors novel, counter intuitive, exciting papers. It is only with the advent of OSIs such as Open Access journals and computer platforms that publicly document any type of research such that replications can freely enter scientific discourse. In this way, meta-analytic researchers can actually open the “file drawer” containing all those non-publishable studies.

Open Communication Improves Scientific Conscientiousness. Public discourse occurs across a dynamic timeline. Publications can take years to complete while Tweets and Status updates take seconds. Though the scientific privileged do not have the same following as media and sports stars do, their expressions of support or disdain for this scientific paradigm revolution are followed by dozens, or hundreds, or thousands of followers. Although I believe we should all be forgiven for public comments made without review (such as an offensive tweet), it is normative pressure that provides the catalysts and mechanisms for us to apologize for offensive remarks. If some computer mediated communications researcher evaluated the back and forth between Open Science proponents and detractors, they will see zealous voices from both sides express potentially offensive comments. In the aftermath, it seemed that Open Science proponents went out of their way to demonstrate that they were not “replication bullies” through apology and clarification. Though mistakes will occur, and angry voices can echo, because Open Science supports open communication, proponents by necessity, address public criticism publicly. As such, it is difficult to see how the Replication Police can allow Data Detectives to engage in bullying behavior without losing support from the scientific public because their findings are only as good as their reputation.

Open Science Initiatives promote Sustainability

Sustainability is a Byproduct of Open Science Tools. The primary tools of open science initiatives are computer based such as platforms for open publication, platforms to document the research process, and platforms to uniformly collect data across locations. Though these systems were not developed specifically to reduce waste, they can. Online versus paper publications, computerized filing versus a metal cabinet, and computerized data collection versus paper and pencil administration of measures are all examples of reduced waste that can occur in this transition. These systems allow for seamless sharing of materials between collaborators, which further reduces time, printing, and delivery costs and allow for new research questions that could not be asked in the paper and pencil era.

Open Science Initiatives Sustain the Workforce. I started getting involved in open science initiatives because I felt like I could not effectively contribute to scientific dialogue because I came from a small school with limited research capacity. Many colleagues at small teaching and research institutions struggle similarly with limited tools, subject pools, time, and access to colleagues. While the contributions of a replication and a novel experiment are widely different, meta-science projects provide me access to the larger scientific debate. Contributors to the International Situation Project or the subsequent International Personality Project can access the data to answer novel questions beyond those intended by the primary researchers. This is an ideal OSI where contributors to a multiple site project can answer novel, impactful questions. OSIs promote sustainability of the workforce by energizing all members of the scientific team from top tier research institutions to primarily undergraduate institutions. Further, by engaging everyone, there is reduction of lost data because there would be fewer small studies that never enter public scientific discourse.

Sustainable Decisions Result from Better Science. Finally, OSIs promote sustainability of resources by helping scientists better communicate good science. The Open Science paradigm shift is a reaction to poor science that results from publication bias. If Ioannidis (2005) is correct that most published findings are false, it is not possible for us to make informed decisions based on strong empirical evidence because we lack the evidence. OSIs are specifically designed to improve the quality of science and therefore the quality of the evidence. When considering people as part of the sustainability equation, this article about open access to science saves lives (Suber & Cubrinklas, 2012) from The Chronicle of Higher Education provides another example. Another example is the recently announced SHARE system designed to notify researchers of related publications so that they can more easily navigate multitudes of publications to find all of them that are most pertinent.

Do OSIs promote Diversity, Social Justice, and Sustainability (if so, is this a good thing)?

My bias is clear. I think that Open Science researchers are challenging a status quo system that is powerful and threatening. I think they are risking much professionally when they encourage changes to publication expectations while engaging in replication science. Not only do OSIs promote each of these separately, I further argue that promoting open discussion of the interconnection between Diversity, Social Justice, and Sustainability leads to a better society. However, this question of the interconnectedness must wait for another blog after more challenges are known.

Just as my home institution is struggling with the conversation about broad integration of these sometimes conflicting principles across the curriculum, Science will struggle to address many conflicting outcomes of the Open Science Revolution. I am a member of the status quo while calling for change. I am an executive editor for a journal that still operates with old habits. I advocate for student contributions to larger science though I know that student contributions can be flawed. I know that conflicts are in the future as we try to increase access to data and the ability to filter those data. This is the beginning of a shift, not the end of it.

I invite you to join this discussion by helping to identify metrics to measure the impact of OSIs on these principles. I invite suggestions from both sides of the argument. If I am wrong and OSIs lead to limited thinking, increased injustice, and poor resource management, then I will give up. I will return to the status quo and disappear. If I am right, then I look forward to reading journals where I hope my children will make their scientific contributions.