none

The paper focuses on two major issues, the “credibility crisis” in Psychology and open science practices, and argues that the two could have a synergistic relationship in Africa, with moves to improve reproducibility and integrity in Psychology benefiting from and contributing to developments to make science more accessible and transparent. Three reviewers assessed the article. The article was considered by all reviewers to be a well-written overview of different dimensions of the “credibility crisis” in Psychology and ways it is being addressed, and of the open science movement, particularly in Africa. The analysis providing a taxonomy of open science developments (organised by accessibility, infrastructure, credibility, and community) and the commentary on how the development of open science can be facilitated in African contexts are considered strengths of the article. Areas where the article could be improved were identified, including that the argument about the mutual benefits that could accrue in African contexts between addressing challenges associated with the “credibility crisis” and encouraging open science could be sharpened and made more specific to Africa. Practical ways in which researchers could adopt open science practices, and how such developments might be measured, could also be improved, with specific comments made about bringing diamond open access into the analysis.

I am on the PsyArxix Scientific Advisory Board. PsyArxiv and preprint servers are mentioned positively by the article. I am a member of the UK Reproducibility Network (UKRN) and chair of the UKRN Institutional Leads group. UKRN is a sister organisation to the African Reproducibility Network (AREN), which is mentioned positively by the article. I am a senior research fellow at the Research on Research Institute (RoRI). RoRI supports the development of MetaRoR, which could be considered an overlay journal. Overlay journals are mentioned positively by the article.

A summary of what the authors were trying to achieve (address the entire article, not just individual points or sections)

This short paper provides an introduction to two issues: the “credibility revolution” of practices in psychology research following the reproducibility crisis, and the state of psychology research in Africa and the factors which are crucial to its development. The paper claims that there are mutual benefits: efforts to support credible and accessible research can benefit psychology in Africa, African psychology can expand and enhance the credibility of psychology research in the rest of the world.

An account of the major strengths and weaknesses of the conceptual framework, methods and results

The paper serves as a very accessible introduction to the “reproducibility crisis” in psychology, and the subsequent “credibility revolution” of research practices which are often (but not exclusively) focussed on transparency and accessibility, and which are applicable in many fields beyond psychology.

The taxonomy of open science innovations into four categories - Accessibility, Infrastructure, Credibility, Community - is a nice way of organising initiatives.

It may be beyond the scope of the remit the authors set themselves, but from a metascience perspective there is an unanswered question of how progress on the challenges set out by the paper would be measured. What are the indicators which we could use to evaluate progress in the different challenge areas or against which to measure benefits ?

One paragraph summarising progress on reproducibility (p8 “The result was an explosion of research on practices to improve the credibility of psychology research”) seems to imply that credibility efforts are coextensive with replication studies (which is surely not what the authors mean) and further to imply that credibility practices are limited in applicability to a restricted domain of mostly online studies (which undersells the benefits of the credibility practices developed within psychology and admirably showcased in this paper).

An appraisal of whether the authors achieved their aims, and whether the results support their conclusions

I am not qualified to comment on whether the portrayal of African psychology is fair or comprehensive. I note that six of the nine authors have affiliations with African institutions.

A discussion of the potential likely impact of the article on the field, and the utility of the conceptual framework, methods and empirical materials/data to the community

The contribution of this paper is to signpost the valuable work that is being done on credibility mechanisms and on research development in Africa.

Any additional context that might help readers interpret or understand the significance of the article

None

Any issues the authors need to address about the availability of data, code, research ethics, or other issues pertaining to the adherence of the article to MetaROR’s publishing policies

N/A

Positionality:

As an experimental psychologist I have been involved in the discussion around credibility since at least 2011. I have no experience or familiarity with African psychology or the human development issues mentioned by the article.

Reviewers are asked to provide specific guidance on the following:

Does the article contribute new insights to the relevant fields?

Yes. Both topics - credibility in research and research in Africa - are huge topics. The brief introductions here are valuable and there is added benefit of bringing the two into explicit dialogue.

Are the key insights clearly communicated in the abstract, introduction, and conclusion?

Yes

Does the introduction section adequately explain necessary background information? Does it set out and justify the motivation for and aim of the study?

Yes

Does the literature review (where applicable) include the relevant research including the most recent research?

Yes, with the caveat that the topics are so large that it is impossible in this amount of space to be comprehensive.

Are any analytical concepts or theoretical frameworks used appropriately introduced and taken up in the empirical analysis (where applicable)?

N/A

Are all research methods clearly described and appropriate? In the case of quantitative submissions, are the methods rigorous and does the study include or point to all materials required to attempt a replication of the results?

N/A

Do the results make sense? Are they clearly formatted and presented? Are graphs easily readable and clearly labeled? Are all figures and tables understandable without reference to the main body of the article? Do they contain proper captions?

N/A

Are the results discussed in the context of previous findings? Are the results similar to previously reported findings? Are differences explained?

There is no mention of previous work on this exact topic (the synergy). Perhaps there isn’t any? Maybe explicit statement to this effect would be good

Are limitations of the study and their implications for interpretation of the results clearly described (where applicable)?

On a similar line, maybe readers would benefit from a statement from the authors on their backgrounds and/or how the author team came together to address this topic?

Are interpretations and conclusions consistent with the empirical materials and data?

N/A

Are all references appropriate? Are necessary references present? Are all references cited in the text included in the reference list?

Nosek et al (2021) is missing or should be Nosek et al (2022), which additionally appears slightly out of alphabetical order in the bibliography

If one or more studies in the article were preregistered, are the hypotheses, research methods, and inference criteria in line with the preregistration?

N/A

none

A summary of what the authors were trying to achieve (address the entire article, not just individual points or sections)

The area of expertise of the reviewer is open access which, for all intents and purposes, is a subset of open science. The nub of this manuscript is the credibility of psychology research using open science to grow that credibility and to bridge the psychology knowledge divide between the global north and Africa. Given the reviewers limited knowledge of the core issue of psychology research, the reviewer will confine his comments to his area of expertise.

An account of the major strengths and weaknesses of the conceptual framework, methods and results

Psychological science is facing a ‘credibility crisis’ because many studies can’t be replicated, prompting reforms for more transparent and rigorous research. While these efforts have progressed in North America and Europe, Africa has seen less impact due to challenges like low funding and poor infrastructure, widening the gap in research capacity. These are the core issues covered in this manuscript. However, what is omitted as a challenge is the brain drain and its impact on research and research production. Many authors talk about knowledge pilgrimage as African researchers have to research issues from a global north perspective to improve chances for publication. There is a nuanced difference between research colonialism (concept used by the authors) and knowledge pilgrimage – the former aligned to helicopter research while the latter is about African researchers manipulating global north research data to get published at the neglect of Africa. 

The authors make the point about power imbalances and the fact that global north researchers use Africa as a point of data collection without recognising the contribution by the African researchers.

The research gap is widened given the reliance on North American and Europeans guidelines and standards. Open science practices offer African researchers tools to improve research quality and join global discussions. Initiatives like the African Reproducibility Network can help build stronger research communities, addressing local issues while contributing to a more inclusive and globally relevant psychology. Strengthening African research could also advance human development across the continent.

It is recommended that the authors do not use illegal entities as sources of information as, in the opinion of the reviewer, brings into question the credibility of the manuscript.

The perceived weaknesses relate to the oversimplification of open science solutions. While open science is framed as a key solution, the manuscript oversimplifies its implementation in Africa. It only superficially acknowledges the barriers to adopting open science practices, such as the lack of stable internet access, digital tools, and the necessary training in many African institutions. The manuscript suggests open science can bridge gaps without delving into the complexities of infrastructure and access that make it difficult for African researchers to fully engage with these practices.

The manuscript should pay a little more attention on how African researchers can practically adopt open science tools. While it mentions open-access platforms and reproducibility networks, it doesn’t provide details on how these can be integrated into the current research systems in Africa, given the resource constraints. Practical guidance on funding models, technical support, or training programs needed to implement open science would provide a more grounded solution. The use of diamond open access is an extremely viable model to grow the production of psychological research.

An appraisal of whether the authors achieved their aims, and whether the results support their conclusions

From an open science perspective, the authors have done relatively well to provide a roadmap for the improved accessibility and credibility of psychology research.

The authors highlight how open science practices, such as PsyArXiv and AfricArXiv, enhance research accessibility by allowing researchers to freely share their findings. This is critical in the context of the paywalls that restrict access to valuable research, particularly for researchers in low-resource settings. They also address the issue of inclusivity or equity. APCs is a new barrier. It is pointed out that many researchers in Africa face barriers due to high article processing fees associated with open access journals. This situation can perpetuate inequities in knowledge production and dissemination. What the authors have missed is the opportunity to investigate diamond open access as a viable alternative for the dissemination of African scholarship.

The preprints solution is a viable pathway for researchers to share their findings before formal publication. This can facilitate greater visibility for their work and allow for earlier feedback, although it’s essential to navigate journal policies carefully. The acknowledgement of overlay journals is important given that it is a relatively new concept. Instead of publishing papers directly, they provide peer review for papers that have already been posted on preprint servers. If the paper passes peer review, the overlay journal "publishes" the paper by linking to it on the preprint server. When one ventures into the arena of quality, this process helps build research production capacity. However, it must be noted that some overlay journals charge a fee for the peer review and publication process.

The other significant issue is that of research credibility. The authors discuss innovative tools and processes that enhance credibility. Initiatives such as registered reports and pre-registration of study protocols enhances the transparency and credibility of research. These methods can mitigate biases related to selective reporting of positive results, thus improving the overall integrity of psychological research. Community initiatives like that of Psychological Science Accelerator, underlines the importance of collaboration in enhancing research credibility. Engaging a diverse range of researchers can lead to more comprehensive studies and foster an environment of shared knowledge.

One of the major challenges is that of low skills levels. The authors bring into the discussion the issue of building capacity – the train-the trainer model is critical for an inclusive process. Further, to maximize the benefits of open science practices, there’s a clear need for training and capacity-building initiatives tailored to the African context. This would empower researchers with the skills and knowledge to effectively engage with these practices.

What is of concern to the reviewer is the indistinctive definition of predatory. The authors should stay away from aligning predatory with open access. The fact that the mode of delivery is electronic material must not be confused with a model for the delivery of predatory scholarship

In the main, the authors have achieved their goal of developing a roadmap for bridging the divide.

Reviewers are asked to provide specific guidance on the following:

Does the article contribute new insights to the relevant fields?

The manuscript provides a high level association between open science practices and driving human development in Africa. This is done through enhancing access to research, improving infrastructure, and boosting credibility. These are established which is confirmed by the authors highlight that platforms like AfricArXiv and OSF provide free access to research materials and tools, reduce barriers imposed by paywalls. There are other initiatives such as Registered Reports, the African Reproducibility Network that foster collaboration and skill-building.

The new insights that are brought forward via the linking of these established tools, practises to the African scenario. Empowering and bridging the divide are the new insights.

Are the key insights clearly communicated in the abstract, introduction, and conclusion?

The abstract should provide a little more detail on how open science and its relationship with bridging the knowledge divide. The introduction gives substantial grounding on what the readers can expect from the rest of the article. However, the conclusion does not pull the golden threads together. A little more can be done with the conclusion and bringing an association between the introduction and conclusion.

Do the results make sense? Are they clearly formatted and presented? Are graphs easily readable and clearly labeled? Are all figures and tables understandable without reference to the main body of the article? Do they contain proper captions?

The discussion is substantial. The two tables are clear and easily readable. What is commended is that the authors do not duplicate what is captured in the two tables.

Are any analytical concepts or theoretical frameworks used appropriately introduced and taken up in the empirical analysis (where applicable)?

There is no theoretical framework or a research methodology section. However, this may be a practise that the reviewer is accustomed to but not necessary. Be that as it may, it will help the manuscript if the authors could articulate the methodology.

Does the literature review (where applicable) include the relevant research including the most recent research?

The review of the literature is substantial. However, there are gaps (e.g. diamond open access) in the review of the literature with regard to more recent development with regard to dissemination of scholarship.

There are a number of very old references and more current references. The old references are important to ground the issue of (de)colonialization which is important for the major part of the discussion. More recent literature would give the authors an understanding of diamond which will help in sourcing alternatives.

I have no competing interests with the authors.

The authors outline the developments of both North America and Europe as well as Africa, and how both parties can benefit from the progress and learnings of the other. However, the authors argue that this can only occur if a wide array of stakeholders invest in science in Africa, including resourcing, training, and development of research tools. I particularly enjoyed reading this article, as the authors outline how science – even in its most basic forms – in fact can and do impact population outcomes and development. It provides a reader naïve of the research situation in Africa, as well as one that is relatively naïve to open science, a concise summary of how these two can interact to yield better outcomes for both – suitable for a wide range of stakeholders.

A key strength of this article is that the authorship consists of experts involved in both fields that are being discussed i.e., open science, and science in Africa. Thus, providing a relatively balanced view of the situation. However, my one concern is that the presentation for the case of each side benefiting the other could be made stronger, particularly how Africa could benefit from open science and the credibility revolution occurring in North America and Europe. At the moment, it reads that the key emphasis is that Africa could benefit greatly from open science learning if it addresses key structural barriers, and that open science could benefit from Africa by expanding its research community. However, the latter is true for many regions of the world.  As Africa is a diverse and rich continent in history and culture, I encourage the authors to examine further and investigate case studies or findings to strengthen this argument. Examples may include where Africa has excelled in transparency, collaboration, or other key tenets of good science. This approach forces us to recognize that in North America and Europe research ‘culture’, shortcomings may be addressed or remedied by looking to other research cultures globally. This would provide a more balanced argument for how both parties can mutually benefit one another - in line with the core intention of the piece. Thank you for the opportunity to review this article.

https://osf.io/preprints/africarxiv/e57bq

Adetula, A., Forscher, P. S., Basnight-Brown, D., Azouaghe, S., Ouherrou, N., CHARYATE, A., … IJzerman, H. (2021, June 21). Synergy Between the Credibility Revolution and Human Development in Africa. https://doi.org/10.31730/osf.io/e57bq

August 8, 2024

January 22, 2025

January 22, 2025

Authors:

• Adeyemi Adetula (Busara Center for Behavioral Economics, Nairobi, Kenya) adeyemiadetula1@gmail.com
• Patrick Forscher (Department of Psychology, United States International University-Africa, Nairobi, Kenya) patrick.forscher@busara.global
• Dana Basnight-Brown (Department of Psychology, Mohammed V University, Rabat, Morocco) dana.basnightbrown@gmail.com
• Soufian Azouaghe (Université Chouaib Doukkali, Université Paul Valéry Montpellier 3) azouaghe.soufian@gmail.com
• Nihal Ouherrou (Université Ibn Tofail) n.ouherrou@gmail.com
• Abdelilah Charyate (University of Groningen, The Netherlands) abdelilah.charyate@uit.ac.ma
• Nina Hansen (Department of Pure and Applied Psychology, Adekunle Ajasin University, Akungba-Akoko, Ondo State, Nigeria) n.hansen@rug.nl
• Gabriel Adetula (LIP/PC2S, Université Grenoble Alpes, Saint-Martin-d’Heres, Auvergne-Rhône-Alpes, France & Institut Universitaire de France (IUF), France) g1b2gbo3detul4@gmail.com
• Hans IJzerman hans@absl.io

1

https://osf.io/preprints/africarxiv/e57bq

10.31730/osf.io/e57bq

Synergy Between the Credibility Revolution and Human Development in Africa

Jennifer Byrne is a member of the editorial team of MetaROR working with Jason Chin, a co-author of the protocol and also a member of the editorial team of MetaROR.

Researchers are willing to trade their results for journal prestige: results from a discrete choice experiment [Version 1]

This protocol aims to address two questions: (1) What do we know about the science underlying impactful legal decisions? (2) How can we assess this evidence efficiently and accurately, such that it is usable for courts? The protocol has been reviewed by three reviewers (reviewer 2 in fact represents a team of three individuals). The reviewers mention various strengths of the protocol. Reviewer 1 emphasises the importance and timeliness of the research questions and praises the interdisciplinary nature of the research team. Reviewer 3 considers the protocol to be thoughtful and detailed, and reviewer 2 notes that the protocol presages an important effort. The reviewers do not see any major shortcomings in the protocol, but they do highlight opportunities to strengthen the protocol, such as considering studies published in languages other than English and adding more detail on how team disagreements will be resolved.

Competing interest: Jennifer Byrne is a member of the editorial team of MetaROR working with Jason Chin, a co-author of the protocol and also a member of the editorial team of MetaROR.