Author response

To: Julie Sheldon

I thank Julie Sheldon for their thoughtful consideration of the manuscript.

Sheldon notes that the Introduction did not provide much information on black bears. In the revision, I have massively expanded the Introduction to cover the physiology and morphology of the black bear; its habitats, diet, and behaviour; natural history; and human- bear interactions. As suggested, I have also compared these black bear characteristics to reported sasquatch characteristics, considering the possibility that sasquatch sightings correlate with black bears (either because they are the same animals or distinct animals with overlapping characteristics including habitat). Because I have had to expand the Introduction following both Sheldon’s and Raveendran’s suggestions, the Introduction is now quite long, which I hope the reviewers can forgive. Sheldon noted in particular the primarily forest habitat of bears. I think this is especially important, and I received a suggestion from another reader to control for forest area in the model. Accordingly, I have replaced the variable for land area in each state/province with forest area in each state/province.

Furthermore, Sheldon noted that most of the bear population estimates were rather dated (from 2001), with others being more recent but being sourced variously from state department resources, conservation societies, and biologists and conservationists quoted in media articles. Sheldon suggested that an updated resource would improve the manuscript. To implement Sheldon’s suggestion, I performed a literature search for such a resource. To the best of my knowledge, the most recent, peer-reviewed, publicly-available single source for bear population estimates is Spencer et al., 2007 (DOI: 10.2192/1537- 6176(2007)18[217:HARTHB]2.0.CO;2), which provides estimates across US states and Canadian provinces for 2006. These data, collected from a survey of North American wildlife agencies, are still relatively dated, but represent the best improvement over the previous data I could find. Following Sheldon’s suggestion, I replaced the previous data from Hristienko and McDonald with the new, more consistent and more recent Spencer et al. data, and I have discarded the estimates from media articles etcetera. Note also that the estimates provided in Hristienko and McDonald were more approximate (i.e., ranges were given) whereas the new estimates given in Spencer et al. are more precise. If the author knows of a more precise and up to date resource than Spencer et al., I would be happy to update the analyses further. However, in this regard I have reached the limit of my art.

Crucially, Sheldon points out that the dates of the bear population data and saquatch sightings did not match (as previously noted in the limitations section of the Discussion). This is an important point, because as Sheldon notes, having accurate data are important for the goal of the study. Sheldon makes the excellent suggestion to restrict the bear and sasquatch data to the same time period. To improve the validity of the analysis, I have implemented Sheldon’s suggestion of date-matching the data. As noted above, the updated bear population data correspond to 2006. Thus, I have restricted the sasquatch sightings to just those made in 2006 (this was a rather painstaking process because the BFRO web database is not sorted by time nor is it downloadable). Likewise, I have updated the human population and forest area estimates to correspond to 2006 (or close).

Sheldon also laments that the Discussion section was short and did not argue in support of the findings of the study. I agree with Sheldon and regret not having elaborated in the Discussion. To this end, I have expanded the Discussion in various respects including arguments for the strengths of the manuscript. Finally, Sheldon noted that in some areas, the choropleth maps did not line up, e.g. there are apparently many sasquatch sightings in Florida but fewer bears (relative to some other states). This is also an important point, and I am grateful that Sheldon raised it. The reason for the apparent mismatch is because a simple comparison of the bears map and bigfoot map does not take into account the confounding by human population and forest area (as Sheldon notes, there is a high concentration of humans in Flordia, and hence many human- bear encounters). I have completely rewritten the paragraph in the Results section on these maps to better explain them, as requested by Sheldon. Following Sheldon’s suggestion, I have used Florida as an example in this paragraph.

I thank Sheldon once again for thoughtfully reviewing the manuscript, and I hope that the changes I have made are satisfactory, and that the reviewer feels that the manuscript may now be verified without reservations. I believe that the manuscript has been greatly improved by Sheldon’s review. Accordingly, I have added Sheldon to the Acknowledgements section of the manuscript.

Reviewer response: I have read through the updated manuscript and response letter. I have no further revisions to suggest and can recommend accepting the paper.

Decision changed: Verified

To: Rahul Raveendran

I also thank Rahul Raveendran for their comprehensive review of the manuscript.

Raveendran suggested that in the Introduction, more details are required about hominology. At Raveendran’s suggestion, I have thoroughly revised the Introduction to provide more explanation of the history and methods of hominology by citing the works of Bayanov and others. Because I have had to expand the Introduction following both Raveendran’s and Sheldon’s suggestions, the Introduction is now quite long, which I hope the reviewers can forgive. Raveendran also noted that the Introduction was missing a chronological flow from paragraph to paragraph. In the revision, I have restructed the entire Introduction to improve the narrative flow overall.

Additionally, Raveendran requested that I expand upon the couple of sentences in the manuscript which identified the American black bear as a likely candidate for many purported sasquatch sightings. Accordingly, I have developed this section into a much larger paragraph including additional references. The paragraph on the previously-published analyses by Blight and Lozier et al. has also been separated into two paragraphs, now with methodological details (including the fundamentals of ecological niche modelling), and these paragraphs have been re-written to properly convey the meaning, as suggested by Raveendran.

The final paragraph of the Introduction was also revised so that the approach taken to execute the study is now stated clearly along with the hypothesis.

Raveendran noted that parts of the Methods section of the manuscript lacked lucidity and were therefore difficult to understand. I agree that the Methods section was unnecessarily convoluted and apologise for the confusion. Following Raveendran’s suggestions, I have completely rewritten and simplified the Methods section to improve the readability.

Concerning the Results section, Raveendran suggested that this be rewritten with a view to make everyone who reads the article understand the results properly. Similar to the Methods section, I have completely rewritten, and massively simplified, the Results as suggested.

Similar to Sheldon’s review above, Raveendran noted that there are mismatches between the sasquatch sighting map and black bear map, for example in the Pacific Northwest (PNW) area. As explained above, the reason for the apparent mismatch is because a simple comparison of the bears map and bigfoot map does not take into account the confounding by human population and forest area. It is this mismatch which necessitates a model that controls for such confounding. Raveendran is right to note that my discussion of the PNW in this regard was not clear, therefore, I have removed the sentences on the PNW and completely reworked the paragaph on the choropleth maps to make all of this clearer.

Raveendran suggested that the Discussion should be improved. Following Raveendran’s suggestions, I have re-written the sentence beginnning “The present study...”; discussed the results in more detail (both with reference to the works cited in the Introduction and with reference to conservation science); and have made better links to the previously-published reports by Lozier et al. and by Blight.

Finally, I have added citations to the Methods section to substantiante the methodological framework of this study with the support of previous conservation biology studies that have used the same or similar model design (negative binomial regression).

I appreciate Raveendran’s assessment that the manuscript required thorough revisions, and I hope that I was able to address the concerns raised by Raveendran in their review, and that the reviewer feels that the manuscript may now be verified without reservations. Like with Sheldon’s review above, I believe that the manuscript has been greatly improved by

Raveendran’s review. Accordingly, I have also added Raveendran to the Acknowledgements section of the manuscript.

Reviewer response: The manuscript is vastly improved. The author has taken considerable efforts to revise it. I have added very minor suggestions for the author to consider. I appreciate the author's sincere effort.

Decision changed: Verified

Author response

To: Julie Sheldon

I thank Julie Sheldon for their thoughtful consideration of the manuscript.

Sheldon notes that the Introduction did not provide much information on black bears. In the revision, I have massively expanded the Introduction to cover the physiology and morphology of the black bear; its habitats, diet, and behaviour; natural history; and human- bear interactions. As suggested, I have also compared these black bear characteristics to reported sasquatch characteristics, considering the possibility that sasquatch sightings correlate with black bears (either because they are the same animals or distinct animals with overlapping characteristics including habitat). Because I have had to expand the Introduction following both Sheldon’s and Raveendran’s suggestions, the Introduction is now quite long, which I hope the reviewers can forgive. Sheldon noted in particular the primarily forest habitat of bears. I think this is especially important, and I received a suggestion from another reader to control for forest area in the model. Accordingly, I have replaced the variable for land area in each state/province with forest area in each state/province.

Furthermore, Sheldon noted that most of the bear population estimates were rather dated (from 2001), with others being more recent but being sourced variously from state department resources, conservation societies, and biologists and conservationists quoted in media articles. Sheldon suggested that an updated resource would improve the manuscript. To implement Sheldon’s suggestion, I performed a literature search for such a resource. To the best of my knowledge, the most recent, peer-reviewed, publicly-available single source for bear population estimates is Spencer et al., 2007 (DOI: 10.2192/1537- 6176(2007)18[217:HARTHB]2.0.CO;2), which provides estimates across US states and Canadian provinces for 2006. These data, collected from a survey of North American wildlife agencies, are still relatively dated, but represent the best improvement over the previous data I could find. Following Sheldon’s suggestion, I replaced the previous data from Hristienko and McDonald with the new, more consistent and more recent Spencer et al. data, and I have discarded the estimates from media articles etcetera. Note also that the estimates provided in Hristienko and McDonald were more approximate (i.e., ranges were given) whereas the new estimates given in Spencer et al. are more precise. If the author knows of a more precise and up to date resource than Spencer et al., I would be happy to update the analyses further. However, in this regard I have reached the limit of my art.

Crucially, Sheldon points out that the dates of the bear population data and saquatch sightings did not match (as previously noted in the limitations section of the Discussion). This is an important point, because as Sheldon notes, having accurate data are important for the goal of the study. Sheldon makes the excellent suggestion to restrict the bear and sasquatch data to the same time period. To improve the validity of the analysis, I have implemented Sheldon’s suggestion of date-matching the data. As noted above, the updated bear population data correspond to 2006. Thus, I have restricted the sasquatch sightings to just those made in 2006 (this was a rather painstaking process because the BFRO web database is not sorted by time nor is it downloadable). Likewise, I have updated the human population and forest area estimates to correspond to 2006 (or close).

Sheldon also laments that the Discussion section was short and did not argue in support of the findings of the study. I agree with Sheldon and regret not having elaborated in the Discussion. To this end, I have expanded the Discussion in various respects including arguments for the strengths of the manuscript. Finally, Sheldon noted that in some areas, the choropleth maps did not line up, e.g. there are apparently many sasquatch sightings in Florida but fewer bears (relative to some other states). This is also an important point, and I am grateful that Sheldon raised it. The reason for the apparent mismatch is because a simple comparison of the bears map and bigfoot map does not take into account the confounding by human population and forest area (as Sheldon notes, there is a high concentration of humans in Flordia, and hence many human- bear encounters). I have completely rewritten the paragraph in the Results section on these maps to better explain them, as requested by Sheldon. Following Sheldon’s suggestion, I have used Florida as an example in this paragraph.

I thank Sheldon once again for thoughtfully reviewing the manuscript, and I hope that the changes I have made are satisfactory, and that the reviewer feels that the manuscript may now be verified without reservations. I believe that the manuscript has been greatly improved by Sheldon’s review. Accordingly, I have added Sheldon to the Acknowledgements section of the manuscript.

Reviewer response: I have read through the updated manuscript and response letter. I have no further revisions to suggest and can recommend accepting the paper.

Decision changed: Verified

To: Rahul Raveendran

I also thank Rahul Raveendran for their comprehensive review of the manuscript.

Raveendran suggested that in the Introduction, more details are required about hominology. At Raveendran’s suggestion, I have thoroughly revised the Introduction to provide more explanation of the history and methods of hominology by citing the works of Bayanov and others. Because I have had to expand the Introduction following both Raveendran’s and Sheldon’s suggestions, the Introduction is now quite long, which I hope the reviewers can forgive. Raveendran also noted that the Introduction was missing a chronological flow from paragraph to paragraph. In the revision, I have restructed the entire Introduction to improve the narrative flow overall.

Additionally, Raveendran requested that I expand upon the couple of sentences in the manuscript which identified the American black bear as a likely candidate for many purported sasquatch sightings. Accordingly, I have developed this section into a much larger paragraph including additional references. The paragraph on the previously-published analyses by Blight and Lozier et al. has also been separated into two paragraphs, now with methodological details (including the fundamentals of ecological niche modelling), and these paragraphs have been re-written to properly convey the meaning, as suggested by Raveendran.

The final paragraph of the Introduction was also revised so that the approach taken to execute the study is now stated clearly along with the hypothesis.

Raveendran noted that parts of the Methods section of the manuscript lacked lucidity and were therefore difficult to understand. I agree that the Methods section was unnecessarily convoluted and apologise for the confusion. Following Raveendran’s suggestions, I have completely rewritten and simplified the Methods section to improve the readability.

Concerning the Results section, Raveendran suggested that this be rewritten with a view to make everyone who reads the article understand the results properly. Similar to the Methods section, I have completely rewritten, and massively simplified, the Results as suggested.

Similar to Sheldon’s review above, Raveendran noted that there are mismatches between the sasquatch sighting map and black bear map, for example in the Pacific Northwest (PNW) area. As explained above, the reason for the apparent mismatch is because a simple comparison of the bears map and bigfoot map does not take into account the confounding by human population and forest area. It is this mismatch which necessitates a model that controls for such confounding. Raveendran is right to note that my discussion of the PNW in this regard was not clear, therefore, I have removed the sentences on the PNW and completely reworked the paragaph on the choropleth maps to make all of this clearer.

Raveendran suggested that the Discussion should be improved. Following Raveendran’s suggestions, I have re-written the sentence beginnning “The present study...”; discussed the results in more detail (both with reference to the works cited in the Introduction and with reference to conservation science); and have made better links to the previously-published reports by Lozier et al. and by Blight.

Finally, I have added citations to the Methods section to substantiante the methodological framework of this study with the support of previous conservation biology studies that have used the same or similar model design (negative binomial regression).

I appreciate Raveendran’s assessment that the manuscript required thorough revisions, and I hope that I was able to address the concerns raised by Raveendran in their review, and that the reviewer feels that the manuscript may now be verified without reservations. Like with Sheldon’s review above, I believe that the manuscript has been greatly improved by

Raveendran’s review. Accordingly, I have also added Raveendran to the Acknowledgements section of the manuscript.

Reviewer response: The manuscript is vastly improved. The author has taken considerable efforts to revise it. I have added very minor suggestions for the author to consider. I appreciate the author's sincere effort.

Decision changed: Verified

Peer review report

Title: If it’s real, could it be an eel?

version: 2

Referee: Dr Derek W Evans

Institution: AFBINI

email: derek.evans@afbini.gov.uk

ORCID iD: 0009-0002-1981-2693

General assessment

Interesting paper and a useful attempt at answering a crypto-zoological questions using real data, although some of the data used is not quite relevant to the cold waters of Ness. There was no figure 2 included with the manuscript. A description of eel behaviour outlining how they do not swim upwards and out of the water akin to “Nessie” breaching would be useful.

Essential revisions that are required to verify the manuscript

A map of locations. Corrected inclusion of Figure 2. Some wider eel biometric data such as that to be found in any of the ICES WG Eel annual reports; reference to The eel book by Tesch 2003 for comments on eel behaviour and comments on biometry.

Other suggestions to improve the manuscript

Inclusion of some images of very large 1m plus eels for comparative purposes

Decision

Verified with reservations: The content is academically sound but has shortcomings that could be improved by further studies and/or minor revisions. By way of edits suggested above

Author response

Response to Dr Don Jellyman:

I thank Dr Jellyman for their polite and complimentary comments on the manuscript. Because no essential revisions or other suggestions were requested by Jellyman, I have not made revisions to the manuscript in response to their review.

Response to Dr Derek W Evans:

I thank also Dr Evans for their careful consideration of the manuscript.

Evans notes that “some of the data used is not quite relevant to the cold waters of Ness”. This is an important limitation. Accordingly, I have expanded the limitations paragraph of the Discussion section to read: “environmental conditions such as temperature and available biomass impact eel growth and length, therefore comparisons to other environments such as Zeeschelde may not be appropriate, i.e., some of the data cited may not be relevant to the relatively cold waters of Loch Ness.”

I apologise for causing some confusion around “Figure 2”. In the manuscript, I refer in multiple places to a “Figure 2” and a “Figure 4” which never appear in the text, e.g. “Oliver et al. (2015, Figure 2)” and “Meulenbroek et al. (2020, Figure 4).” What I meant by these references were the respective figures in those publications, i.e., “Figure 2 of Oliver et al. (2015)” and “Figure 4 of Meulenbroek et al. (2020)”. I see how my orginal wording was entirely confusing and I apologise for not making this at all clear. Correspondingly, I have revised the wording of the manuscript throughout as follows: “Oliver et al. (2015, Figure 2)” → “Figure 2 of Oliver et al. (2015)” “Simon (2007, Figure 2(b))” → “Figure 2(b) of Simon (2007)” “Melia et al. (2006, Fig. 2)” → “Fig. 2 of Melia et al. (2006)” “(Macnamara et al., 2014, Fig. 2)” → “(Fig. 2 of Macnamara et al., 2014)” “Meulenbroek et al. (2020, Figure 4)” → Figure 4 of Meulenbroek et al. (2020)”

I hope that these changes to the figure references above now make it clear that I was citing figures within other published works, rather than a missing figure in my own manuscript. Evans requested that I reference ‘The Eel’ by Tesch (2003). In this revision, I have included additional comments with citations to three chapters of ‘The Eel’, including Kloppmann (Chapter 1: Body Structure and Function, in ‘The Eel’) in the Introduction of the manuscrupt (third paragraph); Tesch and Thorpe (Chapter 2: Developmental stages and distribution of the eel species, in ‘The Eel’) in the Discussion of the manuscript (third paragraph); and Tesch and Thorpe (Chapter 3: Post-larval ecology and behaviour, in ‘The Eel’) in the Discussion of the manuscript (second paragraph).

Another suggestion was made to cite the work of the ICES Working Group on Eels (WGEEL) on eel biometric data, of which I note Evans is a member and so has extensive knowledge of this body of literature. To this end, I have added to the manuscript citations to the latest Report to ICES on the eel stock, fishery and other impacts in UK, 2020–2021, from the Joint EIFAAC/ICES/GFCM Working Group on Eels (WGEEL) Country Reports 2020–2021. I use biometric data from this report to compare Mackal’s eel lengths in Loch Ness to those collected elsewhere in Scotland in the same decade (Discussion, first paragraph) and to estimate ages of 1- and 6- meter eel specimens based on eel growth rates from a Scottish river (Discussion, second paragraph).

Finally, Evans requested a map of locations, which I have now provided links to in the online Supplementary Information (mentioned at the end of the Methods section), and inclusion of some images of very large 1-meter-plus eels for comparative purposes, which I have now also provided links to in the online Supplementary Information (mentioned in the Discussion; because I do not own the copyrights for these images, I do not feel comfortable reproducing them directly in the manuscript).

I hope that these changes are satisfactory, and that the reviewer feels that the manuscript may now be verified without reservations. I am grateful for the reviewers’ feedback, and I believe that their recommendations have greatly improved the manuscript. Accordingly, I have added Evans and Jellyman to the Acknowledgement sections of the manuscript to express my appreciation.

Response from Dr Derek W Evans

Great inclusions and revisions certainly make the paper the finished article and a genuinely interesting read – print as seen

Author response

Response to Dr Don Jellyman:

I thank Dr Jellyman for their polite and complimentary comments on the manuscript. Because no essential revisions or other suggestions were requested by Jellyman, I have not made revisions to the manuscript in response to their review.

Response to Dr Derek W Evans:

I thank also Dr Evans for their careful consideration of the manuscript.

Evans notes that “some of the data used is not quite relevant to the cold waters of Ness”. This is an important limitation. Accordingly, I have expanded the limitations paragraph of the Discussion section to read: “environmental conditions such as temperature and available biomass impact eel growth and length, therefore comparisons to other environments such as Zeeschelde may not be appropriate, i.e., some of the data cited may not be relevant to the relatively cold waters of Loch Ness.”

I apologise for causing some confusion around “Figure 2”. In the manuscript, I refer in multiple places to a “Figure 2” and a “Figure 4” which never appear in the text, e.g. “Oliver et al. (2015, Figure 2)” and “Meulenbroek et al. (2020, Figure 4).” What I meant by these references were the respective figures in those publications, i.e., “Figure 2 of Oliver et al. (2015)” and “Figure 4 of Meulenbroek et al. (2020)”. I see how my orginal wording was entirely confusing and I apologise for not making this at all clear. Correspondingly, I have revised the wording of the manuscript throughout as follows: “Oliver et al. (2015, Figure 2)” → “Figure 2 of Oliver et al. (2015)” “Simon (2007, Figure 2(b))” → “Figure 2(b) of Simon (2007)” “Melia et al. (2006, Fig. 2)” → “Fig. 2 of Melia et al. (2006)” “(Macnamara et al., 2014, Fig. 2)” → “(Fig. 2 of Macnamara et al., 2014)” “Meulenbroek et al. (2020, Figure 4)” → Figure 4 of Meulenbroek et al. (2020)”

I hope that these changes to the figure references above now make it clear that I was citing figures within other published works, rather than a missing figure in my own manuscript. Evans requested that I reference ‘The Eel’ by Tesch (2003). In this revision, I have included additional comments with citations to three chapters of ‘The Eel’, including Kloppmann (Chapter 1: Body Structure and Function, in ‘The Eel’) in the Introduction of the manuscrupt (third paragraph); Tesch and Thorpe (Chapter 2: Developmental stages and distribution of the eel species, in ‘The Eel’) in the Discussion of the manuscript (third paragraph); and Tesch and Thorpe (Chapter 3: Post-larval ecology and behaviour, in ‘The Eel’) in the Discussion of the manuscript (second paragraph).

Another suggestion was made to cite the work of the ICES Working Group on Eels (WGEEL) on eel biometric data, of which I note Evans is a member and so has extensive knowledge of this body of literature. To this end, I have added to the manuscript citations to the latest Report to ICES on the eel stock, fishery and other impacts in UK, 2020–2021, from the Joint EIFAAC/ICES/GFCM Working Group on Eels (WGEEL) Country Reports 2020–2021. I use biometric data from this report to compare Mackal’s eel lengths in Loch Ness to those collected elsewhere in Scotland in the same decade (Discussion, first paragraph) and to estimate ages of 1- and 6- meter eel specimens based on eel growth rates from a Scottish river (Discussion, second paragraph).

Finally, Evans requested a map of locations, which I have now provided links to in the online Supplementary Information (mentioned at the end of the Methods section), and inclusion of some images of very large 1-meter-plus eels for comparative purposes, which I have now also provided links to in the online Supplementary Information (mentioned in the Discussion; because I do not own the copyrights for these images, I do not feel comfortable reproducing them directly in the manuscript).

I hope that these changes are satisfactory, and that the reviewer feels that the manuscript may now be verified without reservations. I am grateful for the reviewers’ feedback, and I believe that their recommendations have greatly improved the manuscript. Accordingly, I have added Evans and Jellyman to the Acknowledgement sections of the manuscript to express my appreciation.

Author response

To Mary Anne Mercer

Introduction: The aims of the study are to “…learn from the efforts…to maintain quality EHS” (p 1) as well as to “understand the variance in service utilization, including the best practices in improving maternal mortality and SBA.” That leads the reader to expect proposed answers to these questions in the conclusion, which didn’t happen.

Thank you for your feedback on this important point. The data shows that there were improvements in maternal and child health, but not in general primary health services. So, the overall aim is to understand the preparedness activities or lack there of took place in TLS as a comparison to the efforts taken to maintain maternal/child health service. The objective has been reworded: ‘This case study examines the enabling factors, strengths, challenges and lessons learned from Timor-Leste (TLS) as it sought to maintain quality essential health services (EHS) during the COVID-19 pandemic.’

Methods: Extensive discussion of methods leaves a few important gaps, such as ages/genders of the few community informants.

The table of documents reviewed could be in an appendix. Thank you for your feedback on this. This was an area that the authors had discussed extensively, and we had originally included both gender and age, but had decided to remove this for the protection of interviewees upon several of their request.

Results: A lot of useful results are presented, but some areas need more explanation. For example, we learn about the “Twinning Partnership for Improvement” (TPI) that seems to have a substantial effort on efforts to continue or improve quality. That entity was not well described and could potentially add a lot to an understanding of what happened to services during the pandemic. How important was the role of other partners? Were they a factor explaining why maternal care services performed better than general outpatient services?

Thank you for this feedback. We have clarified the usefulness of the TPI in the discussion section. It now reads: ‘To maintain quality EHS, planning for quality must be proactively considered across all levels in service delivery packages, municipal health offices and health facilities (44). While the national level sets the priorities, municipal health offices and facilities need to participate in strategic quality planning for implementation to minimize the interoperability gap (47). As such, the authors recommend for countries to consider aligning quality planning and learning systems throughout the health system to be better prepared to maintain quality EHS (44, 47). For example, by involving all three levels of the health system, the activities taken by the maternal and child health program during COVID-19 may have contributed towards minimizing the interoperability gap. The program leaders took an integrated approach to quality by developing national guidance, disseminating, and orientating that guidance to the municipal and facility levels, while also promoting healthy behaviors in the community. Another example is the TPI, which planned quality activities through a learning system between the national, municipal and facility levels and then later shared their lessons learned in implementing IPC improvements during COVID-19 to other municipalities (33, 48). This approach had impact on a referral hospital 18 months later, which helped reduce neonatal sepsis.’ Table 4 shows the ongoing support which was provided by partners, which is can have great impact if done in alignment with the national strategic plans We have also clarified this in the discussion: ‘To better prepare health systems to maintain quality EHS during a public health emergency, the analysis highlighted several key recommendations from TLS that may be of benefit to other countries. First, developing national level strategy and/or policy for quality will set the foundation for the national quality priorities (46). From here, national leaders can inform stakeholders, national and international agencies, and all health systems levels of the quality priorities. For example, table 4 shows that several health partners in TLS had ongoing activities that are supporting quality service delivery before and during COVID-19. Ensuring alignment of these partners to the national quality priorities can greatly contribute towards maintaining quality EHS during any future health emergency.’

The large Table 4 is useful but lacks some important information. Given the decrease in infacility maternal mortality cases – what are the data on overall maternal mortality? Health facilities have some information on home deliveries, which are an important risk for mortality. Were no infant mortality data available?

Thank you for this important feedback. Table 4 is ‘intended to show a mapping between WHO Primary health care measurement framework and indicators (34), the Donabedian Framework (35) and available data in TLS, which was collected from partners, MOH programs, and health facilities. The WHO Primary health care (PHC) framework highlights that health worker density and distribution, and existing policies for example, are health service determinants which reflect the capacity of PHC services. The framework also includes health service delivery indicators (eg. processes and outputs) that may impact health system objectives (outcomes). While the WHO PHC framework monitors progress and performance in PHC, the Donabedian Framework accounts for quality at all levels of care, including secondary and tertiary. Efforts to maintain quality in TLS are highlighted in this table and show quality-focused data before and during COVID-19 that was supported by MOH programs, CQAH and partners. For example, as part of the national direction on quality, TLS had given particular attention to water sanitation and hygiene (WASH) and IPC practices prior to COVID-19, which could have had some impact on maintaining quality EHS. Table 4 also helped direct the authors to formulate the KII questionnaires to better understand what contributed to improved maternal and child health services and outcomes verses the decrease in outpatient, emergency department, and primary care service visits.’ All available data on maternal and child health was included in the table, as it relates to the maintenance of essential health services

Another important issue is identifying what happened, if anything, to health staff availability during the pandemic. In many countries, health worker numbers were substantially decreased because of COVID illness. Those data would be very useful for Table 4. Also, at the bottom of p 7, an FGD participant mentions “worried…about being sent to a COVID-19 treatment center” as a reason for not using health services. That sounds like a fear of being required to go somewhere for treatment? Would be good to clarify.

Thank you for this useful comment and we agree with the reviewer. However, up to date health worker data was not available during COVID-19. We have listed health worker distribution and density as a health system determinate in table 4, which serves as an indication for how well the health system is equipped to support through health workers. We have revised table 4 to make the objective of table 4 stronger and more apparent.

Conclusion: This section is particularly unhelpful in summarizing the results of the case study. It’s essentially a review of what the country should do in the future, rather than what was learned--what the study found had happened during the pandemic. Summarizing a few key findings would be helpful, in areas such as the role of partners, the factors that led to even better maternal care use during the pandemic, or other factors that helped or hindered service provision during the pandemic at the three levels studied. Then the conclusion as to how to use that information for the future would be more meaningful.

Thank you. We have revised the conclusion and it now reads: ‘Maintaining quality EHS is a continuous process that must be considered in health emergency preparedness planning. By setting national level priorities in quality, stakeholders, national and international agencies, and all health facilities can align quality service delivery activities, particularly when resources are strained. Integrating quality planning into preparedness plans can bring awareness and accountability to health systems leaders to ensure that quality EHS will be maintained, while also empowering facility staff to continue advocating for quality during a public health emergency.

This study is a snapshot of quality EHS in TLS during the COVID-19 pandemic and the analysis identified approaches and recommendations that may be of benefit to other countries. Even though more knowledge is needed on how to operationalize plans for maintaining quality EHS, this learning can support countries towards better quality planning for public health emergencies.’

To Supriya Mathew

The paper is well written and discusses an important topic that is also of future use. The study documents the experiences of the Timor-Leste health system as it sought to maintain quality essential health services (EHS) during the COVID-19 pandemic.

Sentence –“ no patients were involved in the recruitment…” page 6 is confusing.

Thank you for this feedback and we agree. We have since deleted and provided an updated statement. “No patients were involved in the recruitment to conduct the study or directly involved in the design of this study. The study will be made available to all participants and shared directly with MOH, municipal health departments and health facilities involved.”

Page 12 -the paper mentions that there was a decrease in service utilization- were there any quotes from the community/patient category that could be added below it?

Thank you for this comment. We have included the following: Table 4 also revealed a large decrease in OPD and emergency room visits (HNGV only). To understand the decrease in service utilization by community members, KIIs revealed that patients were not visiting emergency departments and OPD largely related to the “Lack of trust [in the health system].” [Municipal level, KII participant], “Increased fear [of the hospital].” [Facility level, FGD participant],

When probed further about their ‘lack of trust’ or ‘increased fear’, participants described family and community members, not going to the hospital because they were:

“Worried about being sent to a COVID-19 treatment centre [if becoming COVID-19 positive]” [Facility level, FGD participant].

“Essential Health Services did not work properly during COVID-19 because all staff focused on COVID-19. Chronic patients did not take their medications and they died as a result. Taxi and bus services were not running so patients could not afford to get to the hospitals for treatment.” [Facility level, KII participant]

Page 13 suddenly mentions one family member, is the quote from a community member? Were there any differences between rural and urban quality of care?

Thank you for this reflection. The quote is from a community member. We have revised to clarify: One family member said:

‘The IPC used to be great and implemented in all health facilities, but the consciousness is decreased, we can see limit of PPE and there is no social distance among health workers, and patients with health workers. This makes us not want to come’ [Facility level, KII participant]

Another family member stated:

“[My family member] has been here since morning at 10 am, however [they] did not get clinical care until 08.00 pm. After that [they] got clinical care by putting infusion.” [Facility level, KII participant]

We did not call out urban verses rural in health facilities as the capitol of Dili is mainly the urban populations and the other municipalities are largely rural.

Table 1: year of publication has been mentioned, could it be categorised into pre during and post COVID publications

Thank you for your feedback. This is a fine point. Will consider marking with an asterisk those documents that were post COVID 19.

Table 4: • Data has been provided only from 2019 onwards; is data available for at least 3 years before the COVID period?

This has been considered. We have since updated the table to make the objective of the table more clear.

May be add one more column to indicate insufficient data for comparison between years?

Thank you for the suggestion. The point of the table is not really to compare data as such, although it has been helpful to understand what services were ongoing during covid 19 verses which were not. A main objective of the table is understand what data is available to support ongoing quality activities. We have clarified the objectives of the table: ‘Developed during the document review, table 4 shows a mapping between WHO Primary health care measurement framework and indicators (34), the Donabedian Framework (35) and available data in TLS, which was collected from partners, MOH programs, and health facilities. The WHO Primary health care (PHC) framework highlights that health worker density and distribution, and existing policies for example, are health service determinants which reflect the capacity of PHC services. The framework also includes health service delivery indicators (eg. processes and outputs) that may impact health system objectives (outcomes) (see Figure 1).

While the WHO PHC framework monitors progress and performance in PHC, the Donabedian Framework accounts for quality at all levels of care, including secondary and tertiary. Efforts to maintain quality in TLS are highlighted in this table and show quality-focused data before and during COVID-19 that was supported by MOH programs, CQAH and partners. For example, as part of the national direction on quality, TLS had given particular attention to water sanitation and hygiene (WASH) and IPC practices prior to COVID-19, which could have had some impact on maintaining quality EHS. Table 4 also helped direct the authors to formulate the KII questionnaires to better understand what contributed to improved maternal and child health services and outcomes verses the decrease in outpatient, emergency department, and primary care service visits.’

The paper mentions that the patient satisfaction improved as per patient satisfaction survey. How were the surveys administered (e.g. online)? Were the sample sizes across each years comparable?

The sample sizes are comparable, and the surveys are administered in written format at the health facility.

Author response

To Mary Anne Mercer

Introduction: The aims of the study are to “…learn from the efforts…to maintain quality EHS” (p 1) as well as to “understand the variance in service utilization, including the best practices in improving maternal mortality and SBA.” That leads the reader to expect proposed answers to these questions in the conclusion, which didn’t happen.

Thank you for your feedback on this important point. The data shows that there were improvements in maternal and child health, but not in general primary health services. So, the overall aim is to understand the preparedness activities or lack there of took place in TLS as a comparison to the efforts taken to maintain maternal/child health service. The objective has been reworded: ‘This case study examines the enabling factors, strengths, challenges and lessons learned from Timor-Leste (TLS) as it sought to maintain quality essential health services (EHS) during the COVID-19 pandemic.’

Methods: Extensive discussion of methods leaves a few important gaps, such as ages/genders of the few community informants.

The table of documents reviewed could be in an appendix. Thank you for your feedback on this. This was an area that the authors had discussed extensively, and we had originally included both gender and age, but had decided to remove this for the protection of interviewees upon several of their request.

Results: A lot of useful results are presented, but some areas need more explanation. For example, we learn about the “Twinning Partnership for Improvement” (TPI) that seems to have a substantial effort on efforts to continue or improve quality. That entity was not well described and could potentially add a lot to an understanding of what happened to services during the pandemic. How important was the role of other partners? Were they a factor explaining why maternal care services performed better than general outpatient services?

Thank you for this feedback. We have clarified the usefulness of the TPI in the discussion section. It now reads: ‘To maintain quality EHS, planning for quality must be proactively considered across all levels in service delivery packages, municipal health offices and health facilities (44). While the national level sets the priorities, municipal health offices and facilities need to participate in strategic quality planning for implementation to minimize the interoperability gap (47). As such, the authors recommend for countries to consider aligning quality planning and learning systems throughout the health system to be better prepared to maintain quality EHS (44, 47). For example, by involving all three levels of the health system, the activities taken by the maternal and child health program during COVID-19 may have contributed towards minimizing the interoperability gap. The program leaders took an integrated approach to quality by developing national guidance, disseminating, and orientating that guidance to the municipal and facility levels, while also promoting healthy behaviors in the community. Another example is the TPI, which planned quality activities through a learning system between the national, municipal and facility levels and then later shared their lessons learned in implementing IPC improvements during COVID-19 to other municipalities (33, 48). This approach had impact on a referral hospital 18 months later, which helped reduce neonatal sepsis.’ Table 4 shows the ongoing support which was provided by partners, which is can have great impact if done in alignment with the national strategic plans We have also clarified this in the discussion: ‘To better prepare health systems to maintain quality EHS during a public health emergency, the analysis highlighted several key recommendations from TLS that may be of benefit to other countries. First, developing national level strategy and/or policy for quality will set the foundation for the national quality priorities (46). From here, national leaders can inform stakeholders, national and international agencies, and all health systems levels of the quality priorities. For example, table 4 shows that several health partners in TLS had ongoing activities that are supporting quality service delivery before and during COVID-19. Ensuring alignment of these partners to the national quality priorities can greatly contribute towards maintaining quality EHS during any future health emergency.’

The large Table 4 is useful but lacks some important information. Given the decrease in infacility maternal mortality cases – what are the data on overall maternal mortality? Health facilities have some information on home deliveries, which are an important risk for mortality. Were no infant mortality data available?

Thank you for this important feedback. Table 4 is ‘intended to show a mapping between WHO Primary health care measurement framework and indicators (34), the Donabedian Framework (35) and available data in TLS, which was collected from partners, MOH programs, and health facilities. The WHO Primary health care (PHC) framework highlights that health worker density and distribution, and existing policies for example, are health service determinants which reflect the capacity of PHC services. The framework also includes health service delivery indicators (eg. processes and outputs) that may impact health system objectives (outcomes). While the WHO PHC framework monitors progress and performance in PHC, the Donabedian Framework accounts for quality at all levels of care, including secondary and tertiary. Efforts to maintain quality in TLS are highlighted in this table and show quality-focused data before and during COVID-19 that was supported by MOH programs, CQAH and partners. For example, as part of the national direction on quality, TLS had given particular attention to water sanitation and hygiene (WASH) and IPC practices prior to COVID-19, which could have had some impact on maintaining quality EHS. Table 4 also helped direct the authors to formulate the KII questionnaires to better understand what contributed to improved maternal and child health services and outcomes verses the decrease in outpatient, emergency department, and primary care service visits.’ All available data on maternal and child health was included in the table, as it relates to the maintenance of essential health services

Another important issue is identifying what happened, if anything, to health staff availability during the pandemic. In many countries, health worker numbers were substantially decreased because of COVID illness. Those data would be very useful for Table 4. Also, at the bottom of p 7, an FGD participant mentions “worried…about being sent to a COVID-19 treatment center” as a reason for not using health services. That sounds like a fear of being required to go somewhere for treatment? Would be good to clarify.

Thank you for this useful comment and we agree with the reviewer. However, up to date health worker data was not available during COVID-19. We have listed health worker distribution and density as a health system determinate in table 4, which serves as an indication for how well the health system is equipped to support through health workers. We have revised table 4 to make the objective of table 4 stronger and more apparent.

Conclusion: This section is particularly unhelpful in summarizing the results of the case study. It’s essentially a review of what the country should do in the future, rather than what was learned--what the study found had happened during the pandemic. Summarizing a few key findings would be helpful, in areas such as the role of partners, the factors that led to even better maternal care use during the pandemic, or other factors that helped or hindered service provision during the pandemic at the three levels studied. Then the conclusion as to how to use that information for the future would be more meaningful.

Thank you. We have revised the conclusion and it now reads: ‘Maintaining quality EHS is a continuous process that must be considered in health emergency preparedness planning. By setting national level priorities in quality, stakeholders, national and international agencies, and all health facilities can align quality service delivery activities, particularly when resources are strained. Integrating quality planning into preparedness plans can bring awareness and accountability to health systems leaders to ensure that quality EHS will be maintained, while also empowering facility staff to continue advocating for quality during a public health emergency.

This study is a snapshot of quality EHS in TLS during the COVID-19 pandemic and the analysis identified approaches and recommendations that may be of benefit to other countries. Even though more knowledge is needed on how to operationalize plans for maintaining quality EHS, this learning can support countries towards better quality planning for public health emergencies.’

To Supriya Mathew

The paper is well written and discusses an important topic that is also of future use. The study documents the experiences of the Timor-Leste health system as it sought to maintain quality essential health services (EHS) during the COVID-19 pandemic.

Sentence –“ no patients were involved in the recruitment…” page 6 is confusing.

Thank you for this feedback and we agree. We have since deleted and provided an updated statement. “No patients were involved in the recruitment to conduct the study or directly involved in the design of this study. The study will be made available to all participants and shared directly with MOH, municipal health departments and health facilities involved.”

Page 12 -the paper mentions that there was a decrease in service utilization- were there any quotes from the community/patient category that could be added below it?

Thank you for this comment. We have included the following: Table 4 also revealed a large decrease in OPD and emergency room visits (HNGV only). To understand the decrease in service utilization by community members, KIIs revealed that patients were not visiting emergency departments and OPD largely related to the “Lack of trust [in the health system].” [Municipal level, KII participant], “Increased fear [of the hospital].” [Facility level, FGD participant],

When probed further about their ‘lack of trust’ or ‘increased fear’, participants described family and community members, not going to the hospital because they were:

“Worried about being sent to a COVID-19 treatment centre [if becoming COVID-19 positive]” [Facility level, FGD participant].

“Essential Health Services did not work properly during COVID-19 because all staff focused on COVID-19. Chronic patients did not take their medications and they died as a result. Taxi and bus services were not running so patients could not afford to get to the hospitals for treatment.” [Facility level, KII participant]

Page 13 suddenly mentions one family member, is the quote from a community member? Were there any differences between rural and urban quality of care?

Thank you for this reflection. The quote is from a community member. We have revised to clarify: One family member said:

‘The IPC used to be great and implemented in all health facilities, but the consciousness is decreased, we can see limit of PPE and there is no social distance among health workers, and patients with health workers. This makes us not want to come’ [Facility level, KII participant]

Another family member stated:

“[My family member] has been here since morning at 10 am, however [they] did not get clinical care until 08.00 pm. After that [they] got clinical care by putting infusion.” [Facility level, KII participant]

We did not call out urban verses rural in health facilities as the capitol of Dili is mainly the urban populations and the other municipalities are largely rural.

Table 1: year of publication has been mentioned, could it be categorised into pre during and post COVID publications

Thank you for your feedback. This is a fine point. Will consider marking with an asterisk those documents that were post COVID 19.

Table 4: • Data has been provided only from 2019 onwards; is data available for at least 3 years before the COVID period?

This has been considered. We have since updated the table to make the objective of the table more clear.

May be add one more column to indicate insufficient data for comparison between years?

Thank you for the suggestion. The point of the table is not really to compare data as such, although it has been helpful to understand what services were ongoing during covid 19 verses which were not. A main objective of the table is understand what data is available to support ongoing quality activities. We have clarified the objectives of the table: ‘Developed during the document review, table 4 shows a mapping between WHO Primary health care measurement framework and indicators (34), the Donabedian Framework (35) and available data in TLS, which was collected from partners, MOH programs, and health facilities. The WHO Primary health care (PHC) framework highlights that health worker density and distribution, and existing policies for example, are health service determinants which reflect the capacity of PHC services. The framework also includes health service delivery indicators (eg. processes and outputs) that may impact health system objectives (outcomes) (see Figure 1).

While the WHO PHC framework monitors progress and performance in PHC, the Donabedian Framework accounts for quality at all levels of care, including secondary and tertiary. Efforts to maintain quality in TLS are highlighted in this table and show quality-focused data before and during COVID-19 that was supported by MOH programs, CQAH and partners. For example, as part of the national direction on quality, TLS had given particular attention to water sanitation and hygiene (WASH) and IPC practices prior to COVID-19, which could have had some impact on maintaining quality EHS. Table 4 also helped direct the authors to formulate the KII questionnaires to better understand what contributed to improved maternal and child health services and outcomes verses the decrease in outpatient, emergency department, and primary care service visits.’

The paper mentions that the patient satisfaction improved as per patient satisfaction survey. How were the surveys administered (e.g. online)? Were the sample sizes across each years comparable?

The sample sizes are comparable, and the surveys are administered in written format at the health facility.

Peer review report

Title: An Improved Peer-Review System to Compensate for Scientific Misconduct in Health-Sensitive Topics

version: 7

Referee: Ksenija Bazdaric

Institution: Rijeka University

email: ksenija.bazdaric@fzsri.uniri.hr

ORCID iD: 0000-0002-2977-3686

General assessment

Overall, it is an interesting manuscript/opinion paper that deserves attention. However, I think some points are not realistic at the moment and applicable to most of the medical journals.

Essential revisions that are required to verify the manuscript

A - I would add some data about author’s responsibility in terms of research integrity, not only sharing data and analysis.

A4 - It is not clear how would obligatory preprint improve quality because we have data that only 1% of preprints are reviewed.

J – the mechanisms you suggest are reserved only for major journals who have a lot of paid stuff. How can a small journal (published by a society), where not even editors are paid, not to mention reviewers, ensure better quality?

I1 – it is not completely clear what do you mean here and how is this connected to research integrity.

Other suggestions to improve the manuscript

A2 - frameworks for enhancing quality in preclinical data - please be more specific, insert a reference or a guideline.

A3 - authors are already obliged to put the study protocol in a registry, like clinicaltrials.gov, so I suppose this refers to other types of studies.

Decision

Verified with reservations: The content is academically sound but has shortcomings that could be improved by further studies and/or minor revisions.

Peer review report

Title: An Improved Peer-Review System to Compensate for Scientific Misconduct in Health-Sensitive Topics

version: 7

Referee: Ksenija Bazdaric

Institution: Rijeka University

email: ksenija.bazdaric@fzsri.uniri.hr

ORCID iD: 0000-0002-2977-3686

General assessment

Overall, it is an interesting manuscript/opinion paper that deserves attention. However, I think some points are not realistic at the moment and applicable to most of the medical journals.

Essential revisions that are required to verify the manuscript

A - I would add some data about author’s responsibility in terms of research integrity, not only sharing data and analysis.

A4 - It is not clear how would obligatory preprint improve quality because we have data that only 1% of preprints are reviewed.

J – the mechanisms you suggest are reserved only for major journals who have a lot of paid stuff. How can a small journal (published by a society), where not even editors are paid, not to mention reviewers, ensure better quality?

I1 – it is not completely clear what do you mean here and how is this connected to research integrity.

Other suggestions to improve the manuscript

A2 - frameworks for enhancing quality in preclinical data - please be more specific, insert a reference or a guideline.

A3 - authors are already obliged to put the study protocol in a registry, like clinicaltrials.gov, so I suppose this refers to other types of studies.

Decision

Verified with reservations: The content is academically sound but has shortcomings that could be improved by further studies and/or minor revisions.

Peer review report

Title: Publication Patterns and Perceptions of Open Science in Indian Scholarly Community: Insights from a Survey

version: 2

Referee: Moumita Koley

Institution: Indian Institute of Science, Bangalore, India

email: moumitakoley@iisc.ac.in

ORCID iD: 0000-0003-2394-0663

General assessment

The author of the article provides a limited perspective on Publication Patterns and Perceptions of Open Science in the Indian Scholarly Community. This article sheds some light on open science practices, considering the scarcity of data on this topic. The main drawbacks, the survey conducted in the article is the only methodology used, and it is limited to a small group of researchers. Understanding publication patterns (Open Access publication and use of preprints) should be made using bibliometric studies. Another drawback of the article is that it primarily focuses on researchers from the agriculture field, which is over-represented and makes it misleading to claim that the study represents the entire Indian scholarly community. This is particularly problematic since the physical and chemical sciences dominate the Indian research community, and data from these fields are entirely absent in the article.

Essential revisions that are required to verify the manuscript

This study is at an early stage; more data points and representations of various fields are necessary to claim validity. Moreover, a mixed-method approach is more suitable.

Other suggestions to improve the manuscript

The introduction section mentions several preprint servers, but some are not operational. For example, ArabiXiv is not accepting new submissions, and IndiaRxiv has few submissions. Since there are limited responses from other South Asian countries, the author has chosen to focus on India. However, it is unclear if the percentages of career levels of professionals are representative of India alone.

One drawback of this article is the over-representation of researchers from the agriculture field. In the Indian ecosystem, agricultural research institutes are separate from the general university system and governed by different funding and governance systems. Therefore, the norms and practices can vary significantly.

In the STEM subjects, Chemical Sciences contributed the most publications in India from 2015-2019, followed by Physical Sciences. Biological Sciences had fewer publications during this period(the method used for this statement: a quick search in the Web of Science). However, this data does not represent the publication behaviour of the major constituents (Physical and Chemical Science) of the Indian academic community. Therefore, it is suggested to shift the narrative towards agricultural science.

The statement "patent and scholarly data website, India has produced 19,76,966 scholarly works till date" lacks a timeframe.

The APC section statements are unclear. As far as current knowledge goes, no study has shown a correlation between JIF and APC. The statement "Was it because of JIFs they must publish in Open Access when there is Green Route to Open Access (depositing in subject or institutional repositories)" is unclear. Moreover, while Indian national funding agencies have green OA mandates, they have not been enforced. Several studies indicate that Indian researchers' adoption of green OA is low, so authors have no obligation to publish OA.

The article concludes that advocacy is needed, but it is essential to understand the research assessment frameworks of Indian academia and funding agencies. Without recognition in assessment, researchers are unlikely to adopt Preprints. Preprint submission is becoming the norm in some disciplines, which may improve the situation.

Decision

Requires revisions: The manuscript contains objective errors or fundamental flaws that must be addressed and/or major revisions are suggested.

Peer review report

Title: Publication Patterns and Perceptions of Open Science in Indian Scholarly Community: Insights from a Survey

version: 2

Referee: Subhash C. Lakhotia

Institution: Banaras Hindu University

email: Lakhotia@bhu.ac.in

ORCID iD: 0000-0003-1842-8411

General assessment

This manuscript covers an important issue about publication practices followed by science researchers in India and neighboring countries. The study is based on a survey conducted by authors.

Essential revisions that are required to verify the manuscript

While the inferences are interpreted to reflect perceptions of researchers in the India, their general applicability is greatly limited by the very small sample size (few hundreds) for a big country like India. Moreover, a majority of the limited responders are from a specific area (Agriculture), which further limits a wider applicability of the survey results.

Other suggestions to improve the manuscript

With respect to the presentation of results, the authors have mixed the survey results with what should be done to 'rectify' the weaknesses or inappropriate practices as perceived by the authors. It would be better if the two components are kept distinct so that the carry-home message from the study can be clearly followed.

Decision

Requires revisions: The manuscript contains objective errors or fundamental flaws that must be addressed and/or major revisions are suggested.

Peer review report

Title: If it’s real, could it be an eel?

version: 2

Referee: Dr Derek W Evans

Institution: AFBINI

email: derek.evans@afbini.gov.uk

ORCID iD: 0009-0002-1981-2693

General assessment

Interesting paper and a useful attempt at answering a crypto-zoological questions using real data, although some of the data used is not quite relevant to the cold waters of Ness. There was no figure 2 included with the manuscript. A description of eel behaviour outlining how they do not swim upwards and out of the water akin to “Nessie” breaching would be useful.

Essential revisions that are required to verify the manuscript

A map of locations. Corrected inclusion of Figure 2. Some wider eel biometric data such as that to be found in any of the ICES WG Eel annual reports; reference to The eel book by Tesch 2003 for comments on eel behaviour and comments on biometry.

Other suggestions to improve the manuscript

Inclusion of some images of very large 1m plus eels for comparative purposes

Decision

Verified with reservations: The content is academically sound but has shortcomings that could be improved by further studies and/or minor revisions. By way of edits suggested above

Peer review report

Title: Maintained imbalance of triglycerides, apolipoproteins, energy metabolites and cytokines in long-term COVID-19 syndrome (LTCS) patients

version: 1

Referee: Christopher Gerner

Institution: University of Vienna

email: Christopher.gerner@univie.ac.at

ORCID iD: 0000-0003-4964-0642

General assessment

The manuscript of Berezhnoy et al. is a well written report regarding metabolomics and cytokines in long term COVID-19 syndrome patients. The applied methodology is of some interest and I cannot detect methodological errors. However, I have some concerns which need to be addressed before the manuscript should sent for journal publication.

Most importantly, the manuscript did not adhere to good scientific practice regarding literature research. There are papers about LTCS patients published more than a year ago following highly similar research strategies with quite similar results. It is not sufficient to cite them in subordinate clauses in the Discussion, they need to be cited in the Introduction accordingly, as well when discussing the results. Indeed, relevant similarities in the results would deserve some discussion, such as the dysregulation of cytokines in LTCS.

Another weakness is the structure of data interpretation. It is mentioned in the Introduction that several cell types were reported to show altered metabolism after a COVID-19 infection. I cannot see how plasma analysis should allow to verify such observations as it represents a mixture of all cell types in the body. This obvious challenge regarding data interpretation should be addressed.

This is in line with another weak aspect. The numerous findings a reported without a clear structure regarding potential pathomechanisms. As such, the manuscript is sometimes not easy to read.

To sum up, the manuscript reports interesting analysis results largely corroborating previous results, which deserved publication after some essential improvements.

Essential revisions that are required to verify the manuscript

An improved appreciation of existing literature is essential, as well as an improved data interpretation.

Other suggestions to improve the manuscript

A discussion of the pros and cons of NMR-based metabolomics in contrast to other techniques would be helpful.

Decision

Verified with reservations: The content is academically sound but has shortcomings that could be improved by further studies and/or minor revisions.

Peer review report

Title: Maintained imbalance of triglycerides, apolipoproteins, energy metabolites and cytokines in long-term COVID-19 syndrome (LTCS) patients

version: 1

Referee: Paola Turano

Institution: University of Florence

email: turano@cerm.unifi.it

ORCID iD: 0000-0002-7683-8614

General assessment

This is an integrated study reporting NMR-based metabolomics data and flow cytometry-based cytokine in the blood of 125 individuals (healthy controls (HC; n=73), COVID-19-recovered (n=12), COVID-19 acute (n=7) and LTCS (n=33)).

The main goal appears to be that of demonstrating alterations in the metabolome and immune markers of patients with long COVID. This condition is defined as the continuation or development of new symptoms 3 months after the initial SARS-CoV-2 infection, with these symptoms lasting for at least 2 months with no other explanation.

As admitted by the authors, the 4 groups are very unbalanced in terms of numbers of enrolled subjects; moreover, all numbers are low but those in the recovered groups and even more in the acute phase are extremely low. Therefore, the only reliable comparison appears to be that between HC and LTCS. And this is a pity because the most important comparison to define the signature associated with long-COVID symptoms would have been the one between recovered and LTCS subjects.

Another problem is that there is no information on the status of the LTCS before infection nor during the acute phase. This, combined with the low number of individuals, does not allow to draw a real trajectory of the alterations during the observed time line. It is therefore difficult to be 100% sure that alterations in certain metabolites of lipoproteins are a consequence of LTCS or instead intrinsic characteristics of a group of individual that make them more prone to develop LTCS.

These critical aspects have nothing to do with the experimental approach, which is powerful and carefully performed. Unfortunately, the available cohort is not the best to achieve the goal of a molecular characterization of LTCS.

In any case the present manuscript provides useful hints to be further investigated in future studies and therefore might deserve publication.

Essential revisions that are required to verify the manuscript

If it were possible to enlarge the cohort of patients, confirming the observed trends, this would lead to a significant improvement in the impact of the work. But I understand the practical difficulties in achieving this goal.

Decision

Verified with reservations: The content is academically sound but has shortcomings that could be improved by further studies and/or minor revisions.

Peer review report

Title: Maintained imbalance of triglycerides, apolipoproteins, energy metabolites and cytokines in long-term COVID-19 syndrome (LTCS) patients

version: 1

Referee: Christopher Gerner

Institution: University of Vienna

email: Christopher.gerner@univie.ac.at

ORCID iD: 0000-0003-4964-0642

General assessment

The manuscript of Berezhnoy et al. is a well written report regarding metabolomics and cytokines in long term COVID-19 syndrome patients. The applied methodology is of some interest and I cannot detect methodological errors. However, I have some concerns which need to be addressed before the manuscript should sent for journal publication.

Most importantly, the manuscript did not adhere to good scientific practice regarding literature research. There are papers about LTCS patients published more than a year ago following highly similar research strategies with quite similar results. It is not sufficient to cite them in subordinate clauses in the Discussion, they need to be cited in the Introduction accordingly, as well when discussing the results. Indeed, relevant similarities in the results would deserve some discussion, such as the dysregulation of cytokines in LTCS.

Another weakness is the structure of data interpretation. It is mentioned in the Introduction that several cell types were reported to show altered metabolism after a COVID-19 infection. I cannot see how plasma analysis should allow to verify such observations as it represents a mixture of all cell types in the body. This obvious challenge regarding data interpretation should be addressed.

This is in line with another weak aspect. The numerous findings a reported without a clear structure regarding potential pathomechanisms. As such, the manuscript is sometimes not easy to read.

To sum up, the manuscript reports interesting analysis results largely corroborating previous results, which deserved publication after some essential improvements.

Essential revisions that are required to verify the manuscript

An improved appreciation of existing literature is essential, as well as an improved data interpretation.

Other suggestions to improve the manuscript

A discussion of the pros and cons of NMR-based metabolomics in contrast to other techniques would be helpful.

Decision

Verified with reservations: The content is academically sound but has shortcomings that could be improved by further studies and/or minor revisions.

Peer review report

Title: If it’s there, could it be a bear?

version: 2

Referee: Rahul Raveendran

Institution: Biodiversity Institute, University of Kansas

email: rahulravi777@gmail.com

General assessment

The manuscript needs to be revised thoroughly.

Essential revisions that are required to verify the manuscript

I feel that the introduction can be a little more elaborative. My suggestions are as follows:

• In the first paragraph, the author can give more details about ‘hominology’ by citing the works of Dmitry Bayanov. This is to give a historical account of ‘hominid research’ to the readers who are unfamiliar to this topic.

• Second paragraph has information related to the ‘misdeeds’ of the ‘proponents of hominology’. According to me, there must be continuous flow of information from paragraph to paragraph. Currently, I do not see a proper chronological flow of details in 1st and 2nd paragraph. I request the learned author to check this in such a way that 1st para must provide details about ‘hominids’, ‘hominology’, and the 2nd para must give the scientific explanation about these ‘controversial findings’.

• Line numbers 43-44: This paragraph must be expanded, and possibly include more information about ‘American black bear’ being misrecognized as ‘bigfoot sightings’ with references. If available, provide details regarding the molecular/clinical test results (i.e., references).

• A separate paragraph has to be incorporated to detail the methods adopted by scientists/researchers to link the population density of American black bear and bigfoot sightings.

• Line numbers 49-51: The sentence “No positive correlation between……a small proportion of all sightings” has to be re-written as I think that it does not convey its meaning properly.

• Provide the fundamentals of ecological niche modeling. How such a concept can be adopted in this sort of a study with a strong emphasis on the results of Lozier et al. (2009) would be helpful for the readers.

• In the last paragraph of the introduction, although not in detail, the author should state clearly the approach that was taken to execute the study. For example, details related to the chosen statistical methods with references. And state your hypothesis clearly.

Materials and Methods

• Line numbers 67-77: Please make these sentences more lucid. I feel that this paragraph lacks coherence.

• Line numbers 90-94: Please make these sentences more understandable.

• Line numbers 113-116: Please re-write these sentences to make them more understandable. Results

• Line numbers 121-123: The article states that both the sasquatch sighting and black bear population maps are strongly coloured in the Pacific Northwest area……”. BUT, in PNW, I do not think that bigfoot sightings in British Columbia are proportional to the black bear population.

• Presentation of results is a bit confusing for me. I would suggest to rewrite the results with a view to make everyone who reads this article understands the results properly.

Discussion

• Discussion must be vastly improved

a) It is difficult to understand the very first sentence of the discussion that starts with “The present study regressed ………………..”. Please re-write it.

b) Results of the present study should be discussed in detail, linking previous published reports.

c) The models employed must be discussed in detail with the support of previous reports to substantiate the conceptual correctness of the methodological framework.

Decision

Requires revisions: Major revisions are suggested.

Peer review report

Title: If it’s there, could it be a bear?

version: 2

Referee: Rahul Raveendran

Institution: Biodiversity Institute, University of Kansas

email: rahulravi777@gmail.com

General assessment

The manuscript needs to be revised thoroughly.

Essential revisions that are required to verify the manuscript

I feel that the introduction can be a little more elaborative. My suggestions are as follows:

• In the first paragraph, the author can give more details about ‘hominology’ by citing the works of Dmitry Bayanov. This is to give a historical account of ‘hominid research’ to the readers who are unfamiliar to this topic.

• Second paragraph has information related to the ‘misdeeds’ of the ‘proponents of hominology’. According to me, there must be continuous flow of information from paragraph to paragraph. Currently, I do not see a proper chronological flow of details in 1st and 2nd paragraph. I request the learned author to check this in such a way that 1st para must provide details about ‘hominids’, ‘hominology’, and the 2nd para must give the scientific explanation about these ‘controversial findings’.

• Line numbers 43-44: This paragraph must be expanded, and possibly include more information about ‘American black bear’ being misrecognized as ‘bigfoot sightings’ with references. If available, provide details regarding the molecular/clinical test results (i.e., references).

• A separate paragraph has to be incorporated to detail the methods adopted by scientists/researchers to link the population density of American black bear and bigfoot sightings.

• Line numbers 49-51: The sentence “No positive correlation between……a small proportion of all sightings” has to be re-written as I think that it does not convey its meaning properly.

• Provide the fundamentals of ecological niche modeling. How such a concept can be adopted in this sort of a study with a strong emphasis on the results of Lozier et al. (2009) would be helpful for the readers.

• In the last paragraph of the introduction, although not in detail, the author should state clearly the approach that was taken to execute the study. For example, details related to the chosen statistical methods with references. And state your hypothesis clearly.

Materials and Methods

• Line numbers 67-77: Please make these sentences more lucid. I feel that this paragraph lacks coherence.

• Line numbers 90-94: Please make these sentences more understandable.

• Line numbers 113-116: Please re-write these sentences to make them more understandable. Results

• Line numbers 121-123: The article states that both the sasquatch sighting and black bear population maps are strongly coloured in the Pacific Northwest area……”. BUT, in PNW, I do not think that bigfoot sightings in British Columbia are proportional to the black bear population.

• Presentation of results is a bit confusing for me. I would suggest to rewrite the results with a view to make everyone who reads this article understands the results properly.

Discussion

• Discussion must be vastly improved

a) It is difficult to understand the very first sentence of the discussion that starts with “The present study regressed ………………..”. Please re-write it.

b) Results of the present study should be discussed in detail, linking previous published reports.

c) The models employed must be discussed in detail with the support of previous reports to substantiate the conceptual correctness of the methodological framework.

Decision

Requires revisions: Major revisions are suggested.

Peer review report

Title: If it’s there, could it be a bear?

version: 2

Referee: Julie Sheldon

Institution: University of Tennessee

email: jsheldo3@tennessee.edu

ORCID iD: https://orcid.org/0000-0003-2813-3027

General assessment

This manuscript is a collection of statistical analyses attempting to show that sasquatch sightings correlate with black bear populations, and humans may be mistaking black bears for sasquatch.

The author effectively introduces the topic, provides adequate background on sasquatch, but does not provide much on black bear populations, natural history, or human-bear interactions.

The author performs several statistical tests to support the findings. I am not a statistician, but the tests seem valid. The data used for the statistical analyses, however, are not ideal. The resource (Hristienko and McDonald) provided for obtaining black bear populations was published in 2007 and the data was from 2001 via “subjective extrapolations” and “expert opinions”. Thus, this resource is outdated and suboptimal as black bear populations have changed over time. A more updated resource with more scientific methods in data collection would improve this manuscript since having as accurate as possible bear population estimates is very important for the goal of this study. The author notes this briefly in the limitations. If the human population and sasquatch sighting data matched up with the dates of bear population estimates, it would be more valid (just outdated), but there are no date ranges of human or sasquatch data provided in the manuscript.

In the results, the maps of bigfoot sightings and black bear population do not appear to correlate visually, which downplays the value of the statistical analysis. The stats should support the visual data and vice versa if the study is sound. Perhaps more updated bear population data will improve this.

The discussion is short and briefly brings up important points that can invalidate the study without much discussion or argument supporting the findings of this study.

Essential revisions that are required to verify the manuscript

I recommend the following to improve the manuscript enough to consider it valid:

Date-match the bear population, human population, and bigfoot sightings to improve the validity of the data analysis. One way to do this is to use data from the same 10-year period only.

Improve the sources of bear population information.

Expand the discussion to include reasons and ideas the maps don’t line up like the statistical analyses do – ie bears in Florida and the southeast.

Other suggestions to improve the manuscript

I recommend provide some information on black bear population/natural history in the introduction – ie what sort of habitats do black bears live in. Consider the possibility that sasquatch sightings may correlate with a type of habitat (ie forest), which happen to also correlate with black bear habitat. This may support the idea that sasquatch sightings are bears, or that sasquatch also likes to live in similar habitats as bears.

The author reports that black bears are not prominent in Florida; however, there are > 4,000 black bears bears in Florida, that are reportedly large, and it may be worth considering this as a reason for the concentration of sasquatch sightings in Florida as seen on the map. More accurate black bear data as discussed above may help improve this aspect. Experientially, there is also a high concentration of black bears in the southeastern US, where there is also a high concentration of humans and human-bear encounters. The author does not discuss this along with the number of sasquatch sightings in this region as seen on the map.

Decision

Verified with reservations: The content is academically sound but has shortcomings that could be improved by further studies and/or minor revisions.

Peer review report

Title: If it’s there, could it be a bear?

version: 2

Referee: Julie Sheldon

Institution: University of Tennessee

email: jsheldo3@tennessee.edu

ORCID iD: https://orcid.org/0000-0003-2813-3027

General assessment

This manuscript is a collection of statistical analyses attempting to show that sasquatch sightings correlate with black bear populations, and humans may be mistaking black bears for sasquatch.

The author effectively introduces the topic, provides adequate background on sasquatch, but does not provide much on black bear populations, natural history, or human-bear interactions.

The author performs several statistical tests to support the findings. I am not a statistician, but the tests seem valid. The data used for the statistical analyses, however, are not ideal. The resource (Hristienko and McDonald) provided for obtaining black bear populations was published in 2007 and the data was from 2001 via “subjective extrapolations” and “expert opinions”. Thus, this resource is outdated and suboptimal as black bear populations have changed over time. A more updated resource with more scientific methods in data collection would improve this manuscript since having as accurate as possible bear population estimates is very important for the goal of this study. The author notes this briefly in the limitations. If the human population and sasquatch sighting data matched up with the dates of bear population estimates, it would be more valid (just outdated), but there are no date ranges of human or sasquatch data provided in the manuscript.

In the results, the maps of bigfoot sightings and black bear population do not appear to correlate visually, which downplays the value of the statistical analysis. The stats should support the visual data and vice versa if the study is sound. Perhaps more updated bear population data will improve this.

The discussion is short and briefly brings up important points that can invalidate the study without much discussion or argument supporting the findings of this study.

Essential revisions that are required to verify the manuscript

I recommend the following to improve the manuscript enough to consider it valid:

Date-match the bear population, human population, and bigfoot sightings to improve the validity of the data analysis. One way to do this is to use data from the same 10-year period only.

Improve the sources of bear population information.

Expand the discussion to include reasons and ideas the maps don’t line up like the statistical analyses do – ie bears in Florida and the southeast.

Other suggestions to improve the manuscript

I recommend provide some information on black bear population/natural history in the introduction – ie what sort of habitats do black bears live in. Consider the possibility that sasquatch sightings may correlate with a type of habitat (ie forest), which happen to also correlate with black bear habitat. This may support the idea that sasquatch sightings are bears, or that sasquatch also likes to live in similar habitats as bears.

The author reports that black bears are not prominent in Florida; however, there are > 4,000 black bears bears in Florida, that are reportedly large, and it may be worth considering this as a reason for the concentration of sasquatch sightings in Florida as seen on the map. More accurate black bear data as discussed above may help improve this aspect. Experientially, there is also a high concentration of black bears in the southeastern US, where there is also a high concentration of humans and human-bear encounters. The author does not discuss this along with the number of sasquatch sightings in this region as seen on the map.

Decision

Verified with reservations: The content is academically sound but has shortcomings that could be improved by further studies and/or minor revisions.

Peer review report

Title: Types of Arrhythmias and the risk of sudden cardiac death in dialysis patients: A Systematic Review and Meta-analysis

version: 1

Referee: Milaras Nikias

Institution: National and Kapodistrian University of Athens- Ippokrateion Hospital

email: nikiasmil@med.uoa.gr

ORCID iD: 0000-0001-7312-0976

General assessment

It is now well known that high cardiovascular mortality in ESRD patients is only partly due to atherothrombotic events. Ventricular tachyarrhythmias and electromechanical dissociation account for a significant amount of those deaths as was reported in landmark trials such as the MADIT II. VT or VF might be the mode of death in only a minority of those patients and this is extrapolated from the fact that ICD implantation in this population does not extend survival, whether due to high competing comorbidities or due to electromechanical dissociation being the cause of death. It is true that ESRD patients are underrepresented in such studies due to the high competing factor for non-cardiac death and no safe conclusion can yet be drawn. It remains yet to be seen whether a better risk stratification algorithm through Holter monitoring or programmed ventricular stimulation can unveil those truly at high risk for SCD.

This meta-analysis tries to unveil the mode of death and the high cardiovascular mortality in renal failure through a thorough literature search that included 11 studies. This systematic review/meta-analysis follows current writing and reporting guidelines.

The English used is adequate although some parts of the manuscript could be refined (eg 3rd paragraph in Introduction)

Essential revisions that are required to verify the manuscript

ESRD and ESKD are both discussed in the manuscript. I would personally prefer that the authors devoted more effort in commenting on the meta-analysis results and its implications. The included studies are not adequately annotated in the text, making reading difficult for the statistically unschooled reader who must understand the plots provided.

Decision

Verified with reservations: The content is academically sound but has shortcomings that must be improved.

Peer review report

Title: Types of Arrhythmias and the risk of sudden cardiac death in dialysis patients: A Systematic Review and Meta-analysis

version: 1

Referee: Milaras Nikias

Institution: National and Kapodistrian University of Athens- Ippokrateion Hospital

email: nikiasmil@med.uoa.gr

ORCID iD: 0000-0001-7312-0976

General assessment

It is now well known that high cardiovascular mortality in ESRD patients is only partly due to atherothrombotic events. Ventricular tachyarrhythmias and electromechanical dissociation account for a significant amount of those deaths as was reported in landmark trials such as the MADIT II. VT or VF might be the mode of death in only a minority of those patients and this is extrapolated from the fact that ICD implantation in this population does not extend survival, whether due to high competing comorbidities or due to electromechanical dissociation being the cause of death. It is true that ESRD patients are underrepresented in such studies due to the high competing factor for non-cardiac death and no safe conclusion can yet be drawn. It remains yet to be seen whether a better risk stratification algorithm through Holter monitoring or programmed ventricular stimulation can unveil those truly at high risk for SCD.

This meta-analysis tries to unveil the mode of death and the high cardiovascular mortality in renal failure through a thorough literature search that included 11 studies. This systematic review/meta-analysis follows current writing and reporting guidelines.

The English used is adequate although some parts of the manuscript could be refined (eg 3rd paragraph in Introduction)

Essential revisions that are required to verify the manuscript

ESRD and ESKD are both discussed in the manuscript. I would personally prefer that the authors devoted more effort in commenting on the meta-analysis results and its implications. The included studies are not adequately annotated in the text, making reading difficult for the statistically unschooled reader who must understand the plots provided.

Decision

Verified with reservations: The content is academically sound but has shortcomings that must be improved.

Peer review report

Title: Country Learning on Maintaining Quality Essential Services during CVOID-19 in Timor Leste: A mixed methods qualitative study

version: 2

Referee: Mary Anne Mercer

Institution: University of Washington

email: mamercer@uw.edu

General assessment

A very extensive, complex review of the TLS health system performance during the COVID-19 pandemic.

Introduction: The aims of the study are to “…learn from the efforts…to maintain quality EHS” (p 1) as well as to “understand the variance in service utilization, including the best practices in improving maternal mortality and SBA.” That leads the reader to expect proposed answers to these questions in the conclusion, which didn’t happen.

Methods: Extensive discussion of methods leaves a few important gaps, such as ages/genders of the few community informants. The table of documents reviewed could be in an appendix.

Results: A lot of useful results are presented, but some areas need more explanation. For example, we learn about the “Twinning Partnership for Improvement” (TPI) that seems to have a substantial effort on efforts to continue or improve quality. That entity was not well described and could potentially add a lot to an understanding of what happened to services during the pandemic. How important was the role of other partners? Were they a factor explaining why maternal care services performed better than general outpatient services?

The large Table 4 is useful but lacks some important information. Given the decrease in in-facility maternal mortality cases – what are the data on overall maternal mortality? Health facilities have some information on home deliveries, which are an important risk for mortality. Were no infant mortality data available?

Another important issue is identifying what happened, if anything, to health staff availability during the pandemic. In many countries, health worker numbers were substantially decreased because of COVID illness. Those data would be very useful for Table 4. Also, at the bottom of p 7, an FGD participant mentions “worried…about being sent to a COVID-19 treatment center” as a reason for not using health services. That sounds like a fear of being required to go somewhere for treatment? Would be good to clarify.

Conclusion: This section is particularly unhelpful in summarizing the results of the case study. It’s essentially a review of what the country should do in the future, rather than what was learned--what the study found had happened during the pandemic. Summarizing a few key findings would be helpful, in areas such as the role of partners, the factors that led to even better maternal care use during the pandemic, or other factors that helped or hindered service provision during the pandemic at the three levels studied. Then the conclusion as to how to use that information for the future would be more meaningful.

Essential revisions that are required to verify the manuscript

Professional proofreading would correct a number of typos.

Important error: on page 10, Table 4: I assume that for the Objectives/Indicators row, the proper indicator six rows down should be Postnatal or Postpartum Care coverage (rather than antenatal care, which has already been presented…the 1 week or 1-6 week category makes no sense for antenatal care).

Other suggestions to improve the manuscript

This document is far too detailed for publication in a journal; it would need substantial editing if it were meant for that purpose. However, assuming that it is meant primarily for national use, it does discuss a wide range of issues and practices that were meant to affect the quality and utilization of care during COVID and with revisions would be well worth publishing.

Decision

Verified with reservations: The content is academically sound but has shortcomings that could be improved by further studies and/or minor revisions.

Peer review report

Title: Country Learning on Maintaining Quality Essential Services during CVOID-19 in Timor Leste: A mixed methods qualitative study

version: 2

Referee: Mary Anne Mercer

Institution: University of Washington

email: mamercer@uw.edu

General assessment

A very extensive, complex review of the TLS health system performance during the COVID-19 pandemic.

Introduction: The aims of the study are to “…learn from the efforts…to maintain quality EHS” (p 1) as well as to “understand the variance in service utilization, including the best practices in improving maternal mortality and SBA.” That leads the reader to expect proposed answers to these questions in the conclusion, which didn’t happen.

Methods: Extensive discussion of methods leaves a few important gaps, such as ages/genders of the few community informants. The table of documents reviewed could be in an appendix.

Results: A lot of useful results are presented, but some areas need more explanation. For example, we learn about the “Twinning Partnership for Improvement” (TPI) that seems to have a substantial effort on efforts to continue or improve quality. That entity was not well described and could potentially add a lot to an understanding of what happened to services during the pandemic. How important was the role of other partners? Were they a factor explaining why maternal care services performed better than general outpatient services?

The large Table 4 is useful but lacks some important information. Given the decrease in in-facility maternal mortality cases – what are the data on overall maternal mortality? Health facilities have some information on home deliveries, which are an important risk for mortality. Were no infant mortality data available?

Another important issue is identifying what happened, if anything, to health staff availability during the pandemic. In many countries, health worker numbers were substantially decreased because of COVID illness. Those data would be very useful for Table 4. Also, at the bottom of p 7, an FGD participant mentions “worried…about being sent to a COVID-19 treatment center” as a reason for not using health services. That sounds like a fear of being required to go somewhere for treatment? Would be good to clarify.

Conclusion: This section is particularly unhelpful in summarizing the results of the case study. It’s essentially a review of what the country should do in the future, rather than what was learned--what the study found had happened during the pandemic. Summarizing a few key findings would be helpful, in areas such as the role of partners, the factors that led to even better maternal care use during the pandemic, or other factors that helped or hindered service provision during the pandemic at the three levels studied. Then the conclusion as to how to use that information for the future would be more meaningful.

Essential revisions that are required to verify the manuscript

Professional proofreading would correct a number of typos.

Important error: on page 10, Table 4: I assume that for the Objectives/Indicators row, the proper indicator six rows down should be Postnatal or Postpartum Care coverage (rather than antenatal care, which has already been presented…the 1 week or 1-6 week category makes no sense for antenatal care).

Other suggestions to improve the manuscript

This document is far too detailed for publication in a journal; it would need substantial editing if it were meant for that purpose. However, assuming that it is meant primarily for national use, it does discuss a wide range of issues and practices that were meant to affect the quality and utilization of care during COVID and with revisions would be well worth publishing.

Decision

Verified with reservations: The content is academically sound but has shortcomings that could be improved by further studies and/or minor revisions.

Peer review report

Title: Country Learning on Maintaining Quality Essential Health Services (EHS) during COVID-19 in Timor-Leste: A mixed methods qualitative analysis

version: 2

Referee: Alvaro Alonso-Garbayo

Institution: Independent consultant

email: alvaro_alonso@yahoo.com

ORCID iD: https://orcid.org/0000-0002-4120-

I am currently under a consultancy contract with WHO which represent a CoI as per PeerRef policy. Therefore no decision on the preprint is provided.

General assessment

Methods:

• Lit. review: suggest explaining how data obtained from each document identified was extracted, gathered and analysed

• Suggest including a detailed description of the sample (e.g. demographic characteristics) including gender

• Mistakes in literature gathered/analysed (Table 1):

1) National Health Sector Strategic Plan II 2011-2030 doesn’t match with reference 7 but ref. 9

2) Name of document: Ref. 18 “Ministry of Financial Annual Report” (?)

3) The only document classified as “municipal level” (e.g. “Exceptional and temporary measures of health surveillance in response to C-19 pandemic”) is not municipal but a national law and it was not published in 2020 but in 2021

• It is surprising that given the indicators used to measure quality among which there are reproductive health ones, UNFPA has not been identified as a main source of information for relevant literature and/or included relevant UNFPA staff as a key informant for interviews

• The section “Patient and Public Involvement” doesn’t make clear the link between the literature review and the decision to involve users (which should have been made anyway independently of the results of the literature review). The statements “No patients were involved in the recruitment to conduct the study (?) and patients were not involved in the design of this study (?)” are not clear (review/revise). The last phrase seems to be copy and paste from the study protocol (future tense?). What was the purpose of sharing “the study” with participants? Did they give any feedback? If so, what can be concluded from it?

• Table 4 would be more informative (e.g. covering pre-pandemic and pandemic periods) if more data from 2021 would have been included

• Table 4 requires full revision: some examples of mistakes/issues

1) How data about density of [key] health workers from only one year (2019) can inform efficiency? How density of medical doctors (only data from 2020) provides information about health worker distribution? How that links with efficiency?

2) Physical infrastructure: % of population -or facilities?- with basic hand washing facilities

3) Safety: how a reference dated in 2018 (i.e. Ref. 36) can provide information about 2019, 2020 or 2021?

4) Maternal mortality decrease (from 20 to 16 per 100,000 live births – measuring unit not included in the table) reported only from facilities doesn’t match with the previous statement in the introduction reporting decrease in [overall?] maternal mortality. This is particularly important in Timor-Leste where delivery at home assisted by grandmothers is culturally well rooted and still relatively common practice (particularly in rural/remote areas where access to health services may be difficult). This most probably have pushed pregnant women to choose this option during the pandemic which most probably, and despite the increase in institutional delivery reported (not sure but think that home delivery in TL is considered skilled birth if attended by a skilled professional), has increased the overall MMR.

Discussion:

• While the methods section reports that the selection of participants for KIIs looked for informants with different backgrounds (e.g. rural/urban), discussion about the different perspectives provided by different participants is missing. Most importantly there is no mention to gender perspectives as seems that this characteristic was not sought during recruitment.

Overall:

• The manuscript would benefit of a final proof reading in English to solve orthographic and grammatical issues

Peer review report

Title: Country Learning on Maintaining Quality Essential Services during CVOID-19 in Timor Leste: A mixed methods qualitative study

version: 2

Referee: Supriya Mathew

Institution: Menzies School of Health Research

email: supriya.mathew@menzies.edu.au

ORCID iD: 0000-0002-8078-3708

General assessment

The paper is well written and discusses an important topic that is also of future use. The study documents the experiences of the Timor-Leste health system as it sought to maintain quality essential health services (EHS) during the COVID-19 pandemic.

Sentence –“ no patients were involved in the recruitment…” page 6 is confusing

Page 12 -the paper mentions that there was a decrease in service utilisation- were there any quotes from the community/patient category that could be added below it?

Page 13 suddenly mentions one family member, is the quote from a community member?

Were there any differences between rural and urban quality of care?

Table 1 : year of publication has been mentioned, could it be categorised into pre during and post COVID publications

Table 4:

• Data has been provided only from 2019 onwards; is data available for at least 3 years before the COVID period?

• May be add one more column to indicate insufficient data for comparison between years?

• The paper mentions that the patient satisfaction improved as per patient satisfaction survey. How were the surveys administered (e.g. online)? Were the sample sizes across each years comparable?

Essential revisions that are required to verify the manuscript

Nil

Other suggestions to improve the manuscript

The paper uses too many acronyms which makes it hard to read the paper. It would be good to expand acronyms at least at the beginning of each major section. The paper could be improved by a final proofreading as there are a few typos/grammatical errors in the paper. The reference section needs to be proofread as well.

Examples of typos in the paper:

• Use of past tense: Page 3: ‘there was a decrease in maternal mortality

• Page 8 category of first couple of quotes not mentioned

• Conclusion subtitle spelling is wrong

• Quotes could be edited for readability. For example,

• Page 8 last quote “To guarantee ..” unclear

• Page 9 first quote could be edited slightly for readability

• References 34 author format, ref 18 is the title right?

Decision

Verified: The content is academically sound, only minor amendments (if any) are suggested.

Peer review report

Title: Country Learning on Maintaining Quality Essential Services during CVOID-19 in Timor Leste: A mixed methods qualitative study

version: 2

Referee: Supriya Mathew

Institution: Menzies School of Health Research

email: supriya.mathew@menzies.edu.au

ORCID iD: 0000-0002-8078-3708

General assessment

The paper is well written and discusses an important topic that is also of future use. The study documents the experiences of the Timor-Leste health system as it sought to maintain quality essential health services (EHS) during the COVID-19 pandemic.

Sentence –“ no patients were involved in the recruitment…” page 6 is confusing

Page 12 -the paper mentions that there was a decrease in service utilisation- were there any quotes from the community/patient category that could be added below it?

Page 13 suddenly mentions one family member, is the quote from a community member?

Were there any differences between rural and urban quality of care?

Table 1 : year of publication has been mentioned, could it be categorised into pre during and post COVID publications

Table 4:

• Data has been provided only from 2019 onwards; is data available for at least 3 years before the COVID period?

• May be add one more column to indicate insufficient data for comparison between years?

• The paper mentions that the patient satisfaction improved as per patient satisfaction survey. How were the surveys administered (e.g. online)? Were the sample sizes across each years comparable?

Essential revisions that are required to verify the manuscript

Nil

Other suggestions to improve the manuscript

The paper uses too many acronyms which makes it hard to read the paper. It would be good to expand acronyms at least at the beginning of each major section. The paper could be improved by a final proofreading as there are a few typos/grammatical errors in the paper. The reference section needs to be proofread as well.

Examples of typos in the paper:

• Use of past tense: Page 3: ‘there was a decrease in maternal mortality

• Page 8 category of first couple of quotes not mentioned

• Conclusion subtitle spelling is wrong

• Quotes could be edited for readability. For example,

• Page 8 last quote “To guarantee ..” unclear

• Page 9 first quote could be edited slightly for readability

• References 34 author format, ref 18 is the title right?

Decision

Verified: The content is academically sound, only minor amendments (if any) are suggested.

Peer review report

Title: Country Learning on Maintaining Quality Essential Health Services (EHS) during COVID-19 in Timor-Leste: A mixed methods qualitative analysis

version: 2

Referee: Alvaro Alonso-Garbayo

Institution: Independent consultant

email: alvaro_alonso@yahoo.com

ORCID iD: https://orcid.org/0000-0002-4120-

I am currently under a consultancy contract with WHO which represent a CoI as per PeerRef policy. Therefore no decision on the preprint is provided.

General assessment

Methods:

• Lit. review: suggest explaining how data obtained from each document identified was extracted, gathered and analysed

• Suggest including a detailed description of the sample (e.g. demographic characteristics) including gender

• Mistakes in literature gathered/analysed (Table 1):

1) National Health Sector Strategic Plan II 2011-2030 doesn’t match with reference 7 but ref. 9

2) Name of document: Ref. 18 “Ministry of Financial Annual Report” (?)

3) The only document classified as “municipal level” (e.g. “Exceptional and temporary measures of health surveillance in response to C-19 pandemic”) is not municipal but a national law and it was not published in 2020 but in 2021

• It is surprising that given the indicators used to measure quality among which there are reproductive health ones, UNFPA has not been identified as a main source of information for relevant literature and/or included relevant UNFPA staff as a key informant for interviews

• The section “Patient and Public Involvement” doesn’t make clear the link between the literature review and the decision to involve users (which should have been made anyway independently of the results of the literature review). The statements “No patients were involved in the recruitment to conduct the study (?) and patients were not involved in the design of this study (?)” are not clear (review/revise). The last phrase seems to be copy and paste from the study protocol (future tense?). What was the purpose of sharing “the study” with participants? Did they give any feedback? If so, what can be concluded from it?

• Table 4 would be more informative (e.g. covering pre-pandemic and pandemic periods) if more data from 2021 would have been included

• Table 4 requires full revision: some examples of mistakes/issues

1) How data about density of [key] health workers from only one year (2019) can inform efficiency? How density of medical doctors (only data from 2020) provides information about health worker distribution? How that links with efficiency?

2) Physical infrastructure: % of population -or facilities?- with basic hand washing facilities

3) Safety: how a reference dated in 2018 (i.e. Ref. 36) can provide information about 2019, 2020 or 2021?

4) Maternal mortality decrease (from 20 to 16 per 100,000 live births – measuring unit not included in the table) reported only from facilities doesn’t match with the previous statement in the introduction reporting decrease in [overall?] maternal mortality. This is particularly important in Timor-Leste where delivery at home assisted by grandmothers is culturally well rooted and still relatively common practice (particularly in rural/remote areas where access to health services may be difficult). This most probably have pushed pregnant women to choose this option during the pandemic which most probably, and despite the increase in institutional delivery reported (not sure but think that home delivery in TL is considered skilled birth if attended by a skilled professional), has increased the overall MMR.

Discussion:

• While the methods section reports that the selection of participants for KIIs looked for informants with different backgrounds (e.g. rural/urban), discussion about the different perspectives provided by different participants is missing. Most importantly there is no mention to gender perspectives as seems that this characteristic was not sought during recruitment.

Overall:

• The manuscript would benefit of a final proof reading in English to solve orthographic and grammatical issues

Peer review report

Title: If it’s real, could it be an eel?

version: 2

Referee: Dr Don Jellyman

Institution: National Institute of Water and Atmosphere (New Zealand)

email: don.jellyman@niwa.co.nz

ORCID iD: 0000-0002-6941-2703

General assessment

An interesting assessment that verifies the obvious – that any monster of ~ 6 m cannot be an eel (Anguilla anguilla), although there is a reasonable likelihood that eels of ~ 1 m could account for some of the “sightings” of elongate animals in the loch. However, even though the outcome is unsurprising, the author approaches the subject in a rigorous and systematic way. As such, the manuscript is of value in eliminating eels as possible candidate species for the mythical monster.

The manuscript is well written and referenced.

Essential revisions that are required to verify the manuscript

Nil

Other suggestions to improve the manuscript

Nil

Decision

Verified: The content is academically sound, only minor amendments (if any) are suggested.

Peer review report

Title: If it’s real, could it be an eel?

version: 2

Referee: Dr Don Jellyman

Institution: National Institute of Water and Atmosphere (New Zealand)

email: don.jellyman@niwa.co.nz

ORCID iD: 0000-0002-6941-2703

General assessment

An interesting assessment that verifies the obvious – that any monster of ~ 6 m cannot be an eel (Anguilla anguilla), although there is a reasonable likelihood that eels of ~ 1 m could account for some of the “sightings” of elongate animals in the loch. However, even though the outcome is unsurprising, the author approaches the subject in a rigorous and systematic way. As such, the manuscript is of value in eliminating eels as possible candidate species for the mythical monster.

The manuscript is well written and referenced.

Essential revisions that are required to verify the manuscript

Nil

Other suggestions to improve the manuscript

Nil

Decision

Verified: The content is academically sound, only minor amendments (if any) are suggested.

Peer review report

Title: An Improved Peer-Review System to Compensate for Scientific Misconduct in Health-Sensitive Topics

version: 7

Referee: Cristina Candal-Pedreira

Institution: University of Santiago de Compostela

email: cristina.candal.pedreira@rai.usc.es

ORCID iD: 0000-0002-1703- 3592

General assessment

In this letter, the authors propose a series of actions that the main actors responsible for scientific integrity (researchers, scientific journals, academic institutions, and funding entities) could (and should) implement to prevent and/or detect cases of scientific misconduct. Although the authors refer primarily to high-sensitive publications, in my opinion, many (if not all) of the provided proposals can be applied to any type of publication. Examples of scientific misconduct are commonplace, and the policies implemented so far do not seem to be strong enough to prevent the publication of fraudulent articles. I believe that this article is necessary and very relevant.

Essential revisions that are required to verify the manuscript

In my opinion, this letter presents a very comprehensive list of potential strategies that different stakeholders can undertake to reduce the burden of research misconduct. Of course, there are many other actions that could be implemented, such as promoting post- publication review, imposing sanctions, or auditing research funding from an ethical point of view, among others. However, I consider all the strategies developed by the authors to be important and necessary, so I have no essential revisions to this manuscript.

Other suggestions to improve the manuscript

The text is well written, very clear and easy to follow.

Some comments/suggestions/reflections:

I would suggest introducing the definition of “high sensitivity topic” in the first part of the manuscript. Also, in the title another term is used (health-sensitivity topic), I would homogenize terms.

J2. In addition to solving the problem of coercive citations, open peer review can make the peer review process more transparent by making public how many rounds of review have been done and how the conclusion was reached to publish or reject the article. In addition, reviewers, because they are not anonymous, can take the review more seriously.

R1-R2-R3. Regarding regulatory agencies, funders and institutions, it could also be helpful for them to perform audits of the projects, not only justification of where the money was spent, but also whether the research is being done ethically, including during the phase of dissemination of results.

Decision

Verified: The content is academically sound, only minor amendments (if any) are suggested.

Peer review report

Title: An Improved Peer-Review System to Compensate for Scientific Misconduct in Health-Sensitive Topics

version: 7

Referee: Cristina Candal-Pedreira

Institution: University of Santiago de Compostela

email: cristina.candal.pedreira@rai.usc.es

ORCID iD: 0000-0002-1703- 3592

General assessment

In this letter, the authors propose a series of actions that the main actors responsible for scientific integrity (researchers, scientific journals, academic institutions, and funding entities) could (and should) implement to prevent and/or detect cases of scientific misconduct. Although the authors refer primarily to high-sensitive publications, in my opinion, many (if not all) of the provided proposals can be applied to any type of publication. Examples of scientific misconduct are commonplace, and the policies implemented so far do not seem to be strong enough to prevent the publication of fraudulent articles. I believe that this article is necessary and very relevant.

Essential revisions that are required to verify the manuscript

In my opinion, this letter presents a very comprehensive list of potential strategies that different stakeholders can undertake to reduce the burden of research misconduct. Of course, there are many other actions that could be implemented, such as promoting post- publication review, imposing sanctions, or auditing research funding from an ethical point of view, among others. However, I consider all the strategies developed by the authors to be important and necessary, so I have no essential revisions to this manuscript.

Other suggestions to improve the manuscript

The text is well written, very clear and easy to follow.

Some comments/suggestions/reflections:

I would suggest introducing the definition of “high sensitivity topic” in the first part of the manuscript. Also, in the title another term is used (health-sensitivity topic), I would homogenize terms.

J2. In addition to solving the problem of coercive citations, open peer review can make the peer review process more transparent by making public how many rounds of review have been done and how the conclusion was reached to publish or reject the article. In addition, reviewers, because they are not anonymous, can take the review more seriously.

R1-R2-R3. Regarding regulatory agencies, funders and institutions, it could also be helpful for them to perform audits of the projects, not only justification of where the money was spent, but also whether the research is being done ethically, including during the phase of dissemination of results.

Decision

Verified: The content is academically sound, only minor amendments (if any) are suggested.

Peer review report

Title: A Comparative study in elective repair of large incisional hernias using on- lay mesh vs. sub-lay mesh- A Meta-analysis

version: 1

Referee: Julie Holihan

Institution: UTHealth at Houston

email: Julie.L.Holihan@uth.tmc.edu

ORCID iD: 0000-0001-6764-1687

General assessment

This was a systematic review and meta-analysis comparing onlay to sublay mesh placement in incisional hernia repair. Though this is an important topic, the systematic review seems incomplete and the manuscript is poorly written.

Essential revisions that are required to verify the manuscript

The introduction does not introduce the paper well. Why is it relevant to the question being asked that incisional hernias are more common in females, or speculations as to why this is the case? The study is looking at mesh location, so the introduction should focus on this. What is known about mesh location, why is it important, etc. Currently, intro sort of seems like a conglomeration of random facts about hernias.

Overall, needs editing for numerous grammatical errors. Difficult to read.

The results from search (figure 1) should be referred to in the results section, not the methods. Eligibility criteria is poorly written and unclear.

I think the search terms used are inadequate. This seems to have missed many studies.

Were PRISMA guidelines followed? Should be revised to follow PRISMA guidelines. Methods need to be much more transparent.

It seems incorrect that many of the I2 values are 0. Need to check this with a statistician.

There are results from a sensitivity analysis reported but the methods section does not describe how this was done.

The discussion section just seems to be repeating the results. It should be discussing and interpreting them.

Other suggestions to improve the manuscript

Tables and figures need better labels.

Decision

Requires revisions: The manuscript contains objective errors or fundamental flaws that must be addressed and/or major revisions are suggested.

Peer review report

Title: Maintained imbalance of triglycerides, apolipoproteins, energy metabolites and cytokines in long-term COVID-19 syndrome (LTCS) patients

version: 1

Referee: Paola Turano

Institution: University of Florence

email: turano@cerm.unifi.it

ORCID iD: 0000-0002-7683-8614

General assessment

This is an integrated study reporting NMR-based metabolomics data and flow cytometry-based cytokine in the blood of 125 individuals (healthy controls (HC; n=73), COVID-19-recovered (n=12), COVID-19 acute (n=7) and LTCS (n=33)).

The main goal appears to be that of demonstrating alterations in the metabolome and immune markers of patients with long COVID. This condition is defined as the continuation or development of new symptoms 3 months after the initial SARS-CoV-2 infection, with these symptoms lasting for at least 2 months with no other explanation.

As admitted by the authors, the 4 groups are very unbalanced in terms of numbers of enrolled subjects; moreover, all numbers are low but those in the recovered groups and even more in the acute phase are extremely low. Therefore, the only reliable comparison appears to be that between HC and LTCS. And this is a pity because the most important comparison to define the signature associated with long-COVID symptoms would have been the one between recovered and LTCS subjects.

Another problem is that there is no information on the status of the LTCS before infection nor during the acute phase. This, combined with the low number of individuals, does not allow to draw a real trajectory of the alterations during the observed time line. It is therefore difficult to be 100% sure that alterations in certain metabolites of lipoproteins are a consequence of LTCS or instead intrinsic characteristics of a group of individual that make them more prone to develop LTCS.

These critical aspects have nothing to do with the experimental approach, which is powerful and carefully performed. Unfortunately, the available cohort is not the best to achieve the goal of a molecular characterization of LTCS.

In any case the present manuscript provides useful hints to be further investigated in future studies and therefore might deserve publication.

Essential revisions that are required to verify the manuscript

If it were possible to enlarge the cohort of patients, confirming the observed trends, this would lead to a significant improvement in the impact of the work. But I understand the practical difficulties in achieving this goal.

Decision

Verified with reservations: The content is academically sound but has shortcomings that could be improved by further studies and/or minor revisions.

Author response

We would like to thank the reviewers for their valuable input. In the new version, we have tried to incorporate all of the comments made by Yulia Karmanova and Richmond Dzekoe. As a result, we feel that the quality of the paper has improved substantially. Below, we discuss for each comment (that required revision) which actions were taken to address the reviewer's concerns.

First, we will address the comments of Yulia Karmanova, Research Centre Kairos:

1. I suggest that the authors should involve more assessors in their future research. Two lecturer- researchers and three senior students were involved in the process which I assume is not enough for such large-scale research like this. A bigger team of professional assessors could make valuable contribution when analysing the data and resolving emerging research questions.

Although it was indeed a huge job to assess the entire corpus with only five people, working with a small team also had its advantages in terms of reliability and validity of the research. It helped us address and overcome one of the main difficulties mentioned in qualitative research, viz. the perceived subjectivity of the assessment process (O'Connor & Joffe, 2020). By keeping the team within manageable proportions, we could ensure a like mindset by increasing the inter-rater reliability through calibration sessions. This concern also gave rise to a vast field of research on automated assessment tools. (See "Reply to Comment 1" in the manuscript)

O'Connor, C., & Joffe, H. (2020). Intercoder Reliability in Qualitative Research: Debates and Practical Guidelines. International Journal of Qualitative Methods. \<doi:10.1177/1609406919899220>

2. I would also recommend providing the manuscript with brief comments on the meanings of the parameters in column 4 (Table 3, 4, 5, 6) for readers' clarity. What do t , p and n.s. stand for?

We thank the reviewer for this suggestion. It might indeed help to point out these statistical concepts for a better understanding of the figures in the Results section. We have added footnotes with short clarifying definitions to Table 3, first table in the Results section. These footnotes contain the following information:

In statistics, the t-value measures the size of the difference relative to the variation in your sample data. In other words, T is the calculated difference represented in units of standard error. The greater the magnitude of T, the greater the evidence against the null hypothesis, viz. the assumption that there is no difference in language use between blogs scoring high vs. low in perceived level of ICC.

A p-value is a statistical measurement used to validate a hypothesis against observed data. A p-value measures the probability of obtaining the observed results, assuming that the null hypothesis is true. The lower the p-value, the greater the statistical significance of the observed difference. A p-value of 0.05 or lower is generally considered statistically significant, meaning that the null hypothesis can be rejected. ' N.s.' is simply short for 'not significant'; in other words, a p-value above 0.05.

3. I believe that the manuscript would benefit from correcting minor inaccuracies. I would recommend to:
 - Replace «his» with gender neutral «their», page 6: In these blogs, the language use of students serves as a vehicle of information on the students' development of ICC, offering the reader concrete cues – henceforth referred to as linguistic markers – of his reflective learning process. - Add a space between that and are , page 19: In order to bring more focus to our research, we initially focused on word categories thatare characteristic of properties that can be linked to ICC and cultural sensitivity, such as openness, self- relativity, curiosity and reflection or analytical thinking. - Add missing parentheses, page 22; Deardorff, D. 2006. Identification and Assessment of Intercultural Competence as a Student Outcome of Internationalisation. Journal of Studies in International Education, 10 (3), 241-266.

We have corrected all the above-mentioned typos and inaccuracies concerning gender in the text.

Secondly, we will address the comments of Richmond Dzekoe, Iowa State University:

4. Theoretical Background and Literature review: These sections need a major revision. Move the discussion on studies on the importance of reflection to the literature review section and provide more current references. These sections also read more like annotations. It will be better to focus on particular insights from the studies you cited and the implications of those insights for framing your current study.

We thank the reviewer for his advice. Taking into account his other comments (see comments 8, 15 and 17) we decided to rewrite the introduction to focus more clearly, and from the beginning, on the main aim of our study: to look for linguistic markers of ICC in reflective writing. We hope that, by framing the introduction in a different way, the structure of Section 2 becomes more transparent for the reader, and will be easier to follow.

5. Theoretical Background and Literature review: You mention Byram's (1997) intercultural speaker model and go on to say, "Like most of the current ICC frameworks, Byram's model offers a holistic approach." What are some of the current ICC frameworks you are referring to? Giving some examples here will be helpful for your readers.

We agree with the reviewer that "most of the current ICC frameworks" is a vague and somewhat confusing reference to ICC frameworks in general, and more specifically the ones we already referred to in preceding paragraphs. After reviewing the paragraph 'Language in relation to ICC', we decided to omit the relevant sentence, as it turned out to be superfluous for our reasoning.

6. Theoretical Background and Literature review: The information in Table 2 should be added to your description of the Corpus.

After trying to summarize this information in running text, we concluded that a table is the best way to provide numerical details on the different sub-corpora in a neat and orderly manner. Therefore, we have retained the table in the new version of the text.

7. Theoretical Background and Literature review: In order to support the claim that the use of many "I-words" indicates a more open, curious, and involved stance, it is important to explain more clearly how you differentiate the "I- words" which are descriptive from "I-words" that are reflective in your analysis.

The analysis of our corpus supports our claim that more open, curious and involved authors – sign of high level of ICC – more frequently use I-words. There is no difference, however, in the type, nor the significance (person of reference) of these I-words between the two sub-corpora. In other words, we do not differentiate between descriptive and reflective I-words. We have marked the relevant sections in the manuscript with "Reply to Comment 7".

8. Literature review: Beginning the Literature review with the sub-section "Language in relation to ICC" might provide a better flow of ideas in your lit. review.

Since we have rewritten our introduction to immediately focus on 'linguistic markers for ICC in reflective writing assignments' as our narrative hook (in response to Comments 4, 16 and 17), we think it also becomes easier to understand the structure and flow of ideas of Section 2, Theoretical Background. Therefore, we decided to discard this suggestion.

9. Methods, Results, Discussion: Explain the strengths and limitations of the integrated approach you are using. What does each model add to your integrated framework, and why is this integrated approach the best way to frame your study?

We thank the reviewer for this clear observation. The added value of our combined approach is often suggested in the text but never explicitly stated. In section 3.2 we explain how we combined a holistic approach (by determining the level of ICC for each blog based on a rubric) with a textual analysis of each blog (based on semi-automated approach based on the LIWC lists). By adding a textual analysis to a holistic rubric based on the ICC frameworks of Byram, Deardorff and Pinto, we intend to make the holistic claims (that is, blog perceived as high ICC versus blog perceived as low ICC) more tangible. By focusing at word level on the use of quantifiers, I-words and insight words, teachers can 'materialize' their holistic claims and help students become more nuanced, curious, reflective and open-minded writers and can help them develop their global mindset.

We added the following sentence to the first paragraph of section 4: "By adding a textual analysis to a holistic rubric, we intend to make the perceived level of ICC more tangible. By focusing on language use, teachers can substantiate their holistic claims and help students become more nuanced, curious, reflective and open-minded writers and, consequently, help them develop their intercultural competences". (See "Reply to Comment 9" in the manuscript)

10. Methods, Results, Discussion: In describing the use of the rubrics to score the blogs, you mention calculating inter-rater reliability. How was this reliability calculated, and what was it?

In section 3.2, we mention 'inter-rater reliability' twice. The first time in relation to the use of a rubric: Instead of letting the five assessors freely determine the perceived level of ICC for each blog on the basis of their own knowledge and insights, we have created a rubric (attached to the article): a scoring tool or set of criteria with associated descriptions of certain scores. The use of a rubric is known to increase 'inter-rater reliability'.

The second time we mention 'inter-rater reliability' is when we refer to the calibration sessions we organized to discuss and fine-tune our evaluations based on the rubric, to enhance our (common) understanding of the rubric and ensure or increase our inter-rater reliability. We did not, however, perform calculations based on our (possibly differing) scores to exactly 'calculate' our inter-rater reliability, as in other published studies, e.g., the one by Lucas et al. (2017). Since we do not claim to have made this calculus, we did not change the text in section 3.2.

Lucas, Ch. et al. (2017). Inter-rater reliability of a reflective rubric to assess pharmacy students' reflective thinking. Currents in Pharmacy Teaching and Learning, 9, 989-995.

11. Methods, Results, Discussion: The strong evidence of intercultural competence comes from your analysis of the "Insight words." There is, however, a problem with the analysis of "I-words." As you explain the use of "I-words" as indicators of reflective writing, it will be good to explain more clearly how you differentiate the reflective and descriptive functions of "I-words" in your analysis.

See the above-mentioned comment on the use I-words. We do not distinguish between descriptive and reflective I-words. The difference between blogs with a higher perceived level of ICC and a lower level lies in the frequency in which they use I-words. When we then further look into the type of verbs that follow the personal pronoun I, we notice that the I's in the corpus of high ICC are more frequently combined with verbs marking an analytical approach. These verbs are part of the dictionary of 'insight words' (according to Pennebaker). So, in both cases (combined or not with an 'insight word'), the I's solely refer to a more personal and involved stance. The more frequently authors refer to an I-word, the more involved, curious and open-minded they are. The combination with insight words merely adds to the blog's perceived level of ICC: A more involved, curious and open-minded stance using I- words, plus proof of 'insight' or 'analysis' by the use of insight words, both add up to a higher level of perceived ICC. Please see the highlighted sections in reply to comment 7 in the manuscript.

12. Methods, Results, Discussion: The discussion is weak. Besides a list of limitations, the discussion lacks an insightful engagement with conclusions drawn by previous research. Contextualizing the discussion within already reported insights on this topic from studies such as Belz (2003), Byram (1997), Chan, Wong, & Luo (2020), Deardorff (2006), Elola & Oskoz (2008), Hoefnagels & Schoenmakers (2018) will help you address the main aim of your study which is identifying linguistic markers of ICC in order to provide teachers and other supervisors with tangible cues to help students develop ICC.

Thank you for this critical remark, which – we think − mostly relates to paragraph 4 of Section 4. In order to link our results more explicitly to former research and the gaps we have identified in the previous sections of the text, we have added information that should elucidate the added value of our research to former publications and insights in the domain of ICC. (See "Reply to Comment 12" in the manuscript)

1. Abstract:

In the abstract, it might be a good idea to mention how many students were involved in the study and their level of linguistic proficiency in English.

The blogs were written by a mixed group of students, of which approximately 80% are native Dutch and 20% speak another language. We have no information about the specific level of English each of them has. I have added the number of blogs (1,635) and students (672) to the abstract. (See "Reply to Comment 13" in the manuscript)

14. Abstract: You used the expression "a more analytical approach." Please be more specific and mention that approach by name and what makes it more analytical.

In the abstract we mention "a more analytical approach at text and word level". To be more precise we have changed this into "a text-analytical approach at word level". (See "Reply to Comment 14" in the manuscript)

15. Introduction: In the introduction, please provide more substantial evidence from the literature to support the claim that a "successful career path increasingly depends on ICC." One example from Linked in is not enough.

Since we decided to rewrite our introduction and directly focus on reflective writing to enhance ICC (in response to Comments 4, 8, 16 and 17), we have skipped the 1st paragraph which focused on the importance of ICC in contemporary education and the work field. We did, however, find a more recent source, stating that "intercultural competence plays a crucial role in modern working life, which indicates that the sphere of working life has expanded outside land borders and across cultural boundaries" (Pylväs & Nokelainen, 2021). We would also like to refer to Hoefnagels & Schoenmakers (2018) who – more specifically for the hospitality industry − state that "in a globalized industry, hospitality managers must be able to manage cultural diversity at many different levels. (...) Not only must hospitality managers be effective in their daily interactions with culturally and linguistically diverse guests, but also in communicating with their multicultural domestic staff. And over and above that, hospitality managers might just as well be working for an international hotel group or investor with headquarters on a different continent than their own, thus adding another level of cultural challenge to their working environment." (See "Reply to Comment 15" in the manuscript)

Pylväs, L., & Nokelainen, P. (2021). Academics' perceptions of intercultural competence and professional development after international mobility. International Journal of intercultural Relations, 80, 336-348.

16. Introduction: Please provide a clearer definition of ICC. Besides the mention of Deardorff's (2006) definition, the reader is lost as to what ICC really means in this study and how that definition informs the framing and findings of the study.

Given the fact that we have slightly changed the scope of our introduction and have shifted the focus on ICC and the different theoretical models to Section 2, we have connected Deardorff's definition better to the study at the beginning of the section. (See "Reply to Comment 16" in the manuscript)

1. Introduction:

Tere seems to be a jump from a discussion on "reflective writing" to the "role of language as a source of information for students learning" and a justification of the use of the "Linguistic Inquiry and Word Count framework" to study the use of "I-words" by President Nixon during the Watergate scandal. This structure makes the introduction a bit confusing. Please revise the introduction. Explaining the use of LIWC in other corpus analysis studies for Word Counts might help you provide a stronger justification for using this framework than the Watergate research you cited in the introduction. For current studies that use LIWC please see (Dudău DP and Sava FA (2021). Performing Multilingual Analysis With Linguistic Inquiry and Word Count2015 (LIWC2015).

We thank the reviewer for the reference to Dudău & Sava (2021). We have read the paper and included the reference in our revised introduction (also see comment 4, 8 and 15) to underline the interest in the LIWC2015 framework in recent scientific literature. (See "Reply to Comment 17" in the manuscript)

Dudău, D.P., & Sava, F.A. (2021). Performing Multilingual Analysis With Linguistic Inquiry and Word Count 2015 (LIWC2015). An Equivalence Study of Four Languages. Frontiers in Psychology, 12, article 570568. [doi: 10.3389/fpsyg.2021.570568]

We hope that this letter provides sufficient clarification of the modifications we have made in response to the reviewers' comments. We will upload the new version on Preprints.org and notify the different reviewers in response to the changes they had suggested. If you have any further questions or comments, do not hesitate to contact us.

Referee response:

I appreciate the revision the authors have made to the introduction. Rewriting the introduction helps set a clearer focus for the rest of the paper. However, I still have some reservations about the methodology and how data was collected and analyzed.

I refer specifically to two of my previous comment:

7 “Theoretical Background and Literature review: In order to support the claim that the use of many “I-words” indicates a more open, curious, and involved stance, it is important to explain more clearly how you differentiate the “I- words” which are descriptive from “I-words” that are reflective in your analysis.”

I still find the lack of distinction between “I-words” that might be purely descriptive from “ I –words” that are reflective problematic. To be able to claim that the use of many “I-words” indicates a more open, curious, and involved stance, it is important to code the data in a way that separates descriptive “I-words” (Eg. I am an American) from reflective “I-words” (Eg. I realized that I needed to engage more in cross-cultural communication). The lack of such differentiation will mean all “I-words” in the corpora are reflective.

1. Methods, Results, Discussion: Explain the strengths and limitations of the integrated approach you are using. What does each model add to your integrated framework, and why is this integrated approach the best way to frame your study?

The response the authors gave describes what they intended to do rather than actually providing an answer to the question of the integrated framework's strengths and limitations.

The manuscript needs to address these areas effectively in order to support its central claim and conclusion.

Author response

Dear Mr. Dzekoe

Thank you for clarifying your comments. Please allow us to further address them in the text below.

I appreciate the revision the authors have made to the introduction. Rewriting the introduction helps set a clearer focus for the rest of the paper. However, I still have some reservations about the methodology and how data was collected and analyzed.

I refer specifically to two of my previous comment:

1. “Theoretical Background and Literature review: In order to support the claim that the use of many “I-words” indicates a more open, curious, and involved stance, it is important to explain more clearly how you differentiate the “I- words” which are descriptive from “I-words” that are reflective in your analysis.”

I still find the lack of distinction between “I-words” that might be purely descriptive from “ I –words” that are reflective problematic. To be able to claim that the use of many “I-words” indicates a more open, curious, and involved stance, it is important to code the data in a way that separates descriptive “I-words” (Eg. I am an American) from reflective “I-words” (Eg. I realized that I needed to engage more in cross-cultural communication). The lack of such differentiation will mean all “I-words” in the corpora are reflective.

From your comment we understand that you would like us to make a distinction between descriptive I-words and reflective I-words (when combined with a cognitive verb). This is not possible when using the LIWC framework, however, as entries in the LIWC dictionaries do not contain information about surrounding words, we interpret the significant difference we observed in the frequency of I-words (regardless of the verb that followed them) in accordance with Pennebaker’s claim that that a higher frequency of I-words is sign of a more involved and curious author, two traits that are also important in the theoretical models for ICC. This quantitative outcome allows us thus to link I-words to ICC.

Since our approach combines a quantitative and a qualitative analysis of the blogs, we dug further into the data, looking for extra evidence of that involved stance at text level. There we noticed that these same I-words, in the blogs with a high ICC level, were also often combined with a cognitive verb. These cognitive verbs (part of Pennebaker’s dictionary of Insight Words) are sign of more reflection, and as we will see further in the analysis (Section 3.3.2), this can also be linked to a higher level of ICC.

In other words, we understand the difference between descriptive and reflective I-words, and we address this difference by conducting a qualitative follow-up analysis on combinations of I-words with reflective / cognitive verbs, rather than incorporating the difference into our quantitative analysis (which would be practically impossible given the nature of the LIWC dictionaries). As a consequence, we can state that the link between the use of I-words and ICC, sign of a more involved stance, is sometimes strengthened or corroborated by an extra link, viz. the one between insight words (amongst which cognitive verbs) and ICC, sign of more reflection. (See Reply to Comment #7, in the manuscript)

1. Methods, Results, Discussion: Explain the strengths and limitations of the integrated approach you are using. What does each model add to your integrated framework, and why is this integrated approach the best way to frame your study?

The response the authors gave describes what they intended to do rather than actually providing an answer to the question of the integrated framework's strengths and limitations.

We acknowledge that the added value of our approach still remains implicit in the text. Therefore, we added the following sentences at the end of Section : “This integrated approach, combining a quantitative and qualitative text analysis, allows us to analyze a large corpus of texts in a targeted and fast manner. By adding a second, qualitative step to the statistical outcomes, we are able to interpret the results and to link them, in this case, to the differences in ICC score.” (See Reply to Comment 9, in the manuscript)

The manuscript needs to address these areas effectively in order to support its central claim and conclusion.

We sincerely hope that we were able to clarify the last ambiguities and doubts.

reviewer response

Thank you very much for responding to my concerns and adding a qualitative analysis that helps to speak to the conclusions you draw about the use of the “I-words.” I appreciate you adding this additional step because th LIWC framework itself does not provide this depth of analysis and insight. Also, Pennebaker’s claims were closely tied to his analysis of emotions rather than the cross-cultural factors you investigate in your study. So, again, adding your own qualitative analysis effectively addresses the concerns I had.

I also appreciate the explicit explanation you added on why you adopt an integrative framework.

My final comment is Verified manuscript: The content is scientifically sound, only minor amendments

Peer review report

Title: Crossref as a source of open bibliographic metadata

version: 2

Referee: Simon Porter

Institution: Digital Science

email: s.porter@digital-science.com

ORCID iD: https://orcid.org/0000-0002-6151-8423

General assessment

This is a clear paper that outlines a motivation (assess the metadata completeness of the Crossref record for the purposes of scientometric analysis,) along with providing a set of useful metrics to assess the completeness of each metadata field.

Essential revisions that are required to verify the manuscript

No essential revisions identified

Other suggestions to improve the manuscript

Minor suggestions: Figures in the interactive version of the preprint do not have headings or captions, or a link back to the paper.

On data availability, In the context of the paper, making the code used to process the Crossref’s XML Metadata Plus Snapshot would be a useful contribution enabling scientometric analysis of the Crossref dataset.

The following are offered as suggestions that could be added to the paper at the authors discression, but do not effect the content or the conclusions of the peer review

The authors have chosen to frame metadata completeness of Crossref records as a ‘good in itself,’ leaning on Waltman, L. (2020b) to do the work of setting this up.

Within this framework, the analysis is offered as a set of observations to help publishers understand where they need to do better. It might be the case that Publishers do not intrinsically understand why making certain metadata types available is valuable to the community.

On the question of how Crossref can be used in scientometric analysis, readers are left to make up their own minds on what Crossref can be used for today, vs what it might be capable of providing in the future based on the evidence presented. It would be a stronger conclusion to highlight the types of scientometric analysis that are now possible with Crossref, (for instance bibliometric coupling,) and those that require limits or caveats (analysis by affiliation, abstract.) As this analysis lends itself to being rerun in the future, it would be useful to trace advances (hopefully!) not just in terms of the number of things, but also in terms of how sceintometric analysis capability is progressing because of it.

Decision

Verified: The content is scientifically sound, only minor amendments (if any) are suggested.

Peer review report

Title: Crossref as a source of open bibliographic metadata

version: 2

Referee: Silvio Peroni

Institution: University of Bologna

email: silvio.peroni@unibo.it

ORCID iD: 0000-0003-0530-4305

General assessment

This article describes an analysis of the Crossref dataset to assess if it can be a rich and reliable source for open bibliographic metadata, considering its central role as a primary source of several Open Science infrastructures such as OpenCitations and OpenAlex. The article is very well-written and addresses an important topic for the community. The analysis focuses mainly on the availability of some metadata, namely reference lists, abstracts, ORCIDs, author affiliations, funding information, and license information. Both data and interactive versions of the figures in the article are openly available online, shared in open formats and appropriately cited, supporting well the reproducibility of the analysis. However, it would be crucial to clarify a few aspects, listed in the section below.

Essential revisions that are required to verify the manuscript

Only a few aspects may need to be clarified in the paper.

1. In Section 3.2, there is a link between the fact that large publishers support I4OC and the fact that we have reached a tipping point of one billion open citations available. However, in the cited paper by Hutchins (2021), while there is a strict reference to I4OC, the reaching of the tipping point was not computed directly using Crossref (which is the dataset discussed in the present article) but by combining the data contained OpenCitations’ COCI (derived from Crossref data) and the NIH Open Citation Collection (derived from PubMed data). Thus, while indeed this result has been reached thanks to the enormous contribution of Crossref data, it was necessary to involve other open collections that do not necessarily involve the same publishers participating in I4OC and releasing their bibliographic references via Crossref.

2. In the case of ORCIDs (page 9), it would be good to clarify whether the availability of such identifiers (and other metadata) in Crossref is full responsibility of the publisher or Crossref makes some inference (e.g. using specific tools and/or external sources) either to fill in fields that are not specified or to validate them (at least at a first syntactic level, e.g. by seeing if the check digit of the ORCID identifier is correct or not).

Other suggestions to improve the manuscript

There are two aspects that, if included, would make the analysis even more robust.

First, the authors explicitly tell us that IEEE references have yet to be considered in the analysis since the Crossref dump they have used precedes the release of IEEE bibliographic references in Crossref. It would be great to re-run everything with a newer dump to address this lack, if feasible, considering the publisher's importance and dimension in terms of publications.

Second, it is clear that the authors have developed some tools (e.g. scripts, software, queries) they used to parse the Crossref dump and extract the relevant information from it. However, there is no mention of such tools in the paper. Having them available as open-source material would be ideal since they implement the methods that have been used for processing data and gathering the statistics introduced in the article. Indeed, such a code's availability would increase the analysis's reproducibility. Therefore, even if it is not mandatory for the narrative of the article, I would suggest (if feasible and legally allowed) publishing such sources with a readme file that explains how to run them, providing them with a persistent identifier (either Software Heritage or GitHub+Zenodo can be used for it), and to cite them properly in the article.

Decision

Verified: The content is scientifically sound, only minor amendments (if any) are suggested.

Peer review report

Title: PRIME: Uncovering Circadian Oscillation Patterns and Associations with AD in Untimed Genome-wide Gene Expression across Multiple Brain Regions

version: 1

Referee: Yuka Kotozaki

Institution: Iwate Medical University

email: kotoyuka@iwate-med.ac.jp

ORCID iD: 0000-0002-4659-1200

General assessment

I understand the research question. However, was the sample size you used adequate for applying PRIME to study the oscillatory patterns of non-temporal genome-wide gene expression at 19 sites in the human brain?

I think I need a little more (or more) sample to examine this, in my opinion.

Decision

Requires revisions: The manuscript contains objective errors or fundamental flaws that must be addressed and/or major revisions are suggested.

Peer review report

Title: PRIME: Uncovering Circadian Oscillation Patterns and Associations with AD in Untimed Genome-wide Gene Expression across Multiple Brain Regions

version: 1

Referee: : Jun Li

Institution: University of Michgan

email: junzli@med.umich.edu

ORCID iD: 0000-0001-6727-0812

General assessment

This study tried to use sample-sample correlations to quantify circadian patterns in different brain regions and compare between cases (those affected by Alzheimer's disease) and healthy controls. The premise is that if we do not know the time-of-death of the samples, if the samples do have robust circadian patterns, if we have selected the right genes to capture the circadian patterns in these brain regions, a strong positive correlation between two regions may indicate that both have strong circadian patterns and they are in synchrony.

The key results were shown in Figure 5, which displayed the correlation matrix for 19-19 brain regions calculated using 8 known circadian genes (and several steps of transformation). The interpretation for a pair of brain regions with a high correlation value was that their circadian patterns are strong and in phase. Fig.5 showed that the 19-19 correlation matrix from controls had more instances of large positive correlations than the similarly calculated 19-19 correlations among the cases, therefore the study drew the conclusion that Alzheimer's patients had more disruptions in circadian patterns in the 19 brain regions.

Several issues arose in the strategy of calculating correlations shown in Figure 5 and how they were interpreted.

1. The logic of the algorithm, especially the results for G^2, was confusing. The study first used PCA to reduce the initial N-sample-M-feature matrix to an N-D matrix, so that the data in the M-feature space were embedded in the lower, D-dimensional PC space. Next, the study reduced the D-dimension data to R clusters using NMF, where the N-R matrix, called G, captures the “soft membership metrics of all samples on R clusters in the latent space”. G^2 was used to calculate the N-N matrix in Figure 5. Figure 2 showed the R=6 solution for the mouse liver data. It is not clear why some samples have large membership values across multiple of the 6 clusters, while others have low membership values in all clusters. This brings into question the meaning of the clusters, especially whether they represent circadian patterns at all. How was the G matrix standardized (e.g., one option is to ensure that each sample has a total membership of 100% across the R clusters)? A related question is whether the N-D matrix from PCA can separate the circadian patterns from the large number of other biological and technical effects. Other questions include: why not calculate the N-N correlations using the N-D data? Which genes were used in deriving the N-D result and how they affect the subsequent N-R result?

Since the meaning of G^2 is not clear, it is difficult to assess the validity of calculating the N-N matrix using G^2. Hypothetically, if 50% of the samples are in one cluster for technical reasons, and the other 50% of the samples are scattered in the other R-1 clusters, the correlation with another brain region will depend on the sample distributions among the clusters, not necessarily driven by the level of synchrony of circadian patterns. For instance, patterns in Figure 4 could arise if the two regions experienced the same batch effect among the cases, and a more different effect among the AD samples.

1. The region-region correlation patterns are strongly influenced by the samples used for that pair of regions. The cases and controls are from different groups of human subjects. Not all subjects contributed to the 19 brain regions. As a result, different entries in the 19-19 matrix involved partially overlapping brains, not the same brains.

2. The 8 genes may or may have robust circadian patterns in the brain dataset. Whether they were also affected by other factors, such as age, medical conditions, batch effect or outliers, was not reported.

3. The steps of processing the data. before the standardization of each gene across the samples, have a profound influence on the sample-sample correlation results. For instance, how the samples have been normalized before the genes were standardized, and whether sample normalization was based on all genes or a certain subset of genes, were not clearly described. Additional relevant details are: whether the non-overlapping nature of the samples affected the N-N matrix shown in Fig.5? Are the samples normalized for each region separately? Are they normalized before or after the selection of the "union" samples?

4. A direct way of comparing brain regions for their circadian patterns was to estimate the magnitude and phase of circadian pattern for all samples from that region, using either documented or estimated time of death. For the latter task, the estimation can be done by using prior knowledge of the phase and relative intensity of dozens to hundreds of known circadian genes in the mammalian brain.

5. The correlation matrix shown in Figure 5 is difficult to interpret. Did a negative correlation imply that the two regions involved are of opposite phase? Did a low correlation suggest that the two regions are of orthogonal phase or, equally plausibly, could samples in one or both regions had dampened circadian patterns? The answer also depends on gene selection, as noted above. Further, what could be the biological mechanism that different brain regions may be anti-phase with each other?

6. The application to the mouse liver data was said to represent “cross-species and cross-organ validation”. The situation is complicated: for an easy case - one in which circadian pattern is robust and separable from other sources of variation - one can observe that samples-in-phase are positively correlated; and samples-on-opposite-phase are negatively correlated, without knowing the time order of the samples. But the number of large positive correlation values, which was used as a central metric in this study, can easily vary in another dataset, vary strongly by the genes selected and the pre-processing steps, and by non-circadian factors. For instance, the AD samples may be affected differently than controls by pre-death acute medical condition, sample preparation, and batch effect in processing the RNA for downstream experiments.

7. Figure 1a may have other explanations: whether the gene selection was adequate; whether the circadian patterns were distributed in multiple eigenvectors (Fig.1a only compared the first eigenvector); whether the circadian pattern was dampened in the second cycle or affected by technical factors differently than the first cycle. The mouse liver dataset can be used to observe the transfer of signals in the N-D and N-R space. The caveat is that the human brain datasets will likely have a different blend of circadian effects and other effects.

Other suggestions to improve the manuscript

The phrase "union brain regions" is hard to follow.

Figure.5 had 19 samples in all panels, inconsistent with the legend.

Decision

Requires revisions: The manuscript contains objective errors or fundamental flaws that must be addressed and/or major revisions are suggested.

Peer review report

Title: Burden of HCoV infection in children hospitalized with lower respiratory infection in Cape Town, South Africa

version: 1

Referee: Jessica Price

institution: University of Witwatersrand

email: Jessica.Price@wits.ac.za

ORCID iD: 0000-0002-4020-6850

General assessment

This manuscript it well written, with a clear description of the methods, results and discussion of findings. I think that with minor corrections it would be ready for publication.

Essential revisions that are required to verify the manuscript

Methods:

Study procedure:

1) Please add a reference to the parent study which details the full methods of the parent study.

2) In the third paragraph of this section the authors refer to children young than 18 months with a positive HIV Elisa as being confirmed to have HIV infection. Please double check this - I think it is supposed to be that these children were categorised as HIV- exposed with a confirmatory HIV PCR preformed to determine HIV infection.

Results:

3) The current phrasing of the results suggests that there were no refusals amongst those eligible to participate. Is that accurate?

4) In table 1: HIV status “exposed by negative” – is that children under 18 months or under 6 months? If only those less than 6 months please explained where children between 6-18 months are categorised.

Discussion:

5) 16 patients were found to have different human coronaviruses on the IS and NP samples. Please discuss the implications of this finding. Which would you act on? Does this bring into question the validity of the two methods if they are detecting different viruses in the same patient. I would understand if one method detected additional viruses but to have completely different viruses across the two specimens on the same patient is potentially problematic.

6) The authors note that the study group only incudes hospitalised patients and therefore cannot comment on community transmission/burden. However there have been many community-based surveillance programmes to track respiratory virus burdens and transmission patterns in SA (including work by Sharon Cohen, NICD) – it would be helpful to the readers if the authors could review some of these publications and comment on relevant similarities or differences. (most recent of these publications can be found here: https://crdm.nicd.ac.za/projects/phirst-c/)

7) Please review and redraft this paragraph 5 in the discussion – starting “compared to RSV and other respiratory viruses…”. I could not follow what the authors are trying to say in this paragraph.

8) Please add in a limitations/recommendations section.

Conclusion:

9) The final paragraph of the conclusion raises some interesting questions but does not fit as part of the conclusion. I suggest moving this to be included as part of the discussion, and more fully discussing the questions raised regarding the value of testing for disease if they do not cause severe disease, nor change treatment strategies.

Other suggestions to improve the manuscript

• Stylistic preferences in the introduction: avoid using “for example” when describing work referenced. Either just add the reference number, or use phrasing such as “as shown by (author name) who found …” etc.

• Methods - Statistics: Typo in the last paragraph – please confirm if Stata 13 or stata 16, and add in the necessary stata package reference.

Decision

Requires revisions: The manuscript contains objective errors or fundamental flaws that must be addressed and/or major revisions are suggested.

Peer review report

Title: Burden of HCoV infection in children hospitalized with lower respiratory infection in Cape Town, South Africa

version: 1

Referee: Jessica Price

institution: University of Witwatersrand

email: Jessica.Price@wits.ac.za

ORCID iD: 0000-0002-4020-6850

General assessment

This manuscript it well written, with a clear description of the methods, results and discussion of findings. I think that with minor corrections it would be ready for publication.

Essential revisions that are required to verify the manuscript

Methods:

Study procedure:

1) Please add a reference to the parent study which details the full methods of the parent study.

2) In the third paragraph of this section the authors refer to children young than 18 months with a positive HIV Elisa as being confirmed to have HIV infection. Please double check this - I think it is supposed to be that these children were categorised as HIV- exposed with a confirmatory HIV PCR preformed to determine HIV infection.

Results:

3) The current phrasing of the results suggests that there were no refusals amongst those eligible to participate. Is that accurate?

4) In table 1: HIV status “exposed by negative” – is that children under 18 months or under 6 months? If only those less than 6 months please explained where children between 6-18 months are categorised.

Discussion:

5) 16 patients were found to have different human coronaviruses on the IS and NP samples. Please discuss the implications of this finding. Which would you act on? Does this bring into question the validity of the two methods if they are detecting different viruses in the same patient. I would understand if one method detected additional viruses but to have completely different viruses across the two specimens on the same patient is potentially problematic.

6) The authors note that the study group only incudes hospitalised patients and therefore cannot comment on community transmission/burden. However there have been many community-based surveillance programmes to track respiratory virus burdens and transmission patterns in SA (including work by Sharon Cohen, NICD) – it would be helpful to the readers if the authors could review some of these publications and comment on relevant similarities or differences. (most recent of these publications can be found here: https://crdm.nicd.ac.za/projects/phirst-c/)

7) Please review and redraft this paragraph 5 in the discussion – starting “compared to RSV and other respiratory viruses…”. I could not follow what the authors are trying to say in this paragraph.

8) Please add in a limitations/recommendations section.

Conclusion:

9) The final paragraph of the conclusion raises some interesting questions but does not fit as part of the conclusion. I suggest moving this to be included as part of the discussion, and more fully discussing the questions raised regarding the value of testing for disease if they do not cause severe disease, nor change treatment strategies.

Other suggestions to improve the manuscript

• Stylistic preferences in the introduction: avoid using “for example” when describing work referenced. Either just add the reference number, or use phrasing such as “as shown by (author name) who found …” etc.

• Methods - Statistics: Typo in the last paragraph – please confirm if Stata 13 or stata 16, and add in the necessary stata package reference.

Decision

Requires revisions: The manuscript contains objective errors or fundamental flaws that must be addressed and/or major revisions are suggested.

Discussion, revision and decision

Decision

Verified with reservations: The content is scientifically sound, but has shortcomings that could be improved by further studies and/or minor revisions.

Dr. Bañuelos: Verified manuscript

Dr. Morris: Verified with reservations

Revision

Response to Reviewer 1 (Dr. Bañuelos)

1. Most importantly, I would like to see an introduction that explains the authors’ general arguments about grading changes – including the trajectory of these changes at Dalhousie and why this arc contributes to our knowledge of the history of higher education more broadly. Then, the authors might continually remind us of the arc they present at the outset of their paper – especially when they are highlighting a piece of evidence that illustrates their central argument. To me, the quotes from students and faculty responding to grading changes are among the most interesting parts of the paper and placing these in additional context should make them shine even more brightly!

Our Response: Thank you so much for your thoughtful review. We have added a larger new introduction section of the paper (paragraphs 1-5 in the latest draft are new) that outlines the general importance of the topic, the Canadian context, details on Dalhousie University, and our overall thesis statement (i.e., most decisions were to improve the external communication value of grades). Moreover, we have added three new student quotes form the Dalhousie Gazette to build a stronger picture for student reactions, and to build a better case for our overall thesis statement (i.e., that changes in grading were often to increase the external communication value of grades). Moreover, throughout we have added some details on the overall funding trajectory for institutions in Canada that created some pressure to standardize grading. We think that these changes have improved the manuscript.

1. I’d like to read a little more about Dalhousie itself – why it is either a remarkable or unremarkable place to study changes in grading policies. Is it representative of most Canadian universities and thus, a good example of how grading changes work in this national context? Is it unlike any other institution of higher education and thus, tells us something important about grades that we could not learn from other case studies? I don’t think this kind of description needs to be particularly long, but it should be a little more involved than the brief sentences the authors currently include (p.3, paragraph 1) and should explain the choice of this case.

Our Response: This comment revealed that two additional pieces of context were needed for the introduction: (a) some national context for higher education policy in Canada and (b) some extended description of Dalhousie University when compared to other universities in Canada. To this end, two new paragraphs have been added to the paper (paragraphs 2 & 3 in the current draft).

Notably, Jones (2014) notes that “Canada may have the most decentralized approach to higher education than any other developed country on the planet” (pg 20). With this in mind, any historical review of education policy is by necessity specific to province and institution – that is, the information can be placed in its context, but resists wide generalization to the country as a whole. In the newest draft, we tried to describe the national, provincial, and institutional context in some more detail in paragraphs 2 & 3.

1. I’d also like to know more about the archival materials the authors used. The authors mention that they drew from “Senate minutes, university calendars, and student newspapers” (p. 3), but what kinds of conversations about grades did these materials include? At various points, the authors engage in “speculation” (e.g. p.4) about why a particular change occurred. This is just fine and, in fact, it’s good of the authors to remind us that they are not really sure why some of these shifts happened. But, they might go one step further and tell us why they have to speculate. Were explicit discussions of grading changes – including in inter- and intradepartmental letters and memo, reports, and other documents – not available in these archives? Why are these important discussions absent from the historical record?

Our Response: We have added a new paragraph (paragraph 4) to the paper discussing the sources in some more detail. It is true that the verbatim discussions are frequently absent from the record, especially earlier in history – or if they exist, we have not found them! Instead, we frequently are reviewing meeting minutes or committee reports, which are summaries of discussions. As we now note in the paper, “Thus, the sources used showed what policy changes were implemented, when they were implemented, and a general sense of whether there was opposition to changes; however, there were notable gaps in faculty and student reactions to grade policy changes, as these reactions were frequently not written down and archived.”

This gap was most apparent in the Senate minutes around the 1940s, where I (the first author) could not find any direct discussions of why changes were implemented. Under the 1937-1947 heading, we more clearly indicate that the rationale for the changes was absent from the Senate minutes during this period. I add some further speculation on why these records might be absent, based on summaries from Waite (1998b); specifically, the university president of the time often made unilateral decisions, circumventing Senate, which might account for why the changes are absent from the records.

This will hopefully make the limitations of what can be learned from this approach more apparent.

1. At various points, the authors make references to the outside world – for example, WWII (p. 5), the Veteran’s Rehabilitation Act (pp. 6-7), and British versus American grading schemas (p. 6). But, these references are brief and seem almost off-handed. I know space is limited, but putting these grading changes in their broader context might help make the case for why this study is interesting and important. Are the changes in the 1940s, for example, related to the ascendance of one national graduate education model over another (e.g. American versus British)? Are there any data on how many Canadian undergraduates enrolled in British versus American graduate programs over time? If so, I would share any information you might have on these broader trends.

Our Response: To our knowledge, there isn’t any comparable report to what we’ve written here documenting the transition from British “divisions” to American “letter grades” in Canadian Universities, making our report novel in this regard. It might well be that a similar historical arc exists in many of the 223 public and private universities in Canada, but we don’t believe such data exists in any readily accessible way – excepting perhaps undergoing a similar deep dive into historical documents at each respective institution! So, we do not have the answer to your question: “Are there any data on how many Canadian undergraduates enrolled in British versus American graduate programs over time?” However, we did add one reference which provided a snapshot point of comparison in 1960, noting in the paper “Baldwin (1960) notes that the criteria for “High First Class” grades in the humanities was around 75-80% at Universities of Toronto, Alberta, and British Columbia in 1960, suggesting that Dalhousie’s system was similar to other research-intensive universities around this time.” That said, there are a few major national events related to the funding of universities in Canada that we have elaborated on in the text to address the spirit of your recommendation for describing the national context:

a) In the “Late 1940s” section of the paper, we added: “Though Dalhousie had an unusually high proportion of veterans enrolled relative to other maritime universities during this period (Turner, 2011), the Veteran’s Rehabilitation Act was a turning point for large increases in enrollment and government funding Canada-wide, at least until the economic recession of the 1970s (Jones, 2014).”

b) In the 1990s, there were major government cuts to funding, creating challenging financial times for the university. We discuss the funding pressures that likely contributed to standardization of grading during this time by saying the following in the 1980s-2000s section: “Starting in in the 1980s-1990s there were major government cuts to university funding nation-wide, with the cuts becoming more severe in the 1990s (Jones, 2014; Higher Education Strategy Associates, 2021). Because of the nature of the funding formulas, cuts in Nova Scotia were especially deep. Beyond tuition increases, university administrators knew that obtaining external research grants, Canada Research Chairs, and scholarship funding was one of the few other ways for a university to balance budgets, so there was extra pressure to be competitive in these pools. […] The increased standardization was likely related to increased financial pressures at this time – standardization is an oft-employed tool to deal with ever-increasing class sizes with no additional resources.”

c) In the 2010s section of the paper, we added context to how universities in country-wide have become increasingly dependent on tuition fees for funding: “Following the 2008 recession, federal funding decreased again (Jones, 2014; Higher Education Strategy Associates, 2021); however, this time universities tended to balance budgets by increasing tuition and international student fees. This trend towards increased reliance on tuition for income is especially pronounced in Nova Scotia, which has the highest tuition rates in the country (Higher Education Strategy Associates, 2021). Thus, the university moved closer to a “consumer” model of education, so it makes sense that a driving force for standardization was student complaints.”

1. This is a very nitpicky concern that doesn’t fit well elsewhere, so please take it with a grain of salt. I was surprised at the length of the reference list – it seemed quite short for a historical piece! I wonder, again, if more description of the archival material - including why you looked at these sources, in particular, and what was missing from the record – would help explain this and further convince the reader that you have all your bases covered.

Our Response: In the introduction section, paragraph 4, we describe our sources in more detail including what is likely missing from the record and why we used them. Regarding the length of the reference list, we did add ~12 new references to the list in the course of making various revisions, which partially addresses your concern. Beyond this though, it’s worth noting that some of the sources more extensive than they seem, even though they don’t take up much space in the reference list (e.g., there is one entry for course calendars, but this covers ~100 documents reviewed!). Moreover, there were many dead-ends in the archives that are not cited (e.g., reviewing 10 years of Senate minutes in the 1940s produced little of relevance), so the reference list is curated to only those sources where relevant materials were found.

Reviewer response to revisions

The new introduction to the piece addresses many of my previous questions about the authors’ general arguments, the Dalhousie context, and the source material. Thank you for addressing these! Reading this version, it is much clearer that the key argument is that standardized, centralized grading practices were “to improve the external communication value of the grades, rather than for pedagogical reasons” (p. 6). I also really enjoyed the added quotes from students in the Dalhousie Gazette.

The authors’ response to Reviewer 2 really gave me a better sense of why they wrote this piece and also helped me to more clearly put my finger on what was troubling me in the first round. It still reads a little like a report for an internal audience – which is just fine and, in fact, can be extremely useful for historians of the future. But, as Reviewer 2 notes, this means it does not really seem like a piece of historical scholarship. I do worry that shaping it into this form would take an extensive revision and might not be in the spirit of what the authors intended to do.

A different version of this article might start with this idea that grades were standardized for external audiences and in response to financial pressures. It would then develop a richer story behind the sudden importance of these external audiences and the nature (i.e. source, type) of financial pressures Dalhousie was facing. It would highlight the impact such changes had on students and their future careers/graduate experiences. It could then connect these trends to other similar changes for external audiences and the increasing interconnectedness of American, Canadian, and British systems through graduate education. It might even turn to sociological theories of organizational change and adaptation and make an argument for when (historically) similar forms of decoupling were likely to occur in the Canadian higher education system. Finally, it might connect these grading changes to current trends – including accusations of grade inflation and accepted best practices for measuring learning outcomes.

But, it doesn’t seem that the authors necessarily want to do this, which I can understand and respect. I think there is enormous value in a piece of scholarship like this existing – both for internal audiences and for future historians. Indeed, imagine if every university had a detailed history of its grading policies like this available somewhere online! Comparing such practices across institutions would certainly tell us a lot about why grading currently looks the way it does.

Decision changed

Verified manuscript: The content is scientifically sound, only minor amendments (if any) are suggested.

Response to Reviewer 2 (Dr. Morris)

The authors dove headfirst into Dalhousie’s archives, unpacking the subtle shifts in grading policy. Their work seems to be comparable to archaeologists, digging deep beneath mountains of primary sources to find nuggets of clues into Dalhousie’s grading evolution. I particularly liked when the authors were able to link these changes to student voices, as seen in moments when they referenced student publications.

Ultimately, I kept coming back to one main comment that I wrote in the margins: “So what?” I would humbly suggest that the authors reflect on why this history matters to them. Granted, they do this in the conclusion, where they touch on Schneider & Hutt’s argument that grades evolved to increasingly be a form of external communication with audiences beyond school communities. Sure. But I want more. I wanted to see a new insight that this microhistory of Dalhousie significant to the history of Canada or the history of education more generally.

If the authors are so inclined, there might be several approaches to transform this manuscript. I would suggest the following. First, instead of tracing the entire history of grading at the institution, choose one moment of change that you think is the most important. Perhaps in the 1920s and the lack of transparency in grading, or the post-war shift toward American grading. Second, show me – don’t tell me – what Dalhousie was like at this moment. Paint a picture of the institution with details about student demographics, curriculum, educational goals, the broader town, etc. Make the community come alive. Show me what makes Dalhousie unique from other institutions of higher ed. Once you establish that picture, perhaps you could link the change in grading practices to subtle changes at the university community, thereby establishing a before and after snapshot. This will require considerable amounts of work, and the skills of a historian. You will have to find primary and secondary sources that go far beyond what you’ve relied on thus far.

In the end, I found myself wanting the authors to humanize this manuscript, meaning I wanted them to show me that changes in grading practices have tangible effects on real-life human beings. A humanization of their research would mean going narrower and deeper; or, in other words, eliminating much of what they have documented.

However, if that is too tall of an order, I would ask that the authors clarify for themselves who this manuscript is for. Is this a chronicling of facts for an internal audience at Dalhousie’s faculty, alumni, and students? Fine. But my guess is that even members of the Dalhousie community want to read something relatable.

I am suggesting revisions, although not because of objective errors. History is more of an art, in my opinion. With that in mind, I would suggest that the authors paint a more vivid picture (metaphorically) of Dalhousie, showing me how changes one moment of change in grading practices impacted the lives of human beings.

Our Response: Thank you very much for taking the time to read our paper and provide your thoughts and recommendations. It may be helpful to begin by describing why I (the first author) decided to write this paper. Ultimately, I wrote this paper to satisfy my own personal curiosity and to connect with other people at my own place of employment by exploring our shared history. At present day, Dalhousie has a letter grading scheme with a standardized percentage conversion scheme that all instructors used. I wanted to know why this particular scheme was used, but I quickly realized that nobody at Dalhousie really knew how we ended up grading this way! There was an institutional memory gap, and a puzzle that was irresistible to me. So, I wrote this paper for the most basic of all academic reasons: Pure curiosity. I do very much recognize that the subject matter is very niche, perhaps too niche for a traditional journal outlet. Thus, my publishing plan is to self-publish a manuscript to the Education Resources Information Center (ERIC) database and a preprint server as a way of sharing my work with others who might be interested in what I found. Nonetheless, I believe in the importance and value of peer review, especially since I am writing in a field different than most of my scholarly work. That is why I chose PeerRef as a place to submit, so that I could undergo rigorous peer review to improve the work while still maintaining the niche subject matter and focus that drives my passion and curiosity for the project. Of course, if you feel the whole endeavor is so flawed that it precludes publication anywhere, then we can consider this a “rejection” and I will not make any further edits through PeerRef.<br /> The core of your critique suggested that I should write a fundamentally different paper on different subject matter. While I don’t necessarily disagree that the kind of paper you describe might have broader appeal, it would no longer answer the core research question I wanted an answer to: How has Dalhousie’s grading changed over time? So, I must decline to rewrite the paper to focus on a single timeframe as recommended. All this said, I did try my best to address the spirit of your various concerns to improve the quality of the manuscript. Below, I will outline the various major changes to the manuscript that we made to improve the manuscript along the lines you described, while maintaining our original vision for the structure and focus of the paper. The specific changes are outline below:

a) Two new paragraphs (now paragraphs 1-2 of the revised manuscript) were added to explain the “so what” part of the question. Specifically, we describe why we think the subject matter might be of interest to others and summarize the general dearth of historical information on grading practices in Canada as a whole.

b) Consistent with recommendations from the other reviewer, we now state a core argument (i.e., that most major grading changes were implemented to improve the external communication value of the grades) earlier in the introduction in paragraph 5 and describe how various pieces of evidence throughout the manuscript tie back to that core theme.

c) In an attempt to “humanize” the manuscript more, we added more student quotes from the Dalhousie Gazette throughout the paper so that readers can get a better sense of how students thought about grading practices at various times throughout history. Specifically, three new quotes were added in the following sections: 1901-1936, late 1940s, 1950s-1970s. We also added this short note about the physical location where grades used to be posted: “Naturally, this physical location was dreaded by students, and was colloquially referred to as “The Morgue” (Anonymous Dalhousie Gazette Author, 1937).”

d) Early in the paper, we describe why we chose Dalhousie and the potential audience of interest: “As employees of Dalhousie, we naturally chose this institution as a case study due to accessibility of records and because it has local, community-level interest. The audience was intended to be members of the Dalhousie community; however, it may also be a useful point of comparison for other institutions, should similar histories be written.”

e) We have described some of the limitations of our sources in paragraph 4, which may explain why the manuscript takes the form it does – it has conformed to the information that is available!

f) We have linked events at Dalhousie to the national context in some more detail, by detailing some national events related to the funding of universities in Canada. See our response to Reviewer 1, #4 above for more details on the specific changes.

g) Consistent with your stylistic recommendations, we have changed various spots throughout the paper from the present tense (e.g., “is”) to the past tense (e.g., “was”), and were careful in our new additions to maintain the past tense, when appropriate. If there are any spots that we missed, let us know the page number / section, and we will make further changes, as necessary.

h) We retained the first person in our writing – this may be discipline-specific, but in Psychology (the first author’s home discipline), first person is acceptable in academic writing. If you feel strongly about this, we can go through the manuscript and remove all instances of the first person, but we would prefer to keep it, if at all possible.

Hopefully this helps address the spirit of your concerns, and I look forward to hearing your thoughts in the second round of reviews.

Decision changed

Verified with reservations: The content is scientifically sound, but has shortcomings that could be improved by further studies and/or minor revisions.

Author response

To: Reviewer: Heikki Vapaatalo, MD, PhD, Emeritus professor of Pharmacolog

Dear reviewer Thank you very much for the insightful suggestions, the manuscript improved a lot with the changes performed. Please find the point-by-point answer to the raised questions. In the main text, all changes are highlighted in yellow. I hope that with the changes made the new version is suitable for publication.

Best regards Valquiria Bueno

General assessment

The study is interesting and the title promises for me more than the MS finally contains.

Answer: The manuscript is part of a project aiming to study ACE1 and ACE2 expression in cells from the immune system of aging and young adults. These initial results suggest that ACE1 (and probably ACE2) plays somehow a role in the process of aging.

The background, question and the aim are relevant as explained in the introduction.

Answer: We included a piece of information in the “Introduction” trying to link ACE1 expression in tissue cells and age-related diseases, as it follows:

ACE1 has been suggested to influence age-related diseases (i.e. Alzheimer’s, sarcopenia, cancer) but the associated mechanisms are still under investigation. ACE1 polymorphisms were correlated with susceptibility to Alzheimer’s disease (AD). [15, 16] In addition, it was shown recently that in normal aging ACE1 expression is increased in brain homogenates and this expression is unchanged in early stages of AD. [17] Regarding sarcopenia, Yoshihara et al. [18] found a weak correlation between ACE polymorphism and physical function. In cancer (gastric or colorectal), patients presented higher expression of ECA1 in tumor when compared with healthy tissues. [19, 20]

The major criticism concerns the small size of the material (subjects, n=6), the small age difference (64-67 years) and the lack of younger controls.

Answer: We agree that the small number of studied subjects is a limitation of this study. In spite of the interesting results suggesting that ACE1 expression could be linked to the health status, it was not possible to perform correlation analysis due to the small sample size. Even though there is a small chronological difference between the subjects, the biological aging is very different among them and reflects the genetics, lifestyle, nutrition and comorbidities. Another limitation is the lack of younger controls to compare with the subjects studied. Our next steps are to include younger controls, to increase the number of studied subjects, and if possible to get samples from older subjects (i.e. 70-80, 80 and more years old)

Minor notes:

1)Title: Angiotensin converting enzyme (ACE) expression in leukocytes of older adults

Answer: We evaluated only ACE1 expression, and thus, title, abstract, and main text were changed to ACE1 instead of ACE. We decided to change to title for: Angiotensin converting enzyme (ACE) 1 expression in leukocytes of adults from 64 to 67 years old

2)Introduction: The last chapter, the Author should explain in more detail, how references 11-14 suggest that “ACE play an important role in the aging process”. Does this mean, that ACE is somehow regulating the aging process or in increasing age ACE -levels are changed?

Answer: References 11-14 shows that age-related diseases occurring in older adults are associated with changes in the immune system. To complete the text we added:

ACE1 has been suggested to influence age-related diseases (i.e. Alzheimer’s, sarcopenia, cancer) but the associated mechanisms are still under investigation. ACE1 polymorphisms were correlated with susceptibility to Alzheimer’s disease (AD). [15, 16] In addition, it was shown recently that in normal aging ACE1 expression is increased in brain homogenates and this expression is unchanged in early stages of AD. [17] Regarding sarcopenia, Yoshihara et al. [18] found a weak correlation between ACE polymorphism and physical function. In cancer (gastric or colorectal), patients presented higher expression of ECA1 in tumor when compared with healthy tissues. [19, 20]

Material and Methods:

The N-value of the subjects should be mentioned here, as well the relation of females/males.

Answer: Text was correct as suggested Blood was collected from adults (n=6, four females and two males) aged 64-67 years old in 2015.

Do the Authors really regard 64-67 “older age” nowadays?

Answer: Nowadays the most common term used for individuals older than 65 years is “older adults”.

Why first many years later the assays have been done in comparison to the collection of the blood? Are the samples still useable, not destroyed?

Answer: Samples are part of UNIFESP Biobank and have been kept in adequate conditions. We wanted to test cells from a period anterior to COVID-19 and those samples were the only ones that attended our purpose. We compared samples used in this study with fresh blood samples (cell viability and percentage of CD4+, CD8+ and CD19+) and the results showed good preservation of the cells.

Did the subjects have some diseases and/or drugs because the possibly were from hospital sample bank?

Answer: Samples are part of UNIFESP Biobank, but unfortunately we do not have information about diseases and medicaments.

Express the company details similarly than Amersham, cities and countries.

Answer: Changes were done as required ACE CD143 FITC (R&D Systems, Inc, Minneapolis, USA)

Results:

“Table 1 shows that older adults…..” The comparison between the present data and historical studies belongs to the Discussion.

Answer: Changes were done as required

Give also individual ages and gender of the subjects in the table 1.

Answer: The manuscript version sent to medrxiv@medrxiv.org had age and gender on tables, but due to their request, any possible variable that could identify the studied individual had to be removed. That is why in the present version these variables are not shown.

What means p-values here? Compared with which or interindividual differences in the particular variable? Should be explained

Answer: We used p-value for interindividual differences in each variable since individuals age differently (biological aging) and thus, physiological parameters could be affected by genetics, lifestyle, nutrition, and comorbidities. It is now explained in materials and methods

The numbering of tables and the text seems to me confusing. Only three tables, but in the text mentioned four. Number 4 does not exist.

Answer: For some reason table 2 is missing in the main text, please find the new version with Table 2 included

It would be good to have a list of abbreviations used in the description of the cell types for an unfamiliar reader.

Answer: In each figure and table we are now providing a description of cells evaluated.

Discussion:

A major part of the discussion deals with previous publications and not meaning or clinical significance of the present findings and comparison between the present and earlier studies.

Answer: The discussion was changed as suggested:

Our results show that for the studied population, chronological aging and biological aging don´t go at the same pace. Even individuals having a small chronological difference (64 to 67 years old), they are heterogeneous for physiological parameters such as glucose, urea, glycated hemoglobin (Hbglic), and C-reactive protein (CRP). Changes in the same functional parameters have been reported by Carlsson et al. [22] and Helmerson-Karlqvist [23] in healthy older adults. Carlsson’s study [22] found that CRP value was 2.6 with a coefficient variation of 1.4% whereas in our study, it was observed higher values of CRP in 5 out of 6 individuals. Increased CRP levels has been associated with inflammaging and our findings show that the studied population has changes in functional parameters which are likely associated with an inflammatory profile. [24] The link between RAS and inflammation has been suggested but its role is not completely clear under physiological and pathological conditions. [25, 26] In addition, the association between ACE1 altered expression in tissues (brain, muscle, heart and vessels) and the development and progression of age-related conditions such as Alzheimer’s, sarcopenia, and cardiovascular disease has been suggested but results are controversial. [17, 27, 28, 29, 30] There are few studies showing the association between ACE1 expression in cells from the immune system (monocytes, T cells) and the progression of kidney and cardiovascular disease. [9, 8, 31, 32]. Therefore, considering the lack of information on this issue, we questioned whether ACE1 (CD143) was highly expressed in cells from the immune system during the aging process. We found that ACE1 was expressed in almost 100% of T (CD4+, CD8+) and B lymphocytes and in all phenotypes of these cells. In non-lymphoid cells, ACE1 mean expression was 56,9%. In agreement with our findings, independent studies showed that T cells from healthy donors and monocytes from patients with congestive heart failure expressed ACE1, but there was no investigation on cell phenotype. [25, 26]. Our study is the first to show that either inexperienced (naive) or fully activated (memory) cells expresses ACE1. Our findings suggest a that the expression of ACE1 in lymphoid and non-lymphoid cells reflects the health status since our studied population presented changes in physiological parameters and high levels of ACE1 expression on immune cells. Previous independent studies showed that patients with unstable angina [32] or acute myocardial infarction [33] presented higher expression of ACE1 in T cells and dendritic cells than controls subjects. In addition, markers of the cell (lymphoid and non-lymphoid) functional status such as inflammatory or growth factors production could be modulated by ACE inhibitors (ACEi). Accordingly, mononuclear leukocytes from healthy subjects incubated with endotoxin exhibited high levels of tissue factor activity which was reduced in the presence of captopril in a dose-dependent pattern. This result could be related to the antithrombotic effect of ACEi. [34]. In patients with congestive heart failure, immune cells cultured with LPS secreted high levels of the pro-inflammatory TNF-alpha and these levels were significantly reduced in the presence of captopril. [35]

In those previous studies, also ACE2 has been reported, why not studied here?

Answer: Our next studies will be focused on ACE1 and ACE2 expression in cells from the immune system in both younger and older adults.

In the limitations, the Authors fairly mention the real problem: The small sample size, and I would like to say lack of younger subjects.

Answer: we agree with the limitations pointed and the text was changed as required:

This study have limitations such as the small sample size and the lack of young adults for comparison. As an example, the subject with the highest CRP and albumin also exhibited a high percentage of ACE1 expression on T (CD4+, CD8+), B and non-lymphoid cells in addition to the lowest percentage of CD4+ naive cells, and the highest percentage of CD8+ terminally differentiated (EMRA) and DN B cells. However, due to the small sample size it was not possible to associate the high expression of ACE1 on immune cells with inflammaging and immunosenescence. It would bring important information to correlate physiological parameters/health status with ACE1 expression and to find out whether age and associated chronic diseases could lead to increased ACE1 expression.

The COVID-19 point even tempting today, is too far from this study and unnecessary. Answer: Our point was to emphasize the negative impact of chronic diseases for the outcome of aging population during a viral infection and how ACE1/ACE2 expression could bring information to diagnosis and treatment. Therefore, we would like to maintain this piece of information.

Linguistic checking would improve the MS. Answer: We checked for possible linguistic mistakes

Reviewer, Heikki Vapaatalo:

I read with pleasure the very detailed answers to my comments.

I very warmly recommend acceptance of this MS for publications without any further notes.

Decision changed:

Verified manuscript: The content is scientifically sound, only minor amendments (if any) are suggested.

To: Reviewer: Calogero Caruso

Dear Prof.Caruso Thank you very much for the revision of this manuscript. It is a privilege to have a manuscript reviewed by a research with high expertise on the field of ageing. Please find the answers to your questions and in the main text the changes in bold.

Sincerely yours,

Valquiria Bueno

The paper is essentially anecdotal because it studies the cells of 6 subjects without any comparison with other age groups. There is also a serious limitation because beyond the age and sex there is no information on the donors (how and why they were recruited, what drugs they took, etc.).

It is really a limitation to have only 6 individuals for the study, but they were the only ones fitting in the proposal of the manuscript. The samples were from a central bank of cells at UNIFESP and participants were considered “healthy” but there was not further information in addition to what we displayed on the tables of the manuscript. They were not living on homecares or hospitalized.

Our aim was to evaluate samples from individuals aged 60-69 years previously to COVID-19 and/or vaccination. In addition, there were no samples in the same conditions (PBMCs, -80oC) of young individuals and using fresh blood could bring a result that could not be compared mainly regarding to myeloid cells and B cells as is follows in the below reference. Braudeau C, Salabert-Le Guen N, Chevreuil J, Rimbert M, Martin JC, Josien R. An easy and reliable whole blood freezing method for flow cytometry immuno-phenotyping and functional analyses. Cytometry B Clin Cytom 2021;100(6):652-665. doi: 10.1002/cyto.b.21994.

Our goal from now on is to expand this study with young and old adults samples since it is important to understand whether ageing is associated with an increase in ACE expression on immune cells.

-To infer that chronological and biological ages do not match is inappropriate in the absence of the above information.

This piece of information regarding chronological and biological age was required by another reviewer. I agree that the concept does not match without more information on the donors. However, the information is now referenced and should be considered when older adults are studied. Vasto S, Scapagnini G, Bulati M, Candore G, Castiglia L, Colonna-Romano G, Lio D, Nuzzo D, Pellicano M, Rizzo C, Ferrara N, Caruso C. Biomarkes of aging. Front Biosci (Schol Ed) 2010;2(2):392-402. doi: 10.2741/s72. PMID: 20036955.

-However, the paper is of some interest because there are few studies on the topic.

Thanks for this positive comment. Few studies on the topic was the reason why we decided to send the manuscript for publication even though there were some important information on the donors missing and limited number of individuals.

Essential revisions that are required to verify the manuscript

1) Although we do not have data on donors, placing an age and gender column in all tables adds a minimum of useful information for the reader.

The first table submitted with age, but for requirement of MedRxiv, gender and age could no be linked to the metabolic results to preserve the anonymity of the donors.

2) Inflamm-ageing means low grade of inflammation. The value of CRP 23.1 suggests acute inflammation (also because albumin has high values, while in chronic inflammation its values decrease). Therefore the Ly averages do not have to take this subject into account.

Thank you for this comment. In a review of literature it was found an article (below) with CRP variation from 0.1 to 19.8 (Heumann Z, Youssim I, Kizony R, Friedlander Y, Shochat T, Weiss R, Hochner H, Agmon M. The Relationships of Fibrinogen and C-Reactive Protein With Gait Performance: A 20-Year Longitudinal Study. Front Aging Neurosci 2022;14:761948. doi: 10.3389/fnagi.2022.761948). There is also an article from your group showing CRPs <5g/dL and >5g/dL (Cancemi P, Aiello A, Accardi G, Caldarella R, Candore G, Caruso C, Ciaccio M, Cristaldi L, Di Gaudio F, Siino V, Vasto S. The Role of Matrix Metalloproteinases (MMP-2 and MMP-9) in Ageing and Longevity: Focus on Sicilian Long-Living Individuals (LLIs). Mediators Inflamm 2020;2020:8635158. doi: 10.1155/2020/8635158) that will be used to discuss how ageing impacts CRP levels. Considering the already small number of donors, data were maintained and statistics (mean + SD) with and without 23.1 mg/dL are now shown.

This will be the new version (discussion) about CRP Carlsson’s study [22] found that CRP value was 2.6 with a coefficient variation of 1.4% whereas in our study, it was observed higher values of CRP in 5 out of 6 individuals. In addition, it was shown by Cancemi et al. in an evaluation of individuals from 40 years to older than 95 years (long-living) that CRP increases in an age-dependent manner. Increased CRP levels has been associated with low grade of chronic inflammation (inflammaging) and our findings show that the studied population has changes in functional parameters which are likely associated with an inflammatory profile. [24] However, an individual presented CPR 23.1 mg/dL suggesting acute inflammation instead, but as all donors were not hospitalized or living on homecares, this sample was considered as part of the study. Another study evaluating gait speed found CRPs varying from 0.1 to 19.8mg/dL (Front Aging Neurosci 2022;14:761948.). Our study has an important limitation that is the lack of data on donors such as the use of continuous medicaments or sarcopenia, hypertension, cognition, among others, and thus it was not possible to correlate CRP with age-related conditions.

Table 1. Updated

Other suggestions to improve the manuscript The authors write that their findings suggest that ACE1 could play a role in several processes linked to aging including the generation and activation of autoimmune cells, due to the experimental evidence that inhibitors of ACE suppress the autoimmune process in a number of autoimmune diseases such as EAE, arthritis, autoimmune myocarditis. [49] They do not appear to have these findings in their paper. So, it needs to change the sentence.

Sentence changed to: According to experimental evidence, ACE inhibitors suppress the autoimmune process in a number of autoimmune diseases such as EAE, arthritis, autoimmune myocarditis. [49] Extrapolating these findings to our results, it is possible to suggest that ACE1 play a role in several processes linked to aging including the generation and activation of autoimmune cells.

Rviewer: Calogero Caruso

Decision changed:

Verified manuscript: The content is scientifically sound, only minor amendments (if any) are suggested.

Peer review report

Title: Angiotensin converting enzyme (ACE) 1 expression in leukocytes of adults from 64 to 67 years old

version: 2

Referee: Calogero Caruso MD

Institution: Professor Emeritus, University of Palermo

email: Calogero.caruso@unipa.it

ORCID iD: 0000-0001-8004-2363

General assessment

The paper is essentially anecdotal because it studies the cells of 6 subjects without any comparison with other age groups. There is also a serious limitation because beyond the age and sex there is no information on the donors (how and why they were recruited, what drugs they took, etc.). To infer that chronological and biological ages do not match is inappropriate in the absence of the above information.

However, the paper is of some interest because there are few studies on the topic.

Essential revisions that are required to verify the manuscript

1) Although we do not have data on donors, placing an age and gender column in all tables adds a minimum of useful information for the reader.

2) Inflamm-ageing means low grade of inflammation. The value of CRP 23.1 suggests acute inflammation (also because albumin has high values, while in chronic inflammation its values decrease). Therefore the Ly averages do not have to take this subject into account.

Other suggestions to improve the manuscript

The authors write that their findings suggest that ACE1 could play a role in several processes linked to aging including the generation and activation of autoimmune cells, due to the experimental evidence that inhibitors of ACE suppress the autoimmune process in a number of autoimmune diseases such as EAE, arthritis, autoimmune myocarditis. [49] They do not appear to have these findings in their paper. So, it needs to change the sentence.

Decision

Requires revisions: The manuscript contains objective errors or fundamental flaws that must be addressed and/or major revisions are suggested.

Decision changed:

Verified manuscript: The content is scientifically sound, only minor amendments (if any) are suggested.

Peer review report

Title: Angiotensin converting enzyme (ACE) 1 expression in leukocytes of adults from 64 to 67 years old

version: 2

Referee: Calogero Caruso MD

Institution: Professor Emeritus, University of Palermo

email: Calogero.caruso@unipa.it

ORCID iD: 0000-0001-8004-2363

General assessment

The paper is essentially anecdotal because it studies the cells of 6 subjects without any comparison with other age groups. There is also a serious limitation because beyond the age and sex there is no information on the donors (how and why they were recruited, what drugs they took, etc.). To infer that chronological and biological ages do not match is inappropriate in the absence of the above information.

However, the paper is of some interest because there are few studies on the topic.

Essential revisions that are required to verify the manuscript

1) Although we do not have data on donors, placing an age and gender column in all tables adds a minimum of useful information for the reader.

2) Inflamm-ageing means low grade of inflammation. The value of CRP 23.1 suggests acute inflammation (also because albumin has high values, while in chronic inflammation its values decrease). Therefore the Ly averages do not have to take this subject into account.

Other suggestions to improve the manuscript

The authors write that their findings suggest that ACE1 could play a role in several processes linked to aging including the generation and activation of autoimmune cells, due to the experimental evidence that inhibitors of ACE suppress the autoimmune process in a number of autoimmune diseases such as EAE, arthritis, autoimmune myocarditis. [49] They do not appear to have these findings in their paper. So, it needs to change the sentence.

Decision

Requires revisions: The manuscript contains objective errors or fundamental flaws that must be addressed and/or major revisions are suggested.

Decision changed:

Verified manuscript: The content is scientifically sound, only minor amendments (if any) are suggested.

Author response

To: Reviewer: Heikki Vapaatalo, MD, PhD, Emeritus professor of Pharmacolog

Dear reviewer Thank you very much for the insightful suggestions, the manuscript improved a lot with the changes performed. Please find the point-by-point answer to the raised questions. In the main text, all changes are highlighted in yellow. I hope that with the changes made the new version is suitable for publication.

Best regards Valquiria Bueno

General assessment

The study is interesting and the title promises for me more than the MS finally contains.

Answer: The manuscript is part of a project aiming to study ACE1 and ACE2 expression in cells from the immune system of aging and young adults. These initial results suggest that ACE1 (and probably ACE2) plays somehow a role in the process of aging.

The background, question and the aim are relevant as explained in the introduction.

Answer: We included a piece of information in the “Introduction” trying to link ACE1 expression in tissue cells and age-related diseases, as it follows:

ACE1 has been suggested to influence age-related diseases (i.e. Alzheimer’s, sarcopenia, cancer) but the associated mechanisms are still under investigation. ACE1 polymorphisms were correlated with susceptibility to Alzheimer’s disease (AD). [15, 16] In addition, it was shown recently that in normal aging ACE1 expression is increased in brain homogenates and this expression is unchanged in early stages of AD. [17] Regarding sarcopenia, Yoshihara et al. [18] found a weak correlation between ACE polymorphism and physical function. In cancer (gastric or colorectal), patients presented higher expression of ECA1 in tumor when compared with healthy tissues. [19, 20]

The major criticism concerns the small size of the material (subjects, n=6), the small age difference (64-67 years) and the lack of younger controls.

Answer: We agree that the small number of studied subjects is a limitation of this study. In spite of the interesting results suggesting that ACE1 expression could be linked to the health status, it was not possible to perform correlation analysis due to the small sample size. Even though there is a small chronological difference between the subjects, the biological aging is very different among them and reflects the genetics, lifestyle, nutrition and comorbidities. Another limitation is the lack of younger controls to compare with the subjects studied. Our next steps are to include younger controls, to increase the number of studied subjects, and if possible to get samples from older subjects (i.e. 70-80, 80 and more years old)

Minor notes:

1)Title: Angiotensin converting enzyme (ACE) expression in leukocytes of older adults

Answer: We evaluated only ACE1 expression, and thus, title, abstract, and main text were changed to ACE1 instead of ACE. We decided to change to title for: Angiotensin converting enzyme (ACE) 1 expression in leukocytes of adults from 64 to 67 years old

2)Introduction: The last chapter, the Author should explain in more detail, how references 11-14 suggest that “ACE play an important role in the aging process”. Does this mean, that ACE is somehow regulating the aging process or in increasing age ACE -levels are changed?

Answer: References 11-14 shows that age-related diseases occurring in older adults are associated with changes in the immune system. To complete the text we added:

ACE1 has been suggested to influence age-related diseases (i.e. Alzheimer’s, sarcopenia, cancer) but the associated mechanisms are still under investigation. ACE1 polymorphisms were correlated with susceptibility to Alzheimer’s disease (AD). [15, 16] In addition, it was shown recently that in normal aging ACE1 expression is increased in brain homogenates and this expression is unchanged in early stages of AD. [17] Regarding sarcopenia, Yoshihara et al. [18] found a weak correlation between ACE polymorphism and physical function. In cancer (gastric or colorectal), patients presented higher expression of ECA1 in tumor when compared with healthy tissues. [19, 20]

Material and Methods:

The N-value of the subjects should be mentioned here, as well the relation of females/males.

Answer: Text was correct as suggested Blood was collected from adults (n=6, four females and two males) aged 64-67 years old in 2015.

Do the Authors really regard 64-67 “older age” nowadays?

Answer: Nowadays the most common term used for individuals older than 65 years is “older adults”.

Why first many years later the assays have been done in comparison to the collection of the blood? Are the samples still useable, not destroyed?

Answer: Samples are part of UNIFESP Biobank and have been kept in adequate conditions. We wanted to test cells from a period anterior to COVID-19 and those samples were the only ones that attended our purpose. We compared samples used in this study with fresh blood samples (cell viability and percentage of CD4+, CD8+ and CD19+) and the results showed good preservation of the cells.

Did the subjects have some diseases and/or drugs because the possibly were from hospital sample bank?

Answer: Samples are part of UNIFESP Biobank, but unfortunately we do not have information about diseases and medicaments.

Express the company details similarly than Amersham, cities and countries.

Answer: Changes were done as required ACE CD143 FITC (R&D Systems, Inc, Minneapolis, USA)

Results:

“Table 1 shows that older adults…..” The comparison between the present data and historical studies belongs to the Discussion.

Answer: Changes were done as required

Give also individual ages and gender of the subjects in the table 1.

Answer: The manuscript version sent to medrxiv@medrxiv.org had age and gender on tables, but due to their request, any possible variable that could identify the studied individual had to be removed. That is why in the present version these variables are not shown.

What means p-values here? Compared with which or interindividual differences in the particular variable? Should be explained

Answer: We used p-value for interindividual differences in each variable since individuals age differently (biological aging) and thus, physiological parameters could be affected by genetics, lifestyle, nutrition, and comorbidities. It is now explained in materials and methods

The numbering of tables and the text seems to me confusing. Only three tables, but in the text mentioned four. Number 4 does not exist.

Answer: For some reason table 2 is missing in the main text, please find the new version with Table 2 included

It would be good to have a list of abbreviations used in the description of the cell types for an unfamiliar reader.

Answer: In each figure and table we are now providing a description of cells evaluated.

Discussion:

A major part of the discussion deals with previous publications and not meaning or clinical significance of the present findings and comparison between the present and earlier studies.

Answer: The discussion was changed as suggested:

Our results show that for the studied population, chronological aging and biological aging don´t go at the same pace. Even individuals having a small chronological difference (64 to 67 years old), they are heterogeneous for physiological parameters such as glucose, urea, glycated hemoglobin (Hbglic), and C-reactive protein (CRP). Changes in the same functional parameters have been reported by Carlsson et al. [22] and Helmerson-Karlqvist [23] in healthy older adults. Carlsson’s study [22] found that CRP value was 2.6 with a coefficient variation of 1.4% whereas in our study, it was observed higher values of CRP in 5 out of 6 individuals. Increased CRP levels has been associated with inflammaging and our findings show that the studied population has changes in functional parameters which are likely associated with an inflammatory profile. [24] The link between RAS and inflammation has been suggested but its role is not completely clear under physiological and pathological conditions. [25, 26] In addition, the association between ACE1 altered expression in tissues (brain, muscle, heart and vessels) and the development and progression of age-related conditions such as Alzheimer’s, sarcopenia, and cardiovascular disease has been suggested but results are controversial. [17, 27, 28, 29, 30] There are few studies showing the association between ACE1 expression in cells from the immune system (monocytes, T cells) and the progression of kidney and cardiovascular disease. [9, 8, 31, 32]. Therefore, considering the lack of information on this issue, we questioned whether ACE1 (CD143) was highly expressed in cells from the immune system during the aging process. We found that ACE1 was expressed in almost 100% of T (CD4+, CD8+) and B lymphocytes and in all phenotypes of these cells. In non-lymphoid cells, ACE1 mean expression was 56,9%. In agreement with our findings, independent studies showed that T cells from healthy donors and monocytes from patients with congestive heart failure expressed ACE1, but there was no investigation on cell phenotype. [25, 26]. Our study is the first to show that either inexperienced (naive) or fully activated (memory) cells expresses ACE1. Our findings suggest a that the expression of ACE1 in lymphoid and non-lymphoid cells reflects the health status since our studied population presented changes in physiological parameters and high levels of ACE1 expression on immune cells. Previous independent studies showed that patients with unstable angina [32] or acute myocardial infarction [33] presented higher expression of ACE1 in T cells and dendritic cells than controls subjects. In addition, markers of the cell (lymphoid and non-lymphoid) functional status such as inflammatory or growth factors production could be modulated by ACE inhibitors (ACEi). Accordingly, mononuclear leukocytes from healthy subjects incubated with endotoxin exhibited high levels of tissue factor activity which was reduced in the presence of captopril in a dose-dependent pattern. This result could be related to the antithrombotic effect of ACEi. [34]. In patients with congestive heart failure, immune cells cultured with LPS secreted high levels of the pro-inflammatory TNF-alpha and these levels were significantly reduced in the presence of captopril. [35]

In those previous studies, also ACE2 has been reported, why not studied here?

Answer: Our next studies will be focused on ACE1 and ACE2 expression in cells from the immune system in both younger and older adults.

In the limitations, the Authors fairly mention the real problem: The small sample size, and I would like to say lack of younger subjects.

Answer: we agree with the limitations pointed and the text was changed as required:

This study have limitations such as the small sample size and the lack of young adults for comparison. As an example, the subject with the highest CRP and albumin also exhibited a high percentage of ACE1 expression on T (CD4+, CD8+), B and non-lymphoid cells in addition to the lowest percentage of CD4+ naive cells, and the highest percentage of CD8+ terminally differentiated (EMRA) and DN B cells. However, due to the small sample size it was not possible to associate the high expression of ACE1 on immune cells with inflammaging and immunosenescence. It would bring important information to correlate physiological parameters/health status with ACE1 expression and to find out whether age and associated chronic diseases could lead to increased ACE1 expression.

The COVID-19 point even tempting today, is too far from this study and unnecessary. Answer: Our point was to emphasize the negative impact of chronic diseases for the outcome of aging population during a viral infection and how ACE1/ACE2 expression could bring information to diagnosis and treatment. Therefore, we would like to maintain this piece of information.

Linguistic checking would improve the MS. Answer: We checked for possible linguistic mistakes

Reviewer, Heikki Vapaatalo:

I read with pleasure the very detailed answers to my comments.

I very warmly recommend acceptance of this MS for publications without any further notes.

Decision changed:

Verified manuscript: The content is scientifically sound, only minor amendments (if any) are suggested.

To: Reviewer: Calogero Caruso

Dear Prof.Caruso Thank you very much for the revision of this manuscript. It is a privilege to have a manuscript reviewed by a research with high expertise on the field of ageing. Please find the answers to your questions and in the main text the changes in bold.

Sincerely yours,

Valquiria Bueno

The paper is essentially anecdotal because it studies the cells of 6 subjects without any comparison with other age groups. There is also a serious limitation because beyond the age and sex there is no information on the donors (how and why they were recruited, what drugs they took, etc.).

It is really a limitation to have only 6 individuals for the study, but they were the only ones fitting in the proposal of the manuscript. The samples were from a central bank of cells at UNIFESP and participants were considered “healthy” but there was not further information in addition to what we displayed on the tables of the manuscript. They were not living on homecares or hospitalized.

Our aim was to evaluate samples from individuals aged 60-69 years previously to COVID-19 and/or vaccination. In addition, there were no samples in the same conditions (PBMCs, -80oC) of young individuals and using fresh blood could bring a result that could not be compared mainly regarding to myeloid cells and B cells as is follows in the below reference. Braudeau C, Salabert-Le Guen N, Chevreuil J, Rimbert M, Martin JC, Josien R. An easy and reliable whole blood freezing method for flow cytometry immuno-phenotyping and functional analyses. Cytometry B Clin Cytom 2021;100(6):652-665. doi: 10.1002/cyto.b.21994.

Our goal from now on is to expand this study with young and old adults samples since it is important to understand whether ageing is associated with an increase in ACE expression on immune cells.

-To infer that chronological and biological ages do not match is inappropriate in the absence of the above information.

This piece of information regarding chronological and biological age was required by another reviewer. I agree that the concept does not match without more information on the donors. However, the information is now referenced and should be considered when older adults are studied. Vasto S, Scapagnini G, Bulati M, Candore G, Castiglia L, Colonna-Romano G, Lio D, Nuzzo D, Pellicano M, Rizzo C, Ferrara N, Caruso C. Biomarkes of aging. Front Biosci (Schol Ed) 2010;2(2):392-402. doi: 10.2741/s72. PMID: 20036955.

-However, the paper is of some interest because there are few studies on the topic.

Thanks for this positive comment. Few studies on the topic was the reason why we decided to send the manuscript for publication even though there were some important information on the donors missing and limited number of individuals.

Essential revisions that are required to verify the manuscript

1) Although we do not have data on donors, placing an age and gender column in all tables adds a minimum of useful information for the reader.

The first table submitted with age, but for requirement of MedRxiv, gender and age could no be linked to the metabolic results to preserve the anonymity of the donors.

2) Inflamm-ageing means low grade of inflammation. The value of CRP 23.1 suggests acute inflammation (also because albumin has high values, while in chronic inflammation its values decrease). Therefore the Ly averages do not have to take this subject into account.

Thank you for this comment. In a review of literature it was found an article (below) with CRP variation from 0.1 to 19.8 (Heumann Z, Youssim I, Kizony R, Friedlander Y, Shochat T, Weiss R, Hochner H, Agmon M. The Relationships of Fibrinogen and C-Reactive Protein With Gait Performance: A 20-Year Longitudinal Study. Front Aging Neurosci 2022;14:761948. doi: 10.3389/fnagi.2022.761948). There is also an article from your group showing CRPs <5g/dL and >5g/dL (Cancemi P, Aiello A, Accardi G, Caldarella R, Candore G, Caruso C, Ciaccio M, Cristaldi L, Di Gaudio F, Siino V, Vasto S. The Role of Matrix Metalloproteinases (MMP-2 and MMP-9) in Ageing and Longevity: Focus on Sicilian Long-Living Individuals (LLIs). Mediators Inflamm 2020;2020:8635158. doi: 10.1155/2020/8635158) that will be used to discuss how ageing impacts CRP levels. Considering the already small number of donors, data were maintained and statistics (mean + SD) with and without 23.1 mg/dL are now shown.

This will be the new version (discussion) about CRP Carlsson’s study [22] found that CRP value was 2.6 with a coefficient variation of 1.4% whereas in our study, it was observed higher values of CRP in 5 out of 6 individuals. In addition, it was shown by Cancemi et al. in an evaluation of individuals from 40 years to older than 95 years (long-living) that CRP increases in an age-dependent manner. Increased CRP levels has been associated with low grade of chronic inflammation (inflammaging) and our findings show that the studied population has changes in functional parameters which are likely associated with an inflammatory profile. [24] However, an individual presented CPR 23.1 mg/dL suggesting acute inflammation instead, but as all donors were not hospitalized or living on homecares, this sample was considered as part of the study. Another study evaluating gait speed found CRPs varying from 0.1 to 19.8mg/dL (Front Aging Neurosci 2022;14:761948.). Our study has an important limitation that is the lack of data on donors such as the use of continuous medicaments or sarcopenia, hypertension, cognition, among others, and thus it was not possible to correlate CRP with age-related conditions.

Table 1. Updated

Other suggestions to improve the manuscript The authors write that their findings suggest that ACE1 could play a role in several processes linked to aging including the generation and activation of autoimmune cells, due to the experimental evidence that inhibitors of ACE suppress the autoimmune process in a number of autoimmune diseases such as EAE, arthritis, autoimmune myocarditis. [49] They do not appear to have these findings in their paper. So, it needs to change the sentence.

Sentence changed to: According to experimental evidence, ACE inhibitors suppress the autoimmune process in a number of autoimmune diseases such as EAE, arthritis, autoimmune myocarditis. [49] Extrapolating these findings to our results, it is possible to suggest that ACE1 play a role in several processes linked to aging including the generation and activation of autoimmune cells.

Rviewer: Calogero Caruso

Decision changed:

Verified manuscript: The content is scientifically sound, only minor amendments (if any) are suggested.

Peer review report

Title: Abnormalities in migration of neural precursor cells in familial bipolar disorder

version: 6

Referee: Shani Stern

Institution: University of Haifa

email: sstern@univ.haifa.ac.il

ORCID iD: 0000-0002-2644-7068

General assessment

I have read the study by Sukumaran et al. The authors describe that NPCs derived from bipolar disorder patients using induced pluripotent stem cells show abnormal migration and that transcriptionally there is a dysregulation of a network of genes that relate to on EGF/ERBB proteins. Overall, the study is interesting. However, some points should be addressed. See below.

The following points are important to take into consideration:

1. Some English edits are required. Also, some acronyms appear before their definition (for example MSD).

2. In the introduction, some important studies that describe transcriptomics of BD patients should be described such as Santos et al 2021 and also neuronal phenotypes such as hyperexcitability and physiological instabilities.

3. The transcriptomics is performed only for one of the control lines although there are 3 controls. More control samples should be taken for the gene expression analysis.

4. The figures are not in the correct order.

5. The damaging variants of the patients are not described. Although there are references to previous publications, since this may be central it is good to add a table describing these variants in the patients and in the controls.

6. The differentiation to which type of neurons should be briefly described (although there is a reference to previous publications, but this should be mentioned).

7. The statistical analysis is not very clear. Moreover, the images of the migration assays are not convincing enough that indeed there is a significant difference. How many times was this performed? This should be repeated with several cultures, especially since the number of patients and controls is small. It would be more convincing if this is reproducible over several repetitions.

8. It would be good to also include representative videos in the supplementary.

Decision

Requires revisions: The manuscript contains objective errors or fundamental flaws that must be addressed and/or major revisions are suggested.

Peer review report

Title: Abnormalities in migration of neural precursor cells in familial bipolar disorder

version: 6

Referee: Shani Stern

Institution: University of Haifa

email: sstern@univ.haifa.ac.il

ORCID iD: 0000-0002-2644-7068

General assessment

I have read the study by Sukumaran et al. The authors describe that NPCs derived from bipolar disorder patients using induced pluripotent stem cells show abnormal migration and that transcriptionally there is a dysregulation of a network of genes that relate to on EGF/ERBB proteins. Overall, the study is interesting. However, some points should be addressed. See below.

The following points are important to take into consideration:

1. Some English edits are required. Also, some acronyms appear before their definition (for example MSD).

2. In the introduction, some important studies that describe transcriptomics of BD patients should be described such as Santos et al 2021 and also neuronal phenotypes such as hyperexcitability and physiological instabilities.

3. The transcriptomics is performed only for one of the control lines although there are 3 controls. More control samples should be taken for the gene expression analysis.

4. The figures are not in the correct order.

5. The damaging variants of the patients are not described. Although there are references to previous publications, since this may be central it is good to add a table describing these variants in the patients and in the controls.

6. The differentiation to which type of neurons should be briefly described (although there is a reference to previous publications, but this should be mentioned).

7. The statistical analysis is not very clear. Moreover, the images of the migration assays are not convincing enough that indeed there is a significant difference. How many times was this performed? This should be repeated with several cultures, especially since the number of patients and controls is small. It would be more convincing if this is reproducible over several repetitions.

8. It would be good to also include representative videos in the supplementary.

Decision

Requires revisions: The manuscript contains objective errors or fundamental flaws that must be addressed and/or major revisions are suggested.

Peer review report

Title: Crossref as a source of open bibliographic metadata

version: 2

Referee: Simon Porter

Institution: Digital Science

email: s.porter@digital-science.com

ORCID iD: https://orcid.org/0000-0002-6151-8423

General assessment

This is a clear paper that outlines a motivation (assess the metadata completeness of the Crossref record for the purposes of scientometric analysis,) along with providing a set of useful metrics to assess the completeness of each metadata field.

Essential revisions that are required to verify the manuscript

No essential revisions identified

Other suggestions to improve the manuscript

Minor suggestions: Figures in the interactive version of the preprint do not have headings or captions, or a link back to the paper.

On data availability, In the context of the paper, making the code used to process the Crossref’s XML Metadata Plus Snapshot would be a useful contribution enabling scientometric analysis of the Crossref dataset.

The following are offered as suggestions that could be added to the paper at the authors discression, but do not effect the content or the conclusions of the peer review

The authors have chosen to frame metadata completeness of Crossref records as a ‘good in itself,’ leaning on Waltman, L. (2020b) to do the work of setting this up.

Within this framework, the analysis is offered as a set of observations to help publishers understand where they need to do better. It might be the case that Publishers do not intrinsically understand why making certain metadata types available is valuable to the community.

On the question of how Crossref can be used in scientometric analysis, readers are left to make up their own minds on what Crossref can be used for today, vs what it might be capable of providing in the future based on the evidence presented. It would be a stronger conclusion to highlight the types of scientometric analysis that are now possible with Crossref, (for instance bibliometric coupling,) and those that require limits or caveats (analysis by affiliation, abstract.) As this analysis lends itself to being rerun in the future, it would be useful to trace advances (hopefully!) not just in terms of the number of things, but also in terms of how sceintometric analysis capability is progressing because of it.

Decision

Verified manuscript: The content is scientifically sound, only minor amendments (if any) are suggested.

Peer review report

Title: A single measurement of fecal hemoglobin concentration outperforms polygenic risk score in colorectal cancer risk assessment

version: 1

Referee: Andrea Buron

Institution: Hospital del Mar Medical Research Institute (IMIM), Barcelona, Spain

email: aburon@psmar.cat

ORCID iD: E-5705-2016

General assessment

This is a well-written paper describing the compared utility of FIT and PRS as risk assessment tools, and as means of colorectal cancer screening. The results are very informative given that many screening programmes are currently using FIT and the evidence of using PRS as additional or alternative test in this context is scarce.

One of the main strengths of this study is its population, a cohort of participants in screening colonoscopy program which means that for all participants we have the colonoscopy result as well as the PRS and FIT result.

Essential revisions that are required to verify the manuscript

Some issues need some consideration:

1. The context and the study population is not sufficiently described: how does the German screening colonoscopy program work (exclusion and inclusion criteria, invitation methods, funding and cost to the participant), also main results especially in terms of uptake rates and how any differences by age, sex, etc.

2. Include at least in the discussion possible issues that might affect representability of the study population compared to the general population, and whether this might potentially alter the study results. Ideally, a (annex o supplementary) table describing the characteristics of non-participants in the program as well as non-participants in the study would be very informative in this regard.

Other suggestions to improve the manuscript

1. Authors say that “FIT-based risk assessment is restricted to CRC risk” – there is evidence that FIT tests are useful to predict not only CRC but also pre-neoplasic disease. Also, to a less extent, it has been found to correlate and act as a potential risk marker of other non-digestive diseases.

2. As part of the discussion, authors do not comment on the possibility of using FIT and PRS concomitantly. It would be very interesting to know if their results they could further inform in this direction, on whether adding a PRS test to FIT could improve the prediction and hence better select the population in which to recommend colonoscopy.

Decision

Verified with reservations: The content is scientifically sound but has shortcomings that could be improved by further studies and/or minor revisions.

Peer review report

Title: A single measurement of fecal hemoglobin concentration outperforms polygenic risk score in colorectal cancer risk assessment

version: 1

Referee: Andrea Buron

Institution: Hospital del Mar Medical Research Institute (IMIM), Barcelona, Spain

email: aburon@psmar.cat

ORCID iD: E-5705-2016

General assessment

This is a well-written paper describing the compared utility of FIT and PRS as risk assessment tools, and as means of colorectal cancer screening. The results are very informative given that many screening programmes are currently using FIT and the evidence of using PRS as additional or alternative test in this context is scarce.

One of the main strengths of this study is its population, a cohort of participants in screening colonoscopy program which means that for all participants we have the colonoscopy result as well as the PRS and FIT result.

Essential revisions that are required to verify the manuscript

Some issues need some consideration:

1. The context and the study population is not sufficiently described: how does the German screening colonoscopy program work (exclusion and inclusion criteria, invitation methods, funding and cost to the participant), also main results especially in terms of uptake rates and how any differences by age, sex, etc.

2. Include at least in the discussion possible issues that might affect representability of the study population compared to the general population, and whether this might potentially alter the study results. Ideally, a (annex o supplementary) table describing the characteristics of non-participants in the program as well as non-participants in the study would be very informative in this regard.

Other suggestions to improve the manuscript

1. Authors say that “FIT-based risk assessment is restricted to CRC risk” – there is evidence that FIT tests are useful to predict not only CRC but also pre-neoplasic disease. Also, to a less extent, it has been found to correlate and act as a potential risk marker of other non-digestive diseases.

2. As part of the discussion, authors do not comment on the possibility of using FIT and PRS concomitantly. It would be very interesting to know if their results they could further inform in this direction, on whether adding a PRS test to FIT could improve the prediction and hence better select the population in which to recommend colonoscopy.

Decision

Verified with reservations: The content is scientifically sound but has shortcomings that could be improved by further studies and/or minor revisions.

Peer review report

Title: Post-traumatic Stress Disorder symptom sub-cluster severity predicts gray matter volume changes better than overall symptom severity

version: 1

Referee: Wencai Zhang

Institution: Institute of psychology, Chinese Academy of Sciences

email: Zhangwc@psych.ac.cn

General assessment

The current results provide very limited explanations.

Because there was no comparison of healthy controls, it cannot be concluded that the relationship between symptoms and brain structure is PTSD specific, nor is there a entire-population measure to capture the relationship between symptom severity and gray matter volume.

A total of 12 people, the sample size is too small. The correlation between symptom cluster severity and brain gray matter volume is certainly worth investigating, but a sufficient sample size is needed to obtain reliable results. Because of individual differences, such a small sample is not representative of the population from which the sample came. In this case, it is not reliable for the correlation analysis between the total score and brain gray matter volume. This question is more serious when conducting the correlation analysis between four symptom clusters and brain gray matter volume, the problem of insufficient representation of sample is particularly prominent.

Essential revisions that are required to verify the manuscript

This paper presented a small amount of correlation analysis results between total score and brain gray matter volume and a large number of positive correlation or negative correlation results between symptom clusters and brain gray matter volume. They two are not consistent with each other. The problem here is that, because the results from the total score and symptom clusters can hardly support each other, I think the author needs to discuss, explain or integrate these complex results. If the brain regions associated with the total score and the brain regions associated with the symptom cluster are interpreted separately without integration, it is contradictory to the general view that the PTSD total score means the overall symptom severity.

The authors only discussed and interpreted the results in some brain regions, but not in others. Some explanations are too simple and do not integrate the associations between various brain regions. At present, it is not possible to obtain from these discussions which brain region abnormalities are more valuable for understanding PTSD.

This paper has shortcomings in both sample size, which should be expanded and limitations discussion should be added.

Decision

Requires revisions: The manuscript contains objective errors or fundamental flaws that must be addressed and/or major revisions are suggested.

Peer review report

Title: Post-traumatic Stress Disorder symptom sub-cluster severity predicts gray matter volume changes better than overall symptom severity

version: 1

Referee: Wencai Zhang

Institution: Institute of psychology, Chinese Academy of Sciences

email: Zhangwc@psych.ac.cn

General assessment

The current results provide very limited explanations.

Because there was no comparison of healthy controls, it cannot be concluded that the relationship between symptoms and brain structure is PTSD specific, nor is there a entire-population measure to capture the relationship between symptom severity and gray matter volume.

A total of 12 people, the sample size is too small. The correlation between symptom cluster severity and brain gray matter volume is certainly worth investigating, but a sufficient sample size is needed to obtain reliable results. Because of individual differences, such a small sample is not representative of the population from which the sample came. In this case, it is not reliable for the correlation analysis between the total score and brain gray matter volume. This question is more serious when conducting the correlation analysis between four symptom clusters and brain gray matter volume, the problem of insufficient representation of sample is particularly prominent.

Essential revisions that are required to verify the manuscript

This paper presented a small amount of correlation analysis results between total score and brain gray matter volume and a large number of positive correlation or negative correlation results between symptom clusters and brain gray matter volume. They two are not consistent with each other. The problem here is that, because the results from the total score and symptom clusters can hardly support each other, I think the author needs to discuss, explain or integrate these complex results. If the brain regions associated with the total score and the brain regions associated with the symptom cluster are interpreted separately without integration, it is contradictory to the general view that the PTSD total score means the overall symptom severity.

The authors only discussed and interpreted the results in some brain regions, but not in others. Some explanations are too simple and do not integrate the associations between various brain regions. At present, it is not possible to obtain from these discussions which brain region abnormalities are more valuable for understanding PTSD.

This paper has shortcomings in both sample size, which should be expanded and limitations discussion should be added.

Decision

Requires revisions: The manuscript contains objective errors or fundamental flaws that must be addressed and/or major revisions are suggested.

Peer review report

Title: Burden of HCoV infection in children hospitalized with lower respiratory infection in Cape Town, South Africa

version: 1

Referee: SOCORRO P. LUPISAN, MD MSc

email: socorrolupisan@yahoo.com

ORCID iD: 0000-0002-8916-4380

General assessment

This study reports interesting results, but the description of the methodology needs to be improved.

Essential revisions that are required to verify the manuscript

I provide below some comments/queries to strengthen the methodology, and to increase the clarity in the interpretation of the results.

1. The parent study was not described in the Methods Section. The parent study was disclosed only in the Conclusion: “While this study which is a sub-study restricted itself to the burden of human coronaviruses in children, the main respiratory pathogen under review in the parent study was Bordetella pertussis which only assessed similar risk factors in the same cohort of children. “

2. This sub study is also prospective in nature.

3. Was the Informed Consent Form prepared for the parent study only? Did the sub study investigator get a signed Informed Consent Form for the collection and laboratory analysis of samples for this HCoV study?

4. Study Population: In order for the study to reflect the whole season, recruitment was limited to a maximum of four qualifying participants per day. Did you do stratified and systematic sampling? Please describe your sampling procedures as you have done.

5. In the Discussion it was written: Although the study was sufficiently powered, it had low precision and could not demonstrate statistically significant associations. How did you calculate sample size?

Other suggestions to improve the manuscript

I also have some clarifications which need to be addressed:

1. Would you know the other viruses detected? If yes, include in the results as this will enrich you results.

2. Was ceftriaxone really given prior to admission? At home, as injection?

3. What were the clinical diagnosis of cases with HCoV and no HCoV?

Decision

Requires revisions: The manuscript contains objective errors or fundamental flaws that must be addressed and/or major revisions are suggested.

Peer review report

Title: Burden of HCoV infection in children hospitalized with lower respiratory infection in Cape Town, South Africa

version: 1

Referee: SOCORRO P. LUPISAN, MD MSc

email: socorrolupisan@yahoo.com

ORCID iD: 0000-0002-8916-4380

General assessment

This study reports interesting results, but the description of the methodology needs to be improved.

Essential revisions that are required to verify the manuscript

I provide below some comments/queries to strengthen the methodology, and to increase the clarity in the interpretation of the results.

1. The parent study was not described in the Methods Section. The parent study was disclosed only in the Conclusion: “While this study which is a sub-study restricted itself to the burden of human coronaviruses in children, the main respiratory pathogen under review in the parent study was Bordetella pertussis which only assessed similar risk factors in the same cohort of children. “

2. This sub study is also prospective in nature.

3. Was the Informed Consent Form prepared for the parent study only? Did the sub study investigator get a signed Informed Consent Form for the collection and laboratory analysis of samples for this HCoV study?

4. Study Population: In order for the study to reflect the whole season, recruitment was limited to a maximum of four qualifying participants per day. Did you do stratified and systematic sampling? Please describe your sampling procedures as you have done.

5. In the Discussion it was written: Although the study was sufficiently powered, it had low precision and could not demonstrate statistically significant associations. How did you calculate sample size?

Other suggestions to improve the manuscript

I also have some clarifications which need to be addressed:

1. Would you know the other viruses detected? If yes, include in the results as this will enrich you results.

2. Was ceftriaxone really given prior to admission? At home, as injection?

3. What were the clinical diagnosis of cases with HCoV and no HCoV?

Decision

Requires revisions: The manuscript contains objective errors or fundamental flaws that must be addressed and/or major revisions are suggested.

Peer review report

Title: Angiotensin converting enzyme (ACE) expression in leukocytes of older adults

version: 1

Reviewer: Heikki Vapaatalo, MD, PhD, Emeritus professor of Pharmacology

Institution: Department of Pharmacology, Medical Faculty,University of Helsinki, Finland

email: heikki.vapaatalo@helsinki.fi

General assessment

The study is interesting and the title promises for me more than the MS finally contains.

The background, question and the aim are relevant as explained in the introduction.

The major criticism concerns the small size of the material (subjects, n=6), the small age difference (64-67 years) and the lack of younger controls.

In the following minor notes:

Title: ACE > better ACE1, or does the sophistic, elegant method include both ACE:s? The same should be explained and taken into consideration throughout the text.

Introduction: The last chapter, the Author should explain in more detail, how references 11-14 suggest that “ACE play an important role in the aging process”. ACE plays. Does this mean, that ACE is somehow regulating the aging process or in increasing age ACE -levels are changed?

Material and Methods: The N-value of the subjects should be mentioned here, as well the relation of females/males. Do the Authors really regard 64-67 “older age” nowadays? Lack of younger controls! Why first many years later the assays have been done in comparison to the collection of the blood? Are the samples still useable, not destroyed? Did the subjects have some diseases and/or drugs because the possibly were from hospital sample bank? Express the company details similarly than Amersham, cities and countries.

Results: “Table 1 shows that older adults…..” The comparison between the present data and historical studies belongs to the Discussion. Give also individual ages and gender of the subjects in the table 1. What means p-values here? Compared with which or interindividual differences in the particular variable? Should be explained The numbering of tables and the text seems to me confusing. Only three tables, but in the text mentioned four. Number 4 does not exist. It would be good to have a list of abbreviations used in the description of the cell types for an unfamiliar reader.

Discussion: A major part of the discussion deals with previous publications and not meaning or clinical significance of the present findings and comparison between the present and earlier studies. In those previous studies, also ACE2 has been reported, why not studied here? In the limitations, the Authors fairly mention the real problem: The small sample size, and I would like to say lack of younger subjects. The COVID-19 point even tempting to-day, is too far from this study and unnecessary. Linguistic checking would improve the MS.

As a summary: I recommend the acceptance of the MS for publication after the Authors careful rethinking of the message of the results and correction of the minor comments. I hope that in the future the possible age -related correlations to old age up to >80 years would be possible.

Decision

Verified with reservations: The content is scientifically sound but has shortcomings that could be improved by further studies and/or minor revisions.

Decision changed:

Verified manuscript: The content is scientifically sound, only minor amendments (if any) are suggested.

Peer review report

Title: Angiotensin converting enzyme (ACE) expression in leukocytes of older adults

version: 1

Reviewer: Heikki Vapaatalo, MD, PhD, Emeritus professor of Pharmacology

Institution: Department of Pharmacology, Medical Faculty,University of Helsinki, Finland

email: heikki.vapaatalo@helsinki.fi

General assessment

The study is interesting and the title promises for me more than the MS finally contains.

The background, question and the aim are relevant as explained in the introduction.

The major criticism concerns the small size of the material (subjects, n=6), the small age difference (64-67 years) and the lack of younger controls.

In the following minor notes:

Title: ACE > better ACE1, or does the sophistic, elegant method include both ACE:s? The same should be explained and taken into consideration throughout the text.

Introduction: The last chapter, the Author should explain in more detail, how references 11-14 suggest that “ACE play an important role in the aging process”. ACE plays. Does this mean, that ACE is somehow regulating the aging process or in increasing age ACE -levels are changed?

Material and Methods: The N-value of the subjects should be mentioned here, as well the relation of females/males. Do the Authors really regard 64-67 “older age” nowadays? Lack of younger controls! Why first many years later the assays have been done in comparison to the collection of the blood? Are the samples still useable, not destroyed? Did the subjects have some diseases and/or drugs because the possibly were from hospital sample bank? Express the company details similarly than Amersham, cities and countries.

Results: “Table 1 shows that older adults…..” The comparison between the present data and historical studies belongs to the Discussion. Give also individual ages and gender of the subjects in the table 1. What means p-values here? Compared with which or interindividual differences in the particular variable? Should be explained The numbering of tables and the text seems to me confusing. Only three tables, but in the text mentioned four. Number 4 does not exist. It would be good to have a list of abbreviations used in the description of the cell types for an unfamiliar reader.

Discussion: A major part of the discussion deals with previous publications and not meaning or clinical significance of the present findings and comparison between the present and earlier studies. In those previous studies, also ACE2 has been reported, why not studied here? In the limitations, the Authors fairly mention the real problem: The small sample size, and I would like to say lack of younger subjects. The COVID-19 point even tempting to-day, is too far from this study and unnecessary. Linguistic checking would improve the MS.

As a summary: I recommend the acceptance of the MS for publication after the Authors careful rethinking of the message of the results and correction of the minor comments. I hope that in the future the possible age -related correlations to old age up to >80 years would be possible.

Decision

Verified with reservations: The content is scientifically sound but has shortcomings that could be improved by further studies and/or minor revisions.

Decision changed:

Verified manuscript: The content is scientifically sound, only minor amendments (if any) are suggested.

Peer review report

Title: A single measurement of fecal hemoglobin concentration outperforms polygenic risk score in colorectal cancer risk assessment

version: 1

Referee: Iris Lansdorp-Vogelaar

Institution: Erasmus MC, Rotterdam, the Netherlands

email: i.vogelaar@erasmusmc.nl

ORCID iD: 0000-0002-9438-2753

General assessment

This is a well conducted study, clearly written study. The main strengths of the study include the novelty of the topic, its large sample size and that physicians and lab analysts were blinded to each other’s outcomes. There are few weaknesses. First, FIT and SNPs in essence service different purposes. Although I agree with the authors that FIT can be used both ways, this deserves more explicit explanation in the discussion section. Second, I disagree with the exclusion of non-advanced adenomas as relevant findings. Given that the authors suggest using FIT/SNPs for risk prediction at younger ages, non-advanced adenomas are also relevant because with time they could develop into colorectal cancer.

Essential revisions that are required to verify the manuscript

1. There is an essential difference between SNPs and FIT: SNPs predict risk of developing colorectal lesions, whereas FIT signals presence of colorectal lesions. In the current manuscript, SNPs are essentially compared on their performance to detect advanced neoplasms, which is not their intention. Yet, I agree with the authors that in order to predict development of colorectal cancer, one would expect the presence of precursor lesions >10 years prior and thus the performance can be compared that way. However, this is not immediately obvious. I therefore feel very strongly that this difference in initial purpose should be more explicitly explained in the discussion section, and also the argument why this comparison is reasonable nevertheless.

2. Given the above point, I feel that non-advanced adenomas should be included as relevant findings. The purpose of SNPs/FIT in this paper is to predict colorectal cancer risk for stratified screening approaches. In that case, non-advanced adenomas are relevant for future cancer risk in 10+ years. Omitting these basically makes the tests focused on early detection rather than risk prediction and stratification.

Other suggestions to improve the manuscript

1. Genotyping was done on a random age- and sex- matched sample of participants without advanced neoplasms before applying exclusion criteria. As a consequence, there actually was an age- and sex-difference between participants with and without advanced neoplasms in the study. Would it not have been better to age- and sex- match after exclusion criteria were applied.

2. Provide statistical tests for difference in baseline characteristics between participants with and without advanced neoplasms in Table 1.

3. Have the authors evaluated a combined approach of FIT and SNPs to see if that improves risk prediction and outperforms use of either one separately.

Decision

Requires revisions: The manuscript contains objective errors or fundamental flaws that must be addressed and/or major revisions are suggested.

Peer review report

Title: A single measurement of fecal hemoglobin concentration outperforms polygenic risk score in colorectal cancer risk assessment

version: 1

Referee: Iris Lansdorp-Vogelaar

Institution: Erasmus MC, Rotterdam, the Netherlands

email: i.vogelaar@erasmusmc.nl

ORCID iD: 0000-0002-9438-2753

General assessment

This is a well conducted study, clearly written study. The main strengths of the study include the novelty of the topic, its large sample size and that physicians and lab analysts were blinded to each other’s outcomes. There are few weaknesses. First, FIT and SNPs in essence service different purposes. Although I agree with the authors that FIT can be used both ways, this deserves more explicit explanation in the discussion section. Second, I disagree with the exclusion of non-advanced adenomas as relevant findings. Given that the authors suggest using FIT/SNPs for risk prediction at younger ages, non-advanced adenomas are also relevant because with time they could develop into colorectal cancer.

Essential revisions that are required to verify the manuscript

1. There is an essential difference between SNPs and FIT: SNPs predict risk of developing colorectal lesions, whereas FIT signals presence of colorectal lesions. In the current manuscript, SNPs are essentially compared on their performance to detect advanced neoplasms, which is not their intention. Yet, I agree with the authors that in order to predict development of colorectal cancer, one would expect the presence of precursor lesions >10 years prior and thus the performance can be compared that way. However, this is not immediately obvious. I therefore feel very strongly that this difference in initial purpose should be more explicitly explained in the discussion section, and also the argument why this comparison is reasonable nevertheless.

2. Given the above point, I feel that non-advanced adenomas should be included as relevant findings. The purpose of SNPs/FIT in this paper is to predict colorectal cancer risk for stratified screening approaches. In that case, non-advanced adenomas are relevant for future cancer risk in 10+ years. Omitting these basically makes the tests focused on early detection rather than risk prediction and stratification.

Other suggestions to improve the manuscript

1. Genotyping was done on a random age- and sex- matched sample of participants without advanced neoplasms before applying exclusion criteria. As a consequence, there actually was an age- and sex-difference between participants with and without advanced neoplasms in the study. Would it not have been better to age- and sex- match after exclusion criteria were applied.

2. Provide statistical tests for difference in baseline characteristics between participants with and without advanced neoplasms in Table 1.

3. Have the authors evaluated a combined approach of FIT and SNPs to see if that improves risk prediction and outperforms use of either one separately.

Decision

Requires revisions: The manuscript contains objective errors or fundamental flaws that must be addressed and/or major revisions are suggested.

Peer review report

Title: Patients’ satisfaction and quality of clinical laboratory services provision at public health facilities in northeast Ethiopia

version: 1

Reviewer: I wish for this review to remain anonymous. While certainly imperfect, I believe that well- conducted reviews anonymous are preferable to signed reviews and free of the bias that may affect reviewers in a relatively small field.

General assessment

The authors report on an ambitious study that sought to rigorously assess the level of patient satisfaction with a representative sample of laboratory service facilities in the Amhara region of Ethiopia. The relevance of this topic is clearly explained and the role of patient satisfaction in the assessment and life cycle of laboratory services is likely underappreciated – particularly in low-resource settings. The manuscript is reasonably well written but would benefit from some English-language copyediting, as well as editing for length as the manuscript contains several redundant passages.

Overall, the assessment of customer satisfaction as a metric of lab quality is potentially important, and not easily captured by accreditation processes such as SLIPTA or ISO. As such this is a valuable endeavour that will stimulate the field in my view.

In the present study, the authors directly address the fact that patients experience may not reflect the quality or safety of a diagnostic laboratory. They did so by conducting their own measures of laboratory quality assessment, with the aim of establishing whether patient satisfaction is associated with such measures.

Given that this aspects is in my view the core of the study, it is important that the methods used for the quality assessments be better explained and expanded. It is laudable that the authors undertook what appears to be an external quality assurance audit of malaria and TB slides examined in the last 3 months. It is important to understand exactly who performed this examination and what their qualifications were. Moreover, other details on the methods are important such as whether the slides were re-stained at the time of the audit.

Similarly, the section on facility assessment (line 177) suggests that the investigators performed a full SLIPTA audit on participating centres. This would require a huge amount of work from both auditors and the facilities in order to be a valid account. This should be described in much more detail. I was surprised to find few references detailing Ethiopian laboratory implementation or strengthening experiences (of which there are a few instructive published examples).

Finally, the finding that satisfaction is most strongly associated with objective measures of quality – such as use of fresh gloves (pre-analytical quality), EQA results of microscopy (analytical quality) and TAT (post-analytical quality) is interesting and supports the idea that quality is not a compartmental issue, but rather a local culture that permeates all laboratory activities. This is a finding that deserves to be highlighted, even if it is unclear that patient satisfaction should be used as a surrogate for more direct measures of lab quality. The emphasis on the lack association with the SLIPTA score is overstated in my view because there wasn’t sufficient variation in these scores – i.e. they were all rather poor - to yield an association.

Decision

Verified with reservations: The content is scientifically sound but has shortcomings that could be improved by further studies and/or minor revisions.

Peer review report

Title: Patients’ satisfaction and quality of clinical laboratory services provision at public health facilities in northeast Ethiopia

version: 1

Reviewer: I wish for this review to remain anonymous. While certainly imperfect, I believe that well- conducted reviews anonymous are preferable to signed reviews and free of the bias that may affect reviewers in a relatively small field.

General assessment

The authors report on an ambitious study that sought to rigorously assess the level of patient satisfaction with a representative sample of laboratory service facilities in the Amhara region of Ethiopia. The relevance of this topic is clearly explained and the role of patient satisfaction in the assessment and life cycle of laboratory services is likely underappreciated – particularly in low-resource settings. The manuscript is reasonably well written but would benefit from some English-language copyediting, as well as editing for length as the manuscript contains several redundant passages.

Overall, the assessment of customer satisfaction as a metric of lab quality is potentially important, and not easily captured by accreditation processes such as SLIPTA or ISO. As such this is a valuable endeavour that will stimulate the field in my view.

In the present study, the authors directly address the fact that patients experience may not reflect the quality or safety of a diagnostic laboratory. They did so by conducting their own measures of laboratory quality assessment, with the aim of establishing whether patient satisfaction is associated with such measures.

Given that this aspects is in my view the core of the study, it is important that the methods used for the quality assessments be better explained and expanded. It is laudable that the authors undertook what appears to be an external quality assurance audit of malaria and TB slides examined in the last 3 months. It is important to understand exactly who performed this examination and what their qualifications were. Moreover, other details on the methods are important such as whether the slides were re-stained at the time of the audit.

Similarly, the section on facility assessment (line 177) suggests that the investigators performed a full SLIPTA audit on participating centres. This would require a huge amount of work from both auditors and the facilities in order to be a valid account. This should be described in much more detail. I was surprised to find few references detailing Ethiopian laboratory implementation or strengthening experiences (of which there are a few instructive published examples).

Finally, the finding that satisfaction is most strongly associated with objective measures of quality – such as use of fresh gloves (pre-analytical quality), EQA results of microscopy (analytical quality) and TAT (post-analytical quality) is interesting and supports the idea that quality is not a compartmental issue, but rather a local culture that permeates all laboratory activities. This is a finding that deserves to be highlighted, even if it is unclear that patient satisfaction should be used as a surrogate for more direct measures of lab quality. The emphasis on the lack association with the SLIPTA score is overstated in my view because there wasn’t sufficient variation in these scores – i.e. they were all rather poor - to yield an association.

Decision

Verified with reservations: The content is scientifically sound but has shortcomings that could be improved by further studies and/or minor revisions.

Peer review report

Reviewer: Richmond Dzekoe

Institution: Iowa State University

email: rsdzekoe@iastate.edu

General assessment

This study explores and addresses an important issue of how to identify linguistic Markers of Intercultural Competence in students’ writing. The study sheds some important light on the potential of “I-words,” “In-sight words,” and “quantifiers” to indicate ICC in students’ writing. However, some significant issues need to be addressed to justify the conclusions drawn in this study. The study, therefore, requires a Major Revision. Please see my specific suggestions for revision.

Essential revisions that are required to verify the manuscript

Theoretical Background and Literature review

1. These sections need a major revision. Move the discussion on studies on the importance of reflection to the literature review section and provide more current references. These sections also read more like annotations. It will be better to focus on particular insights from the studies you cited and the implications of those insights for framing your current study.

2. You mention Byram’s (1997) intercultural speaker model and go on to say, “Like most of the current ICC frameworks, Byram’s model offers a holistic approach.” What are some of the current ICC frameworks you are referring to? Giving some examples here will be helpful for your readers.

3. The information in Table 2 should be added to your description of the Corpus.

In order to support the claim that the use of many “I-words” indicates a more open, curious, and involved stance, it is important to explain more clearly how you differentiate the “I- words” which are descriptive from “I-words” that are reflective in your analysis.

Methods, Results, Discussion

1. Explain the strengths and limitations of the integrated approach you are using. What does each model add to your integrated framework, and why is this integrated approach the best way to frame your study?

2. In describing the use of the rubrics to score the blogs, you mention calculating inter- rater reliability. How was this reliability calculated, and what was it?

3. The strong evidence of intercultural competence comes from your analysis of the “Insight words.” There is, however, a problem with the analysis of “I-words.” As you explain the use of “I-words” as indicators of reflective writing, it will be good to explain more clearly how you differentiate the reflective and descriptive functions of “I-words” in your analysis.

4. The discussion is weak. Besides a list of limitations, the discussion lacks an insightful engagement with conclusions drawn by previous research. Contextualizing the discussion within already reported insights on this topic from studies such as Belz (2003), Byram (1997), Chen, Wong, & Luo (2020), Deardorff (2006), Elola & Oskoz (2008), Hoefnagels & Schoenmakers (2018) will help you address the main aim of your study which is identifying linguistic markers of ICC in order to provide teachers and other supervisors with tangible cues to help students develop ICC.

Other suggestions to improve the manuscript

1. In the abstract, it might be a good idea to mention how many students were involved in the study and their level of linguistic proficiency in English.

2. You used the expression “a more analytical approach.” Please be more specific and mention that approach by name and what makes it more analytical. Introduction

3. In the introduction, please provide more substantial evidence from the literature to support the claim that a “successful career path increasingly depends on ICC.” One example from Linked in is not enough.

4. Please provide a clearer definition of ICC. Besides the mention of Deardorff’s (2006) definition, the reader is lost as to what ICC really means in this study and how that definition informs the framing and findings of the study.

5. There seems to be a jump from a discussion on “reflective writing” to the “role of language as a source if information for students learning” and a justification of the use of the “Linguistic Inquiry and Word Count framework” to study the use of “I-words” by President Nixon during the Watergate scandal. This structure makes the introduction a bit confusing. Please revise the introduction. Explaining the use of LIWC in other corpus analysis studies for Word Counts might help you provide a stronger justification for using this framework than the Watergate research you cited in the introduction. For current studies that use LIWC please see (Dud ̆au DP and Sava FA (2021). Performing Multilingual Analysis With Linguistic Inquiry and Word Count2015 (LIWC2015).

6. Beginning the Literature review with the sub-section “Language in relation to ICC” might provide a better flow of ideas in your lit. review.

7. The information in Table 2 should be added to your description of the Corpus. In order to support the claim that the use of many “I-words” indicates a more open, curious, and involved stance, it is important to explain more clearly how you differentiate the “I-words” which are descriptive from “I-words” that are reflective in your analysis.

Decision

Requires revisions: The manuscript contains objective errors or fundamental flaws that must be addressed and/or major revisions are suggested.

Peer review report

Reviewer: Dr. Kilani Hajer,

Institution: Tunisian Institute of Veterinary Research

email: hajerkilani@yahoo.fr

General assessment

The article is scientifically correct, the English language is acceptable; discussion is rich and linked to their results. The article can be accepted after minor revision.

Essential revisions that are required to verify the manuscript

1- Abstract, you said that you found ‘Twelve E. coli’, then you wrote ‘Phylogroup B1 of environment origin was the most predominant (58%, n=12) among the isolates, followed by commensal phylogroup A (16%), phylogroup C (8%), D (8%), and E (8%)’. The number of these phylogroups do not correspond to the total isolates that you collected (n=12)?? Please verify. I realized that you mean by percentage among 12 isolates, therefore it is better for example to write ‘most predominant (n=7; 58%), also for the other phylogroups

2- If the number of studied isolates is 12, please never use percentages, since it is not appropriate for number under at least 30.

3- In the abstract as well in the manuscript, the occurrence of integrons do not means that these integrons contains gene cassettes that encode antimicrobial resistance. Many studies reported the occurrence of class 1 integrons with empty variable regions. Therefore, please take this in mind and modify your discussion.

4- Introduction, you wrote ‘..are typically comprised of phylogenetic groups A, B, and D’. I believe you mean B1 phylogroup, correct this.

5- You wrote ‘..prepared, as Mandal et al. (2014) said’. PLEASE delete the word ‘said’

6- Write ‘Purified 16S rRNA amplicons (Figure 2) were digested with three different restriction enzymes’ correct this also in Figure 2.

7- You wrote ‘In this research, E. coli was isolated at a rate of 8%’. Please, you investigate 60 samples and you find 12 isolates, so 12/60 X100 = 20%. So how you speak about 8%?

8- As I mentioned above since you find only 12 isolates, please avoid to provide percentage, please provide the number of isolates when appropriate and especially in the section ‘Distribution of phylogroups of E. coli in street foods’

9- You provide figure 4, but it is not indicated in your manuscript. If it is not necessary, you can delete it.

10- You wrote ‘..Genes carried by integrons usually express multiple resistance mechanisms, such as resistance to beta-lactams…’ in reality beta-lactamase genes are not part of integrons, they are not gene cassette, they can be near integrons but not as gene casette.

Other suggestions to improve the manuscript

Try to avoid unnecessary figures, and shorten the discussion if possible

Decision

Verified with reservations: The content is scientifically sound but has shortcomings that could be improved by further studies and/or minor revisions.

Peer review report

Reviewer: Dr. Kilani Hajer

Institution: Tunisian Institute of Veterinary Research

email: hajerkilani@yahoo.fr

General assessment

The article is scientifically correct, the English language is acceptable; discussion is rich and linked to their results. The article can be accepted after minor revision.

Essential revisions that are required to verify the manuscript

1- Abstract, you said that you found ‘Twelve E. coli’, then you wrote ‘Phylogroup B1 of environment origin was the most predominant (58%, n=12) among the isolates, followed by commensal phylogroup A (16%), phylogroup C (8%), D (8%), and E (8%)’. The number of these phylogroups do not correspond to the total isolates that you collected (n=12)?? Please verify. I realized that you mean by percentage among 12 isolates, therefore it is better for example to write ‘most predominant (n=7; 58%), also for the other phylogroups

2- If the number of studied isolates is 12, please never use percentages, since it is not appropriate for number under at least 30.

3- In the abstract as well in the manuscript, the occurrence of integrons do not means that these integrons contains gene cassettes that encode antimicrobial resistance. Many studies reported the occurrence of class 1 integrons with empty variable regions. Therefore, please take this in mind and modify your discussion.

4- Introduction, you wrote ‘..are typically comprised of phylogenetic groups A, B, and D’. I believe you mean B1 phylogroup, correct this.

5- You wrote ‘..prepared, as Mandal et al. (2014) said’. PLEASE delete the word ‘said’

6- Write ‘Purified 16S rRNA amplicons (Figure 2) were digested with three different restriction enzymes’ correct this also in Figure 2.

7- You wrote ‘In this research, E. coli was isolated at a rate of 8%’. Please, you investigate 60 samples and you find 12 isolates, so 12/60 X100 = 20%. So how you speak about 8%?

8- As I mentioned above since you find only 12 isolates, please avoid to provide percentage, please provide the number of isolates when appropriate and especially in the section ‘Distribution of phylogroups of E. coli in street foods’

9- You provide figure 4, but it is not indicated in your manuscript. If it is not necessary, you can delete it.

10- You wrote ‘..Genes carried by integrons usually express multiple resistance mechanisms, such as resistance to beta-lactams…’ in reality beta-lactamase genes are not part of integrons, they are not gene cassette, they can be near integrons but not as gene casette.

Other suggestions to improve the manuscript

Try to avoid unnecessary figures, and shorten the discussion if possible

Decision

Verified with reservations: The content is scientifically sound but has shortcomings that could be improved by further studies and/or minor revisions.

Peer review report

Reviewer: Suleyman Sedar Koca

ORCID: 0000-0003-4995-430X

email: kocassk@yahoo.com

General assessment

The latest version (bioRxiv V3) of the manuscript is acceptable/publishable as it stands.

Essential revisions that are required to verify the manuscript

There are no suggestions for revisions

Decision

Verified manuscript: The content is scientifically sound, only minor amendments (if any) are suggested.

Peer review report

Reviewer: Božena Horváthová

Institution: University of Constantine the Philosopher in Nitra

email: bhorvathova@ukf.sk

ORCID: 0000-0002-0611-2623

General comments

The manuscript is well written and follows the standards of academic writing, the research is thorough and the conclusions are comprehensive.

Serious concerns

• Do you have any serious concerns about the manuscript such as fraud, plagiarism, unethical practices? No

• Have authors’ provided the necessary ethics approval (from authors’ institution or an ethics committee)? Yes

Language quality

• How would you rate the English language quality? High quality

validity and reproducibility

• Does the manuscript contain any objective errors, fundamental flaws, or is key information missing? No

Suggestions

• Do you have any other suggestions, feedback, or comments on how the study could be improved? No

Decision

Verified manuscript: The content is scientifically sound, only minor amendments (if any) are suggested.

Peer review report

Reviewer: Yulia Karmanova

Institution: Research Centre Kairos

email: yulia.karmanova@gmail.com

General assessment

In my honest opinion the topic of intercultural competence (ICC) should be of great interest not only to researchers involved in linguistics and pedagogics but to a general reader as well. By developing ICC, that represents a set of skills needed when encountering people from various backgrounds, one can learn valuable communication skills, flexibility in behaviour and become more aware of a lack of one’s tact and tolerance.

The manuscript is well written in an engaging and lively style, it provides excellent context about linguistic cues of ICC that will help educators steer and stimulate the ICC development of their students.

The manuscript cites relevant and sufficient literature that provides a very useful resource for current practitioners.

I do not identify fundamental flaws in the manuscript, there is nothing illogical or irrational, although I have a few suggestions for minor improvements. Please see my comments below for further details.

Essential revisions that are required to verify the manuscript

No essential revisions. The manuscript clearly describes the research methods of data collection and analysis as well as other meaningful parameters. Section number 3 (Research Method and Results) is recipe-like, the study can be reproduced.

The data collected for the research is impressive: 1,635 blogs (on average 400 words each) written by 672 students majoring in Hotel Management.

The data and analysis provided in the manuscript are not deprived of clarity and logic. No additional experiments are needed to validate the results presented in the manuscript.

Discussion and conclusion section aligns with objectives stated in the first section.

The authors of the manuscript made a valuable contribution by identifying linguistic markers for ICC in the language use of students blogging about intercultural experiences: I-perspective lexemes, insight verbs and quantifiers. These language cues make ICC more «tangible» and as a result provide teachers with concrete tools for giving students more targeted ICC assessments in their reflective writing tasks. By giving certain linguistic prompts to students, educators may form a more thoughtful and personalised approach in describing their intercultural experience.

Other suggestions to improve the manuscript

The content of the manuscript is scientifically sound but has minor shortcomings that could be improved by further revisions.

I do agree with the limitations of the research mentioned by the authors, especially with the lack of the explanatory value of a significant difference in frequency of use of the linguistic markers which I think can be resolved in future studies of this topic.

I suggest that the authors should involve more assessors in their future research. Two lecturer-researchers and three senior students were involved in the process which I assume is not enough for such large-scale research like this. A bigger team of professional assessors could make valuable contribution when analysing the data and resolving emerging research questions.

I would also recommend providing the manuscript with brief comments on the meanings of the parameters in column 4 (Table 3, 4, 5, 6) for readers’ clarity. What do t, p and n.s. stand for?

I believe that the manuscript would benefit from correcting minor inaccuracies. I would recommend to:
 replace «his» with gender neutral «their», page 6: In these blogs, the language use of students serves as a vehicle of information on the students’ development of ICC, offering the reader concrete cues – henceforth referred to as linguistic markers – of his reflective learning process.

• add a space between that and are, page 19: In order to bring more focus to our research, we initially focused on word categories thatare characteristic of properties that can be linked to ICC and cultural sensitivity, such as openness, self- relativity, curiosity and reflection or analytical thinking.

• add missing parentheses, page 22; Deardorff, D. 2006. Identification and Assessment of Intercultural Competence as a Student Outcome of Internationalisation. Journal of Studies in International Education, 10 (3), 241-266.

All in all, I find the topic of the manuscript fascinating and the research question relevant and essential to the field.

Decision

Verified manuscript: The content is scientifically sound, only minor amendments (if any) are suggested.

Discussion, revision and decision

Decision

Verified with reservations: The content is scientifically sound, but has shortcomings that could be improved by further studies and/or minor revisions.

Dr. Bañuelos: Verified manuscript

Dr. Morris: Verified with reservations

Revision

Response to Reviewer 1 (Dr. Bañuelos)

1. Most importantly, I would like to see an introduction that explains the authors’ general arguments about grading changes – including the trajectory of these changes at Dalhousie and why this arc contributes to our knowledge of the history of higher education more broadly. Then, the authors might continually remind us of the arc they present at the outset of their paper – especially when they are highlighting a piece of evidence that illustrates their central argument. To me, the quotes from students and faculty responding to grading changes are among the most interesting parts of the paper and placing these in additional context should make them shine even more brightly!

Our Response: Thank you so much for your thoughtful review. We have added a larger new introduction section of the paper (paragraphs 1-5 in the latest draft are new) that outlines the general importance of the topic, the Canadian context, details on Dalhousie University, and our overall thesis statement (i.e., most decisions were to improve the external communication value of grades). Moreover, we have added three new student quotes form the Dalhousie Gazette to build a stronger picture for student reactions, and to build a better case for our overall thesis statement (i.e., that changes in grading were often to increase the external communication value of grades). Moreover, throughout we have added some details on the overall funding trajectory for institutions in Canada that created some pressure to standardize grading. We think that these changes have improved the manuscript.

1. I’d like to read a little more about Dalhousie itself – why it is either a remarkable or unremarkable place to study changes in grading policies. Is it representative of most Canadian universities and thus, a good example of how grading changes work in this national context? Is it unlike any other institution of higher education and thus, tells us something important about grades that we could not learn from other case studies? I don’t think this kind of description needs to be particularly long, but it should be a little more involved than the brief sentences the authors currently include (p.3, paragraph 1) and should explain the choice of this case.

Our Response: This comment revealed that two additional pieces of context were needed for the introduction: (a) some national context for higher education policy in Canada and (b) some extended description of Dalhousie University when compared to other universities in Canada. To this end, two new paragraphs have been added to the paper (paragraphs 2 & 3 in the current draft).

Notably, Jones (2014) notes that “Canada may have the most decentralized approach to higher education than any other developed country on the planet” (pg 20). With this in mind, any historical review of education policy is by necessity specific to province and institution – that is, the information can be placed in its context, but resists wide generalization to the country as a whole. In the newest draft, we tried to describe the national, provincial, and institutional context in some more detail in paragraphs 2 & 3.

1. I’d also like to know more about the archival materials the authors used. The authors mention that they drew from “Senate minutes, university calendars, and student newspapers” (p. 3), but what kinds of conversations about grades did these materials include? At various points, the authors engage in “speculation” (e.g. p.4) about why a particular change occurred. This is just fine and, in fact, it’s good of the authors to remind us that they are not really sure why some of these shifts happened. But, they might go one step further and tell us why they have to speculate. Were explicit discussions of grading changes – including in inter- and intradepartmental letters and memo, reports, and other documents – not available in these archives? Why are these important discussions absent from the historical record?

Our Response: We have added a new paragraph (paragraph 4) to the paper discussing the sources in some more detail. It is true that the verbatim discussions are frequently absent from the record, especially earlier in history – or if they exist, we have not found them! Instead, we frequently are reviewing meeting minutes or committee reports, which are summaries of discussions. As we now note in the paper, “Thus, the sources used showed what policy changes were implemented, when they were implemented, and a general sense of whether there was opposition to changes; however, there were notable gaps in faculty and student reactions to grade policy changes, as these reactions were frequently not written down and archived.”

This gap was most apparent in the Senate minutes around the 1940s, where I (the first author) could not find any direct discussions of why changes were implemented. Under the 1937-1947 heading, we more clearly indicate that the rationale for the changes was absent from the Senate minutes during this period. I add some further speculation on why these records might be absent, based on summaries from Waite (1998b); specifically, the university president of the time often made unilateral decisions, circumventing Senate, which might account for why the changes are absent from the records.

This will hopefully make the limitations of what can be learned from this approach more apparent.

1. At various points, the authors make references to the outside world – for example, WWII (p. 5), the Veteran’s Rehabilitation Act (pp. 6-7), and British versus American grading schemas (p. 6). But, these references are brief and seem almost off-handed. I know space is limited, but putting these grading changes in their broader context might help make the case for why this study is interesting and important. Are the changes in the 1940s, for example, related to the ascendance of one national graduate education model over another (e.g. American versus British)? Are there any data on how many Canadian undergraduates enrolled in British versus American graduate programs over time? If so, I would share any information you might have on these broader trends.

Our Response: To our knowledge, there isn’t any comparable report to what we’ve written here documenting the transition from British “divisions” to American “letter grades” in Canadian Universities, making our report novel in this regard. It might well be that a similar historical arc exists in many of the 223 public and private universities in Canada, but we don’t believe such data exists in any readily accessible way – excepting perhaps undergoing a similar deep dive into historical documents at each respective institution! So, we do not have the answer to your question: “Are there any data on how many Canadian undergraduates enrolled in British versus American graduate programs over time?” However, we did add one reference which provided a snapshot point of comparison in 1960, noting in the paper “Baldwin (1960) notes that the criteria for “High First Class” grades in the humanities was around 75-80% at Universities of Toronto, Alberta, and British Columbia in 1960, suggesting that Dalhousie’s system was similar to other research-intensive universities around this time.” That said, there are a few major national events related to the funding of universities in Canada that we have elaborated on in the text to address the spirit of your recommendation for describing the national context:

a) In the “Late 1940s” section of the paper, we added: “Though Dalhousie had an unusually high proportion of veterans enrolled relative to other maritime universities during this period (Turner, 2011), the Veteran’s Rehabilitation Act was a turning point for large increases in enrollment and government funding Canada-wide, at least until the economic recession of the 1970s (Jones, 2014).”

b) In the 1990s, there were major government cuts to funding, creating challenging financial times for the university. We discuss the funding pressures that likely contributed to standardization of grading during this time by saying the following in the 1980s-2000s section: “Starting in in the 1980s-1990s there were major government cuts to university funding nation-wide, with the cuts becoming more severe in the 1990s (Jones, 2014; Higher Education Strategy Associates, 2021). Because of the nature of the funding formulas, cuts in Nova Scotia were especially deep. Beyond tuition increases, university administrators knew that obtaining external research grants, Canada Research Chairs, and scholarship funding was one of the few other ways for a university to balance budgets, so there was extra pressure to be competitive in these pools. […] The increased standardization was likely related to increased financial pressures at this time – standardization is an oft-employed tool to deal with ever-increasing class sizes with no additional resources.”

c) In the 2010s section of the paper, we added context to how universities in country-wide have become increasingly dependent on tuition fees for funding: “Following the 2008 recession, federal funding decreased again (Jones, 2014; Higher Education Strategy Associates, 2021); however, this time universities tended to balance budgets by increasing tuition and international student fees. This trend towards increased reliance on tuition for income is especially pronounced in Nova Scotia, which has the highest tuition rates in the country (Higher Education Strategy Associates, 2021). Thus, the university moved closer to a “consumer” model of education, so it makes sense that a driving force for standardization was student complaints.”

1. This is a very nitpicky concern that doesn’t fit well elsewhere, so please take it with a grain of salt. I was surprised at the length of the reference list – it seemed quite short for a historical piece! I wonder, again, if more description of the archival material - including why you looked at these sources, in particular, and what was missing from the record – would help explain this and further convince the reader that you have all your bases covered.

Our Response: In the introduction section, paragraph 4, we describe our sources in more detail including what is likely missing from the record and why we used them. Regarding the length of the reference list, we did add ~12 new references to the list in the course of making various revisions, which partially addresses your concern. Beyond this though, it’s worth noting that some of the sources more extensive than they seem, even though they don’t take up much space in the reference list (e.g., there is one entry for course calendars, but this covers ~100 documents reviewed!). Moreover, there were many dead-ends in the archives that are not cited (e.g., reviewing 10 years of Senate minutes in the 1940s produced little of relevance), so the reference list is curated to only those sources where relevant materials were found.

Reviewer response to revisions

The new introduction to the piece addresses many of my previous questions about the authors’ general arguments, the Dalhousie context, and the source material. Thank you for addressing these! Reading this version, it is much clearer that the key argument is that standardized, centralized grading practices were “to improve the external communication value of the grades, rather than for pedagogical reasons” (p. 6). I also really enjoyed the added quotes from students in the Dalhousie Gazette.

The authors’ response to Reviewer 2 really gave me a better sense of why they wrote this piece and also helped me to more clearly put my finger on what was troubling me in the first round. It still reads a little like a report for an internal audience – which is just fine and, in fact, can be extremely useful for historians of the future. But, as Reviewer 2 notes, this means it does not really seem like a piece of historical scholarship. I do worry that shaping it into this form would take an extensive revision and might not be in the spirit of what the authors intended to do.

A different version of this article might start with this idea that grades were standardized for external audiences and in response to financial pressures. It would then develop a richer story behind the sudden importance of these external audiences and the nature (i.e. source, type) of financial pressures Dalhousie was facing. It would highlight the impact such changes had on students and their future careers/graduate experiences. It could then connect these trends to other similar changes for external audiences and the increasing interconnectedness of American, Canadian, and British systems through graduate education. It might even turn to sociological theories of organizational change and adaptation and make an argument for when (historically) similar forms of decoupling were likely to occur in the Canadian higher education system. Finally, it might connect these grading changes to current trends – including accusations of grade inflation and accepted best practices for measuring learning outcomes.

But, it doesn’t seem that the authors necessarily want to do this, which I can understand and respect. I think there is enormous value in a piece of scholarship like this existing – both for internal audiences and for future historians. Indeed, imagine if every university had a detailed history of its grading policies like this available somewhere online! Comparing such practices across institutions would certainly tell us a lot about why grading currently looks the way it does.

Decision changed

Verified manuscript: The content is scientifically sound, only minor amendments (if any) are suggested.

Response to Reviewer 2 (Dr. Morris)

The authors dove headfirst into Dalhousie’s archives, unpacking the subtle shifts in grading policy. Their work seems to be comparable to archaeologists, digging deep beneath mountains of primary sources to find nuggets of clues into Dalhousie’s grading evolution. I particularly liked when the authors were able to link these changes to student voices, as seen in moments when they referenced student publications.

Ultimately, I kept coming back to one main comment that I wrote in the margins: “So what?” I would humbly suggest that the authors reflect on why this history matters to them. Granted, they do this in the conclusion, where they touch on Schneider & Hutt’s argument that grades evolved to increasingly be a form of external communication with audiences beyond school communities. Sure. But I want more. I wanted to see a new insight that this microhistory of Dalhousie significant to the history of Canada or the history of education more generally.

If the authors are so inclined, there might be several approaches to transform this manuscript. I would suggest the following. First, instead of tracing the entire history of grading at the institution, choose one moment of change that you think is the most important. Perhaps in the 1920s and the lack of transparency in grading, or the post-war shift toward American grading. Second, show me – don’t tell me – what Dalhousie was like at this moment. Paint a picture of the institution with details about student demographics, curriculum, educational goals, the broader town, etc. Make the community come alive. Show me what makes Dalhousie unique from other institutions of higher ed. Once you establish that picture, perhaps you could link the change in grading practices to subtle changes at the university community, thereby establishing a before and after snapshot. This will require considerable amounts of work, and the skills of a historian. You will have to find primary and secondary sources that go far beyond what you’ve relied on thus far.

In the end, I found myself wanting the authors to humanize this manuscript, meaning I wanted them to show me that changes in grading practices have tangible effects on real-life human beings. A humanization of their research would mean going narrower and deeper; or, in other words, eliminating much of what they have documented.

However, if that is too tall of an order, I would ask that the authors clarify for themselves who this manuscript is for. Is this a chronicling of facts for an internal audience at Dalhousie’s faculty, alumni, and students? Fine. But my guess is that even members of the Dalhousie community want to read something relatable.

I am suggesting revisions, although not because of objective errors. History is more of an art, in my opinion. With that in mind, I would suggest that the authors paint a more vivid picture (metaphorically) of Dalhousie, showing me how changes one moment of change in grading practices impacted the lives of human beings.

Our Response: Thank you very much for taking the time to read our paper and provide your thoughts and recommendations. It may be helpful to begin by describing why I (the first author) decided to write this paper. Ultimately, I wrote this paper to satisfy my own personal curiosity and to connect with other people at my own place of employment by exploring our shared history. At present day, Dalhousie has a letter grading scheme with a standardized percentage conversion scheme that all instructors used. I wanted to know why this particular scheme was used, but I quickly realized that nobody at Dalhousie really knew how we ended up grading this way! There was an institutional memory gap, and a puzzle that was irresistible to me. So, I wrote this paper for the most basic of all academic reasons: Pure curiosity. I do very much recognize that the subject matter is very niche, perhaps too niche for a traditional journal outlet. Thus, my publishing plan is to self-publish a manuscript to the Education Resources Information Center (ERIC) database and a preprint server as a way of sharing my work with others who might be interested in what I found. Nonetheless, I believe in the importance and value of peer review, especially since I am writing in a field different than most of my scholarly work. That is why I chose PeerRef as a place to submit, so that I could undergo rigorous peer review to improve the work while still maintaining the niche subject matter and focus that drives my passion and curiosity for the project. Of course, if you feel the whole endeavor is so flawed that it precludes publication anywhere, then we can consider this a “rejection” and I will not make any further edits through PeerRef.<br /> The core of your critique suggested that I should write a fundamentally different paper on different subject matter. While I don’t necessarily disagree that the kind of paper you describe might have broader appeal, it would no longer answer the core research question I wanted an answer to: How has Dalhousie’s grading changed over time? So, I must decline to rewrite the paper to focus on a single timeframe as recommended. All this said, I did try my best to address the spirit of your various concerns to improve the quality of the manuscript. Below, I will outline the various major changes to the manuscript that we made to improve the manuscript along the lines you described, while maintaining our original vision for the structure and focus of the paper. The specific changes are outline below:

a) Two new paragraphs (now paragraphs 1-2 of the revised manuscript) were added to explain the “so what” part of the question. Specifically, we describe why we think the subject matter might be of interest to others and summarize the general dearth of historical information on grading practices in Canada as a whole.

b) Consistent with recommendations from the other reviewer, we now state a core argument (i.e., that most major grading changes were implemented to improve the external communication value of the grades) earlier in the introduction in paragraph 5 and describe how various pieces of evidence throughout the manuscript tie back to that core theme.

c) In an attempt to “humanize” the manuscript more, we added more student quotes from the Dalhousie Gazette throughout the paper so that readers can get a better sense of how students thought about grading practices at various times throughout history. Specifically, three new quotes were added in the following sections: 1901-1936, late 1940s, 1950s-1970s. We also added this short note about the physical location where grades used to be posted: “Naturally, this physical location was dreaded by students, and was colloquially referred to as “The Morgue” (Anonymous Dalhousie Gazette Author, 1937).”

d) Early in the paper, we describe why we chose Dalhousie and the potential audience of interest: “As employees of Dalhousie, we naturally chose this institution as a case study due to accessibility of records and because it has local, community-level interest. The audience was intended to be members of the Dalhousie community; however, it may also be a useful point of comparison for other institutions, should similar histories be written.”

e) We have described some of the limitations of our sources in paragraph 4, which may explain why the manuscript takes the form it does – it has conformed to the information that is available!

f) We have linked events at Dalhousie to the national context in some more detail, by detailing some national events related to the funding of universities in Canada. See our response to Reviewer 1, #4 above for more details on the specific changes.

g) Consistent with your stylistic recommendations, we have changed various spots throughout the paper from the present tense (e.g., “is”) to the past tense (e.g., “was”), and were careful in our new additions to maintain the past tense, when appropriate. If there are any spots that we missed, let us know the page number / section, and we will make further changes, as necessary.

h) We retained the first person in our writing – this may be discipline-specific, but in Psychology (the first author’s home discipline), first person is acceptable in academic writing. If you feel strongly about this, we can go through the manuscript and remove all instances of the first person, but we would prefer to keep it, if at all possible.

Hopefully this helps address the spirit of your concerns, and I look forward to hearing your thoughts in the second round of reviews.

Decision changed

Verified with reservations: The content is scientifically sound, but has shortcomings that could be improved by further studies and/or minor revisions.

Peer review report

Reviewer: Wade H. Morris Institution: Georgia State University email: morriswh@gmail.com

General comments

The authors dove headfirst into Dalhousie’s archives, unpacking the subtle shifts in grading policy. Their work seems to be comparable to archaeologists, digging deep beneath mountains of primary sources to find nuggets of clues into Dalhousie’s grading evolution. I particularly liked when the authors were able to link these changes to student voices, as seen in moments when they referenced student publications.

Ultimately, I kept coming back to one main comment that I wrote in the margins: “So what?” I would humbly suggest that the authors reflect on why this history matters to them. Granted, they do this in the conclusion, where they touch on Schneider & Hutt’s argument that grades evolved to increasingly be a form of external communication with audiences beyond school communities. Sure. But I want more. I wanted to see a new insight that this microhistory of Dalhousie significant to the history of Canada or the history of education more generally.

If the authors are so inclined, there might be several approaches to transform this manuscript. I would suggest the following. First, instead of tracing the entire history of grading at the institution, choose one moment of change that you think is the most important. Perhaps in the 1920s and the lack of transparency in grading, or the post-war shift toward American grading. Second, show me – don’t tell me – what Dalhousie was like at this moment. Paint a picture of the institution with details about student demographics, curriculum, educational goals, the broader town, etc. Make the community come alive. Show me what makes Dalhousie unique from other institutions of higher ed. Once you establish that picture, perhaps you could link the change in grading practices to subtle changes at the university community, thereby establishing a before and after snapshot.

This will require considerable amounts of work, and the skills of a historian. You will have to find primary and secondary sources that go far beyond what you’ve relied on thus far.

In the end, I found myself wanting the authors to humanize this manuscript, meaning I wanted them to show me that changes in grading practices have tangible effects on real-life human beings. A humanization of their research would mean going narrower and deeper; or, in other words, eliminating much of what they have documented.

However, if that is too tall of an order, I would ask that the authors clarify for themselves who this manuscript is for. Is this a chronicling of facts for an internal audience at Dalhousie’s faculty, alumni, and students? Fine. But my guess is that even members of the Dalhousie community want to read something relatable.

Section 1 – Serious concerns

• Do you have any serious concerns about the manuscript such as fraud, plagiarism, unethical or unsafe practices? No

• Have authors’ provided the necessary ethics approval (from authors’ institution or an ethics committee)? not applicable

Section 2 – Language quality

• How would you rate the English language quality? There are a few nagging stylistic quirks. Most obvious to me, the authors switch into and out of present tense. Stick with past tense. Also, I would suggest that they remove the first person.

Section 3 – validity and reproducibility

• Does the manuscript contain any objective errors, fundamental flaws, or is key information missing?

Not that I am aware.

Section 4 – Decision

Requires revisions: The manuscript contains objective errors or fundamental flaws that must be addressed and/or major revisions are suggested.

I am suggesting revisions, although not because of objective errors. History is more of an art, in my opinion. With that in mind, I would suggest that the authors paint a more vivid picture (metaphorically) of Dalhousie, showing me how changes one moment of change in grading practices impacted the lives of human beings.

Peer review report

Reviewer: Nidia Bañuelos Institution: University of Wisconsin-Madison email: nbanuelos@wisc.edu

General comments

Questions of how and why grading practices change over time, how students, faculty, and administrators respond to grades, and the external pressures on grading practices (e.g. war, graduate school requirements) are inherently interesting! The authors have clearly done a careful job of tracking these – often minute, and likely, difficult to follow – changes at Dalhousie University. The manuscript is well-written and relatively easy to follow.

My biggest concern, reflected in the more detailed comments below, is that the authors could do a better job of explaining to the reader why these changes are interesting, important, and relevant to historians of higher education more broadly – even those who aren’t at Dalhousie. They do some of this at the very end of the paper and, indeed, this summing up of their findings and explanation of their relevance was my favorite part of the manuscript. I would suggest reorganizing the paper so that these bigger takeaways appear in the introduction and so that the reader is reminded of them at each major section break of the paper. For example, when the authors present a quote from a student who is concerned that grades have little to do with learning outcomes, they might remind us that one of their main arguments is that “decisions about university grading schemes had very little to do with actual pedagogy” (p. 15).

As it is written, the manuscript sometimes reads like a list of facts about grading changes. But, I think a reframing that focuses on the general importance of these changes could make the entire piece more engaging. More on this below…

Section 1 – Serious concerns

• Do you have any serious concerns about the manuscript such as fraud, plagiarism, unethical or unsafe practices? No

• Have authors’ provided the necessary ethics approval (from authors’ institution or an ethics committee)? not applicable

Section 2 – Language quality

• How would you rate the English language quality? High quality

Section 3 – validity and reproducibility

• Does the manuscript contain any objective errors, fundamental flaws, or is key information missing?

While I don’t notice any “objective errors”, I do think the paper has a major flaw (i.e. little explanation of the broader significance of this case study) and could benefit from additional information about the institutional context, the archival material, and external influences on grading trends. (Please see below.)

Section 4 – Suggestions

• Based on your answer in section 3 how could the author improve the study?

I. Most importantly, I would like to see an introduction that explains the authors’ general arguments about grading changes – including the trajectory of these changes at Dalhousie and why this arc contributes to our knowledge of the history of higher education more broadly. Then, the authors might continually remind us of the arc they present at the outset of their paper – especially when they are highlighting a piece of evidence that illustrates their central argument. To me, the quotes from students and faculty responding to grading changes are among the most interesting parts of the paper and placing these in additional context should make them shine even more brightly!

II. I’d like to read a little more about Dalhousie itself – why it is either a remarkable or unremarkable place to study changes in grading policies. Is it representative of most Canadian universities and thus, a good example of how grading changes work in this national context? Is it unlike any other institution of higher education and thus, tells us something important about grades that we could not learn from other case studies? I don’t think this kind of description needs to be particularly long, but it should be a little more involved than the brief sentences the authors currently include (p.3, paragraph 1) and should explain the choice of this case.

III. I’d also like to know more about the archival materials the authors used. The authors mention that they drew from “Senate minutes, university calendars, and student newspapers” (p. 3), but what kinds of conversations about grades did these materials include? At various points, the authors engage in “speculation” (e.g. p.4) about why a particular change occurred. This is just fine and, in fact, it’s good of the authors to remind us that they are not really sure why some of these shifts happened. But, they might go one step further and tell us why they have to speculate. Were explicit discussions of grading changes – including in inter- and intradepartmental letters and memo, reports, and other documents – not available in these archives? Why are these important discussions absent from the historical record?

IV. At various points, the authors make references to the outside world – for example, WWII (p. 5), the Veteran’s Rehabilitation Act (pp. 6-7), and British versus American grading schemas (p. 6). But, these references are brief and seem almost off-handed. I know space is limited, but putting these grading changes in their broader context might help make the case for why this study is interesting and important. Are the changes in the 1940s, for example, related to the ascendance of one national graduate education model over another (e.g. American versus British)? Are there any data on how many Canadian undergraduates enrolled in British versus American graduate programs over time? If so, I would share any information you might have on these broader trends.

Similarly, the authors make brief mention of the internal reaction to grade changes – quoting students or faculty minutes. But, it would be wonderful (space permitting) to have even more information the internal impact of these changes. Did they change faculty instructional practices? Did they seem to have any effect on students’ orientation to their learning? Did standardization reflect an increasing interdependence of departments, or did it contribute to their lessening autonomy? If the archival record doesn’t permit us to know these things, then this might be a limitation the authors note at the end of the manuscript. I noticed that the authors reference a secondary source on Dalhousie student experiences repeatedly (Waite, 1998). Even a little more from this text or another secondary source like it could help the reader better understand the impact of grade changes.

• Do you have any other suggestions, feedback, or comments for the Author?

This is a very nitpicky concern that doesn’t fit well elsewhere, so please take it with a grain of salt. I was surprised at the length of the reference list – it seemed quite short for a historical piece! I wonder, again, if more description of the archival material - including why you looked at these sources, in particular, and what was missing from the record – would help explain this and further convince the reader that you have all your bases covered.

Section 5 – Decision

Requires revisions: The manuscript contains objective errors or fundamental flaws that must be addressed and/or major revisions are suggested.

Peer review report

Reviewer: Ludo Waltman

Institution: Leiden University

email: waltmanlr@cwts.leidenuniv.nl

ORCID: https://orcid.org/0000-0001-8249-1752

I would like to note that I am an expert in the field of bibliometrics and that my review therefore focuses on the bibliometric aspects of this paper. I am not an expert on scoping reviews or on curcumin. I hope that other reviewers have expertise on these aspects of the paper.

I am one of the developers of the VOSviewer software, which may perhaps be seen as a competing interest.

General comments

Please find below my detailed comments on the paper, including suggestions for improvements.

“A study by Loannidis et al.”: ‘Loannidis’ should be ‘Ioannidis’.

The author uses the Web of Science Core Collection. This database consists of a number of citation indexes (e.g., Science Citation Index, Social Sciences Citation Index, etc.). Depending on their subscription, different Web of Science users have access to different citation indexes. Please mention which citation indexes were used.

The description of the search query in Section 2 is unclear, because the search query doesn’t seem to be restricted to COVID-19 research. The full search query should be reported in the main text of the paper (not only in Appendix 1).

The VOSviewer visualizations presented in the paper are hard to read (especially Figures 1 and 2). The font size used in the visualizations needs to be increased. This can be done using the ‘Scale’ slider in VOSviewer. The author may also consider making interactive versions of the visualizations available online, so that readers can explore these visualizations in their web browser. A visualization can be made available online using the ‘Share’ button on the ‘File’ tab in VOSviewer.

I found Section 3.2 to be quite confusing. This section is presented as a ‘bibliometric analysis of citations’. However, it is not clear to me whether Figures 1 and 2 show visualizations of citation networks or visualizations of co-authorship networks. Also, the results presented in Section 3.2 rely strongly on the total link strength attribute in VOSviewer. If the author wants to use this attribute, it needs to be explained to the reader how the total link strength is defined and how it can be interpreted. However, I think it is better not to use the total link strength. Presenting statistics based on publication and citation counts is more useful, since these statistics are easier to interpret.

“Regarding keyword analysis, VOSviewer software features two options, one for keywords provided by the authors and the second for keywords provided by authors in addition to others extracted from title and abstract”: This is not correct. VOSviewer users need to choose between analyzing keywords (keywords provided by authors and/or keywords assigned algorithmically by Web of Science) and analyzing terms extracted from titles and/or abstracts. Combining these two analyses is not possible in VOSviewer. Also, while the author mentions a number of frequently occuring keywords, a visualization of the keyword co-occurrence network seems to be missing.

To reduce the number of clusters in a VOSviewer visualization, the author increased the minimum cluster size. Instead of (or in addition to) increasing the minimum cluster size, my advice is to reduce the value of the resolution parameter. This can be done on the ‘Analysis’ tab in VOSviewer.

The discussion section needs major improvements. This section provides a lot information that could better be presented in the introduction or methods sections.

The conclusion section is very brief. The section needs to be extended and improved. The conclusion that the “VOSviewer software is very successful” doesn’t seem relevant, since the paper is not about evaluating the VOSviewer software.

There is room for improving the writing style of the paper. In particular, my suggestion is to avoid the use of exclamation marks and the unnecessary use of capitals in the middle of a sentence (e.g., “found that Till 1 August 2021” should be “found that till 1 August 2021”). Also, there should be no colon at the end of a section heading.

According to the data availability statement, “all data produced in the present study are available upon reasonable request to the authors”. Making data available upon request is poor practice. It is preferable to make data openly available in a data repository (e.g., Zenodo). However, the author should check whether data sharing is allowed in the case of Web of Science data. It probably violates the terms of use of Web of Science. If data sharing is not allowed, this needs to be reported in the data availability statement.

According to the copyright statement, reuse is not allowed without permission. It is good practice to allow preprints to be reused provided that authors are properly acknowledged. I therefore recommend to attach a CC-BY license to the paper.

Serious concerns

• Do you have any serious concerns about the manuscript such as fraud, plagiarism, unethical or unsafe practices? No

• Have authors’ provided the necessary ethics approval (from authors’ institution or an ethics committee)? not applicable

Language quality

• How would you rate the English language quality? Low to medium quality, but I understand the content

validity and reproducibility

• Is the rationale for, and objectives of, the scoping review clearly stated? Yes

• Are the methods documented and analysis provided so that replication can be conducted? No

• Is the statistical analysis and its interpretation appropriate? Not applicable

• Are quality of the figures and tables satisfactory? No

• Are the conclusions adequately supported by the results presented in the review? Not applicable, since the conclusion section is extremely short

• Are there any fundamental flaws or errors that make the scoping review invalid?

Please see my comments above.

Decision

Requires revisions: The manuscript contains objective errors or fundamental flaws that must be addressed and/or major revisions are suggested.

Peer review report

Reviewer: Ludo Waltman

Institution: Leiden University

email: waltmanlr@cwts.leidenuniv.nl

ORCID: https://orcid.org/0000-0001-8249-1752

I would like to note that I am an expert in the field of bibliometrics and that my review therefore focuses on the bibliometric aspects of this paper. I am not an expert on scoping reviews or on curcumin. I hope that other reviewers have expertise on these aspects of the paper.

I am one of the developers of the VOSviewer software, which may perhaps be seen as a competing interest.

General comments

Please find below my detailed comments on the paper, including suggestions for improvements.

“A study by Loannidis et al.”: ‘Loannidis’ should be ‘Ioannidis’.

The author uses the Web of Science Core Collection. This database consists of a number of citation indexes (e.g., Science Citation Index, Social Sciences Citation Index, etc.). Depending on their subscription, different Web of Science users have access to different citation indexes. Please mention which citation indexes were used.

The description of the search query in Section 2 is unclear, because the search query doesn’t seem to be restricted to COVID-19 research. The full search query should be reported in the main text of the paper (not only in Appendix 1).

The VOSviewer visualizations presented in the paper are hard to read (especially Figures 1 and 2). The font size used in the visualizations needs to be increased. This can be done using the ‘Scale’ slider in VOSviewer. The author may also consider making interactive versions of the visualizations available online, so that readers can explore these visualizations in their web browser. A visualization can be made available online using the ‘Share’ button on the ‘File’ tab in VOSviewer.

I found Section 3.2 to be quite confusing. This section is presented as a ‘bibliometric analysis of citations’. However, it is not clear to me whether Figures 1 and 2 show visualizations of citation networks or visualizations of co-authorship networks. Also, the results presented in Section 3.2 rely strongly on the total link strength attribute in VOSviewer. If the author wants to use this attribute, it needs to be explained to the reader how the total link strength is defined and how it can be interpreted. However, I think it is better not to use the total link strength. Presenting statistics based on publication and citation counts is more useful, since these statistics are easier to interpret.

“Regarding keyword analysis, VOSviewer software features two options, one for keywords provided by the authors and the second for keywords provided by authors in addition to others extracted from title and abstract”: This is not correct. VOSviewer users need to choose between analyzing keywords (keywords provided by authors and/or keywords assigned algorithmically by Web of Science) and analyzing terms extracted from titles and/or abstracts. Combining these two analyses is not possible in VOSviewer. Also, while the author mentions a number of frequently occuring keywords, a visualization of the keyword co-occurrence network seems to be missing.

To reduce the number of clusters in a VOSviewer visualization, the author increased the minimum cluster size. Instead of (or in addition to) increasing the minimum cluster size, my advice is to reduce the value of the resolution parameter. This can be done on the ‘Analysis’ tab in VOSviewer.

The discussion section needs major improvements. This section provides a lot information that could better be presented in the introduction or methods sections.

The conclusion section is very brief. The section needs to be extended and improved. The conclusion that the “VOSviewer software is very successful” doesn’t seem relevant, since the paper is not about evaluating the VOSviewer software.

There is room for improving the writing style of the paper. In particular, my suggestion is to avoid the use of exclamation marks and the unnecessary use of capitals in the middle of a sentence (e.g., “found that Till 1 August 2021” should be “found that till 1 August 2021”). Also, there should be no colon at the end of a section heading.

According to the data availability statement, “all data produced in the present study are available upon reasonable request to the authors”. Making data available upon request is poor practice. It is preferable to make data openly available in a data repository (e.g., Zenodo). However, the author should check whether data sharing is allowed in the case of Web of Science data. It probably violates the terms of use of Web of Science. If data sharing is not allowed, this needs to be reported in the data availability statement.

According to the copyright statement, reuse is not allowed without permission. It is good practice to allow preprints to be reused provided that authors are properly acknowledged. I therefore recommend to attach a CC-BY license to the paper.

Serious concerns

• Do you have any serious concerns about the manuscript such as fraud, plagiarism, unethical or unsafe practices? No

• Have authors’ provided the necessary ethics approval (from authors’ institution or an ethics committee)? not applicable

Language quality

• How would you rate the English language quality? Low to medium quality, but I understand the content

validity and reproducibility

• Is the rationale for, and objectives of, the scoping review clearly stated? Yes

• Are the methods documented and analysis provided so that replication can be conducted? No

• Is the statistical analysis and its interpretation appropriate? Not applicable

• Are quality of the figures and tables satisfactory? No

• Are the conclusions adequately supported by the results presented in the review? Not applicable, since the conclusion section is extremely short

• Are there any fundamental flaws or errors that make the scoping review invalid?

Please see my comments above.

Decision

Requires revisions: The manuscript contains objective errors or fundamental flaws that must be addressed and/or major revisions are suggested.

Discussion, revision and decision

Decision

Verified manuscript — The content is scientifically sound, only minor amendments (if any) are suggested.

Revision

Reviewer: Dacre Knight

(1) Included the World Health Organization (WHO) and the National Institute for Health and CARE Excellence (NICE) definitions of PASC. See: Lines 51; 332-339

(2) The PECO criteria is listed (and not just implied) in the body of the manuscript. See: Lines 207-237.

Decision changed — Verified manuscript: The content is scientifically sound.

Reviewer: Yin Qianlan

(3) The purpose of the study was revised for clarity. See: Lines 114-171

Reviewer did not respond. Therefore, a third reviewer (Daniel Griffin) was asked to review the manuscript. They gave the decision, verified manuscript.

Discussion, revision and decision

Decision

Verified manuscript — The content is scientifically sound, only minor amendments (if any) are suggested.

Revision

Reviewer: Dacre Knight

(1) Included the World Health Organization (WHO) and the National Institute for Health and CARE Excellence (NICE) definitions of PASC. See: Lines 51; 332-339

(2) The PECO criteria is listed (and not just implied) in the body of the manuscript. See: Lines 207-237.

Decision changed — Verified manuscript: The content is scientifically sound.

Reviewer: Yin Qianlan

(3) The purpose of the study was revised for clarity. See: Lines 114-171

Reviewer did not respond. Therefore, a third reviewer (Daniel Griffin) was asked to review the manuscript. They gave the decision, verified manuscript.

Peer review report

Reviewer: Daniel Griffin, MD PhD, <br /> Institution: Columbia University ORCID: 0000-0001-5853-6906 email: danielgriffinmd@gmail.com, dgriffin@cumc.columbia.edu

Please describe your research in a sentence or a few key words

COIVD-19, general infectious disease, immunology, virology

General comments

The authors lay out a reasonable protocol for this type of investigation.

Section 1 – Serious concerns

• Do you have any serious concerns about the manuscript such as fraud, plagiarism, unethical or unsafe practices? No
• Have authors’ provided the necessary ethics approval (from authors’ institution or an ethics committee)? Yes

Section 2 – Language quality

• How would you rate the English language quality? High quality

Section 3 – validity and reproducibility

• Is the reasons for conducting the study and its objectives clearly explained? Yes

• Is the study design appropriate? Yes

• Are sufficient details provided so that the method can be replicated? Yes

• Are datasets available so that others could use them? not applicable

Section 4 – Suggestions

• Based on your answers in section 3 how could the author improve the protocol? Fine as is.

• Do you have any other feedback or comments for the Author?

The authors lay out a reasonable protocol for this type of investigation that is based on a fairly standard approach with the standard GRADE grading.

Section 5 – Decision

Verified manuscript: The content is scientifically sound, only minor amendments (if any) are suggested.

Peer review report

Reviewer: Daniel Griffin, MD PhD, <br /> Institution: Columbia University ORCID: 0000-0001-5853-6906 email: danielgriffinmd@gmail.com, dgriffin@cumc.columbia.edu

Please describe your research in a sentence or a few key words

COIVD-19, general infectious disease, immunology, virology

General comments

The authors lay out a reasonable protocol for this type of investigation.

Section 1 – Serious concerns

• Do you have any serious concerns about the manuscript such as fraud, plagiarism, unethical or unsafe practices? No
• Have authors’ provided the necessary ethics approval (from authors’ institution or an ethics committee)? Yes

Section 2 – Language quality

• How would you rate the English language quality? High quality

Section 3 – validity and reproducibility

• Is the reasons for conducting the study and its objectives clearly explained? Yes

• Is the study design appropriate? Yes

• Are sufficient details provided so that the method can be replicated? Yes

• Are datasets available so that others could use them? not applicable

Section 4 – Suggestions

• Based on your answers in section 3 how could the author improve the protocol? Fine as is.

• Do you have any other feedback or comments for the Author?

The authors lay out a reasonable protocol for this type of investigation that is based on a fairly standard approach with the standard GRADE grading.

Section 5 – Decision

Verified manuscript: The content is scientifically sound, only minor amendments (if any) are suggested.

Peer review report

Reviewer: Richard L. Guerrant Institution: University of Virginia email: guerrant@vrginia.edu

Section 1 – Serious concerns

• Do you have any serious concerns about the manuscript such as fraud, plagiarism, unethical or unsafe practices? No

• Have authors’ provided the necessary ethics approval (from authors’ institution or an ethics committee)? not applicable

Section 2 – Language quality

• How would you rate the English language quality? Low to medium quality, but I understand the content

Section 3 – validity and reproducibility

• Does the work cite relevant and sufficient literature? Yes
• Is the study design appropriate and are the methods used valid? No — See comments
• Are the methods documented and analysis provided so that the study can be replicated? No — See comments
• Is the source data that underlies the result available so that the study can be replicated? not applicable
• Is the statistical analysis and its interpretation appropriate? Yes
• Is quality of the figures and tables satisfactory? Yes
• Are the conclusions adequately supported by the results? No — See comments
• Are there any objective errors or fundamental flaws that make the research invalid? Yes, see comments below

Section 4 – Suggestions

• Based on your answers in section 3 how could the author improve the study?

Major concerns include:

1. Quantification of coliforms is key; if done as suggested by MPN method, actual projected counts should be put into a table and discussed by food type, pathogenic potential.

2. Were any actual pathogens detected? (ex ETEC, EPEC, EAEC, EHEC); probes are not those typically used for that. Were any Shigella, Salmonella? Are other pathogens detected? If not, why not?

3. Many statements are expressed as causal facts, but this is far from proven. Examples include ‘foodborne infections …have accelerated … resistance’ on page 1; ‘infections … are due to fast food ingredients;’ on page 2; these points require citation of documented data.

4. At the bottom of page 2, the objective of ‘determining the origin’ is not addressed.

5. Do you have any other feedback or comments for the Author? No

Tighten statements, like the opening line of abstract to say more directly, ‘burden…compromises health.’

Section 5 – Decision

Requires revisions: The manuscript contains objective errors or fundamental flaws that must be addressed and/or major revisions are suggested.

Peer review report

Reviewer: Richard L. Guerrant Institution: University of Virginia email: guerrant@vrginia.edu

Section 1 – Serious concerns

• Do you have any serious concerns about the manuscript such as fraud, plagiarism, unethical or unsafe practices? No

• Have authors’ provided the necessary ethics approval (from authors’ institution or an ethics committee)? not applicable

Section 2 – Language quality

• How would you rate the English language quality? Low to medium quality, but I understand the content

Section 3 – validity and reproducibility

• Does the work cite relevant and sufficient literature? Yes
• Is the study design appropriate and are the methods used valid? No — See comments
• Are the methods documented and analysis provided so that the study can be replicated? No — See comments
• Is the source data that underlies the result available so that the study can be replicated? not applicable
• Is the statistical analysis and its interpretation appropriate? Yes
• Is quality of the figures and tables satisfactory? Yes
• Are the conclusions adequately supported by the results? No — See comments
• Are there any objective errors or fundamental flaws that make the research invalid? Yes, see comments below

Section 4 – Suggestions

• Based on your answers in section 3 how could the author improve the study?

Major concerns include:

1. Quantification of coliforms is key; if done as suggested by MPN method, actual projected counts should be put into a table and discussed by food type, pathogenic potential.

2. Were any actual pathogens detected? (ex ETEC, EPEC, EAEC, EHEC); probes are not those typically used for that. Were any Shigella, Salmonella? Are other pathogens detected? If not, why not?

3. Many statements are expressed as causal facts, but this is far from proven. Examples include ‘foodborne infections …have accelerated … resistance’ on page 1; ‘infections … are due to fast food ingredients;’ on page 2; these points require citation of documented data.

4. At the bottom of page 2, the objective of ‘determining the origin’ is not addressed.

5. Do you have any other feedback or comments for the Author? No

Tighten statements, like the opening line of abstract to say more directly, ‘burden…compromises health.’

Section 5 – Decision

Requires revisions: The manuscript contains objective errors or fundamental flaws that must be addressed and/or major revisions are suggested.

Peer review report

Reviewer: Samuel Mayeden Institution: Ghana Health Service email: csmayeden@gmail.com

Section 1 – Serious concerns

• Do you have any serious concerns about the manuscript such as fraud, plagiarism, unethical or unsafe practices? No
• Have authors’ provided the necessary ethics approval (from authors’ institution or an ethics committee)? Yes

Section 2 – Language quality

• How would you rate the English language quality? High quality

Section 3 – validity and reproducibility

• Does the work cite relevant and sufficient literature? Yes
• Is the study design appropriate and are the methods used valid? Yes
• Are the methods documented and analysis provided so that the study can be replicated? Yes
• Is the source data that underlies the result available so that the study can be replicated? Yes
• Is the statistical analysis and its interpretation appropriate? Yes
• Is quality of the figures and tables satisfactory? Yes
• Are the conclusions adequately supported by the results? Yes
• Are there any objective errors or fundamental flaws that make the research invalid? No

Section 4 – Suggestions

• Do you have any other feedback or comments for the Author?

• Line 35, 36 and 37: Authors should be consistent in using odds ratios and/or p values to indicate associations.

• Line 37 (Conclusion): Should align with objectives and results stated in abstract

• 42: Keywords should emerge from the text of the abstract. Kindly review the keywords to align with the body of the abstract

• Introduction: Has no concrete global, regional or local level statistics on the subject. It will be good to indicate some statistics of existing work to appropriately conceptualize your work.

• The introduction is too long. It will be good to keep it within 2 pages

• Line 73: It helps to move citation to the end of the sentence

• Line 132: Indicate from which department(s) of the health facilities the participants were recruited

• Methods: Needs lots of clarification. Authors have to indicate the process of sampling more clearly

• What influenced the selection of the 32 facilities out of the 252

• You indicated 15 clients from health centres and 22 clients from hospitals. Are these the authors’ assumptions? Was this calculated from the facility registers? If so what amount of data from which period was used to make the estimates

• Line 154: Did each facility have a sampling frame? What was k?

• Line 172: Which department were patient exiting from. Was it outpatients or laboratory only cases, or inpatients as well.

• Line 174: Was scale adapted or developed by authors

• Line 183-185: how many slides were reviewed. Why did you choose only malaria and TB.

• Line 196: Is the score decided by researchers or it has been adopted or adapted

• Line 205: Reference star grading system

• Line 256: state p value as p<0.001

Section 5 – Decision

Verified manuscript: The content is scientifically sound, only minor amendments (if any) are suggested.

Peer review report

Reviewer: Samuel Mayeden Institution: Ghana Health Service email: csmayeden@gmail.com

Section 1 – Serious concerns

• Do you have any serious concerns about the manuscript such as fraud, plagiarism, unethical or unsafe practices? No
• Have authors’ provided the necessary ethics approval (from authors’ institution or an ethics committee)? Yes

Section 2 – Language quality

• How would you rate the English language quality? High quality

Section 3 – validity and reproducibility

• Does the work cite relevant and sufficient literature? Yes
• Is the study design appropriate and are the methods used valid? Yes
• Are the methods documented and analysis provided so that the study can be replicated? Yes
• Is the source data that underlies the result available so that the study can be replicated? Yes
• Is the statistical analysis and its interpretation appropriate? Yes
• Is quality of the figures and tables satisfactory? Yes
• Are the conclusions adequately supported by the results? Yes
• Are there any objective errors or fundamental flaws that make the research invalid? No

Section 4 – Suggestions

• Do you have any other feedback or comments for the Author?

• Line 35, 36 and 37: Authors should be consistent in using odds ratios and/or p values to indicate associations.

• Line 37 (Conclusion): Should align with objectives and results stated in abstract

• 42: Keywords should emerge from the text of the abstract. Kindly review the keywords to align with the body of the abstract

• Introduction: Has no concrete global, regional or local level statistics on the subject. It will be good to indicate some statistics of existing work to appropriately conceptualize your work.

• The introduction is too long. It will be good to keep it within 2 pages

• Line 73: It helps to move citation to the end of the sentence

• Line 132: Indicate from which department(s) of the health facilities the participants were recruited

• Methods: Needs lots of clarification. Authors have to indicate the process of sampling more clearly

• What influenced the selection of the 32 facilities out of the 252

• You indicated 15 clients from health centres and 22 clients from hospitals. Are these the authors’ assumptions? Was this calculated from the facility registers? If so what amount of data from which period was used to make the estimates

• Line 154: Did each facility have a sampling frame? What was k?

• Line 172: Which department were patient exiting from. Was it outpatients or laboratory only cases, or inpatients as well.

• Line 174: Was scale adapted or developed by authors

• Line 183-185: how many slides were reviewed. Why did you choose only malaria and TB.

• Line 196: Is the score decided by researchers or it has been adopted or adapted

• Line 205: Reference star grading system

• Line 256: state p value as p<0.001

Section 5 – Decision

Verified manuscript: The content is scientifically sound, only minor amendments (if any) are suggested.

Discussion, revision and decision

Overall decision — Requires revisions: The manuscript contains objective errors or fundamental flaws that must be addressed and/or major revisions are suggested.

Responses to the reviewers

Reviewer Charlotte Martial

Decision — Requires revisions: The manuscript contains objective errors or fundamental flaws that must be addressed and/or major revisions are suggested.

1-The work does not cite relevant and sufficient literature. For instance, the authors do not cite a recent publication which aimed to induce OBE using hypnosis (Martial et al., Scientific Reports, 2019).

Reply: We re-read the procedure of this paper we were aware of, but we didn’t find any reference related to an OBE induced by hypnotic suggestions. Here follows an excerpt of the procedure as reported in the cited paper: “The hypnotic instruction included a 5-min induction procedure with eye fixation (ultimately closing the eyes) and progressive muscle relaxation. Permissive and indirect suggestions were used to develop and deepen the hypnotic state. The participant was then invited to re-experience their NDE memory (20 min) and their pleasant autobiographical memory (20 min) in a counterbalanced order, ..”

Thanks for the reply. I do not agree, though. As described in the paper, Martial and colleagues (2019) focused on two prototypical features of the NDE during the hypnosis session, that is, feeling of peacefulness and out-of-body experience. Consequently, I think it is highly relevant for the present paper.

Reply: we accepted this suggestion and added a paragraph about this application of hypnosis.

2-Another example: The authors could have at least briefly discussed Parnia’s AWARE prospective study reporting OBE in NDE experiencers (Parnia et al., 2014).

Reply: Thank you for this suggestion, now added in the introduction

3-Besides, they sometimes cite some inappropriate references; I would like to invite the authors to cite rigorous and serious articles in the field. For example, they used Jourdan (2011) reference to draw a parallel with NDEs, however, Jourdan (2011) is not a reference article in the field…

Reply: Jourdan’s (2011) paper “Near Death Experiences and the 5th Dimensional Spatio-Temporal Perspective. Journal of Cosmology, 14, 4743-4762 deals with NDE. It is not clear why it cannot be considered a reference article in the field.

Dr Jourdan and most of his articles are scientifically controversial. They are many other references more rigorous that you could cite in order to be scientifically stronger.

Reply: we have a different opinion about Jourdan’s scientific findings. As written in the paper, we referred to Jourdan’s paper only as a source to draw the questions to be used with the participants when in the OBE status.

4- Importantly, it is worth mentioning that, so far, no rigorous empirical scientific study has shown evidence of veridical perceptions during OBE; according to me, it is not clear in the manuscript.

Reply: On pag. 2 we added “and verify by a third-person perspective what they declare to perceive in this particular state of consciousness”.

I thank the authors. However, this is still not clearer in the manuscript. I suggest that the authors clearly state that so far, no rigorous empirical scientific study has shown evidence of veridical perceptions during OBE. This is highly important for a manuscript aiming to study OBE in laboratory settings.

Reply: added this suggestion at the end of the Introduction

5-Some limitations of the study are not mentioned in the discussion section

Reply: On pag. 14 there is a whole paragraph “Study limitations”.

Well, let me clarify: some of the limitations of the study are not discussed in the manuscript.

Reply: the “Study limitations” is added on pag. 15

6- Some details of the study are missing. Notably, the description of the Table 6 is incomplete: what do the bold numbers mean

Reply: we revised the presentation of the data in Table 6 and clarified why some data were presented in bold.

7- Another example: how did they recruit the participants –who are co-authors of the paper? Are they researchers?

Reply: We have revised the section related to the participants’ selection

8- page13: what do they mean by “general consensus”? This is not clear to me

Reply: We revised the text in “The answers of all five participants to the eleven questions presented in Table 4 and 5, were quite similar in content”

9-The conclusions they draw in the discussion are not based on the present findings; they extrapolate

Reply: Given in the paper there is not a “Conclusions” paragraph, to what “conclusions” do you refer to?

I mean the interpretation of their results … As an example, based on what do you conclude that “…in this state of consciousness they can act without the limitations that the physical body imposes on motion and perception especially with regard to vision”. It is not clear that you here mean that their subjective experience is related to their imaginary. Please reformulate.

Reply: This is not a conclusion, it is a simple summary of participants’ phenomenological reports “….from all of the participants’ reports, it appears that in this state of consciousness they can act ….”

what do you mean by “a three-dimensional universe”?

Reply: we corrected this sentence that now is “Time seems to be absent or similar to a sequence of video-clips, suggesting that they are living in a four-dimensional universe, very similar to that described as the “Block Universe” which represents space-time as a fixed whole in which past and future events already exist in the space-time continuum, in accordance with the special theory of relativity”.

Based on which data can you conclude that what they experienced is “very similar to or the same as that described in NDEs”.

Reply: we deleted this sentence

Reviewer Aminata Bicego

Decision — Verified with reservations: The content is scientifically sound, but has shortcomings that could be improved by further studies and/or minor revisions.

Thank you for your accurate review and suggestions.

Abstract :

L13 : what do the authors mean by “hypnotic induction”, is it hypnotic experience ? During a hypnosis session, the therapist starts by an induction and then makes some suggestions according to the goal that has been decided. It is not clear to what the “hypnosis induction” refers to.

Reply : We changed « hypnotic induction » as « hypnotic experience », as suggested.

L16 : “under hypnosis” should be changed as it suggests that the individual who is in hypnosis is passive. I would suggest “in hypnosis”.

Reply : changed as suggested

Introduction: L 54-56 : there has been some recent literature on hypnosis and OBEs or NDEs. It would be interesting to add newer literature on that topic.

Reply : we searched with google scholar if there were new studies about hypnosis and OBEs, but we didn’t find any. The were two studies related to hypnosis and NDEs, but these experiences are not related to our study (see also our response to reviewer Charlotte Martial).

L75 : if the study’s aim is to confirm Tressoldi and Del Prete (2007), then this study should be explicitly explained in detail. That way the reader understands better the present study.

Reply : We added some further details related to Tressoldi and Del Prete (2007) study, as suggested.

L76 : could the authors specify the suggestion used.

Reply : the suggestions used in the study are presented in the Supplementary Materials

L78 : the induction is not the only and principal part of the hypnotic sessions that impacts an individual perceptions but rather the suggestion that is used. This paragraph could be clearer in terms of methodology

Reply : the suggestion to induce an OBE in Tressoldi and Del Prete (2007) study, are too long to be added in the text, but are very similar to that used in the present study.

L79 to L95 : this should be in methods.

Reply : moved in the Methods under the paragraph « Study aims »

Materials and Methods : L110 : can the authors define and detail “clinical level of medical or psychiatric disease”.

Reply : we added « compatible with the clinical criteria defined in the main international clinical guidelines, such as DSM V or ICD-10,

L111 : took, should be written “take”. Throughout the manuscript there are language mistakes that have to be checked.

Reply : we checked the syntax as suggested.

L111 : can the authors be more specific on the “personal experience with hypnosis” ? What do you mean by experience ?

Reply : In Table 1, we changed the experience value adding the number of previous hypnotic sessions.

Procedure : in general the procedure is not very clear. Some results appear in the section when they should be in the result section

Reply : we moved the sentence « In total, twenty-eight sessions were necessary to complete the study. Eighteen sessions were carried out by phone with the participants at home” in the results section.

L125 : Can the authors explain why some participants had one or two sessions ?

Reply : In the Procedure we wrote "We placed the images in two different rooms as long as it was possible to require the identification of two different images in a single session. In only two cases, the participants had sufficient time and concentration to identify two images in a single session reducing their overall sessions to four instead of six"

L127 : Can the supplementary Material be numbered. L129 to 132 : Have all participants been seen in real life? Were all of them called? If not, was it always the same people in person or on the phone? This part should be more specific.

Reply : In the Results section we clarified that « In total, twenty-eight sessions were necessary to complete the study. Eighteen sessions were carried out by phone with the participants at home »

L130 : hypnotized should not be used. Same comment as comment L16.

Reply : corrected as suggested

L140 : was the order of the picture position randomized ? If so it should be mentioned here.

Reply : we wrote « …the six target images and randomized their order of presentation, the room where each one was placed and their orientation, either facing up or upside down »

L164: Who did the authors know that the participants where in an OBE state ? What were the criteria ? Especially on the phone ?

Reply: On pag. 5, we clarified that the hypnotist referred to the participants phenomenological account in response to the suggestions described in the Supplementary Material.

L174 : did the authors create the questions ? Do they come from a questionnaire ? This should be specified.

Reply : we clarified « devised by the authors of this study »

L188 : this should be in results Reply : moved in the Data analysis paragraph as suggested.

L191 : “... and give suggestion...” : It is confusing to use that word, comments, might be more appropriate.

Reply : corrected as suggested.

L198: what are the eleven questions ?

Reply : we corrected and added « all participants were emailed the same six questions related to their perceptual and cognitive experience when in OBE..

L208 : all the questionnaires used should also be described in material. The reader has no information on the minimal phenomenal selfhood, nor has he information about the “characteristics of spatial and temporal perception reported in NDEs”.

Reply: all questions were original and devised by the authors. None questionnaire of other authors was used.

L221 : how did the 3 decoys were selected ?

Reply : we added « low arousal pictures »

L225 : I only see 2 authors, not three.

Reply : we corrected as « co-author LP and another independent judge affiliate with the EcvanLab… »

L234 : a section with the statistical analyses should be written before the results.

Reply : we added a section « Statistical method »

L241 : 52.4 % is not metionned in the table, this is confusing.

Reply : Sorry, we missed the overall results in the last line, now added

L245 : this should be in the statistical analyses part, it is not a result per se.

L260 : the table 3 should be more specific: define ES + formula, CI, BF, H1, H0. A table should be able to be read by itself.

Reply : In the Table 3 presentation and title, we clarified all the statistics

L277 : This part merits some clarifications : Did the approval from the Ethical committee approved this part ? If so, did the participants signed an informed consent ?

Reply: The Ethical approval included the informed consent to be signed by participants

What should they refer to when they answered the questions ? Did the ones that had an OBE had to refere to that episode ? And those who did not life an OBE ? What was the experience of reference ?

Reply: Participants induced in OBE, should refer to their phenomenological experience in this state included the period when they were requested to identify the targets.

It there is no reference for the controls without an OBE experience then is seems logical that they do not answer like the others.

Reply : controls participants were selected among those who have only read about OBEs

L304 : the % is not correct, is should be 46.6%

Reply : Corrected

L338 : With what material was the comparison made ? With the material from Jourdan (2011) or the participants that were contacted after the study ? This part should be clearer If the comparison is made with Jourdan, then a table with the similarity and differences could be added.

Reply : We changed the title of Table 6 in order to clarify its data.

L382 : “but his aim was not to confirm his knowledge, but to compare it with the participants’ experience” : this was not mentionned before. It is hard to understand as we have no information in the hypnotist knowledge or the comparison that is mentionned. Could the authors clarify ?

Reply : The hypnotist knew very well the OBE phenomenology (now added in his description on page 4. However, the questions for participants were formulated in order not to confirm the hypnotist’s previous knowledge.

L387 : Another limitation it the response expectancy during hypnosis. There is a large literature on the subject this should be discused in the limites. More so because all the subject had good knowledge about OBEs

Reply : In the paragraph « Reliability of participants’ reports », we discuss precisely these issues.

L412 : Is it acceptable to disclose the participants identity ?

Reply : we deleted their names

Page 5 : The authors mention in a foot note that some other informatio will be avaiable in a future publication but the publication has since been published. Furthermore, that publication is cited in the bibliography this is confusing.

Reply : we corrected this discrepancy

Table S1 : it is hard for the reader to understand to what the table refers to. GAPED has to be explained.

Reply ; The GAPED acronym is described in the Materials paragraph on pag. 6

Peer review report

Reviewer: Wei Zhang Institution: The First Affiliated Hospital of Hebei North University, Hebei Province, China. email: 15369318318@163.com

Section 1 – Serious concerns

• Do you have any serious concerns about the manuscript such as fraud, plagiarism, unethical or unsafe practices? No
• Have authors’ provided the necessary ethics approval (from authors’ institution or an ethics committee)? *Yes

Section 2 – Language quality

• How would you rate the English language quality? High quality

Section 3 – validity and reproducibility

• Does the work cite relevant and sufficient literature? Yes
• Is the study design appropriate and are the methods used valid? Yes
• Are the methods documented and analysis provided so that the study can be replicated? Yes
• Is the source data that underlies the result available so that the study can be replicated? not applicable
• Is the statistical analysis and its interpretation appropriate? Yes
• Is quality of the figures and tables satisfactory? No
• Are the conclusions adequately supported by the results? not applicable
• Are there any objective errors or fundamental flaws that make the research invalid?

Table 1 - The number of people in the gender section is wrong.

Section 4 – Suggestions

• Based on your answers in section 3 how could the author improve the study?

（1）There is no "conclusion" in the abstract.

（2）In order to make the study group comparable with the control group, should the patients in the control group be CSKP infected patients? Or the control group should at least be patients with bacterial infection.

（3）The font in Table 2 is inconsistent with the original text.

（4）The table format should be "three line table".

（5）The number "0" can be added before some decimal points in Table 2.

（6）The references are generally too old. It is suggested to increase the proportion of references published within 5 years.

（7）It is suggested to write more specific experimental methods for amplifying KPC gene, such as primers, PCR reaction conditions and reaction system of KPC gene.

（8）Have you done the drug sensitivity test of Klebsiella pneumoniae? If so, it is recommended to write the results in the article.

Section 5 – Decision

Verified with reservations: The content is scientifically sound but has shortcomings that could be improved by minor revisions.

Peer review report

Reviewer: Wei Zhang Institution: The First Affiliated Hospital of Hebei North University, Hebei Province, China. email: 15369318318@163.com

Section 1 – Serious concerns

• Do you have any serious concerns about the manuscript such as fraud, plagiarism, unethical or unsafe practices? No
• Have authors’ provided the necessary ethics approval (from authors’ institution or an ethics committee)? *Yes

Section 2 – Language quality

• How would you rate the English language quality? High quality

Section 3 – validity and reproducibility

• Does the work cite relevant and sufficient literature? Yes
• Is the study design appropriate and are the methods used valid? Yes
• Are the methods documented and analysis provided so that the study can be replicated? Yes
• Is the source data that underlies the result available so that the study can be replicated? not applicable
• Is the statistical analysis and its interpretation appropriate? Yes
• Is quality of the figures and tables satisfactory? No
• Are the conclusions adequately supported by the results? not applicable
• Are there any objective errors or fundamental flaws that make the research invalid?

Table 1 - The number of people in the gender section is wrong.

Section 4 – Suggestions

• Based on your answers in section 3 how could the author improve the study?

（1）There is no "conclusion" in the abstract.

（2）In order to make the study group comparable with the control group, should the patients in the control group be CSKP infected patients? Or the control group should at least be patients with bacterial infection.

（3）The font in Table 2 is inconsistent with the original text.

（4）The table format should be "three line table".

（5）The number "0" can be added before some decimal points in Table 2.

（6）The references are generally too old. It is suggested to increase the proportion of references published within 5 years.

（7）It is suggested to write more specific experimental methods for amplifying KPC gene, such as primers, PCR reaction conditions and reaction system of KPC gene.

（8）Have you done the drug sensitivity test of Klebsiella pneumoniae? If so, it is recommended to write the results in the article.

Section 5 – Decision

Verified with reservations: The content is scientifically sound but has shortcomings that could be improved by minor revisions.

Discussion, revision and decision

Author response

To: Adam Marcus, co-founder Retraction Watch & Alison Abritis, PhD, researcher at Retraction Watch

Major Problems: I found serious deficits in both for this article, and thus I have serious concerns as to the usefulness of this article. Therefore, I have not proceeded in a line-by-line, as I consider the overall problems to be grave enough to require attention and revision before getting to lesser items of clarity.

I would like to point out that the authors show a marvelous attention to their work, and they have much to contribute to the field of retraction studies, and I do honestly look forward to their future work. However, in order for the field to move ahead with accuracy and validity, we must no longer just rely on superficial number crunching, and must start including the complexities of publishing in our analyses, as difficult and labor-intensive as it might be.

We do not consider that our article presents serious problems nor that it would be useless.

It is possible that a different view on the subject, some tendency to forbearance (understandable) for the difficult life of the publishing industry, along with some difficulties in understanding the ideas presented in the article, may have led to a series of points of view that we would like to comment on below.

We would first like to thank the reviewers for their comments, some of which will allow us to improve and nuance, using objective elements, the analysis of this bumpy field represented by the ecosystem of retracted publications. Because we have based our study on data from freely accessible sources of information, we will not insist too much on commenting on this issue.

The authors stated that they used the search protocol (and therefore presumably the same dataset) as described in Toma & Padureanu, 2021, and do not indicate any process to compensate for its weaknesses. In the referenced study, the authors (same as for this article) utilized a PubMed search using only “Retracted Publication” in Publication Type. This search method is immediately insufficient, as some retracted articles are not bannered or indexed as retracted in PubMed. This issue is well-understood among scholars who search databases for retractions, and by now one would expect that these searches would strive to be more comprehensive.

A better method, if one insists on restricting the search to PubMed, would have been to use Publication Type to search for “retracted publication,” and then to search for “retraction of publication,” and to compare the output to eliminate duplications. There are even more comprehensive ways to search PubMed, especially since some articles are retitled as “Withdrawn” – Elsevier, for example, uses the term instead of “Retracted” for papers removed within a year of their publication date – but do not come in searches for either publication type. Even better would have been to use databases with more comprehensive indexing of retractions.

In an ideal world, if any effort were to be made, it would be aimed at better indexing and managing existing databases, not at generating query strategies to make up for their shortcomings.

Thank you very much for the suggestions on the search strategy. We do not consider that the use of "Retracted Publication [PT]" should be compensated in any way but, if it should be compensated, we wouldn't want to add "Retraction of publication". We consider that using a search protocol more specific to systematic reviews is not very useful in our case: data are added/updated continuously (sometimes late), incorrect indexing can be corrected, the number of retracted articles increases from month to month; the same strategy can give different results at different times regardless of its complexity. Putting extra effort into detecting problematic articles without knowing the benefit but expecting it only highlights issues that can be improved at the publisher/editor(content delivery) and database level(indexing).

The dataset analyzed is a snapshot of a particular time interval and nothing more. Even during the analysis we found, in the case of one publisher, the addition of details to the initially incomplete retraction notes. Hence the need for follow-up studies. Therefore in the case of retractions, unlike the reviewer, we prefer an approach based on simple and easily reproducible strategies, widely accessible sources of information, and several steps. The first step in this strategy is the "number crunching" stage which includes this article.

1. The authors are using the time from publication to retraction based on the notice dates and using them to indicate efficacy of oversight by publishers. However, this approach is seriously problematic. It takes no notice of when the publisher was first informed that the article was potentially compromised. Publishers who respond rapidly to information that affects years/decades old publications will inevitably show worse scores than those who are advised upon an article’s faults immediately upon its publication, but who drag their heels a few months in dealing with the problem.

Indeed, the article uses the time between publication and retraction(exposure time – ET) as one of the SDTP score components for assessing editorial/publisher performance. Data on when a publisher or editor has been informed of problems with an article, apart from being relatively rare, is not a substitute for a retraction note. Moreover, the use of such information may induce a risk of bias.

We mention in the article the need to use reporting standards for retraction notes, and one element that might be useful is, indeed, the date on which the publisher or editor was informed of problems with an article. Unfortunately, as the author of this review knows very well, information precedes investigation; the retraction note contains (or should contain) much more data than the initial information about the quality problems of an article.

Our article aims to suggest a score for measuring publication performance in the context of retracted articles that would also allow an assessment of the dynamics of the activity of correcting the scientific record and, more importantly, how publishers engage in post-publication quality control. ET is only one component of this score.

It is quite clear from the data presented in the article that a publisher/journal that emphasizes systematic back-checking will have an increasingly longer average lifespan of retracted articles, logically higher than one that does not do this type of checking. We don't see precisely where the reviewer thinks there is a problem: once the checking is done, the ET will decrease, and a publisher that takes concrete steps to correct the literature will ultimately have a better reputation. This does not mean that a higher ET is laudable, it suggests that there is a post-publication quality control but also that the peer review process has let problematic articles through and that the control of these articles has been carried out late. This is an argument for more active involvement of publishers (as potential generators of editorial policies) in post-publication control.

Second, there is little consistency in dealing with retractions between publishers, within the same publishers or even within the same journal. Under the same publisher, one journal editor may be highly responsive during their term, while the next editor may not be. Most problems with articles quite often are first addressed by contacting the authors and/or journal editors, and publishers – especially those with hundreds of journals – may not have any idea of the ensuing problem for weeks or months, if at all. Therefore, the larger publishers would be far more likely to show worse scores than publishers with few journals to manage oversight.

It is exactly this inconsistency that we highlight in the article. Differing policies, attitudes, and responsiveness does not mean that a publisher cannot/should not ask questions about the effectiveness of internal processes and resources used for post-publication quality control or the implementation of uniform measures across journals in its portfolio.

Third, the dates on retraction notices are not always representative of when an article was watermarked or otherwise indicated as retracted. Elsevier journals often overwrite the html page of the original article with the retraction notice, leaving the original article’s date of publication alone. A separate retraction notice may not be published until days, weeks or even years after the article has been retracted. Springer and Sage have done this as well, as have other publishers – though not to the same extent (yet).

Historically, The Journal of Biological Chemistry would publish a retraction notice and link it immediately to the original article, but a check of the article’s PDF would show it having been retracted days to weeks earlier. They have recently been acquired by Elsevier, so it is unknown how this trend will play out. And keep in mind, in some ways this is in itself not a bad thing – as it gives the user quicker notice that an article is unsuitable for citation, even while the notice itself is still undergoing revisions. It just makes tracking the time of publication to retraction especially difficult.

We used the same date for all articles in our study (the one listed in PubMed), thus ensuring a uniform criterion for all publishers. If this date was not in PubMed we used the date from the retraction notes on the journal website but this was for a small number of articles. How different publishers handle retraction processes or the delay with which these are published is primarily related to internal editorial procedures, and these delays are reflected in the ET. In our experience, most articles retracted by Elsevier are available online, supplemented, and not replaced by retraction notes, which we think is an excellent policy.

1. As best as can be determined, the authors are taking the notices at face value, and that has been repeatedly shown to be flawed. Many notices are written as a cooperative effort between the authors and journal, regardless of who initiated the retraction and under the looming specter of potential litigation.

Shown to be flawed by who? Indeed, in our study, we refer to the retraction notes published by the journals. The fact that they are incomplete or formulated under the threat of litigation only supports our view that publishers and editors need to make a more significant effort to correct the biomedical literature, including avoiding litigation when the retraction note clearly describes the reasons for retraction. The way the retraction note is worded should be an editorial prerogative and should primarily aim at correcting scientific literature, not at appeasing egos, careers, or financial interests.

Trying to establish who initiated a retraction process strictly by analyzing the notice language is destined to produce faulty conclusions. Looking just at PubPeer comments, questions about the data quality may be raised days/month/years before a retraction, with indications of having contacted the journal or publisher. And yet, an ensuing notice may be that the authors requested the retraction because of concerns about the data/image – where the backstory clearly shows that impetus for the retraction was prompted by a journal’s investigation of outside complaints. As an example, the recent glut of retractions of papers coming from paper mills often suggest the authors are requesting the retraction. This interpretation would be false, however, as those familiar with the backstory are aware that the driving force for many of these retractions were independent investigators contacting the journals/publishers for retraction of these manuscripts.

Once again, the author of this review does not seem to fully understand our study, apparently favouring information published on third-party websites over that the journals officially assumed. The retraction notes represent the material available to a researcher doing documentation on a particular topic. The clarity and information contained in the note is the editor's or publisher’s responsibility, reflecting their performance and concern for the integrity of the science. Interpretation of a retraction note/analyzing an article occurs in this context. Not everyone has time for further investigation or to search third-party sites for information that is, with a notable exception, the result of a selection bias.

Assigning the reason for retraction from only the text of the notice will absolutely skew results. As already stated, in many cases, journal editors and authors work together to produce the language. Thus, the notice may convey an innocuous but unquestionable cause (e.g., results not reproducible) because the fundamental reason (e.g., data/image was fabricated or falsified) is too difficult to prove to a reasonable degree. Even the use of the word “plagiarism” is triggering for authors’ reputations – and notices have been crafted to avoid any suggestion of such, with euphemisms that steer well clear of the “p” word. Furthermore, it has been well-documented that some retractions required by institutional findings of misconduct have used language in the notice indicating simple error or other innocuous reasons as the definitive cause.

We understand your point of view and the situations presented may be accurate. However, from our point of view, the only valid reference remains the retraction note published on the journal's website. The existence of wording difficulties and various other problems that may arise are more likely to do with a tendency of the reviewer to make excuses for journals reluctant to indicate precisely what the reasons for retracting the article are. There are plenty of retraction notes in which the images with problems (including whether they were plagiarized, reused, manipulated, fabricated, etc.) are indicated with great precision, there are equally plenty of notes in which the word plagiarism is used without hesitation, indicating the sources, how they were informed, what was plagiarized. No matter how many hesitant publishers/editors there are, it should not be forgotten that there are many journals/publishers who take their role seriously, acknowledge and learn from their mistakes, thus providing a real service to the scientific community.

The authors also discuss changes in the quality of notices increasing or decreasing in publishers – but without knowing the backstory. Having more words in a notice or giving one or two specific causes cannot in itself be an indicator of the quality (i.e., accuracy) of said notice.

"Knowing the backstory" is not part of our objectives, and neither is assessing the quality of the retraction notes. This is also very difficult to do due to the lack of an accepted standard format. We are trying to propose a score composed of several parameters resulting from existing (or non-existing) data in the retraction notes so that we can have a picture of retractions at publisher level. Knowing the backstory is not relevant, reading and interpreting the official retraction note is relevant.

1. The authors tend to infer that the lack of a retraction in a journal implies a degree of superiority over journals with retractions. Although they qualify it a bit ( “Are over 90% of journals without a retracted article perfect? It is a question that is quite difficult to answer at this time, but we believe that the opinion that, in reality, there are many more articles that should be retracted (Oransky et al. 2021) is justified and covered by the actual figures.”), the inference is naive. First, they have not looked at the number of corrections within these journals. Even ignoring that these corrections may be disproportionate within different journals and require responsive editorial staff, some journals have gone through what can only be called great contortions to issue corrections rather than retractions.

We believe that this is a case of reviewer confusion generated either by the insufficiently precise wording of the text or a lack of understanding of our study objectives. We are trying to point out that more than 90% of the journals in the NLM catalogue-PubMed subset have not retracted a single article. We are not trying to say that journals without retracted articles are superior to the others. As explained in the article, we referred to retraction notes, not corrections.

Second, the lack of retractions in a journal speaks nothing to the quality of the articles therein. Predatory journals generally avoid issuing retractions, even when presented with outright proof of data fabrication or plagiarism. Meanwhile, high-quality journals are likely to have more, and possibly more astute, readers, who could be more adept at spotting errors that require retraction.

Of course, the quality level of articles in a journal is not determined by the number of articles removed.

Third, smaller publishers/journals may not have the fiscal resources to deal with the issues that come with a retraction. As an example, even though there was an institutional investigation finding data fabrication, at least one journal declined to issue a retraction for an article by Joachim Boldt (who has more than 160 retractions for misconduct) after his attorneys made threats of litigation.

Threats of lawsuits are instead a failure of a publisher/journal to adapt to the realities of the publishing business or to the risk of misconduct. This is something that needs to change.

Simply put, the presence or lack of a retraction in a journal is no longer a reasonable speculation about the quality of the manuscripts or the efficiency of the editorial process.

We have not attempted to suggest this, we have only analyzed the retracted articles and their associated retraction notes. On the other hand, the way a journal/publisher handles the retraction of problematic articles still reflects, to some extent, the quality/performance of the editorial processes.

1. I am concerned that the authors appear to have made significant errors in their analysis of publishers. For example, they claim that neither PLOS nor Elsevier retracted papers in 2020 for problematic images. That assertion is demonstrably false.

This is wrong. In our dataset, there are eleven PLOS articles related to human health with the publication year 2019 and 2020. None of these have images as retraction reasons.

Regarding the 21 Elsevier articles published in 2020, there is nothing in the retraction notes to indicate that the article was retracted because of the images. In 2 retraction notes there is mention of the comments made by Dr. Bik (The Tadpole Paper Mill - Science Integrity Digest) but the text of these (retraction notes) stops at the authors' inability to provide the raw data underlying the article.

Our study is based only on the content of the retraction notes published and assumed by the journal, not on opinions/comments appearing on other sites, which, for unknown/unmentioned reasons, are not officially assumed in the retraction note. Therefore, we consider the statement in the review to be questionable at best, as the use of material other than the retraction notes has severe implications for the internal and external validity of the study and the suggestion to use such methods is, in our opinion, wrong. We would also like to draw attention to the fact that many retraction notes are explicitely mentioning the request to provide raw images and the authors' inability to provide them.

Anyway, as far as images are concerned, our article suggested that there are publishers which seem to adopt image analysis technologies faster than others. The numbers are not really relevant in this case but the trend is: it describes the publishing activity complexity better than the numbers.

Reviewer response

We appreciate the authors’ zeal in standing by their work.

In regard to the deficits in the search process, the author states, “We do not consider that the use of ‘Retracted Publication [PT]’ should be compensated in any way but, if it should be compensated, we wouldn't want to add ‘Retraction of publication’”

There is a lack of appreciation for the complexities of indexing retracted materials in an indexing site such as PubMed. To have a comprehensive search, one should not be choosing to use either “Retracted Publication [PT]” OR “Retraction of Publication [PT].” One would use both, and then filter out the duplicates, because some retractions are indexed by retraction notices, some only have “Retracted” added to the indexed title and the publication type changed to “Retracted Publication.” Use of only one or the other guarantees that the search is far less comprehensive than it should be.

The authors state, “In an ideal world, if any effort were to be made, it would be aimed at better indexing and managing existing databases, not at generating query strategies to make up for their shortcomings.”

There is at least one database (http://retractiondatabase.org) that has a far more comprehensive indexing of retractions and is publicly available for use.

In Item 3, where it is pointed out that retraction notices themselves are inaccurate and cannot be taken at face value as to the reason behind the retraction, the authors responded, “Shown to be flawed by who?” — By an article cited in the manuscript:

Fang, Ferric C.; Steen, R. Grant; Casadevall, Arturo (2012): Misconduct accounts for the majority of retracted scientific publications. In Proceedings of the National Academy of Sciences of the United States of America 109 (42), pp. 17028–17033. DOI: 10.1073/pnas.1212247109.

“To understand the reasons for retraction, we consulted reports from the Office of Research Integrity and other published resources (7, 8), in addition to the retraction announcements in scientific journals. Use of these additional sources of information resulted in the reclassification of 118 of 742 (15.9%) retractions in an earlier study (4) from error to fraud.” Followed by “These factors have contributed to the systematic underestimation of the role of misconduct and the overestimation of the role of error in retractions (3, 4), and speak to the need for uniform standards regarding retraction notices (5).”

The authors then choose to state that it is the “editorial prerogative” – and that when notices “are incomplete or formulated under the threat of litigation [it] only supports our view that publishers and editors need to make a more significant effort to correct the biomedical literature, including avoiding litigation when the retraction note clearly describes the reasons for retraction.”

Following our attempt to explain why understanding the real reason behind a retraction is important to study the publication of notices, the authors respond: “Once again, the author of this review does not seem to fully understand our study, apparently favouring information published on third-party websites over that the journals officially assumed.”

First, yes, we do understand the study. We read a lot of these. Second, the “third-party websites” we prefer include the Office of Research Integrity and the Retraction Watch blog, where background investigations into the causes of retraction notices are described. If the authors are challenging the reference to PubPeer, keep in mind that journals initiate investigations based on comments on that website, and have taken to citing the website in their notices.

Had the authors not chosen to categorize the reasons for retraction, their reasoning may have had more support – but they did, and in doing so, by just using the notice with no further review, their findings address only the notice itself, with no context.

We recommend that the manuscript be substantially revised with strong attention to the comments we made in our original review.

Discussion, revision and decision

Author response

To: Adam Marcus, co-founder Retraction Watch & Alison Abritis, PhD, researcher at Retraction Watch

Major Problems: I found serious deficits in both for this article, and thus I have serious concerns as to the usefulness of this article. Therefore, I have not proceeded in a line-by-line, as I consider the overall problems to be grave enough to require attention and revision before getting to lesser items of clarity.

I would like to point out that the authors show a marvelous attention to their work, and they have much to contribute to the field of retraction studies, and I do honestly look forward to their future work. However, in order for the field to move ahead with accuracy and validity, we must no longer just rely on superficial number crunching, and must start including the complexities of publishing in our analyses, as difficult and labor-intensive as it might be.

We do not consider that our article presents serious problems nor that it would be useless.

It is possible that a different view on the subject, some tendency to forbearance (understandable) for the difficult life of the publishing industry, along with some difficulties in understanding the ideas presented in the article, may have led to a series of points of view that we would like to comment on below.

We would first like to thank the reviewers for their comments, some of which will allow us to improve and nuance, using objective elements, the analysis of this bumpy field represented by the ecosystem of retracted publications. Because we have based our study on data from freely accessible sources of information, we will not insist too much on commenting on this issue.

The authors stated that they used the search protocol (and therefore presumably the same dataset) as described in Toma & Padureanu, 2021, and do not indicate any process to compensate for its weaknesses. In the referenced study, the authors (same as for this article) utilized a PubMed search using only “Retracted Publication” in Publication Type. This search method is immediately insufficient, as some retracted articles are not bannered or indexed as retracted in PubMed. This issue is well-understood among scholars who search databases for retractions, and by now one would expect that these searches would strive to be more comprehensive.

A better method, if one insists on restricting the search to PubMed, would have been to use Publication Type to search for “retracted publication,” and then to search for “retraction of publication,” and to compare the output to eliminate duplications. There are even more comprehensive ways to search PubMed, especially since some articles are retitled as “Withdrawn” – Elsevier, for example, uses the term instead of “Retracted” for papers removed within a year of their publication date – but do not come in searches for either publication type. Even better would have been to use databases with more comprehensive indexing of retractions.

In an ideal world, if any effort were to be made, it would be aimed at better indexing and managing existing databases, not at generating query strategies to make up for their shortcomings.

Thank you very much for the suggestions on the search strategy. We do not consider that the use of "Retracted Publication [PT]" should be compensated in any way but, if it should be compensated, we wouldn't want to add "Retraction of publication". We consider that using a search protocol more specific to systematic reviews is not very useful in our case: data are added/updated continuously (sometimes late), incorrect indexing can be corrected, the number of retracted articles increases from month to month; the same strategy can give different results at different times regardless of its complexity. Putting extra effort into detecting problematic articles without knowing the benefit but expecting it only highlights issues that can be improved at the publisher/editor(content delivery) and database level(indexing).

The dataset analyzed is a snapshot of a particular time interval and nothing more. Even during the analysis we found, in the case of one publisher, the addition of details to the initially incomplete retraction notes. Hence the need for follow-up studies. Therefore in the case of retractions, unlike the reviewer, we prefer an approach based on simple and easily reproducible strategies, widely accessible sources of information, and several steps. The first step in this strategy is the "number crunching" stage which includes this article.

1. The authors are using the time from publication to retraction based on the notice dates and using them to indicate efficacy of oversight by publishers. However, this approach is seriously problematic. It takes no notice of when the publisher was first informed that the article was potentially compromised. Publishers who respond rapidly to information that affects years/decades old publications will inevitably show worse scores than those who are advised upon an article’s faults immediately upon its publication, but who drag their heels a few months in dealing with the problem.

Indeed, the article uses the time between publication and retraction(exposure time – ET) as one of the SDTP score components for assessing editorial/publisher performance. Data on when a publisher or editor has been informed of problems with an article, apart from being relatively rare, is not a substitute for a retraction note. Moreover, the use of such information may induce a risk of bias.

We mention in the article the need to use reporting standards for retraction notes, and one element that might be useful is, indeed, the date on which the publisher or editor was informed of problems with an article. Unfortunately, as the author of this review knows very well, information precedes investigation; the retraction note contains (or should contain) much more data than the initial information about the quality problems of an article.

Our article aims to suggest a score for measuring publication performance in the context of retracted articles that would also allow an assessment of the dynamics of the activity of correcting the scientific record and, more importantly, how publishers engage in post-publication quality control. ET is only one component of this score.

It is quite clear from the data presented in the article that a publisher/journal that emphasizes systematic back-checking will have an increasingly longer average lifespan of retracted articles, logically higher than one that does not do this type of checking. We don't see precisely where the reviewer thinks there is a problem: once the checking is done, the ET will decrease, and a publisher that takes concrete steps to correct the literature will ultimately have a better reputation. This does not mean that a higher ET is laudable, it suggests that there is a post-publication quality control but also that the peer review process has let problematic articles through and that the control of these articles has been carried out late. This is an argument for more active involvement of publishers (as potential generators of editorial policies) in post-publication control.

Second, there is little consistency in dealing with retractions between publishers, within the same publishers or even within the same journal. Under the same publisher, one journal editor may be highly responsive during their term, while the next editor may not be. Most problems with articles quite often are first addressed by contacting the authors and/or journal editors, and publishers – especially those with hundreds of journals – may not have any idea of the ensuing problem for weeks or months, if at all. Therefore, the larger publishers would be far more likely to show worse scores than publishers with few journals to manage oversight.

It is exactly this inconsistency that we highlight in the article. Differing policies, attitudes, and responsiveness does not mean that a publisher cannot/should not ask questions about the effectiveness of internal processes and resources used for post-publication quality control or the implementation of uniform measures across journals in its portfolio.

Third, the dates on retraction notices are not always representative of when an article was watermarked or otherwise indicated as retracted. Elsevier journals often overwrite the html page of the original article with the retraction notice, leaving the original article’s date of publication alone. A separate retraction notice may not be published until days, weeks or even years after the article has been retracted. Springer and Sage have done this as well, as have other publishers – though not to the same extent (yet).

Historically, The Journal of Biological Chemistry would publish a retraction notice and link it immediately to the original article, but a check of the article’s PDF would show it having been retracted days to weeks earlier. They have recently been acquired by Elsevier, so it is unknown how this trend will play out. And keep in mind, in some ways this is in itself not a bad thing – as it gives the user quicker notice that an article is unsuitable for citation, even while the notice itself is still undergoing revisions. It just makes tracking the time of publication to retraction especially difficult.

We used the same date for all articles in our study (the one listed in PubMed), thus ensuring a uniform criterion for all publishers. If this date was not in PubMed we used the date from the retraction notes on the journal website but this was for a small number of articles. How different publishers handle retraction processes or the delay with which these are published is primarily related to internal editorial procedures, and these delays are reflected in the ET. In our experience, most articles retracted by Elsevier are available online, supplemented, and not replaced by retraction notes, which we think is an excellent policy.

1. As best as can be determined, the authors are taking the notices at face value, and that has been repeatedly shown to be flawed. Many notices are written as a cooperative effort between the authors and journal, regardless of who initiated the retraction and under the looming specter of potential litigation.

Shown to be flawed by who? Indeed, in our study, we refer to the retraction notes published by the journals. The fact that they are incomplete or formulated under the threat of litigation only supports our view that publishers and editors need to make a more significant effort to correct the biomedical literature, including avoiding litigation when the retraction note clearly describes the reasons for retraction. The way the retraction note is worded should be an editorial prerogative and should primarily aim at correcting scientific literature, not at appeasing egos, careers, or financial interests.

Trying to establish who initiated a retraction process strictly by analyzing the notice language is destined to produce faulty conclusions. Looking just at PubPeer comments, questions about the data quality may be raised days/month/years before a retraction, with indications of having contacted the journal or publisher. And yet, an ensuing notice may be that the authors requested the retraction because of concerns about the data/image – where the backstory clearly shows that impetus for the retraction was prompted by a journal’s investigation of outside complaints. As an example, the recent glut of retractions of papers coming from paper mills often suggest the authors are requesting the retraction. This interpretation would be false, however, as those familiar with the backstory are aware that the driving force for many of these retractions were independent investigators contacting the journals/publishers for retraction of these manuscripts.

Once again, the author of this review does not seem to fully understand our study, apparently favouring information published on third-party websites over that the journals officially assumed. The retraction notes represent the material available to a researcher doing documentation on a particular topic. The clarity and information contained in the note is the editor's or publisher’s responsibility, reflecting their performance and concern for the integrity of the science. Interpretation of a retraction note/analyzing an article occurs in this context. Not everyone has time for further investigation or to search third-party sites for information that is, with a notable exception, the result of a selection bias.

Assigning the reason for retraction from only the text of the notice will absolutely skew results. As already stated, in many cases, journal editors and authors work together to produce the language. Thus, the notice may convey an innocuous but unquestionable cause (e.g., results not reproducible) because the fundamental reason (e.g., data/image was fabricated or falsified) is too difficult to prove to a reasonable degree. Even the use of the word “plagiarism” is triggering for authors’ reputations – and notices have been crafted to avoid any suggestion of such, with euphemisms that steer well clear of the “p” word. Furthermore, it has been well-documented that some retractions required by institutional findings of misconduct have used language in the notice indicating simple error or other innocuous reasons as the definitive cause.

We understand your point of view and the situations presented may be accurate. However, from our point of view, the only valid reference remains the retraction note published on the journal's website. The existence of wording difficulties and various other problems that may arise are more likely to do with a tendency of the reviewer to make excuses for journals reluctant to indicate precisely what the reasons for retracting the article are. There are plenty of retraction notes in which the images with problems (including whether they were plagiarized, reused, manipulated, fabricated, etc.) are indicated with great precision, there are equally plenty of notes in which the word plagiarism is used without hesitation, indicating the sources, how they were informed, what was plagiarized. No matter how many hesitant publishers/editors there are, it should not be forgotten that there are many journals/publishers who take their role seriously, acknowledge and learn from their mistakes, thus providing a real service to the scientific community.

The authors also discuss changes in the quality of notices increasing or decreasing in publishers – but without knowing the backstory. Having more words in a notice or giving one or two specific causes cannot in itself be an indicator of the quality (i.e., accuracy) of said notice.

"Knowing the backstory" is not part of our objectives, and neither is assessing the quality of the retraction notes. This is also very difficult to do due to the lack of an accepted standard format. We are trying to propose a score composed of several parameters resulting from existing (or non-existing) data in the retraction notes so that we can have a picture of retractions at publisher level. Knowing the backstory is not relevant, reading and interpreting the official retraction note is relevant.

1. The authors tend to infer that the lack of a retraction in a journal implies a degree of superiority over journals with retractions. Although they qualify it a bit ( “Are over 90% of journals without a retracted article perfect? It is a question that is quite difficult to answer at this time, but we believe that the opinion that, in reality, there are many more articles that should be retracted (Oransky et al. 2021) is justified and covered by the actual figures.”), the inference is naive. First, they have not looked at the number of corrections within these journals. Even ignoring that these corrections may be disproportionate within different journals and require responsive editorial staff, some journals have gone through what can only be called great contortions to issue corrections rather than retractions.

We believe that this is a case of reviewer confusion generated either by the insufficiently precise wording of the text or a lack of understanding of our study objectives. We are trying to point out that more than 90% of the journals in the NLM catalogue-PubMed subset have not retracted a single article. We are not trying to say that journals without retracted articles are superior to the others. As explained in the article, we referred to retraction notes, not corrections.

Second, the lack of retractions in a journal speaks nothing to the quality of the articles therein. Predatory journals generally avoid issuing retractions, even when presented with outright proof of data fabrication or plagiarism. Meanwhile, high-quality journals are likely to have more, and possibly more astute, readers, who could be more adept at spotting errors that require retraction.

Of course, the quality level of articles in a journal is not determined by the number of articles removed.

Third, smaller publishers/journals may not have the fiscal resources to deal with the issues that come with a retraction. As an example, even though there was an institutional investigation finding data fabrication, at least one journal declined to issue a retraction for an article by Joachim Boldt (who has more than 160 retractions for misconduct) after his attorneys made threats of litigation.

Threats of lawsuits are instead a failure of a publisher/journal to adapt to the realities of the publishing business or to the risk of misconduct. This is something that needs to change.

Simply put, the presence or lack of a retraction in a journal is no longer a reasonable speculation about the quality of the manuscripts or the efficiency of the editorial process.

We have not attempted to suggest this, we have only analyzed the retracted articles and their associated retraction notes. On the other hand, the way a journal/publisher handles the retraction of problematic articles still reflects, to some extent, the quality/performance of the editorial processes.

1. I am concerned that the authors appear to have made significant errors in their analysis of publishers. For example, they claim that neither PLOS nor Elsevier retracted papers in 2020 for problematic images. That assertion is demonstrably false.

This is wrong. In our dataset, there are eleven PLOS articles related to human health with the publication year 2019 and 2020. None of these have images as retraction reasons.

Regarding the 21 Elsevier articles published in 2020, there is nothing in the retraction notes to indicate that the article was retracted because of the images. In 2 retraction notes there is mention of the comments made by Dr. Bik (The Tadpole Paper Mill - Science Integrity Digest) but the text of these (retraction notes) stops at the authors' inability to provide the raw data underlying the article.

Our study is based only on the content of the retraction notes published and assumed by the journal, not on opinions/comments appearing on other sites, which, for unknown/unmentioned reasons, are not officially assumed in the retraction note. Therefore, we consider the statement in the review to be questionable at best, as the use of material other than the retraction notes has severe implications for the internal and external validity of the study and the suggestion to use such methods is, in our opinion, wrong. We would also like to draw attention to the fact that many retraction notes are explicitely mentioning the request to provide raw images and the authors' inability to provide them.

Anyway, as far as images are concerned, our article suggested that there are publishers which seem to adopt image analysis technologies faster than others. The numbers are not really relevant in this case but the trend is: it describes the publishing activity complexity better than the numbers.

Reviewer response

We appreciate the authors’ zeal in standing by their work.

In regard to the deficits in the search process, the author states, “We do not consider that the use of ‘Retracted Publication [PT]’ should be compensated in any way but, if it should be compensated, we wouldn't want to add ‘Retraction of publication’”

There is a lack of appreciation for the complexities of indexing retracted materials in an indexing site such as PubMed. To have a comprehensive search, one should not be choosing to use either “Retracted Publication [PT]” OR “Retraction of Publication [PT].” One would use both, and then filter out the duplicates, because some retractions are indexed by retraction notices, some only have “Retracted” added to the indexed title and the publication type changed to “Retracted Publication.” Use of only one or the other guarantees that the search is far less comprehensive than it should be.

The authors state, “In an ideal world, if any effort were to be made, it would be aimed at better indexing and managing existing databases, not at generating query strategies to make up for their shortcomings.”

There is at least one database (http://retractiondatabase.org) that has a far more comprehensive indexing of retractions and is publicly available for use.

In Item 3, where it is pointed out that retraction notices themselves are inaccurate and cannot be taken at face value as to the reason behind the retraction, the authors responded, “Shown to be flawed by who?” — By an article cited in the manuscript:

Fang, Ferric C.; Steen, R. Grant; Casadevall, Arturo (2012): Misconduct accounts for the majority of retracted scientific publications. In Proceedings of the National Academy of Sciences of the United States of America 109 (42), pp. 17028–17033. DOI: 10.1073/pnas.1212247109.

“To understand the reasons for retraction, we consulted reports from the Office of Research Integrity and other published resources (7, 8), in addition to the retraction announcements in scientific journals. Use of these additional sources of information resulted in the reclassification of 118 of 742 (15.9%) retractions in an earlier study (4) from error to fraud.” Followed by “These factors have contributed to the systematic underestimation of the role of misconduct and the overestimation of the role of error in retractions (3, 4), and speak to the need for uniform standards regarding retraction notices (5).”

The authors then choose to state that it is the “editorial prerogative” – and that when notices “are incomplete or formulated under the threat of litigation [it] only supports our view that publishers and editors need to make a more significant effort to correct the biomedical literature, including avoiding litigation when the retraction note clearly describes the reasons for retraction.”

Following our attempt to explain why understanding the real reason behind a retraction is important to study the publication of notices, the authors respond: “Once again, the author of this review does not seem to fully understand our study, apparently favouring information published on third-party websites over that the journals officially assumed.”

First, yes, we do understand the study. We read a lot of these. Second, the “third-party websites” we prefer include the Office of Research Integrity and the Retraction Watch blog, where background investigations into the causes of retraction notices are described. If the authors are challenging the reference to PubPeer, keep in mind that journals initiate investigations based on comments on that website, and have taken to citing the website in their notices.

Had the authors not chosen to categorize the reasons for retraction, their reasoning may have had more support – but they did, and in doing so, by just using the notice with no further review, their findings address only the notice itself, with no context.

We recommend that the manuscript be substantially revised with strong attention to the comments we made in our original review.

Peer review report

Reviewer: Adam Marcus, co-founder Retraction Watch & Alison Abritis, PhD, researcher at Retraction Watch

General comments

Major Problems: I found serious deficits in both for this article, and thus I have serious concerns as to the usefulness of this article. Therefore, I have not proceeded in a line-by-line, as I consider the overall problems to be grave enough to require attention and revision before getting to lesser items of clarity.

I would like to point out that the authors show a marvelous attention to their work, and they have much to contribute to the field of retraction studies, and I do honestly look forward to their future work. However, in order for the field to move ahead with accuracy and validity, we must no longer just rely on superficial number crunching, and must start including the complexities of publishing in our analyses, as difficult and labor-intensive as it might be.

1) The authors stated that they used the search protocol (and therefore presumably the same dataset) as described in Toma & Padureanu, 2021, and do not indicate any process to compensate for its weaknesses. In the referenced study, the authors (same as for this article) utilized a PubMed search using only “Retracted Publication” in Publication Type. This search method is immediately insufficient, as some retracted articles are not bannered or indexed as retracted in PubMed. This issue is well-understood among scholars who search databases for retractions, and by now one would expect that these searches would strive to be more comprehensive.

A better method, if one insists on restricting the search to PubMed, would have been to use Publication Type to search for “retracted publication,” and then to search for “retraction of publication,” and to compare the output to eliminate duplications. There are even more comprehensive ways to search PubMed, especially since some articles are retitled as “Withdrawn” – Elsevier, for example, uses the term instead of “Retracted” for papers removed within a year of their publication date – but do not come in searches for either publication type. Even better would have been to use databases with more comprehensive indexing of retractions.

2) The authors are using the time from publication to retraction based on the notice dates and using them to indicate efficacy of oversight by publishers. However, this approach is seriously problematic. It takes no notice of when the publisher was first informed that the article was potentially compromised. Publishers who respond rapidly to information that affects years/decades old publications will inevitably show worse scores than those who are advised upon an article’s faults immediately upon its publication, but who drag their heels a few months in dealing with the problem.

Second, there is little consistency in dealing with retractions between publishers, within the same publishers or even within the same journal. Under the same publisher, one journal editor may be highly responsive during their term, while the next editor may not be. Most problems with articles quite often are first addressed by contacting the authors and/or journal editors, and publishers – especially those with hundreds of journals – may not have any idea of the ensuing problem for weeks or months, if at all. Therefore, the larger publishers would be far more likely to show worse scores than publishers with few journals to manage oversight.

Third, the dates on retraction notices are not always representative of when an article was watermarked or otherwise indicated as retracted. Elsevier journals often overwrite the html page of the original article with the retraction notice, leaving the original article’s date of publication alone. A separate retraction notice may not be published until days, weeks or even years after the article has been retracted. Springer and Sage have done this as well, as have other publishers – though not to the same extent (yet).

Historically, The Journal of Biological Chemistry would publish a retraction notice and link it immediately to the original article, but a check of the article’s PDF would show it having been retracted days to weeks earlier. They have recently been acquired by Elsevier, so it is unknown how this trend will play out. And keep in mind, in some ways this is in itself not a bad thing – as it gives the user quicker notice that an article is unsuitable for citation, even while the notice itself is still undergoing revisions. It just makes tracking the time of publication to retraction especially difficult.

3) As best as can be determined, the authors are taking the notices at face value, and that has been repeatedly shown to be flawed. Many notices are written as a cooperative effort between the authors and journal, regardless of who initiated the retraction and under the looming specter of potential litigation.

Trying to establish who initiated a retraction process strictly by analyzing the notice language is destined to produce faulty conclusions. Looking just at PubPeer comments, questions about the data quality may be raised days/month/years before a retraction, with indications of having contacted the journal or publisher. And yet, an ensuing notice may be that the authors requested the retraction because of concerns about the data/image – where the backstory clearly shows that impetus for the retraction was prompted by a journal’s investigation of outside complaints. As an example, the recent glut of retractions of papers coming from paper mills often suggest the authors are requesting the retraction. This interpretation would be false, however, as those familiar with the backstory are aware that the driving force for many of these retractions were independent investigators contacting the journals/publishers for retraction of these manuscripts.

Assigning the reason for retraction from only the text of the notice will absolutely skew results. As already stated, in many cases, journal editors and authors work together to produce the language. Thus, the notice may convey an innocuous but unquestionable cause (e.g., results not reproducible) because the fundamental reason (e.g., data/image was fabricated or falsified) is too difficult to prove to a reasonable degree. Even the use of the word “plagiarism” is triggering for authors’ reputations – and notices have been crafted to avoid any suggestion of such, with euphemisms that steer well clear of the “p” word. Furthermore, it has been well-documented that some retractions required by institutional findings of misconduct have used language in the notice indicating simple error or other innocuous reasons as the definitive cause.

The authors also discuss changes in the quality of notices increasing or decreasing in publishers – but without knowing the backstory. Having more words in a notice or giving one or two specific causes cannot in itself be an indicator of the quality (i.e., accuracy) of said notice.

4) The authors tend to infer that the lack of a retraction in a journal implies a degree of superiority over journals with retractions. Although they qualify it a bit ( “Are over 90% of journals without a retracted article perfect? It is a question that is quite difficult to answer at this time, but we believe that the opinion that, in reality, there are many more articles that should be retracted (Oransky et al. 2021) is justified and covered by the actual figures.”), the inference is naive. First, they have not looked at the number of corrections within these journals. Even ignoring that these corrections may be disproportionate within different journals and require responsive editorial staff, some journals have gone through what can only be called great contortions to issue corrections rather than retractions.

Second, the lack of retractions in a journal speaks nothing to the quality of the articles therein. Predatory journals generally avoid issuing retractions, even when presented with outright proof of data fabrication or plagiarism. Meanwhile, high-quality journals are likely to have more, and possibly more astute, readers, who could be more adept at spotting errors that require retraction.

Third, smaller publishers/journals may not have the fiscal resources to deal with the issues that come with a retraction. As an example, even though there was an institutional investigation finding data fabrication, at least one journal declined to issue a retraction for an article by Joachim Boldt (who has more than 160 retractions for misconduct) after his attorneys made threats of litigation.

Simply put, the presence or lack of a retraction in a journal is no longer a reasonable speculation about the quality of the manuscripts or the efficiency of the editorial process.

5) I am concerned that the authors appear to have made significant errors in their analysis of publishers. For example, they claim that neither PLOS nor Elsevier retracted papers in 2020 for problematic images. That assertion is demonstrably false.

Decision

Requires revisions: The manuscript contains objective errors or fundamental flaws that must be addressed and/or major revisions are suggested.

Peer review report

Reviewer: Adam Marcus, co-founder Retraction Watch & Alison Abritis, PhD, researcher at Retraction Watch

General comments

Major Problems: I found serious deficits in both for this article, and thus I have serious concerns as to the usefulness of this article. Therefore, I have not proceeded in a line-by-line, as I consider the overall problems to be grave enough to require attention and revision before getting to lesser items of clarity.

I would like to point out that the authors show a marvelous attention to their work, and they have much to contribute to the field of retraction studies, and I do honestly look forward to their future work. However, in order for the field to move ahead with accuracy and validity, we must no longer just rely on superficial number crunching, and must start including the complexities of publishing in our analyses, as difficult and labor-intensive as it might be.

1) The authors stated that they used the search protocol (and therefore presumably the same dataset) as described in Toma & Padureanu, 2021, and do not indicate any process to compensate for its weaknesses. In the referenced study, the authors (same as for this article) utilized a PubMed search using only “Retracted Publication” in Publication Type. This search method is immediately insufficient, as some retracted articles are not bannered or indexed as retracted in PubMed. This issue is well-understood among scholars who search databases for retractions, and by now one would expect that these searches would strive to be more comprehensive.

A better method, if one insists on restricting the search to PubMed, would have been to use Publication Type to search for “retracted publication,” and then to search for “retraction of publication,” and to compare the output to eliminate duplications. There are even more comprehensive ways to search PubMed, especially since some articles are retitled as “Withdrawn” – Elsevier, for example, uses the term instead of “Retracted” for papers removed within a year of their publication date – but do not come in searches for either publication type. Even better would have been to use databases with more comprehensive indexing of retractions.

2) The authors are using the time from publication to retraction based on the notice dates and using them to indicate efficacy of oversight by publishers. However, this approach is seriously problematic. It takes no notice of when the publisher was first informed that the article was potentially compromised. Publishers who respond rapidly to information that affects years/decades old publications will inevitably show worse scores than those who are advised upon an article’s faults immediately upon its publication, but who drag their heels a few months in dealing with the problem.

Second, there is little consistency in dealing with retractions between publishers, within the same publishers or even within the same journal. Under the same publisher, one journal editor may be highly responsive during their term, while the next editor may not be. Most problems with articles quite often are first addressed by contacting the authors and/or journal editors, and publishers – especially those with hundreds of journals – may not have any idea of the ensuing problem for weeks or months, if at all. Therefore, the larger publishers would be far more likely to show worse scores than publishers with few journals to manage oversight.

Third, the dates on retraction notices are not always representative of when an article was watermarked or otherwise indicated as retracted. Elsevier journals often overwrite the html page of the original article with the retraction notice, leaving the original article’s date of publication alone. A separate retraction notice may not be published until days, weeks or even years after the article has been retracted. Springer and Sage have done this as well, as have other publishers – though not to the same extent (yet).

Historically, The Journal of Biological Chemistry would publish a retraction notice and link it immediately to the original article, but a check of the article’s PDF would show it having been retracted days to weeks earlier. They have recently been acquired by Elsevier, so it is unknown how this trend will play out. And keep in mind, in some ways this is in itself not a bad thing – as it gives the user quicker notice that an article is unsuitable for citation, even while the notice itself is still undergoing revisions. It just makes tracking the time of publication to retraction especially difficult.

3) As best as can be determined, the authors are taking the notices at face value, and that has been repeatedly shown to be flawed. Many notices are written as a cooperative effort between the authors and journal, regardless of who initiated the retraction and under the looming specter of potential litigation.

Trying to establish who initiated a retraction process strictly by analyzing the notice language is destined to produce faulty conclusions. Looking just at PubPeer comments, questions about the data quality may be raised days/month/years before a retraction, with indications of having contacted the journal or publisher. And yet, an ensuing notice may be that the authors requested the retraction because of concerns about the data/image – where the backstory clearly shows that impetus for the retraction was prompted by a journal’s investigation of outside complaints. As an example, the recent glut of retractions of papers coming from paper mills often suggest the authors are requesting the retraction. This interpretation would be false, however, as those familiar with the backstory are aware that the driving force for many of these retractions were independent investigators contacting the journals/publishers for retraction of these manuscripts.

Assigning the reason for retraction from only the text of the notice will absolutely skew results. As already stated, in many cases, journal editors and authors work together to produce the language. Thus, the notice may convey an innocuous but unquestionable cause (e.g., results not reproducible) because the fundamental reason (e.g., data/image was fabricated or falsified) is too difficult to prove to a reasonable degree. Even the use of the word “plagiarism” is triggering for authors’ reputations – and notices have been crafted to avoid any suggestion of such, with euphemisms that steer well clear of the “p” word. Furthermore, it has been well-documented that some retractions required by institutional findings of misconduct have used language in the notice indicating simple error or other innocuous reasons as the definitive cause.

The authors also discuss changes in the quality of notices increasing or decreasing in publishers – but without knowing the backstory. Having more words in a notice or giving one or two specific causes cannot in itself be an indicator of the quality (i.e., accuracy) of said notice.

4) The authors tend to infer that the lack of a retraction in a journal implies a degree of superiority over journals with retractions. Although they qualify it a bit ( “Are over 90% of journals without a retracted article perfect? It is a question that is quite difficult to answer at this time, but we believe that the opinion that, in reality, there are many more articles that should be retracted (Oransky et al. 2021) is justified and covered by the actual figures.”), the inference is naive. First, they have not looked at the number of corrections within these journals. Even ignoring that these corrections may be disproportionate within different journals and require responsive editorial staff, some journals have gone through what can only be called great contortions to issue corrections rather than retractions.

Second, the lack of retractions in a journal speaks nothing to the quality of the articles therein. Predatory journals generally avoid issuing retractions, even when presented with outright proof of data fabrication or plagiarism. Meanwhile, high-quality journals are likely to have more, and possibly more astute, readers, who could be more adept at spotting errors that require retraction.

Third, smaller publishers/journals may not have the fiscal resources to deal with the issues that come with a retraction. As an example, even though there was an institutional investigation finding data fabrication, at least one journal declined to issue a retraction for an article by Joachim Boldt (who has more than 160 retractions for misconduct) after his attorneys made threats of litigation.

Simply put, the presence or lack of a retraction in a journal is no longer a reasonable speculation about the quality of the manuscripts or the efficiency of the editorial process.

5) I am concerned that the authors appear to have made significant errors in their analysis of publishers. For example, they claim that neither PLOS nor Elsevier retracted papers in 2020 for problematic images. That assertion is demonstrably false.

Decision

Requires revisions: The manuscript contains objective errors or fundamental flaws that must be addressed and/or major revisions are suggested.

Peer review report

Reviewer: Yin Qianlan Institution: Navy Medical University email: yinqianlan@smmu.edu.cn

Section 1 – Serious concerns

• Do you have any serious concerns about the manuscript such as fraud, plagiarism, unethical or unsafe practices? No
• Have authors’ provided the necessary ethics approval (from authors’ institution or an ethics committee)? Yes

Section 2 – Language quality

• How would you rate the English language quality? High quality

Section 3 – validity and reproducibility

• Is the reasons for conducting the study and its objectives clearly explained? No

• Is the study design appropriate? Yes

• Are sufficient details provided so that the method can be replicated? Yes

• Are datasets available so that others could use them? not applicable

Section 4 – Suggestions

• Based on your answers in section 3 how could the author improve the protocol?

As an important part of a review is the declaration of the purpose, the introduction should be the core of the article. However, after reading the beginning of the paper, I could realize the seriousness of COVID-19, but I cannot see the key point of the research. There is a lot of data to emphasize the worse results, but I don’t know how this data contributed to the relationship between the major topic of Post-acute sequelae of COVID-19 and adverse psychiatric outcomes, for example, the introduction about the effect of therapies. Hence, more organized structure for the introduction of could be more concise and easier for readers.

Section 5 – Decision

Requires revisions: The manuscript contains objective errors or fundamental flaws that must be addressed and/or major revisions are suggested.

Peer review report

Reviewer: Yin Qianlan Institution: Navy Medical University email: yinqianlan@smmu.edu.cn

Section 1 – Serious concerns

• Do you have any serious concerns about the manuscript such as fraud, plagiarism, unethical or unsafe practices? No
• Have authors’ provided the necessary ethics approval (from authors’ institution or an ethics committee)? Yes

Section 2 – Language quality

• How would you rate the English language quality? High quality

Section 3 – validity and reproducibility

• Is the reasons for conducting the study and its objectives clearly explained? No

• Is the study design appropriate? Yes

• Are sufficient details provided so that the method can be replicated? Yes

• Are datasets available so that others could use them? not applicable

Section 4 – Suggestions

• Based on your answers in section 3 how could the author improve the protocol?

As an important part of a review is the declaration of the purpose, the introduction should be the core of the article. However, after reading the beginning of the paper, I could realize the seriousness of COVID-19, but I cannot see the key point of the research. There is a lot of data to emphasize the worse results, but I don’t know how this data contributed to the relationship between the major topic of Post-acute sequelae of COVID-19 and adverse psychiatric outcomes, for example, the introduction about the effect of therapies. Hence, more organized structure for the introduction of could be more concise and easier for readers.

Section 5 – Decision

Requires revisions: The manuscript contains objective errors or fundamental flaws that must be addressed and/or major revisions are suggested.

Peer review report

Reviewer: Argentina Felisbela Muianga,<br /> Institution: Instituto Nacional de Saúde-Maputo-Mozambique email: valiosa.muianga@gmail.com, Argentina.muianga@ins.gov.mz

General comments

The manuscript addresses a relevant subject that is still, in a way, little prioritized, therefore little developed and therefore raises the need for further in-depth studies to develop a good RDT that can detect infection in the most important phase of infection and ensure a better real-time response.

Section 1 – Serious concerns

• Do you have any serious concerns about the manuscript such as fraud, plagiarism, unethical or unsafe practices? No
• Have authors’ provided the necessary ethics approval (from authors’ institution or an ethics committee)? No, I did not see any approach on ethical aspects of the study. This should updated.

Section 2 – Language quality

• How would you rate the English language quality? High quality

Section 3 – validity and reproducibility

• Is the rationale for, and objectives of, the scoping review clearly stated? Yes

• Are the methods documented and analysis provided so that replication can be conducted? Yes

• Is the statistical analysis and its interpretation appropriate? Yes

• Are quality of the figures and tables satisfactory? Yes

• Are the conclusions adequately supported by the results presented in the review? Yes

• Are there any fundamental flaws or errors that make the scoping review invalid? No

Section 4 – Suggestions

• Based on your answers in section 3 how could the author improve the study?

I did not see information regarding to the type of sample used on the different platforms, since for the RDT the type of samples influences accessibility.

How was the prototype evaluated, in terms of type of samples and environmental conditions?

• Do you have any other feedback or comments for the Author?

No

Section 5 – Decision

Verified manuscript: The content is scientifically sound, only minor amendments (if any) are suggested.

Peer review report

Reviewer: Argentina Felisbela Muianga,<br /> Institution: Instituto Nacional de Saúde-Maputo-Mozambique email: valiosa.muianga@gmail.com, Argentina.muianga@ins.gov.mz

General comments

The manuscript addresses a relevant subject that is still, in a way, little prioritized, therefore little developed and therefore raises the need for further in-depth studies to develop a good RDT that can detect infection in the most important phase of infection and ensure a better real-time response.

Section 1 – Serious concerns

• Do you have any serious concerns about the manuscript such as fraud, plagiarism, unethical or unsafe practices? No
• Have authors’ provided the necessary ethics approval (from authors’ institution or an ethics committee)? No, I did not see any approach on ethical aspects of the study. This should updated.

Section 2 – Language quality

• How would you rate the English language quality? High quality

Section 3 – validity and reproducibility

• Is the rationale for, and objectives of, the scoping review clearly stated? Yes

• Are the methods documented and analysis provided so that replication can be conducted? Yes

• Is the statistical analysis and its interpretation appropriate? Yes

• Are quality of the figures and tables satisfactory? Yes

• Are the conclusions adequately supported by the results presented in the review? Yes

• Are there any fundamental flaws or errors that make the scoping review invalid? No

Section 4 – Suggestions

• Based on your answers in section 3 how could the author improve the study?

I did not see information regarding to the type of sample used on the different platforms, since for the RDT the type of samples influences accessibility.

How was the prototype evaluated, in terms of type of samples and environmental conditions?

• Do you have any other feedback or comments for the Author?

No

Section 5 – Decision

Verified manuscript: The content is scientifically sound, only minor amendments (if any) are suggested.

Peer review report

Reviewer: Sadia Ali Pereira,<br /> Institution: Instituto Nacional de Saúde, Mozambique email: Sadia.abdul.ali@gmail.com / sadia.pereira@ins.gov.mz

Section 1 – Serious concerns

• Do you have any serious concerns about the manuscript such as fraud, plagiarism, unethical or unsafe practices? No
• Have authors’ provided the necessary ethics approval (from authors’ institution or an ethics committee)? not applicable

Section 2 – Language quality

• How would you rate the English language quality? High quality

Section 3 – validity and reproducibility

• Is the rationale for, and objectives of, the scoping review clearly stated? Yes

• Are the methods documented and analysis provided so that replication can be conducted? No

• Is the statistical analysis and its interpretation appropriate? Yes

• Are quality of the figures and tables satisfactory? No

It is difficult to read figure 4, on page 29 there is no legend of the figure, in figure 1 the pallet colours could have greater variation.

• Are the conclusions adequately supported by the results presented in the review? No

• Are there any fundamental flaws or errors that make the scoping review invalid? Not any major errors detected

Section 4 – Suggestions

• Based on your answers in section 3 how could the author improve the study?

I would suggest improving the methodology and the criteria for articles selection/inclusion

• Do you have any other feedback or comments for the Author?

Will be helpful if the authors could better discussion the risk bias and applicability. Why select studies with patients with signs for CHIKV? What about asymptomatic? Not proper discussed the possibility of cross reactivity between other alphaviruses.

Section 5 – Decision

Verified with reservations: The content is scientifically sound but has shortcomings that could be improved by further studies and/or minor revisions.

Peer review report

Reviewer: Sadia Ali Pereira,<br /> Institution: Instituto Nacional de Saúde, Mozambique email: Sadia.abdul.ali@gmail.com / sadia.pereira@ins.gov.mz

Section 1 – Serious concerns

• Do you have any serious concerns about the manuscript such as fraud, plagiarism, unethical or unsafe practices? No
• Have authors’ provided the necessary ethics approval (from authors’ institution or an ethics committee)? not applicable

Section 2 – Language quality

• How would you rate the English language quality? High quality

Section 3 – validity and reproducibility

• Is the rationale for, and objectives of, the scoping review clearly stated? Yes

• Are the methods documented and analysis provided so that replication can be conducted? No

• Is the statistical analysis and its interpretation appropriate? Yes

• Are quality of the figures and tables satisfactory? No

It is difficult to read figure 4, on page 29 there is no legend of the figure, in figure 1 the pallet colours could have greater variation.

• Are the conclusions adequately supported by the results presented in the review? No

• Are there any fundamental flaws or errors that make the scoping review invalid? Not any major errors detected

Section 4 – Suggestions

• Based on your answers in section 3 how could the author improve the study?

I would suggest improving the methodology and the criteria for articles selection/inclusion

• Do you have any other feedback or comments for the Author?

Will be helpful if the authors could better discussion the risk bias and applicability. Why select studies with patients with signs for CHIKV? What about asymptomatic? Not proper discussed the possibility of cross reactivity between other alphaviruses.

Section 5 – Decision

Verified with reservations: The content is scientifically sound but has shortcomings that could be improved by further studies and/or minor revisions.

Peer review report

Reviewer: Veena Nair Institution: UW Madison email: vnair@uwhealth.org

Section 1 – Serious concerns

• Do you have any serious concerns about the manuscript such as fraud, plagiarism, unethical or unsafe practices? No
• Have authors’ provided the necessary ethics approval (from authors’ institution or an ethics committee)? Yes

Section 2 – Language quality

• How would you rate the English language quality? High quality

Section 3 – validity and reproducibility

• Does the work cite relevant and sufficient literature? Yes
• Is the study design appropriate and are the methods used valid? Yes
• Are the methods documented and analysis provided so that the study can be replicated?

See comments below at the end of Section 3; some more methodological details could be provided.

• Is the statistical analysis and its interpretation appropriate? Yes
• Is quality of the figures and tables satisfactory? No

In the demographics table, could you please add number of left versus right stroke patients, distribution of stroke location too. Thank you.

In figure 1 I don’t see a L for left; perhaps missing in the pdf I have.

• Are there any objective errors or fundamental flaws that make the research invalid? How could the author improve the study?

It would be helpful to see the demographic characteristics of the normative database. Do the subjects in that database have a similar distribution of age and sex.

Scanner variance is a known confound that studies must deal with; should we be concerned that the normative database was collected on a 7T whereas the current study’s data were all collected on a 3T? are there ways to do some kind of harmonization? Likewise, the multimodal Glasser atlas is based off of 21-35 old young healthy adults; but the variability in the current study population is large.

In the Disconnectome map, what does each ‘map’ represent? Is it a structural connectivity map? What do the connections represent (fiber cross-sectional area, number of fibers etc.?). More details would be helpful.

Based on Figure 4, prediction accuracy on the test set is low, at 22%; can this be improved by improvements in normalization? Perhaps by using a more age-appropriate template than the MNI152? [see for e.g., Mayo clinic template for older adults].

Section 4 – Suggestions

• Do you have any other feedback or comments for the Author?

This is a very interesting study and the research question is an important one and very relevant to the field. Thank you.

Section 5 – Decision

Verified with reservations:The content is scientifically sound but has shortcomings that could be improved by further studies and/or minor revisions.

Peer review report

Reviewer: Veena Nair Institution: UW Madison email: vnair@uwhealth.org

Section 1 – Serious concerns

• Do you have any serious concerns about the manuscript such as fraud, plagiarism, unethical or unsafe practices? No
• Have authors’ provided the necessary ethics approval (from authors’ institution or an ethics committee)? Yes

Section 2 – Language quality

• How would you rate the English language quality? High quality

Section 3 – validity and reproducibility

• Does the work cite relevant and sufficient literature? Yes
• Is the study design appropriate and are the methods used valid? Yes
• Are the methods documented and analysis provided so that the study can be replicated?

See comments below at the end of Section 3; some more methodological details could be provided.

• Is the statistical analysis and its interpretation appropriate? Yes
• Is quality of the figures and tables satisfactory? No

In the demographics table, could you please add number of left versus right stroke patients, distribution of stroke location too. Thank you.

In figure 1 I don’t see a L for left; perhaps missing in the pdf I have.

• Are there any objective errors or fundamental flaws that make the research invalid? How could the author improve the study?

It would be helpful to see the demographic characteristics of the normative database. Do the subjects in that database have a similar distribution of age and sex.

Scanner variance is a known confound that studies must deal with; should we be concerned that the normative database was collected on a 7T whereas the current study’s data were all collected on a 3T? are there ways to do some kind of harmonization? Likewise, the multimodal Glasser atlas is based off of 21-35 old young healthy adults; but the variability in the current study population is large.

In the Disconnectome map, what does each ‘map’ represent? Is it a structural connectivity map? What do the connections represent (fiber cross-sectional area, number of fibers etc.?). More details would be helpful.

Based on Figure 4, prediction accuracy on the test set is low, at 22%; can this be improved by improvements in normalization? Perhaps by using a more age-appropriate template than the MNI152? [see for e.g., Mayo clinic template for older adults].

Section 4 – Suggestions

• Do you have any other feedback or comments for the Author?

This is a very interesting study and the research question is an important one and very relevant to the field. Thank you.

Section 5 – Decision

Verified with reservations:The content is scientifically sound but has shortcomings that could be improved by further studies and/or minor revisions.

Discussion, revision and decision

Requires revisions: The manuscript contains objective errors or fundamental flaws that must be addressed and/or major revisions are suggested.

Responses to the reviewers

Reviewer Charlotte Martial

1-The work does not cite relevant and sufficient literature. For instance, the authors do not cite a recent publication which aimed to induce OBE using hypnosis (Martial et al., Scientific Reports, 2019).

Reply: We re-read the procedure of this paper we were aware of, but we didn’t find any reference related to an OBE induced by hypnotic suggestions. Here follows an excerpt of the procedure as reported in the cited paper: “The hypnotic instruction included a 5-min induction procedure with eye fixation (ultimately closing the eyes) and progressive muscle relaxation. Permissive and indirect suggestions were used to develop and deepen the hypnotic state. The participant was then invited to re-experience their NDE memory (20 min) and their pleasant autobiographical memory (20 min) in a counterbalanced order, ..”

Thanks for the reply. I do not agree, though. As described in the paper, Martial and colleagues (2019) focused on two prototypical features of the NDE during the hypnosis session, that is, feeling of peacefulness and out-of-body experience. Consequently, I think it is highly relevant for the present paper.

2-Another example: The authors could have at least briefly discussed Parnia’s AWARE prospective study reporting OBE in NDE experiencers (Parnia et al., 2014).

Reply: Thank you for this suggestion, now added in the introduction

3-Besides, they sometimes cite some inappropriate references; I would like to invite the authors to cite rigorous and serious articles in the field. For example, they used Jourdan (2011) reference to draw a parallel with NDEs, however, Jourdan (2011) is not a reference article in the field…

Reply: Jourdan’s (2011) paper “Near Death Experiences and the 5th Dimensional Spatio-Temporal Perspective. Journal of Cosmology, 14, 4743-4762 deals with NDE. It is not clear why it cannot be considered a reference article in the field.

Dr Jourdan and most of his articles are scientifically controversial. They are many other references more rigorous that you could cite in order to be scientifically stronger.

4- Importantly, it is worth mentioning that, so far, no rigorous empirical scientific study has shown evidence of veridical perceptions during OBE; according to me, it is not clear in the manuscript.

Reply: On pag. 2 we added “and verify by a third-person perspective what they declare to perceive in this particular state of consciousness”.

I thank the authors. However, this is still not clearer in the manuscript. I suggest that the authors clearly state that so far, no rigorous empirical scientific study has shown evidence of veridical perceptions during OBE. This is highly important for a manuscript aiming to study OBE in laboratory settings.

5-Some limitations of the study are not mentioned in the discussion section

Reply: On pag. 14 there is a whole paragraph “Study limitations”.

Well, let me clarify: some of the limitations of the study are not discussed in the manuscript.

6- Some details of the study are missing. Notably, the description of the Table 6 is incomplete: what do the bold numbers mean

Reply: we revised the presentation of the data in Table 6 and clarified why some data were presented in bold.

7- Another example: how did they recruit the participants –who are co-authors of the paper? Are they researchers?

Reply: We have revised the section related to the participants’ selection

8- page13: what do they mean by “general consensus”? This is not clear to me

Reply: We revised the text in “The answers of all five participants to the eleven questions presented in Table 4 and 5, were quite similar in content”

9-The conclusions they draw in the discussion are not based on the present findings; they extrapolate

Reply: Given in the paper there is not a “Conclusions” paragraph, to what “conclusions” do you refer to?

I mean the interpretation of their results … As an example, based on what do you conclude that “…in this state of consciousness they can act without the limitations that the physical body imposes on motion and perception especially with regard to vision”. It is not clear that you here mean that their subjective experience is related to their imaginary. Please reformulate. Another example: what do you mean by “a three-dimensional universe”? Another example: Based on which data can you conclude that what they experienced is “very similar to or the same as that described in NDEs”.

Reviewer Aminata Bicego Thank you for your accurate review and suggestions.

Abstract :

L13 : what do the authors mean by “hypnotic induction”, is it hypnotic experience ? During a hypnosis session, the therapist starts by an induction and then makes some suggestions according to the goal that has been decided. It is not clear to what the “hypnosis induction” refers to.

Reply : We changed « hypnotic induction » as « hypnotic experience », as suggested.

L16 : “under hypnosis” should be changed as it suggests that the individual who is in hypnosis is passive. I would suggest “in hypnosis”.

Reply : changed as suggested

Introduction: L 54-56 : there has been some recent literature on hypnosis and OBEs or NDEs. It would be interesting to add newer literature on that topic.

Reply : we searched with google scholar if there were new studies about hypnosis and OBEs, but we didn’t find any. The were two studies related to hypnosis and NDEs, but these experiences are not related to our study (see also our response to reviewer Charlotte Martial).

L75 : if the study’s aim is to confirm Tressoldi and Del Prete (2007), then this study should be explicitly explained in detail. That way the reader understands better the present study.

Reply : We added some further details related to Tressoldi and Del Prete (2007) study, as suggested.

L76 : could the authors specify the suggestion used.

Reply : the suggestions used in the study are presented in the Supplementary Materials

L78 : the induction is not the only and principal part of the hypnotic sessions that impacts an individual perceptions but rather the suggestion that is used. This paragraph could be clearer in terms of methodology

Reply : the suggestion to induce an OBE in Tressoldi and Del Prete (2007) study, are too long to be added in the text, but are very similar to that used in the present study.

L79 to L95 : this should be in methods.

Reply : moved in the Methods under the paragraph « Study aims »

Materials and Methods : L110 : can the authors define and detail “clinical level of medical or psychiatric disease”.

Reply : we added « compatible with the clinical criteria defined in the main international clinical guidelines, such as DSM V or ICD-10,

L111 : took, should be written “take”. Throughout the manuscript there are language mistakes that have to be checked.

Reply : we checked the syntax as suggested.

L111 : can the authors be more specific on the “personal experience with hypnosis” ? What do you mean by experience ?

Reply : In Table 1, we changed the experience value adding the number of previous hypnotic sessions.

Procedure : in general the procedure is not very clear. Some results appear in the section when they should be in the result section

Reply : we moved the sentence « In total, twenty-eight sessions were necessary to complete the study. Eighteen sessions were carried out by phone with the participants at home” in the results section.

L125 : Can the authors explain why some participants had one or two sessions ?

Reply : In the Procedure we wrote "We placed the images in two different rooms as long as it was possible to require the identification of two different images in a single session. In only two cases, the participants had sufficient time and concentration to identify two images in a single session reducing their overall sessions to four instead of six"

L127 : Can the supplementary Material be numbered. L129 to 132 : Have all participants been seen in real life? Were all of them called? If not, was it always the same people in person or on the phone? This part should be more specific.

Reply : In the Results section we clarified that « In total, twenty-eight sessions were necessary to complete the study. Eighteen sessions were carried out by phone with the participants at home »

L130 : hypnotized should not be used. Same comment as comment L16.

Reply : corrected as suggested

L140 : was the order of the picture position randomized ? If so it should be mentioned here.

Reply : we wrote « …the six target images and randomized their order of presentation, the room where each one was placed and their orientation, either facing up or upside down »

L164: Who did the authors know that the participants where in an OBE state ? What were the criteria ? Especially on the phone ?

Reply: On pag. 5, we clarified that the hypnotist referred to the participants phenomenological account in response to the suggestions described in the Supplementary Material.

L174 : did the authors create the questions ? Do they come from a questionnaire ? This should be specified.

Reply : we clarified « devised by the authors of this study »

L188 : this should be in results Reply : moved in the Data analysis paragraph as suggested.

L191 : “... and give suggestion...” : It is confusing to use that word, comments, might be more appropriate.

Reply : corrected as suggested.

L198: what are the eleven questions ?

Reply : we corrected and added « all participants were emailed the same six questions related to their perceptual and cognitive experience when in OBE..

L208 : all the questionnaires used should also be described in material. The reader has no information on the minimal phenomenal selfhood, nor has he information about the “characteristics of spatial and temporal perception reported in NDEs”.

Reply: all questions were original and devised by the authors. None questionnaire of other authors was used.

L221 : how did the 3 decoys were selected ?

Reply : we added « low arousal pictures »

L225 : I only see 2 authors, not three.

Reply : we corrected as « co-author LP and another independent judge affiliate with the EcvanLab… »

L234 : a section with the statistical analyses should be written before the results.

Reply : we added a section « Statistical method »

L241 : 52.4 % is not metionned in the table, this is confusing.

Reply : Sorry, we missed the overall results in the last line, now added

L245 : this should be in the statistical analyses part, it is not a result per se.

L260 : the table 3 should be more specific: define ES + formula, CI, BF, H1, H0. A table should be able to be read by itself.

Reply : In the Table 3 presentation and title, we clarified all the statistics

L277 : This part merits some clarifications : Did the approval from the Ethical committee approved this part ? If so, did the participants signed an informed consent ?

Reply: The Ethical approval included the informed consent to be signed by participants

What should they refer to when they answered the questions ? Did the ones that had an OBE had to refere to that episode ? And those who did not life an OBE ? What was the experience of reference ?

Reply: Participants induced in OBE, should refer to their phenomenological experience in this state included the period when they were requested to identify the targets.

It there is no reference for the controls without an OBE experience then is seems logical that they do not answer like the others.

Reply : controls participants were selected among those who have only read about OBEs

L304 : the % is not correct, is should be 46.6%

Reply : Corrected

L338 : With what material was the comparison made ? With the material from Jourdan (2011) or the participants that were contacted after the study ? This part should be clearer If the comparison is made with Jourdan, then a table with the similarity and differences could be added.

Reply : We changed the title of Table 6 in order to clarify its data.

L382 : “but his aim was not to confirm his knowledge, but to compare it with the participants’ experience” : this was not mentionned before. It is hard to understand as we have no information in the hypnotist knowledge or the comparison that is mentionned. Could the authors clarify ?

Reply : The hypnotist knew very well the OBE phenomenology (now added in his description on page 4. However, the questions for participants were formulated in order not to confirm the hypnotist’s previous knowledge.

L387 : Another limitation it the response expectancy during hypnosis. There is a large literature on the subject this should be discused in the limites. More so because all the subject had good knowledge about OBEs

Reply : In the paragraph « Reliability of participants’ reports », we discuss precisely these issues.

L412 : Is it acceptable to disclose the participants identity ?

Reply : we deleted their names

Page 5 : The authors mention in a foot note that some other informatio will be avaiable in a future publication but the publication has since been published. Furthermore, that publication is cited in the bibliography this is confusing.

Reply : we corrected this discrepancy

Table S1 : it is hard for the reader to understand to what the table refers to. GAPED has to be explained.

Reply ; The GAPED acronym is described in the Materials paragraph on pag. 6

Peer review report

Reviewer: : Aminata Bicego Institution: University of Liège email: abicego@uliege.be

Section 1 – Serious concerns

• Do you have any serious concerns about the manuscript such as fraud, plagiarism, unethical or unsafe practices? No
• Have authors’ provided the necessary ethics approval (from authors’ institution or an ethics committee)? Yes

Section 2 – Language quality

• How would you rate the English language quality? - Low to medium quality, but I understand the content

Section 3 – validity and reproducibility

• Does the work cite relevant and sufficient literature? No
• Is the study design appropriate and are the methods used valid? Yes
• Are the methods documented and analysis provided so that the study can be replicated? no
• Is the source data that underlies the result available so that the study can be replicated? Yes
• Is the statistical analysis and its interpretation appropriate? Yes
• Is quality of the figures and tables satisfactory? No
• Are the conclusions adequately supported by the results? Yes
• Are there any objective errors or fundamental flaws that make the research invalid? No*

Section 4 – Suggestions

• In your opinion how could the author improve the study?

Abstract :

L13 : what do the authors mean by “hypnotic induction”, is it hypnotic experience ? During a hypnosis session, the therapist starts by an induction and then makes some suggestions according to the goal that has been decided. It is not clear to what the “hypnosis induction” refers to.

L16 : “under hypnosis” should be changed as it suggests that the individual who is in hypnosis is passive. I would suggest “in hypnosis”.

Introduction:

L 54-56 : there has been some recent literature on hypnosis and OBEs or NDEs. It would be interesting to add newer literature on that topic.

L75 : if the study’s aim is to confirm Tressoldi and Del Prete (2007), then this study should be explicitly explained in detail. That way the reader understands better the present study.

L76 : could the authors specify the suggestion used.

L78 : the induction is not the only and principal part of the hypnotic sessions that impacts an individual perceptions but rather the suggestion that is used. This paragraph could be clearer in terms of methodology

L79 to L95 : this should be in methods.

Materials and Methods :

L110 : can the authors define and detail “clinical level of medical or psychiatric disease”.

L111 : took, should be written “take”. Throughout the manuscript there are language mistakes that have to be checked.

L111 : can the authors be more specific on the “personal experience with hypnosis” ? What do you mean by experience ?

Procedure :

in general the procedure is not very clear. Some results appear in the section when they should be in the result section

L125 : Can the authors explain why some participants had one or two sessions ?

L127 : Can the supplementary Material be numbered.

L129 to 132 : Have all participants been seen in real life? Were all of them called? If not, was it always the same people in person or on the phone? This part should be more specific.

L130 : hypnotized should not be used. Same comment as comment L16.

L140 : was the order of the picture position randomized ? If so it should be mentioned here.

L164: Who did the authors know that the participants where in an OBE state ? What were the criteria ? Especially on the phone ?

L174 : did the authors create the questions ? Do they come from a questionnaire ? This should be specified.

L188 : this should be in results

L191 : “... and give suggestion...” : It is confusing to use that word, comments, mught be more appropriate.

L198: what are the eleven questions ?

L208 : all the questionnaires used should also be described in material. The reader has no information on the minimal phenomenal selfhood, nor has he information about the “characteristics of spatial and temporal perception reported in NDEs”.

L221 : how did the 3 decoys were selected ?

L225 : I only see 2 authors, not three.

L234 : a section with the statistical analyses should be written before the results.

L241 : 52.4 % is not metionned in the table, this is confusing.

L245 : this should be in the statistical analyses part, it is not a result per se.

L260 : the table 3 should be more specific: define ES + formula, CI, BF, H1, H0. A table should be able to be read by itself.

L277 : This part merits some clarifications : - Did the approval from the Ethical committee approved this part ? If so, did the participants signed an informed consent ? - What should they refer to when they answered the questions ? Did the ones that had an OBE had to refere to that episode ? And those who did not life an OBE ? What was the experience of reference ? - It there is no reference for the controls without an OBE expereince then is seems logical that they do not answer like the others.

L304 : the % is not correct, is should be 46.6%

L338 : With what material was the comparison made ? With the material from Jourdan (2011) or the participants that were contacted after the study ? This part should be clearer If the cmparison is made with Jourdan, then a table with the similaritie and differences could be added.

L382 : “but his aim was not to confirm his knowledge, but to compare it with the participants’ experience” : this was not mentionned before. It is hard to understand as we have no information in the hypnostist knowledge or the comparison that is mentionned. Could the authors clarify ?

L387 : Another limitation it the response expectancy during hypnosis. There is a large literature on the subject this should be discused in the limites. More so because all the subject had good knowledge about OBEs

L412 : Is it acceptable to disclose the participants identity ?

Page 5 : The authors mention in a foot note that some other informatio will be avaiable in a future publication but the publication has since been published. Furthermore, that publication is cited in the bibliography this is confusing.

Table S1 : it is hard for the reader to understand to what the table refers to. GAPED has to be explained.

Section 5 – Decision

Verified with reservations: The content is scientifically sound, but has shortcomings that could be improved by further studies and/or minor revisions.

Peer review report

Reviewer: : Charlotte Martial Institution: University of Liège email: cmartial@uliege.be

Section 1 – Serious concerns

• Do you have any serious concerns about the manuscript such as fraud, plagiarism, unethical or unsafe practices? No
• Have authors’ provided the necessary ethics approval (from authors’ institution or an ethics committee)? Yes

Section 2 – Language quality

• How would you rate the English language quality? High quality (but English typos)

Section 3 – validity and reproducibility

• Does the work cite relevant and sufficient literature? No
• Is the study design appropriate and are the methods used valid? Yes
• Are the methods documented and analysis provided so that the study can be replicated? Yes
• Is the source data that underlies the result available so that the study can be replicated? Yes
• Is the statistical analysis and its interpretation appropriate? No
• Is quality of the figures and tables satisfactory? No
• Are the conclusions adequately supported by the results? No
• Are there any objective errors or fundamental flaws that make the research invalid? No*

Section 4 – Suggestions

• In your opinion how could the author improve the study?

I read with great interest this manuscript, studying a fascinating topic. Although this study is highly interesting and intriguing, I think that some major improvements could be made. Notably:

The work does not cite relevant and sufficient literature. For instance, the authors do not cite a recent publication which aimed to induce OBE using hypnosis (Martial et al., Scientific Reports, 2019). Another example: The authors could have at least briefly discussed Parnia’s AWARE prospective study reporting OBE in NDE experiencers (Parnia et al., 2014). Besides, they sometimes cite some inappropriate references; I would like to invite the authors to cite rigorous and serious articles in the field. For example, they used Jourdan (2011) reference to draw a parallel with NDEs, however, Jourdan (2011) is not a reference article in the field…

Importantly, it is worth mentioning that, so far, no rigorous empirical scientific study has shown evidence of veridical perceptions during OBE; according to me, it is not clear in the manuscript.

Some limitations of the study are not mentioned in the discussion section.

Some details of the study are missing. Notably, the description of the Table 6 is incomplete: what do the bold numbers mean? Another example: how did they recruit the participants –who are co-authors of the paper? Are they researchers?

The conclusions they draw in the discussion are not based on the present findings; they extrapolate.

• Do you have any other feedback or comments for the Author?

page13: what do they mean by “general consensus”? This is not clear to me

I would like to invite the authors to correct English typo and spelling errors (example: “We also thankS”)

Section 5 – Decision

Requires revisions: The manuscript contains objective errors that must be addressed

Peer review report

Reviewer: Dacre Knight, MD Institution: Mayo Clinic email: Knight.dacre@mayo.edu

Section 1 – Serious concerns

• Do you have any serious concerns about the manuscript such as fraud, plagiarism, unethical or unsafe practices? No
• Have authors’ provided the necessary ethics approval (from authors’ institution or an ethics committee)? not applicable

Section 2 – Language quality

• How would you rate the English language quality? High quality

Section 3 – validity and reproducibility

• Is the reasons for conducting the study and its objectives clearly explained? Yes

• Is the study design appropriate? Yes

• Are sufficient details provided so that the method can be replicated? Yes

• Are datasets available so that others could use them? not applicable

Section 4 – Suggestions

• Based on your answers in section 3 how could the author improve the protocol?

There is a more specific definition of PASC that should be included (with reference). Need to list specific medical databases to search, not just “various”. PECO criteria needs to be listed, not only implied that it will be used.

• Do you have any other suggestions, feedback, or comments for the Author?

GRADE approach will be useful, as is mentioned along with narrative synthesis if needed. Strengths and limits seem accurate, good to list.

Section 5 – Decision

Verified with reservations: The content is scientifically sound but has shortcomings that could be improved by further studies and/or minor revisions.