A version of this preprint has been published in the Open Access journal GigaScience (see https://doi.org/10.1093/gigascience/giae008 ), where the paper and peer reviews are published openly under a CC-BY 4.0 license.

Reviewer Qiuyue Yuan

The authors conducted a study where they generated multi-omics datasets, including whole-genome sequencing and RNA sequencing , for rare neuroendocrine tumors in the lungs, small intestine, and large cells.

They used patient-derived tumor organoids and performed quality control analysis on the datasets. Additionally, they developed a random forest classifier specifically for detecting mutations in the RNA-seq data.

The pipeline used in this study is well-organized, but I have a few queries that I would like to clarify before recommending it for publication.Major concerns:The data processing and quality control procedures would be valuable for other researchers working with similar datasets. It would be beneficial to add these procedures to the GitHub repository (https://github.com/IARCbioinfo/MS_panNEN_organoids).

Furthermore, it would be helpful to provide insights into what constitutes good quality reads, such as the number of unique reads and the ratio of duplicate reads.Regarding the random forest (RF) model, it is mentioned that there are 10 features. Could you clarify if these features are from the public information, or are all the features extracted solely from the RNA-seq data?

Also, does the RF model work for WGS data as well?Was there any specific design implemented to address the issue of imbalanced positive and negative samples?RNA-seq are not used to generate the gene expression here, which would waste important information.Minor concerns:In Figure 6C, what does "Mean minimum depth" refer to?Is the most important feature identified by the RF model a good predictor?

A version of this preprint has been published in the Open Access journal GigaScience (see https://doi.org/10.1093/gigascience/giae008 ), where the paper and peer reviews are published openly under a CC-BY 4.0 license.

Reviewer Saurabh V Laddha

Alcala et al., did an excellent work on rare cancer type by creating PDTOs molecular fingerprint which has a direct impact for researcher working on these rare cancer type. As a data note, this is excellent resource and covering huge gap in this rare cancer field.These PDTOs holds high impact specially for such cancers which are slow growing and not easy culture in lab. Authors covered details regarding each technique used in this study and figures are clear to understand with exceptional writing.Minor comments:- Did authors compare the PDTOs to tumor molecular dataset ? This will be the key to understand how closely and qualitatively PTDOs are related to actual tumor datasets molecular profile. It is not clear in the current version and it will be helpful to readers to decide whether PTDOs molecular fingerprint system are valuable to them. This is not required for this manuscript to address but a note will be helpful to make valulabe decision to use such resources and with what limitations.- Authors covered longitudinal samples in this system for 1 to 2 timepoints. What changes did they observe (molecularly) looking at this data from a longitudinal timepoints view will be helpful for readers. Also, based on author's experience for longitudinal sampling, do authors have key suggestions for researcher ? a brief discussion will be helpful.- Authors did comprehensive small variant analysis from WGS and RNAseq. Did you authors find known somatic variations for these samples ? mainly comparing against the known published mutational landscape. A note of this will be helpful.- A comment on limitations of PTDOs and molecular fingerprint created from such PDTOs will be valuable.- Authors briefly comment on using such molecular datasets from PDTOs and combining with other datasets to improve on power statistics to discover informative molecular features of these cancers. This points towards my first point on how similar PDTOs are to tumor molecular profile.

A version of this preprint has been published in the Open Access journal GigaScience (see https://doi.org/10.1093/gigascience/giae008 ), where the paper and peer reviews are published openly under a CC-BY 4.0 license.

Reviewer Masashi Fujita

In this manuscript, Alcala et al. have reported on the whole genome sequencing (WGS) and RNA sequencing (RNA-seq) of 23 patient-derived tumor organoids of neuroendocrine neoplasms.

This is a detailed report on the quality control of WGS, RNA-seq, and sample swap. The methods are solid and well-described. The raw sequencing data have been deposited in a public repository. This dataset could be a valuable resource for exploring the biology and treatment of this rare type of tumor.

Here are my comments to the authors:

Could you please clarify whether the organoids described in this manuscript will be distributed? If so, could you provide the contact address and any restrictions, such as a material transfer agreement?You have deposited the RNA-seq gene expression matrix in the public repository European Genome-phenome Archive (dataset ID: EGAD00001009994).

However, the file is under controlled access. This limits the availability of data, especially for scientists who just want a quick glance at the data. Since the gene expression matrix does not contain personally identifiable information, I wonder if you could make the file open access.

You have reported how you detected somatic mutations in the organoids. However, you did not share the list of detected mutations. Sharing this list would help scientists who do not have a computational background. Open access is preferable in this case, but controlled access is also acceptable because germline variants could be misclassified as somatic.

The primary site of mLCNEC23 is unknown. Could you infer its primary site based on gene expression patterns or driver mutations?I have concerns about the generalizability of your random forest model because it was trained using only 22 somatic mutations. Could you assess your prediction model using publicly available datasets of cancer genomes (e.g., TCGA)?

Editors Assessment:

MPDSCOVID-19 has been developed as a one-stop solution for drug discovery research for COVID-19, running on the Molecular Property Diagnostic Suite (MPDS) platform. This is built upon the open-source Galaxy workflow system, integrating many modules and data specific to COVID-19. Data integrated includes SARS-CoV-2 targets, genes and their pathway information; information on repurposed drugs against various targets of SARS-CoV-2, mutational variants, polypharmacology for COVID-19, drug-drug interaction information, Protein-Protein Interaction (PPI), host protein information, epidemiology, and inhibitors databases. After improvements to the technical description of the platform, testing helped demonstrate the potential to drive open-source computational drug discovery with the platform.

This evaluation refers to version 1 of the preprint

This work has been published in GigaByte Journal under a CC-BY 4.0 license (https://doi.org/10.46471/gigabyte.114), and has published the reviews under the same license. These are as follows.

Reviewer 1. Prashanth N Suravajhala

Is there a clear statement of need explaining what problems the software is designed to solve and who the target audience is? Yes. The authors could describe Minimum Information about bioinformatics investigation (MIABI) guidelines. Is the source code available, and has an appropriate Open Source Initiative license been assigned to the code? github and Zenodo, yes!

I tested git, forked it and as I didn't test the graphical version, ensured all python libraries are working! Is the documentation provided clear and user friendly? Yes. Yes, a white paper could be more friendly! Is there a clearly-stated list of dependencies, and is the core functionality of the software documented to a satisfactory level? Yes. yes with README version! Have any claims of performance been sufficiently tested and compared to other commonly-used packages? Yes, as described by the authors Are there (ideally real world) examples demonstrating use of the software? Yes. The Molecular Property Dynamic Suite (MPDS) is a welcome initiative which would serve chemical space for research community. While the authors aimed to deploy it in Galaxy, there is no Galaxy reference cited in first few introductory lines. A strong rationale on Galaxy-MPDS connect could be a value addition The port 8085/8080 are ephemeral and it would be nice if the authors deploy it on a more permanent base An absolute strength for the suite is availability of source code so that end-users can fine tune and reinstantiate the codes. In the architecture, could the end user have a chance to deploy biopython modules for drug discovery/modeling

In Page 4, the authors can define what are the tools precisely used in MPDS 2.3 section: The PPI is not abbreviated as first use The results are exploited well for disease dependent/independent use. However, the major challenge for ligand use/preparation is the use of ncRNAs. Could MPDS provide such instances where ncRNAs could be used fpr targeted ligands? L28 in section 4.1: Pluralis for features ( as one of is used) Also a word or two on aadhar card for perhaps naive users may be mentioned and it may be italicized as it may be a domestic word. Does MPDS suite augur well for Anvaya that Government of India launched, or Tavexa or Taverna? A word to two on local setting up of cloud instance may be a nice addition

Scores on a scale of 0-5 with 5 being the best

Language: 4 Novelty: 4.5 Brevity: 4 Scope and relevance: 4 Language/Brevity checks: Page 9 L6: fulfill misspelt webserver are two words, IMHO

Page 10: CADD which IS available

Tabl S2/S4: from THE coronavirdiae space between anticoronavirusdrugs

Figure S3: remove OF (identifying OF existing) Supporting information may be corrected High resolution Figures esp GA, Figures 2-4 may be inserted

Reviewer 2. Abdul Majeed

Is the language of sufficient quality? Yes. Some changes are needed to make the writing more scientific. Is the code executable? Unable to test Is installation/deployment sufficiently outlined in the paper and documentation, and does it proceed as outlined? Unable to test

Additional comments: In this paper, the authors introduced a Molecular Property Diagnostic Suite (MPDS), which is a Galaxy-based web portal that was conceived and developed as an open-source disease-specific web portal. MPDS is a customized web portal developed for COVID-19, which is a one-stop solution for drug discovery research. I read the article; it is well-written and well-presented. The enclosed contents can be very useful for researchers working in this field (e.g., COVID-19 systems development). However, I propose some comments/concerns to the current version that need correction during the revision. 1- In the abstract, please provide the technical description of the method’s working. Also, please mention the entities which can benefit from the system. 2- The introduction section doesn’t present the challenges/problems of the existing tools. Please discuss the challenges of the previous such tools and how are they addressed through this new system. 3- I could not find the concrete details of data modalities supported in the system. The authors are advised to include such details. 4- The authors mentioned the use of ML, but I couldn’t find any potential usage of ML models. Please add such analysis during the revision. 5- Also, please add some performance results like time complexity, storage, I/O cost, etc. 6- One comprehensive diagram should be included to better illustrate the working of the proposed tool. 7- Please add limitations of the proposed tool in the revised work. 8- Please add the potential implications of this tool in the context of current/future pandemics.

Re-review: I have carefully checked the revised work and the author's responses. The authors have made the desired modifications. I have no major concerns on this paper. In the previous review round, Comment #: 3 has not been properly responded by the authors. By data modality, I meant tabular data, graph data, audio data, video data, etc. Authors should add this aspect clearly in the paper about each data modality processed in their system. In Figure 4, some contents (e.g., protein information, PPI interaction, etc.) are unreadable. The abbreviations are not consistently written in terms of small and capital letters. In the paper, the authors are advised to clearly describe the purpose of this tool, who will benefit and in what capacity, why these kinds of tools are needed, etc. I suggest adding such information in abstract to clearly convey the message to readers. In the title, please recheck one word, Open Access or Open Source. The journals are open access while the software are usually open source .

Reviewer 3. Agastya P Bhati

Is there a clear statement of need explaining what problems the software is designed to solve and who the target audience is? Yes. As noted in my comments, it would be beneficial to clarify what new capabilities are provided by this new portal over and above what is already available currently. Is the source code available, and has an appropriate Open Source Initiative license been assigned to the code? No. There is a github repository (https://github.com/gnsastry/MPDS-18Compound_Library), however, I am unable to access it currently. As Open Source Software are there guidelines on how to contribute, report issues or seek support on the code? Yes. A github repository provides such capabilities. However, it is inaccessible currently. Is the code executable? Unable to test Is installation/deployment sufficiently outlined in the paper and documentation, and does it proceed as outlined? Unable to test Have any claims of performance been sufficiently tested and compared to other commonly-used packages? Is automated testing used or are there manual steps described so that the functionality of the software can be verified? No Additional comments: Molecular Property Diagnostic Suite for COVID-19 (MPDSCOVID19) is an open-source disease specific web portal aiming to provide a collection of all tools and databases relevant for COVID-19 that are available online along with a few in-house scripts at a single portal. It is built upon another platform called "Galaxy" that provides similar services for data intensive biomedical research. MPDSCOVID19 is in continuation to two other similar disease-specific portals that this group has published earlier - for Tuberculosis and Diabetes mellitus. Overall, MPDSCOVID19 is an interesting and useful resource that could be helpful for biomedical community in conducting COVID-19 related research. It brings together all the databases and relevant tools that may make a researcher's life easier as exemplified through the various case studies included.

I recommend publishing this article after the following revisions noted. Please note that any mention of page numbers below is referring to the reviewer PDF version.

Major revisions:

(1) One main issue in this manuscript is the lack of a clear description of the "new" capabilities provided by MPDSCOVID19 over and above what Galaxy provides. I think a clear distinction between the capabilities/features of Galaxy and MPDSCOVID19 would help improve the manuscript substantially and help readers better understand the capabilities of this new COVID-19 portal.

Further, a description of the additions in the new portal over the earlier TB and Diabetes portals is mentioned on page 7. However, I think more details on such advancements/additions would be beneficial. It could be in the form of a table.

(2) It is mentioned that a major advancement in this new portal is the inclusion of ML/AI models/approaches, however no details have been provided. It would be beneficial to briefly describe what ML based capabilities are included in MPDS and how they can be used by any general user. An additional case study demonstrating the same would be helpful.

(3) MPDS portal provides a collection of tools and databases for COVID-19. However, such resources are ever-growing and hence constant updating of the portal's capabilities/resources would be a necessary requirement for its sustainability. There is no mention of any such plans. Do authors have a modus operandi for the same? Have there been further releases of the previous MPDS portals for TB and Diabetes that may be relevant here?

(4) Page 6 - lines 3-4: I suggest replacing "are going to witness" with "are witnessing". There are several recent advancements in applying ML/AI based approaches to improve different aspects of drug discovery. I recommend including a few references here to this effect. Below are some relevant examples:

(a) 10.1021/acs.jcim.0c00915 (b) 10.1021/acs.jcim.1c00851 (c) 10.1038/s41598-023-28785-9 (d) 10.1098/rsfs.2021.0018 (e) 10.1145/3472456.3473524 (f) 10.1145/3468267.3470573

(5) Page 7 - line 8: I am assuming that the terms like "updates", "additions", etc., used in this paragraph are comparing MPDS with its older versions. If so, it would be beneficial to clarify this explicitly. In addition, I suggest that the authors include a brief literature survey to describe what other tools and/or webservers are available already and how MPDS compares with them. This has not been done so far.

(6) The github repository is currently inaccessible publicly. This needs rectification.

Minor revisions:

(1) Page 4: Before introducing MPDSCOVID19 it makes sense to briefly describe Galaxy and its main features. For instance moving forward lines 19-20 (page 4) and lines 3-6 (page 5) to line 12 (page 4).

(2) Page 5 - line 22: I suggest that authors mention the total number of databases/servers that are covered by MPDSCOVID19 as of now. From Table S1, it appears that there are 15 currently (items 5 and 7 are repeated so the 13 seems the wrong total - needs rectification).

(3) Page 5 - line 30: It would make sense to specify details of the MPDS local server. For instance, how many cores/GPUs are available and what are their hardware architectures? Also, it would be beneficial for the users to know if it is possible to use MPDS tools on their own or public infrastructures for large scale implementations. I suggest authors comment on this aspect too.

(4) Page 6 - lines 16-19: The sentence "Galaxy platform.......extend the availability." needs some rephrasing. It is too long and the hard to comprehend.

(5) Page 7 - line 18: I don't understand the word "colloids". Please clarify.

(6) Page 8 - line 30: "section 2.3" is referred to but I don't see any section numbering the PDF provided. This needs rectification.

Re-review: I am satisfied with the changes made to the manuscript and recommend publishing it in its current form if all other reviewers are happy with that.

Editors Assessment:

This Data Release paper presents an updated genome assembly of the doubled haploid perennial ryegrass (Lolium perenne L.) genotype Kyuss (Kyuss v2.0). To correct for structural the authors de novo assembled the genome again with ONT long-reads and generated 50-fold coverage high-throughput chromosome conformation capture (Hi-C) data to assist pseudo-chromosome construction. After being asked for some more improvements to gene and repeat annotation the authors now demonstrate the new assembly is more contiguous, more complete, and more accurate than Kyuss v1.0 and shows the correct pseudo-chromosome structure. This more accurate data have great potential for downstream genomic applications, such as read mapping, variant calling, genome-wide association studies, comparative genomics, and evolutionary biology. These future analyses being able to benefit forage and turf grass research and breeding.

This evaluation refers to version 1 of the preprint

This work has been published in GigaByte Journal under a CC-BY 4.0 license (https://doi.org/10.46471/gigabyte.112), and has published the reviews under the same license. These are as follows.

Reviewer 1. Qing Liu

This updated double haploid perennial ryegrass (Lolium perenne L.) showed contig N50 of 120 Mb, total BUSCO score=99%, which verified that the improved assembly can serve a reference for Lolium species using 50-fold coverage Hi-C data. The article is well edited except for below revision points. The minor revision is suggested for the current version. 1 Please elucidate the Kyuss v2.0, whether its reference is the same as Kyuss v1.0, if same or separate reference please elucidate. 2 In Table 3 of page 6, What the repeat element number for each family, could authors listed in number and proportion in order to clear the family category, for example, is the number of rolling-circles the same for Heltrons? 3 Tandem repeat or satellite or centromere location data, could author provide for the updated assembly of the Lolium species. 4 For Figure 1, what the heterozygosity and k-mer estimated genome size, I can’t find the data. 5 In Figure 3A, lowercase letter a, b, c , d and e are suggested to subsittute the A, B, C, D and E in order to avoid Figure 3A and Figure 3AA

Reviewer 2. Istvan Nagy

Are all data available and do they match the descriptions in the paper? No. Minor revision in the manuscript body is suggested. Gene annotation and repeat annotation data need some minor revision) See details in the "Additional Comments" section. Additional Comments: The submitted dataset reports and improved chromosome-level assembly and annotation of the doubled-haploid line Kyuss of Lolium perenne. The present v2.0 assembly is showing significant improvements as compared to the Kyuss v1.0 assembly published by the same group in 2021: The new assembly incorporates 99% of the estimated genome size in seven pseudo-chromosomes and the >99% BUSCO completeness of the gene space is also impressive.

Below are mine remarks and suggestions to the present version of manuscript:

Genome assembly and polishing It's indicated that for the primary assembly of the present work the same source of ONT reads were used as for the previous Kyuss v1.0 assembly. However, in the present manuscript the authors report clearly better assembly quality as opposed to the Kyuss v1.0 assembly. The question remains open, whether the authors achieved better results by changing/optimizing the primary assembly parameters, and/or applying a step-wise, iterating strategy with repeated rounds of long-read and short-read corrections? By any means, a more detailed description/specification of assembly parameters would be desirable.

Genome annotation In the provided annotation file "kyuss_v2.gff" in the majority of cases gene IDs consisting of the reference chromosome ID and of an ongoing number, like "KYUSg_chr1.188" are used. However in a few cases gene IDs like "KYUSt_contig_1275.207" are also used. This inconsistency might create confusions for future users of Kyuss_2 resources, and while the later type of gene IDs might be useful for internal usage, they became meaningless, as instead of contigs now pseudo-chromosomes (and some unplaced scaffolds) are used as references. The authors should modified the gff files and use a consistent naming scheme for all genes. Further, transcript DNA sequences as well as transcript protein sequences with consistent naming schemes should also be provided.

Repeat annotation The authors should modify Table 3 by specifying and breaking down repeat categories according to the Unified Classification System of transposable elements, by giving Order and Superfamily specifications (like LTR/Gipsy and LTR/Copia etc, in accord with the provided gff file "kyuss_v2_repeatmask.gff").

According to the provided repeat annotation BED file, more than 750K repeat features have been annotated on the Kyuss_2 genome. Of these repeat features 57815 are overlapping with gene features and 25843 of these overlaps are longer than 100 bp. This indicate that a substantial portion of the 38765 annotated genes might represent sequences coding for transposon proteins and/or transposon related ORFs. I suggest that the authors revise the gene annotation data (and at least remove gene annotation entries that show ~100% overlap with repeat features).

Assembly quality assessment "The quality score(QV) estimated by Polca for Kyuss v2.0 was 50, suggesting a 99.999% base-level accuracy with the probability of one sequencing error per 100 kb. The estimated accuracy of Kyuss v1.0 is 99.990% (QV40, Table 1), which is 10 times lower than Kyuss v2.0, suggesting that Kyuss v2 is more accurate than Kyuss v1.0." In my opinion, this sentence needs clarification as readers might have difficulties to properly interpret this - especially considering the facts that the same long-read data was used for both for the v1 as well a for the v2 assembly versions, the short-read mapping rate was the same (99.55%) for both versions and the K-mer completeness analysis results differed only slightly (99,39% vs. 99.48%).

See more on this in GigaBlog http://gigasciencejournal.com/blog/citizen-scientists-can-see-the-wood-for-the-trees/

Editors Assessment:

This is a Data Release paper describing data sets derived from the Pomar Urbano project cataloging edible fruit-bearing plants in Brazil. Including data sourced from the citizen science iNaturalist app, tracking the distribution and monitoring of these plants within urban landscapes (Brazilian state capitals). The data was audited and peer reviewed and put into better context, and there is a companion commentary in GigaScience journal better explaining the rationale for the study. Demonstrating this data providing a platform for understanding the diversity of fruit-bearing plants in select Brazilian cities and contributing to many open research questions in the existing literature on urban foraging and ecosystem services in urban environments.

This evaluation refers to version 1 of the preprint

This work has been published in GigaByte Journal under a CC-BY 4.0 license (https://doi.org/10.46471/gigabyte.108 and see also the accompanying commentary in GigaScience: https://doi.org/10.1093/gigascience/giae007 ), and has published the reviews under the same license as follows:

Reviewer 1. Corey T. Callaghan

Are the data and metadata consistent with relevant minimum information or reporting standards?

Yes. More information should be given on the relevance to GBIF. And why the dataset is necessary to 'stand alone'. The main reason I guess is because in this context cultivated organisms are really valuable as a lot of your target organisms will indeed be cultivated.

Is the data acquisition clear, complete and methodologically sound?

No. More detail should be provided about the difference in research grade and cultivated organisms on iNaturalist. The RG could be downloaded from GBIF, but I understand the need to go around that given that the cultivated organisms are also valuable in this context.

Is the validation suitable for this type of data?

No. There should be more information provided on the CV model. And more information provided on the importance of identifiers in iNaturalist ecosystem. They are critically important. Right now, it reads as if the CV model generally accurately identifies organisms, but this isn't necessarily true, and there is no reference given. However, the identifiers are necessary to help data processing and identification of the organisms submitted to iNaturalist. I also think the biases of cultivated organisms not being identified as readily by iNaturalist identifiers should be discussed somewhere in the manuscript.

Additional Comments:

I appreciated the description of this dataset and particularly liked the 'context' section and think it did a good job of setting up the need for such data. I would use iNaturalist throughout as opposed to iNat since iNat is a bit more colloquial.

Reviewer 2. Patrick Hurley

This is a very interesting paper and approach to examining questions related to the presence of edible plants in Brazilian cities. As such, it addresses--whether intentionally or not--open questions within the existing literatures of urban foraging and urban ecosystem services (Shackleton et al. 2017, ), among others, including:

1. how the existing species composition of cities create already existing edible/useful landscapes (see Hurley et al. 2015, Hurley and Emery 2018, Hurley et al. 2022), or what the authors appear to describe as "orchards", and including the use of open data sources to support these activities (Stark et al. 2019),

2. the ways that urban forests support cultural ecosystem services (Plieininger et al. 2015), 2a. dietary need/food security (Synk et al. 2017, Bunge et al. 2019, Gaither et al. 2020, Sardeshpande & Shackleton 2023), including in Brazil (Brito et al 2020), and diversity (Gareake & Shackleton 2020), 2b. sharing of ecological knowledge (Landor-Yamagata 2018), and 2c. social-ecological resilience (Sardeshpande et al. 2021) as well as 2d. reconnect urban residents to nature/biodiversity (Palliwoda et al. 2017, Fisher and Kowarik 2020, Schunko and Brandner 2022).

3. I note that while most of the literatures above focus on foods and edibility, Hurley et al. 2015 and Hurley and Emery consider the relationship of urban forests for other, not food-related uses and thus the material connections and uses by people within art and other cultural objects.

4. I also note that some scholars are beginning to focus on the question of urban governance and the inclusion of urban fruit trees (Kowalski & Conway 2023), building off of the rapidly expanding literature on urban food forestry (Clark and Nicholas 2011) and edible green infrastructure. The difference between these literatures and those I've suggested above is that they generally focus on policy and planting interventions to insert, add, or otherwise enhance the edibility of these spaces (as opposed to the above stream analyzing how people interact with what is already there, whether those species are intended for harvest by people, or not, and thus it seems like this piece better links to those issues .

5. It would be helpful to see at least some of these links between the present research and its focus on methods for using a particularly valuable dataset linked to/with efforts to address the conceptual questions that are raised by the authors. For example, in relation to item #1 above, I might suggest dropping the use of "orchard" and describe the species being analyzed as representative of an "actually existing food forests" within these cities (building on the existing literature Items 1 through 3), while indicating the insights it might provide to those interested in interventions to shape future cities and their species composition to enhance human benefits (items 4 and 5). Likewise, it would be helpful to reference the items in 2a through 2d where they appear in the Context section, building on the very high level citations already (e.g., current citations #5 FAO and #6 Salbitano).

To be clear, much of what I'm asking for here can be, I think, addressed through additions of single sentences or phrases throughout the context section, along with brief reference to these within the brief discussions under "Reuse Potenial".

Or perhaps this is too in-depth for this journal. If that's the case, then I do think that reference to several key articles is needed, specifically to signal the insights this piece has for this ongoing work to understand how urban forests function for human benefit. Those would be:

Shackleton et al. 2017, Hurley & Emery 2018, Garekea & Shackleton 2020, Fisher & Kowarik 2020, Sardeshpande et al. 2021.

Most critically, the work of Stark et al. 2019 should be acknowledged.

My sincere thanks to the authors to learn from this work and my apologies for the delay in completing this review.

Works Cited Above

Bunge, A., Diemont, S. A., Bunge, J. A., & Harris, S. (2019). Urban foraging for food security and sovereignty: quantifying edible forest yield in Syracuse, New York using four common fruit-and nut-producing street tree species. Journal of Urban Ecology, 5(1), juy028.

Fischer, L. K., & Kowarik, I. (2020). Connecting people to biodiversity in cities of tomorrow: Is urban foraging a powerful tool?. Ecological Indicators, 112, 106087.

Garekae, H., & Shackleton, C. M. (2020). Foraging wild food in urban spaces: the contribution of wild foods to urban dietary diversity in South Africa. Sustainability, 12(2), 678.

Hurley, P. T., Emery, M. R., McLain, R., Poe, M., Grabbatin, B., & Goetcheus, C. L. (2015). Whose urban forest? The political ecology of foraging urban nontimber forest products. Sustainability in the global city: Myth and practice, 187-212.

Hurley, P. T., & Emery, M. R. (2018). Locating provisioning ecosystem services in urban forests: Forageable woody species in New York City, USA. Landscape and Urban Planning, 170, 266-275.

Hurley, P. T., Becker, S., Emery, M. R., & Detweiler, J. (2022). Estimating the alignment of tree species composition with foraging practice in Philadelphia's urban forest: Toward a rapid assessment of provisioning services. Urban Forestry & Urban Greening, 68, 127456.

Kowalski, J. M., & Conway, T. M. (2023). The routes to fruit: Governance of urban food trees in Canada. Urban Forestry & Urban Greening, 86, 128045.

Landor-Yamagata, J. L., Kowarik, I., & Fischer, L. K. (2018). Urban foraging in Berlin: People, plants and practices within the metropolitan green infrastructure. Sustainability, 10(6), 1873.

Palliwoda, J., Kowarik, I., & von der Lippe, M. (2017). Human-biodiversity interactions in urban parks: The species level matters. Landscape and Urban Planning, 157, 394-406.

Plieninger, T., Bieling, C., Fagerholm, N., Byg, A., Hartel, T., Hurley, P., ... & Huntsinger, L. (2015). The role of cultural ecosystem services in landscape management and planning. Current Opinion in Environmental Sustainability, 14, 28-33.

Sardeshpande, M., Hurley, P. T., Mollee, E., Garekae, H., Dahlberg, A. C., Emery, M. R., & Shackleton, C. (2021). How people foraging in urban greenspace can mobilize social–ecological resilience during Covid-19 and beyond. Frontiers in Sustainable Cities, 3, 686254.

Sardeshpande, M., & Shackleton, C. (2023). Fruits of the city: The nature, nurture and future of urban foraging. People and Nature, 5(1), 213-227.

Schunko, C., & Brandner, A. (2022). Urban nature at the fingertips: Investigating wild food foraging to enable nature interactions of urban dwellers. Ambio, 51(5), 1168-1178.

Shackleton, C. M., Hurley, P. T., Dahlberg, A. C., Emery, M. R., & Nagendra, H. (2017). Urban foraging: A ubiquitous human practice overlooked by urban planners, policy, and research. Sustainability, 9(10), 1884.

Stark, P. B., Miller, D., Carlson, T. J., & De Vasquez, K. R. (2019). Open-source food: Nutrition, toxicology, and availability of wild edible greens in the East Bay. PLoS One, 14(1), e0202450.

Synk, C. M., Kim, B. F., Davis, C. A., Harding, J., Rogers, V., Hurley, P. T., ... & Nachman, K. E. (2017). Gathering Baltimore’s bounty: Characterizing behaviors, motivations, and barriers of foragers in an urban ecosystem. Urban Forestry & Urban Greening, 28, 97-102.

This work has been peer reviewed in GigaScience (https://doi.org/10.1093/gigascience/giae001), which carries out open, named peer-review. These reviews are published under a CC-BY 4.0 license and were as follows:

Reviewer 2: Linlin Zhuo

In this manuscript, the authors introduce a network permutation framework to quantify the effects of node degree on edge prediction. The importance of degree in the edge detection task is self-evident, and the quantification of this effect is undoubtedly groundbreaking. The experimental results on a variety of datasets demonstrate the advanced nature of the method proposed by the authors. However, some parts require further explanation from the authors and can be considered for acceptance in a later stage.

1.The imbalance of the degree distribution has a significant impact on the results of the edge detection task. In this manuscript, the author proposes a framework to quantify this impact. It is important to note that the manuscript does not explicitly mention the specific form in which the quantification is reflected, such as whether it is presented as an indicator or in another form. Therefore, further explanation from the author is needed to clarify this aspect.

2.The authors propose that researchers employ marginal priors as a reference point to discern the contributions attributed to node degree from those arising from specific performance. It would be helpful if the authors could elaborate further on the methodology or provide a sample demonstration to clarify the implementation of this approach.

3.For the XSwap algorithm, I wonder that if the authors could provide a more detailed explanation of its workings, including a step-by-step implementation of the improved XSwap. Furthermore, it would be beneficial if the authors could highlight the significance of the improved XSwap algorithm in biomedical tasks.

4.The author presents the pseudocode of the XSwap algorithm in Figure 2, along with the improved pseudocode after the author's enhancements. Both pseudocodes are accompanied by explanatory text. However, I believe that expressing them in the form of a figure would make it more visually appealing and intuitive.

5.The authors introduce the edge prior to quantify the probability of two nodes being connected solely based on their degree. I request the authors to provide a detailed explanation of the specific implementation of the edge prior.

6.In the "Prediction tasks" section, the author utilizes three prediction tasks to assess the performance of the edge prior. It is recommended to segment correctly for better display of the content.

7.The focus of the article might not be prominent enough. It is advisable for the author to provide further elaboration on the advanced nature of the proposed framework and its significance in practical tasks. This would help emphasize the main contributions of the research and its relevance in real-world applications.

This work has been peer reviewed in GigaScience (https://doi.org/10.1093/gigascience/giae001), which carries out open, named peer-review. These reviews are published under a CC-BY 4.0 license and were as follows:

Reviewer 1: Babita Pandey

The manuscript "The probability of edge existence due to node degree: a baseline for network-based predictions" presents novel work. But some of the sections are written very briefly, so it is difficult to understand. The section that needs revision are: Degree-grouping, The edge prior encapsulates degree, Degree can underly a large fraction of performance and Analytical approximation of the edge prior. The result section needs revision.

Some other concerns are: Academic adhar, Jaccard coefficient, preferential atachment etc are link prediction methods. Why auther has termed them as edge prediction features.

Editors Assessment:

One limiting factor in the adoption of spatial omics research are workflow systems for data preprocessing, and to address these authors developed the SAW tool to process Stereo-seq data. The analysis steps of spatial transcriptomics involve obtaining gene expression information from space and cells. Existing tools face issues with large data sets, such as intensive spatial localization, RNA alignment, and excessive memory usage. These issues affect the process's applicability and efficiency. To address this, this paper presents a high-performance open-source workflow called SAW for Stereo-Seq. This includes mRNA position reconstruction, genome alignment, matrix generation, clustering, and result file generation for personalized analysis. During review the authors have added examples of MID correction in the article to make the process easier to understand. And In the future, more accurate algorithms or deep learning models may further improve the accuracy of this pipeline.

*This evaluation refers to version 1 of the preprint *

This work has been published in GigaByte Journal under a CC-BY 4.0 license (https://doi.org/10.46471/gigabyte.111) as part of our Spatial Omics Methods and Applications series (https://doi.org/10.46471/GIGABYTE_SERIES_0005), and has published the reviews under the same license as follows:

Reviewer 1. Zexuan Zhu

It would be helpful if some examples can be provided to illustrate the key steps, e.g., the gene region annotation process and MID correction. Some information of the references is missing. Please carefully check the format of the references.

Decision: Minor Revision

Reviewer 2. Yanjie Wei

In this manuscript, the authors introduce a comprehensive Stereo-seq spatial transcriptomics analysis workflow, termed SAW. This workflow encompasses mRNA spatial position reconstruction, genome alignment, gene expression matrix generation, and clustering, culminating in the production of universally formatted results files for subsequent personalized analysis. SAW is particularly optimized for large field Stereo-seq spatial transcriptomics.

The authors provide an in-depth elucidation of SAW's workflow and the optimization techniques employed for each module. However, several aspects warrant further discussion:

1. The authors outline a strategy to reduce memory consumption during the mapping of CID tagged reads to corresponding coordinates by partitioning the mask file and fastq files. The manuscript, however, lacks a detailed description of how these files are divided. It would be beneficial if the authors could furnish additional information regarding this partitioning method.

2. The gene expression matrix, a crucial output of the SAW process, lacks sufficient evaluation to substantiate its accuracy. The count tool generates this matrix through three primary steps: gene region annotation, MID correction, and MID deduplication. During the gene annotation phase, a hard threshold (50% of the read overlapping with exon) is used to determine if a read is exonic. The basis for this threshold, however, remains unclear.

3. In the testing section, the authors evaluated the workflow on 2 S1 chips with approximately 1 million reads. The optimized workflow demonstrated a 1.8-fold speed increase compared to the non-optimized version. Table 2 only presents the total runtime before and after optimization. It would be advantageous if the authors could enrich this table by including the runtime of critical modules, such as read mapping, which accounts for 70% of the total runtime.

This work has been published in GigaByte Journal under a CC-BY 4.0 license (https://doi.org/10.46471/gigabyte.110) as part of our Spatial Omics Methods and Applications series (https://doi.org/10.46471/GIGABYTE_SERIES_0005), and has published the reviews under the same license as follows:

Reviewer 1. Chunquan Li

Stereo-seq, an advanced spatial transcriptomics technique, allows detailed analysis of large tissues at the single-cell level with precise subcellular resolution. Author's prior software was groundbreaking, generating robust single-cell spatial gene expression profiles using cell nuclei staining images and statistical methods. They've enhanced their software to STCellbin, using cell nuclei images to align cell membrane/wall staining images. This update employs improved cell segmentation, ensuring accurate boundaries and more dependable single-cell spatial gene expression profiles. Successful tests on mouse liver and Arabidopsis seed datasets demonstrate STCellbin's effectiveness, enabling a deeper insight into the role of single-cell characteristics in tissue biology. However, I do have some suggestions and questions about certain parts of the manuscript. 1. The authors should show the advantages and performance of STCellbin compared to other methods, such as its computational efficiency, accuracy, and suitability for various image types. 2. To comprehensively assess the performance of STCellbin, the authors should consider integrating other commonly used cell segmentation evaluation metrics, such as F1-score, Dice coefficient, and so forth. 3. To ensure the completeness and reproducibility of the data analysis, more detailed information regarding the clustering analysis of the single-cell spatial gene expression maps generated through STCellbin is requested. This information should encompass methods, parameters, and results such as cluster type annotations. 4. The authors can use simpler and clearer language and terminology to describe the image registration process in the methods section, ensuring that readers can easily understand the workflow and principles of image registration.

Reviewer 2. Zhaowei Wang

In this manuscript, the authors propose STCellbin to generate single-cell gene expression profiles for high-resolution spatial transcriptomics based on cell boundary images. The experiment results on mouse liver and Arabidopsis seed datasets prove the good performance of STCellbin. The topic is significant and the proposed method is feasible. However, there are still some concerns and problems to be improved and clarified.

(1) STCellbin is an update version of StereoCell, but the explanation of StereoCell is not sufficient. The authors should give a more detailed explanation of StereoCell, such as its input and main process. (2) The authors list some existing dyeing methods in Lines 52-53, Page 3. They should clarify that these methods are used for nuclei staining, which differentiate them from the cell membrane/wall staining methods of following content. It can provide a more accurate explanation for readers and users. (3) The authors share the GitHub repository of STCellbin, and I noticed that when executing STCellbin, the input only requires the path of image data and spatial gene expression data, the path of the output results, and the chip number. Are there other adjustable parameters? (4) In Page 5, Line 85, “steps” should be “step”, and in Page 8, Line 145, “must” would be better revised to “should”. Moreover, the writing of “stained image” and “staining image” should be consistent.

Editors Assessment:

This paper describes a new spatial transcriptomics method that that utilizes cell nuclei staining images and statistical methods to generate high-confidence single-cell spatial gene expression profiles for Stereo-seq data. STCellbin is an update of StereoCell, now using a more advanced cell segmentation technique, so more accurate cell boundaries can be obtained, allowing more reliable single-cell spatial gene expression profiles to be obtained. After peer review more comparisons were added and more description given on what was upgraded in this version to convince the reviewers. Demonstrating it is a more reliable method, particularly for analyzing high-resolution and large-field-of-view spatial transcriptomic data. And extending the capability to automatically process Stereo-seq cell membrane/wall staining images for identifying cell boundaries.

This evaluation refers to version 2 of the preprint

Editors Assessment:

For better data quality assessment of large spatial transcriptomics datasets this new BatchEval software has been developed as a batch effect evaluation tool. This generates a comprehensive report with assessment findings, including basic information of integrated datasets, a batch effect score, and recommended methods for removing batch effects. The report also includes evaluation details for the raw dataset and results from batch effect removal methods. Through peer review and clarification of a number of points it now looks convincing that this tool helps researchers identify and remove batch effects, ensuring reliable and meaningful insights from integrated datasets. Potentially making the tool valuable for researchers who need to analyze large datasets of this type, as it provides an easy and reliable way to assess data quality and ensures that downstream analyses are robust and reliable.

This evaluation refers to version 1 of the preprint

This work has been published in GigaByte Journal under a CC-BY 4.0 license (https://doi.org/10.46471/gigabyte.108) as part of our Spatial Omics Methods and Applications series (https://doi.org/10.46471/GIGABYTE_SERIES_0005), and has published the reviews under the same license as follows:

**Reviewer 1. Chunquan Li **

1. Page 1, Lines 14-16. The authors indicate that “it is crucial to thoroughly investigate the batch effects in the dataset before integrating and processing the data”. The term “thoroughly” may be not accurate enough. The current method can alleviate the batch effects, but it can’t thoroughly solve the related problems. In addition, this work proposes a batch evaluation tool, such “reasonably evaluate the batch effects” may be more accurate than “thoroughly investigate the batch effects”.
2. In Figure 1, does the first box is “integrated datasets”?
3. Page 5, Line 168, and Page 6, Lines 169-175, the content of these two paragraphs is similar, with some redundant descriptions. It is recommended to organize and write them into one paragraph.
4. There is Table 1 in the table list, but Table 1 is missing in the main text.
5. Page 8, Discussion section, it is better to discuss the differences between the proposed tool and a similar tool “batchQC”, especially the advantages of the proposed tool.
6. Some other minor issues: Page 1, Line 22, “to do so” should be “to do it”. Page 3, Line 100, Ref. [13] should be cited when it first appears on Line 97. Page 4, Line 114 and Page 5, Line 146, “UMAP” should be given its full name when it first appears and abbreviated directly in the following text. The variable should be in italics, such as “p” on Page 4, Line 119, “H” on Page 6, Line 184.

Reviewer 2. W. Evan Johnson and Howard Fan

Is the source code available, and has an appropriate Open Source Initiative license (https://opensource.org/licenses) been assigned to the code?

Yes. However, the code could use substantial improvements.

Is installation/deployment sufficiently outlined in the paper and documentation, and does it proceed as outlined?

No. The manuscript is missing a section describing the software and its implementation.

Is there enough clear information in the documentation to install, run and test this tool, including information on where to seek help if required?

Yes. But it took a while to get it installed.

Have any claims of performance been sufficiently tested and compared to other commonly-used packages?

No. I think the most glaring deficiency in the paper is the lack of comparison with other methods. For example, there is no comparison of the tools available in BatchEval compared to other methods, such as BatchQC. Also, they mention that BatchQC might not work on larger datasets, but they perform no performance evaluation for BatchEval, and no comparison with BatchQC to demonstrate improved performance.

Are there (ideally real world) examples demonstrating use of the software?

Yes. Missed opportunity--I think the most exciting thing I observed from the paper was that the example data were from spatial transcriptomics data! To my knowledge, existing batch effect methods are not directly adapted to manage these data (although they did mention tools like BatchQC cannot handle large datasets, which may be true). But they don’t mention anything about batch adjustment/evaluation in spatial data in the manuscript. I feel that if the authors address this niche it would increase the value/impact of their work!

Additional Comments:

This review was conducted and written by Evan Johnson, who developed the competing BatchQC software.

The authors provide an interesting toolkit for assessing batch effects in genomics data. The paper was clear and well-written, albeit I had a few concerns (see below). We were also able to download the associated software and test it out (comments below as well).

I think the most exciting thing I observed from the paper was that the example data were from spatial transcriptomics data! To my knowledge, existing batch effect methods are not directly adapted to manage these data (although they did mention tools like BatchQC cannot handle large datasets, which may be true). But they don’t mention anything about batch adjustment/evaluation in spatial data in the manuscript. I feel that if the authors address this niche it would increase the value/impact of their work!

In addition, this toolkit is written in Python, while BatchQC and other tools are written in R, so this is an advantage of the method as well—it addresses an audience that uses Python for gene expression analysis (not as big as the R community, but substantial). Their Python toolkit might also be more accessible to implementation in a pipeline workflow (for a core or large project) than R-based tools like BatchQC—this might be important to mention this as well.

I think the most glaring deficiency in the paper is the lack of comparison with other methods. For example, there is no comparison of the tools available in BatchEval compared to other methods, such as BatchQC. Also, they mention that BatchQC might not work on larger datasets, but they perform no performance evaluation for BatchEval, and no comparison with BatchQC to demonstrate improved performance.

Similarly, the authors claim: “Manimaran [10] has developed user-friendly software for evaluating batch effects. However, the software does not take into account nonlinear batch effects and may not be able to provide objective conclusions.” I don’t understand what the authors mean by “may not be able to provide objective conclusions” – BatchQC provides – several visual and numerical evaluations of batch effect – more so than even the proposed BatchEval does. Did the authors mean something else, maybe that the lack of non-linear correction may lead to less accurate conclusions?

A related concern: does BatchEval provide non-linear adjustments? I may have missed this, but it seems that BatchEval is not providing non-linear adjustments either. Also, regarding non-linear adjustments, the authors should show in an example the problems with a lack non-linear adjustments and show that pre-transforming the data before using BatchQC does not perform as well as the non-linear BatchEval adjustments.

In Equation 10, should “batchScore” be BatchEvalScore?

Also, in the bottom of Figure on page 15, should the “BatchQCScore” also be BatchEvalScore??

The manuscript is missing a section describing the software and its implementation.

I asked my research scientist, who recently graduated with his PhD in Bioinformatics, to assess the software and examples. First of all, much of the software is named “BatchQC”. I think this is confusing, since the method is really named BatchEval and it will be confused with BatchQC which is another existing/competing software. Furthmore, it took him a significant effort to install the BatchEval software and get is working on our cluster. I would recommend the authors make their software more accessible and easier to install.

The output of the software was a nice .html report diagnosing the batch effects in the data—very useful (attached is a combined .pdfs of the .htmls that we generated). We were also able to generate a report for the harmony adjusted example using their code. One major disadvantage was that these reports are separate files, and this could get very complicated comparing cases using multiple batch effect methods that will all be in separate reports (refer to a recent single cell batch comparison that compared more than a dozen methods – Tran et al. Genome Biology, 2020 – it would be hard to use BatchEval for this comparison).

Also, it seems that the user is required to conduct the batch correction themselves, BatchEval does not help with the correction except for their example code for Harmony.

Finally, on comparing the raw and Harmony adjusted datasets, inspection of the visual assessments (e.g. PCA) show some improvement—although not a perfect correction. But must of the numerical assessments are still the sample. The BatchEvalScore in both cases leads to the conclusion “Need to do batch effect removal”. What’s missing is the difference or improvement that Harmony makes on its correction. Maybe this is just because Harmony doesn’t fully remove the batch effects? Or is there something not working in the code? Might be good to see another example where the batch effect correction improves the BatchEvalScore significantly.

Additional Files: https://gigabyte-review.rivervalleytechnologies.com/journal/gx/download-files?YXJ0aWNsZT00NDImZmlsZT0xNzEmdHlwZT1nZW5lcmljJnZpZXc9dHJ1ZQ~~

Re-review:

I find this paper to be much improved in this version. The authors have clearly worked hard to address my concerns and have addressed them in a satisfactory manner. I fully support the publication of this paper, and I believe their tools are a nice addition to the field.

Reference 6 has been retracted due to potential manipulation of the publication process. The publisher of this paper cannot vouch for its reliability, but in this case this citation does not change the conclusions of the work published here. Though we thought we would highlight this to let readers know.

Reference 6 has been retracted due to potential manipulation of the publication process. The publisher of this paper cannot vouch for its reliability, but in this case this citation does not change the conclusions of the work published here. Though we thought we would highlight this to let readers know.

Reference 6 has been retracted due to potential manipulation of the publication process. The publisher of this paper cannot vouch for its reliability, but in this case this citation does not change the conclusions of the work published here. Though we thought we would highlight this to let readers know.

Reviewer 2. Stefano Monti

The manuscript addresses the very challenging problem of integrating multiple spatially resolved transcriptomics datasets, and proposes a novel algorithm based on multiple deep learning techniques, including DNN encoders, and self supervised and contrastive learning. Evaluation on several datasets is presented alongside comparison to multiple existing methods using several integration metrics (LISI, ARI, etc.). The presented method appears to outperform existing methods according to multiple criteria, and thus it represents a significant contribution to the field worth publishing.

The write-up is adequate, although the description of the method very "abstract", and it would benefit from more specificity in describing the inputs and outputs of each step, how some of the models are shared (e.g., is the DNN encoder shared only across sections/samples or also across the original (Fig 1C, top) and perturbed (Fig 1C, bottom) inputs? Likewise for the Graph Encoder), and the intuition behind each of the steps included.

Some specific comments: * It would be helpful if the results sections describing each of the applications (DLPFC datasets, Olfactory bulb datasets, etc.) were more detailed in the description of the datasets to be combined. What are the inputs (how many samples, are sections the same as samples?, how many slices per sample, etc). * Unless I'm mistaken, the labeling of Fig S1 is wrong. I think fig S1a is the UMap and S1b is the "manual annotation" rather than the other way around?

Reviewer 1. Lamda Moses.

This papers presents spatiAlign, a package that batch corrects spatial transcriptomics data and performs spatially informed clustering. Spatial information is incorporated in the graph layers in the variational graph autoencoder which performs dimension reduction, and in the reduced dimensional space, self-supervised contrastive learning is used to batch correct and to assign cells/spots to clusters. The autoencoder then reconstructs a batch corrected gene count matrix for downstream use with methods that require a full gene count matrix. The method seems reasonable for this task and is well-described, more intuitively in the Results section and in more details in the Methods section.

Then spatiAlign is benchmarked against several popular and state of the art methods for batch correction, including two recently published methods that use spatial information (GraphST and PRECAST) and several not using spatial information but commonly used (e.g. Seurat, Harmony, COMBAT). The choice of existing methods to benchmark is fair. The LISI F1 score is a reasonable metric to quantify performance in both batch correction and cluster separation when the spatial clusters in the brain datasets used in benchmarking are already annotated. The iLISI (batch correction) and cLISI (cluster separation), analogous to precision and recall in the original F1, are shown separately in the supplement. The F1 score is around 0.8 for spatiAlign, which is pretty good. When there is no a priori annotation, the iLISI is used to quantify how well different batches mix and Moran's I is used to indicate spatial coherence of the clusters, which are then validated with differential expression. spatiAlign is also demonstrated to integrate data from different technologies—Stereo-seq and Visium—which have different spatial resolutions. Finally, spatiAlign is demonstrated on the developing mouse brain integrating data across multiple time points.

The language of this paper is good and does not require extensive editing for clarity. The spatiAlign package can be installed with pip and has a minimal tutorial on the documentation website.

Overall, I find this paper well-written and a valuable contribution to this field. There are many methods that perform batch correction without using spatial information, and several that align different tissue sections, some using transcriptome information, but without correcting for batch effect in the transcriptomes. Not all methods that take spatial information into account give a batch corrected full gene count matrix as an output. The metrics reasonably demonstrate superior performance of spatiAlign compared to other methods benchmarked on the datasets used.

Below are my questions and comments that may improve this paper:

1. All the benchmarking datasets are from the brain, though different parts of the brain, from human and mouse, with different morphologies. The brain has a stereotypical structure. As spatiAlign uses the spatial neighborhood graph rather than the original coordinates, can it be applied to tissues without such stereotypical structure, such as tumors, skeletal muscle, colon, liver, lung, and adipose tissue? Benchmarking on a dataset from a tissue without a stereotypical structure would make a stronger case, to be more representative of the full breadth of spatial transcriptomics datasets.
2. Biological variability is mentioned, such as from different regions of hippocampus and different stages of development. Many studies have a disease or experiment group and a control group, often with multiple subjects in each group. There are biological differences among the subjects and technical batch effects between sections, but the differences between case and control are of interest, so we have different kinds of batches. Benchmarking on a case/control study would be really helpful. How well does spatiAlign preserve biological differences between case and control while correcting for technical batch effects?
3. The Methods section says, "Inspired by unsupervised contrastive clustering[32], we map each spot/cell i into an embedding space with d dimensions, where d is equal to the number of pseudoprototypical clusters." In Tutorial 2 on the documentation website, the latent dimension is set to be 100. Why is this number chosen? Can you clarity how to choose the number of latent dimensions? How does this affect downstream results?
4. Since you use the k nearest neighbor graph when constructing the spatial neighborhood graph that feeds into the variational graph autoencoder, what are the reasons why k=15 is chosen? Should it be different for array-based technologies such as Visium and Stereo-seq and imaging-based technologies with single cell resolution such as MERFISH? Furthermore, due to different spatial resolutions, the spatial neighborhood graph has different biological meanings for Visium and MERFISH.
5. All the benchmarking datasets are from array-based technologies: Visium, Slide-seq, and Stereo-seq. Imaging-based technologies are getting commercialized and getting more widely adopted, especially MERFISH and Molecular Cartography. It would be great if you benchmark using an imaging-based dataset and perhaps integrate an imaging-based and an array-based dataset, to be more representative of the full breadth of spatial transcriptomics technologies. This should also take into consideration that imaging-based datasets typically only profile a few hundred genes while array-based datasets are transcriptome-wide. This might be too much for this paper, but should at least be mentioned in the Discussions section.
6. Is the code used to reproduce the figures available?
7. Generally, the y axes of bar charts for F1 scores, ARI, normalized iLISI, and normalized cLISI are really confusing when they don't start at 0 and end at 1. This exaggerates how much better spatiAlign performs compared to other methods when the other methods aren't that much worse based on the numbers, such as in Figure 2c.
8. In Supplementary Figure S4b, do you actually mean 1 - cLISI? If a smaller cLISI is better, then spatiAlign performs the worst in this case, and should have a low F1 score in Figure 2c.
9. It would be helpful to include a computational time and memory usage benchmark.
10. The join count statistic is a spatial autocorrelation statistic designed for binary data, and may thus be more appropriate than Moran's I to indicate spatial coherence of clusters, although Moran's I does convey the message of spatial coherence here.
11. The documentation website can be improved by making a description of all parameters of the functions available, to explain what each parameter means and what kind of input and output is expected.
12. It would be helpful to include preprocessing in the tutorial on the documentation website. Do we need to log normalize the data first and why? Does the data need to be scaled?

Below are minor technical comments: 1. The notation for the LISI F1 score in the Methods sections is very confusing. Based on context and the definition of the F1 score, you probably meant to put parentheses around 1 - cLISInorm . 2. Typo in "SCAlEX" in Supplementary Figure S5a; you seem to mean "SCALEX". It's more aesthetically pleasing to be consistent in capitalizing according to the original names of the packages in Supplementary Figure S5.

Re-review

For the most part, the authors have satisfactorily addressed concerns raised by the reviewers. Below are my followup comments on the revised manuscript: 1. The authors missed the point of my second comment on case/control studies. What I was asking for is performance of spatiAlign and other related packages when integrating case datasets and control datasets while preserving biological differences of interest to the study. For example, data from healthy liver (control) and hepatic steatosis (case) are integrated. Case and control samples were collected from different patients and may be mounted on different slides. How well does spatiAlign preserve differences between healthy and steatosis, while correcting for technical batch effect? In Figure S7, the two sub-slices are still from the same disease condition. Case/control studies should at least be mentioned in the Discussions section. 2. The authors have provided thoughtful explanations on data scaling, number of latent dimensions, and number of neighbors in the k nearest neighbor graph in the response to reviewers. However, these explanations are not found in the manuscript or on the documentation website. Because these explanations are very relevant to users, it would be helpful to add them to either the manuscript or the documentation website. 3. For the bar charts, I suggest assigning a fixed color to each data integration method and keeping it consistent throughout this study. Right now the bar charts don't have a consistent color scheme even within the same figure. Keeping a consistent color scheme can reduce the mental burden of readers since the colors are a stand-in for the different methods. Also, a colorblind-friendly palette should be used. 4. I agree with Reviewer 3 that the grammar in this paper should be improved. For example, in lines 75-76, "in which gene expression is adjustment" should be "in which gene expression is adjusted". In lines 82-83, the "adjusted" in "laminar organization with adjusted, and clear boundaries between regions" does not make sense given the context referring to Figure 2f. In line 332, "the benchmarking methods" should be "the benchmarked methods", because the methods are being benchmarked and the methods themselves are not meant for benchmarking. Grammar in the newly added section from line 344 onwards should be corrected.

Editors Assessment:

The snake pipefish, Entelurus aequoreus, is a species of fish that dwells in open seagrass habitats in the northern Atlantic. As a pipefish, it is a member of the Syngnathidae family of fish which also includes seahorses and seadragons. In recent years it has expanded its population size and range into arctic waters. To better understand these demographic changes genomic data is useful, and to address this a high-quality reference genome has been produced. Building on a previous short-read reference, a near chromosome-scale genome assembly for the snake pipefish was assembled using PacBio CLR and Hi-C reads. After revisions the authors provided more details on the assembly metrics, the final assembly has a length of 1.6 Gbp, with scaffold and contig N50s of 62.3 Mbp and 45.0 Mbp respectively. Demographic inference analysis of the snake pipefish genome using this data enables tracing of population changes over the past 1 million years, and this reference will allow further analyses and studies relating these to changes in climate.

**This evaluation refers to version 1 of the preprint *

This work has been published in GigaByte Journal under a CC-BY 4.0 license (https://doi.org/10.46471/gigabyte.105), and has published the reviews under the same license as follows:

Reviewer 1. Yanhong Zhang

Are all data available and do they match the descriptions in the paper? No. There is no BioProject available for review at the link. Are the data and metadata consistent with relevant minimum information or reporting standards?

No. "the GigaDB repository:DOI:XXXXX." I am not sure that the authors have upload the data.

Is the data acquisition clear, complete and methodologically sound? No. I am not sure that the authors have uploaded the data.

Is there sufficient data validation and statistical analyses of data quality? No. I need more information.

Is the validation suitable for this type of data? No. I need more information.

Is there sufficient information for others to reuse this dataset or integrate it with other data? No. I need more information.

Any Additional Overall Comments to the Author:

In line 41, you mean “50-100 kya”?

The authors need to provide more details about the genomic data： Genome size estimation based on K-mer spectrum？ Statistics of genomic characteristics from K-mer? Statistics of Hi-C sequencing raw data, such as raw bases, clean bases. Statistics of the pseduchromosome assemblies using Hi-C data. The result of BUSCO assessment, how about complete BUSCOs? complete single-copy？ Statistics of gene predictions in the snake pipefish Statistics of the noncoding RNA in the snake pipefish genome. The author claims that all other data, including the repeat and gene annotation, was uploaded to the GigaDB repository: DOI: XXXXX. I cannot find these data. “DOI: XXXXX”? What does that mean?

Reviewer 2. Sarah Flanagan

Are all data available and do they match the descriptions in the paper?

No. I received an NCBI link which took me to the raw data files and a BioSample description, but it did not link to the assembled and annotated genome.

Is there sufficient detail in the methods and data-processing steps to allow reproduction?

Yes. Only one point was not clear to me in the methods -- please clarify in the text which data was used to generate consensus genome sequences using vcfutils (lines 240-241). How did this differ from the assembled and annotated genome?

Any Additional Overall Comments to the Author:

In the abstract and introduction, the description of the habitat of the species is confusing and it was not clear from the manuscript as written that there are two ecotypes, one that is pelagic and one that is coastal. Consider re-phrasing these sections (lines 31-32, 57-59, and 61-62) to better describe the habitat of this species.

Please also consider increasing the font size of the labels in Figure 1 -- the details are very difficult to read.

Editors Assessment: Understanding the distribution of Anopheles mosquito species is essential for planning and implementing malaria control programmes, a task undertaken in this study that assesses the composition and distribution of the Anopheles in different districts of Kinshasa in the Democratic Republic of Congo. Mosquitoes were collected using CDC light traps, and then identified by morphological and molecular means. In total 3,839 Anopheles were collected, and data was digitised, validated and shared via the GBIF database under a CC0 waiver. The project monitoring the monthly dynamics of four species of Anopheles, showing a fluctuation in their respective frequencies during the study period. Review improved the metadata by adding more accurate date information, and this data can provide important information for further basic and advanced studies on the ecology and phenology of these vectors in West Africa.

*This evaluation refers to version 1 of the preprint

This work has been published in GigaByte Journal under a CC-BY 4.0 license (https://doi.org/10.46471/gigabyte.104), and has published the reviews under the same license. This is part of the GigaByte Vectors of Human Disease series, and this and the other papers are hosted here. https://doi.org/10.46471/GIGABYTE_SERIES_0002

The peer reviews are as follows.

Reviewer 1. Paul Taconet

Are all data available and do they match the descriptions in the paper? No

Additional Comments 1/ The CDC light trap catch data are available in the GBIF release, but the larva collection data are not included in the release. These larva collection data should be either included in the GBIF release, or it should be made clear in the manuscript that this data is not published. 2/ in the dataset, the data are indicated to be reported at the species level (taxonRank = Species) but there are no An. coluzzii reported. However, in table 3 of the manuscript, some An. coluzzii are reported. This is inconsistent. My guess is that the data reported in the dataset are those out of the morphological identification, hence for An. gambiae at the COMPLEX level, and not the species. This shoud in any case be clarified and corrected : are the data in the dataset provided at the complex or at the species level ? If complex, the ScientificName and taxonRank columns should be corrected. In addition, in the dataset, you could add an "identificationRemarks" column providing the source of identification (morphological or molecular). 3/ in the dataset, for the species scientific name, I suggest to use the names as presented in : Harbach, R.E. 2013. Mosquito Taxonomic Inventory, https://mosquito-taxonomic-inventory.myspecies.info/ . Or at least, to provide the "nameAccordingTo" column. 4/ The data available are of type 'occurrence' ( only in 1 file - the "occurrence" file). For a better presentation of the data and in order to be in line with the GBIF data architecture, I would suggest to transform them into "sampling event" data (consisting in 1 'event core' file, 1 'occurence' file, and potentially extension files), which is more suited to this kind of data acquired from sampling events (see https://ipt.gbif.org/manual/en/ipt/latest/sampling-event-data) and containing external measurements (eg. temperature, see next point). This would enable the user to quickly understand the dates and locations of the sampling events. 5/ Temperature and humidity are included in the main 'occurence' file (column "dynamicProperties") : - to which reality these data correspond (mean during the night of collection ? ), and how were these data collected (instrument, etc.) ? this information is not provided in the manuscript. - Instead of putting this data in the "occurence" file, I would suggest to add a "measurement" file in the GBIF data release, containing these meteorological data. Doing so would enable to include metadata about these measurements (instrument, etc.) See e.g. https://www.gbif.org/sites/default/files/gbif_IPT-sample-data-primer_en.pdf page 6 6/ in the dataset, for some of the collected mosquitoes, you put "organismRemarks" = "unfed" . How did you collect this information ? I could not see any mention to this feeding identification, neither in the manuscript nor in the dataset. 7/ in the dataset, in the column "SamplingProtocol", there are spelling errors -> "CDC ligth trap cathes" should be corrected to "CDC light trap catches "

Are the data and metadata consistent with relevant minimum information or reporting standards? No. See comments above.

Is the data acquisition clear, complete and methodologically sound? No. See comments above

Any Additional Overall Comments to the Author: Thanks for this nice work and the effort you put to open your data. See comments below and above to improve the work. 1/ comments for figure 1 (map) : the background layer is not very appropriate, as we miss landscape context. Maybe better to put an Open Street Map background layer, or a satellite image.

Reviewer 2. Chris Hunter.

Are all data available and do they match the descriptions in the paper?

No. The larva data are not included in the GBIF dataset. Some of the descriptions of the data in the manuscript do not match the data available from GBIF. Any Additional Overall Comments to the Author:

Major comments (Author action required): 1 - The manuscript describes larva collection and molecular identification of those species, but I cannot see any indication that those data are included in the GBIF dataset. Please clarify whether they are included or not, and if not please add them. 2 - The numbers cited in Table 1 do not match those shown in the GBIF dataset, e.g. the total of indoor/outdoor sampling events quoted in MS table 1 = 2180 / 1659, whereas in GBIF dataset there are 2304 indoor and 1535 outdoor sites listed? Please check your calculations and/or the data submitted to GBIF.

Minor comments (Author action suggested): 1 - There are 59 events in the GBIF data that do not have a date. Please check those data and update if you have those dates available. 2 - The events are all included in the GBIF sampling event dataset, however “individualCount” data are not included, please explain why those counts are not included as observation dataset(s)? i.e. why is there no number of individual mosquitos included in the dataset? 3 - The full DwC-GBIF dataset does include an indication of the indoor/outdoor location of the sampling sites in the "eventRemark" column, but if you are making updates to the dataset may I suggest using the column heading “habitat” to include that information in GBIF either instead or as well. 4 - Ideally, the molecular identification data should be shared. I don’t have access to the “protocol of Scott [29]” but my assumption is that the PCR products are differentiated by size via running on a gel? If so, and you have the digital images of those gels please let the GigaByte editors know and they will help you share them via the GigaDB database.

Please see the linked file "Data-Review-of-DRR-202310-03.pdf" for more details about the above concerns.

https://gigabyte-review.rivervalleytechnologies.com/journal/gx/download-files?YXJ0aWNsZT00ODEmZmlsZT0xODMmdHlwZT1nZW5lcmljJnZpZXc9dHJ1ZQ~~

Reviewer 3 Megan Hagenauer - Original Submission

Review of "A Multi-omics Data Analysis Workflow Packaged as a FAIR Digital Object" by Niehues et al. for GigaScience08-31-2023I want to begin by apologizing for the tardiness of this review - my whole family caught Covid during the review period, and it has taken several weeks for us to be functional again.OverviewAs a genomics data analyst, I found this manuscript to be a fascinating, inspiring, and, quite honestly, intimidating, view into the process of making analysis code and workflow truly meet FAIR standards. I have added recommendations below for elements to add to the manuscript that would help myself and other analysts use your case study to plan out our own workflows and code release. These recommendations fall quite solidly into the "Minor Revision" category and may require some editorial oversight as this article type is new to me. Please note that I only had access to the main text of the manuscript while writing this review.Specific Comments1) As a case study, it would be useful to have more explicit discussion of the expertise and effort involved in the FAIR code release and the anticipated cost/benefit ratio:As a data analyst, I have a deep, vested interest in reproducible science and improved workflow/code reusability, but also a limited bandwidth. For me, your overview of the process of producing a FAIR code release was both inspiring and daunting, and left me with many questions about the feasibility of following in your footsteps. The value of your case study would be greatly enhanced by discussing cost/benefit in more detail:a. What sort of expertise or training was required to complete each step in the FAIR release? E.g.,i. Was your use of tools like Github, Jupyter notebook, WorkflowHub, and DockerHub something that could be completed by a scientist with introductory training in these tools, or did it require higher level use?ii. Was there any particular training required for the production of high quality user documentation or metadata? (e.g., navigating ontologies?)b. With this expertise/training in place, how much time and effort do you estimate that it took to complete each step of adapting your analysis workflow and code release to meet FAIR standards?i. Do you think this time and effort would differ if an analyst planned to meet FAIR standards for analysis code prior to initiating the analysis versus deciding post-hoc to make the release of previously created code fit FAIR standards?c. The introduction provides an excellent overview of the potential benefits of releasing FAIR analysis code/workflows. How did these benefits end up playing out within your specific case study?i. e.g., I thought this sentence in your discussion was a particularly important note about the benefits of FAIR analysis code in your study: "Developing workflows with partners across multiple institutions can pose a challenge and we experienced that a secure shared computing environment was key to the success of this project."ii. Has the FAIR analysis workflow also been useful for collaboration or training in your lab?iii. How many of the analysis modules (or other aspects of the pipeline) do you plan on reusing? In general, what do you think is the size for the audience for reuse of the FAIR code? (e.g., how many people do you think will have been saved significant amounts of work by you putting in this effort?)iv. … Or is the primary benefit mostly just improving the transparency/reproducibility of your science?d. If there is any way to easily overview these aspects of your case study (effort/time, expertise, immediate benefits) in a table or figure, that would be ideal. This is definitely the content that I would be skimming your paper to find.2) As a reusable code workflow, it would be useful to provide additional information about the data input and experimental design, so that readers can determine how easily the workflow could be adapted to their own datasets. This information could be added to the text or to Fig 1. E.g.,i. The dimensionality of the input (sample size, number of independent variables & potential co-variates, number of dependent variables in each dataset, etc)ii. Data types for the independent variables, co-variates, and dependent variables (e.g., categorical, numeric, etc)iii. Any collinearity between independent variables (e.g., nesting, confounding).3) As documentation of the analysis, it would be useful to provide additional information about how the analysis workflow may influence the interpretation of the results.a. It would be especially useful to know which aspects of the analysis were preplanned or following a standard procedure/protocol, and which aspects of the analysis were customized after reviewing the data or results. This information can help the reader assess the risk of overfitting or HARKing.b. It would also be useful to call out explicitly how certain analysis decisions change the interpretation of the results. In particular, the decision to use dimension reduction techniques within the analysis of both the independent and dependent variables, and then focus only on the top dimensions explaining the largest sources of variation within the datasets, is especially important to justify and describe its impact on the interpretation of the results. Is there reason to believe that externalizing behavior should be related to the largest sources of variation within buccal DNA methylation or urinary metabolites? Within genetic analyses, the assumption tends to be the opposite - that genetic variation related to behavior (such as externalizing) is likely to be present in a small percent of the genome, and that the top sources of variation within the genetics dataset are uninteresting (related to population) and therefore traditionally filtered out of the data prior to analysis. Within transcriptomics, if a tissue is involved in generating the behavior, some of the top dimensions explaining the largest sources of variation in the dataset may be related to that behavior, but the absolute largest sources of variation are almost always technical artifacts (e.g., processing batches, dissection batches) or impactful sources of biological noise (e.g., age, sex, cell type heterogeneity in the tissue). Is there reason to believe that cheek cells would have their main sources of epigenetic variation strongly related to externalizing behavior? (maybe as a canary in a coal mine for other whole organism events like developmental stress exposure?). Is there reason to believe that the primary variation in urinary metabolites would be related to externalizing behavior? (perhaps as a stand-in for other largescale organismal states that might be related to the behavior - hormonal states? metabolic states? inflammation?). Since the goal of this paper is to provide a case study for creating a FAIR data analysis workflow, it is less important that you have strong answers for these questions, and more important that you are transparent about how the answers to these questions change the interpretation of your results. Adding a few sentences to the discussion is probably sufficient to serve this purpose. Thank you for your hard work helping advance our field towards greater transparency and reproducibility. I look forward to seeing your paper published so that I can share it with the other analysts in our lab.

Reviewer 2 Dominique Batista - Original Submission

Very good paper on the FAIR side. You detail what were the challenges, in particular when it comes to the selection of ontologies and terms.It is unclear if the generation of the ISA metadata is included in the workflow. Can a user generate the metadata for the synthetic dataset or their own data using the workflow ?Adding a GitHub action running the workflow with the synthetic data would help reusability but is not required for the publication of the paper.

This work has been published in GigaScience Journal under a CC-BY 4.0 license (https://doi.org/10.1093/gigascience/giad115), and has published the reviews under the same license. These are as follows.

Reviewer 1 Carole Goble - Original Submission

This work reports a multi-omics data analysis workflow packaged as a RO-Crate, an implementation of a FAIR Digital Object.We limit our comments to the technical aspects of the Research Object and workflow packaging. The scientific validity of the omics analysis itself is outside our expertise.The paper is comprehensive and the background grounding in the current state of the art is excellent and thorough. The paper is an excellent exemplar of the future of data analysis reporting for FAIR and reproducible computational methods, and the amount of work impressive. We congratulate the authors.WorkflowHub entry https://workflowhub.eu/workflows/402?version=5# gives a comprehensive report of the Nextflow workflow and its multiple versions, all the files including the R scripts and the synthetic data. The RO-Crate rendering looks correct and version-locking the R containers is following best practice(https://github.com/Xomics/ACTIONdemonstrator_workflow/blob/main/nextflow.config#L44)T he paper also highlights the amount of work needed to make such a pipeline to be both metadata machine processable and metadata human readable.To make this pipeline reproducible requires a mixture of notebooks submitted as supplementary materials, the Nextflow workflow with its R scripts represented as an RO-Crate in WorkflowHub and a README is linked to the container recipes in https://github.com/Xomics/Docker_containers and then another Documentation.md file. There seems to be the potential for duplicated effort in reporting the necessary metadata describing the workflow that could be highlighted in the Discussion as a steer to the digital object community.- Could the ROCrate approach be widened beyond the current Workflow RO-Crate, and would there be value in streamlining the metadata, or is this just an artefact of the need for multiple descriptions and ease of publishing. If the JSON within the RO-Crate was more richly annotated, could some of the Documentation.md be avoided altogether, and is that even desirable?- The README includes the container/software packaging and is not linked from the RO-Crate (and there isn't an obvious property to link to it yet). Could these be RO-Crates too?- The notebooks in the supplementary files could also be registered in WorkflowHub and linked to the Nextflow workflow (see https://workflowhub.eu/workflows?filter%5Bworkflow_type%5D=jupyter).- Is it feasible and desirable to have a single RO-Crate linked to many other RO-Crates to represent the whole reproducible pipeline in full?In the discussion the FAIR principles verification through different practices and approaches would be more helpful if it was more precise. Comments seem to be limited to the Workflow RO-Crate and use of ontologies for machine readability. As highlighted in table 1 there is more to FAIR software & workflows than this.Minor remarksKey points- We here demonstrate the implementation multiomics data -> We here demonstrate an implementation of an multi-omics data.Background- The documentation of dependencies is highlighted as a prerequisite for software interoperability. In the FAIR4RS principles I2 also highlights qualified references to other objects - presumably other software or installation requirements. This highlights the relationship between software interoperability and software portability. It seems that dependencies more relate to portability rather than interoperability.- "Based on the FDO concept, the RO-Crate approach was specified". This is a confusing statement. ROCrates have been recognised as an implementation approach for the FDO concept as proposed by the FDO Forum. For more discussion on FDO and the Linked Data approach promoted by RO-Crates see https://arxiv.org/abs/2306.07436. However, RO-Crates are not based in the FDO - they are based on the Research Object packaging work that emerged from the EU Wf4ever project, (see https://doi.org/10.1016/j.future.2011.08.004 from 2013).- It is better to describe the RO-Crate metadata file as " It contains all contextual and non-contextual related data to re-run the workflow". Instead of "It can additionally contain data on which the workflow can be run."Workflow Implementation- At the beginning of the last paragraph, "Besides the workflow and the synthetic data set" replace with "As well as the workflow and the synthetic data set".- https://workflowhub.eu/workflows/402?version=5# gives a very nice pictorial overview of the workflow that you may consider including in the paper itself.

Reviewer 2 Ryan J. Urbanowicz Revision 2

At this point I earnestly wish to see this paper published, and in acknowledging my own potential bias as a developer of STREAMLINE and participant in the development of TPOT, I am still recommending minor revision. At minimum, for me to recommend acceptance of this paper the following small but critical issue needs to be addressed, otherwise I must recommend reject. I believe this concern is well justified by scientific standards. I also still strongly recommend the authors re-consider the other non-critical issues reiterated below as a way to make their paper stronger and to be better received by the scientific community. If the journal editor disagrees with my assessment, I would still be happy to see this work published, however I must stand by my assertions below.Critical Issue:Limitations section: The authors updated the text - "excells in it's core objective of addressing classification tasks." To "it excels in its primary objective of addressing pipeline development for classification tasks.The use of the word 'excells' is the key problem, as this word is defined as "to do or be better than others". While the change in phrasing correctly no longer implies that MLme performed better than the other evaluated AutoML tools, it does still imply that it is the best in developing a pipeline for classification tasks, but no specific evidence is provided in the paper to support this assertion. I.e. there were no studies comparing how easy the tool was for users to apply than other autoML, and no detailed comparison of what pipeline elements could be included by MLme vs other autoML or pipeline development tools. The fact that MLme doesn't include hyperparameter optimization is in itself a limitation that I think would prevent MLme from being claimed as excelling or superior in pipeline development to other tools/platforms, even if it's easier to use that other tools. As phrased in the reviewer response, the authors could say that MLme is well-equipped to handle pipeline development as this would be a fair statement. All together I'd strongly encourage the authors not to make statements about the superior aspects of MLme without clearly backing up these statements with direct comparisons. Instead I'd suggest highlighting elements of MLme that are 'unique' or provide more functionality in contrast with other tools. In the reviewer response the authors make the claim that MLme is superior in terms of ease of use for visualization and exploratory analysis. If they want to make that statement in the paper backed up by accurate comparisons to other tools, I'd agree with that addition.Non-Critical Issues that I feel still should be addressed:1. Table S1 has been updated to remove the inaccuracies I previously pointed out, however this alone does not change the broader concern I had regarding the intention of this table (which is to highlight the parts of MLme that appear better than other AutoML tools without fairly pointing out the limitations of MLme in contrast with other tools). As a supplemental materials table, I do not feel this is critical, but I think making a table that more fairly reflects strengths and limitations of different tools would greatly strengthen this paper.2. The pipeline design in Figure 2 and and S10 are both high-level and still do not provide enough detail/clarity to understand exactly what happens and in what order when applying the autoML element of MLme. They key words here being transparency and reproducibility. The supplemental materials could describe a detailed walk through of what the autoML does at each step. At minimum this could also be clearly addressed in the software documentation on GitHub.3. While I understand the need for brevity, I think the addition of a sentence that indicates specifically what AutoML tools are most similar to MLme is a reasonable request that better places MLme in the context of the greater AutoML research space.

**Reviewer 2 Ryan J. Urbanowicz ** Revision 1

Overall I think the authors have made some good improvements to this paper, although it does not seem like the main body of the paper has changed much with most of the updates going into supplemental materials. However, I think this work is worthy of publication once the following items are addressed. (which I still feel strongly should be addressed, but should be fairly easy to do so).

1. Limitations section: While the authors added some basic comparisons to a few other AutoML tools, I do not see how they are justified in saying that MLme 'excells' in it's core objective of addressing classification tasks. This implies it is better performing a classification than other methods, which is not at all backed up here, and indeed would be very difficult to prove as it would require a huge amount of analyes over a broad range of simulated and real world benchmark datasets, and incomparison to many or all orther other autoML tools. At best i think the authors can say here that it is at least comparable in performance to AutoML tools (X, Y, Z) in its ability to conduct classification analyses. And according to Figure S9 this is only across 7 datasets, and focused only on the F1 score which could also be missleading or cherry picked. At best I believe the authors can say in the paper that "Initial evaluation across 7 datasets suggested that MLMe performed comparably to TPOT and Hyperopt-sklearn with respect to F1 score performance. This suggests that MLme is effective as an automated ML tool for classification tasks. " (or something similar).

2. While the authors lengthened the supplemental materials table comparing ML algorithms (mainly by adding some other autoML tools, this table is intentionally presenting the capabilities of tools in a way that make it appear like MLme does the most (with the exception of the 'features' column) . For example, what about a column to indicate if an autoML tool has an automated pipeline discovery component (like TPOT)? In terms of AutoML, this table is structured to highlight the benefits of MLme, rather than give a fair comparison of AutoML tools (which is my major concern here). In terms of AutoML performance and usability there is alot more to these different tools than the 6 columns presented. In this table 'features' seems like an afterthought, but is arguably the most important aspect of an AutoML.

3. Additionally, the information presented in the autoML comparison table does not seem to be entirely accurate, or at least how the columns are defined is not made entirely clear. Looking at STREAMLINE, which can be run by users with no coding experience (as a google colab notebook), it has a code free option (just not a GUI), STREAMLINE also generates more than two exploratory analysis plots, and more results visualizations plots than indicated). While I agree that MLme has many more ease of use functionality in comparison to STREAMLINE (which is a very nice plus), a reader might look at this table and think they need to know how to code in order to use STREAMLINE, which is not the case. Could the authors at least define their criteria for the "code free" column. As it's presented now it seems to be the same exact criteria as for GUI (in which case this is redundant). The same is true for the legend for the table where '*' indicates that coding experience is required for designing a custom pipeline. This requires more clarification, as STREAMLINE can be customized easily without coding experience by simply changing options in the Google Colab notebook, and TPOT automatically discovers new analysis pipelines which isn't reflected at all.

4. While I appreciate the authors adding a citation for STREAMLINE and some other autoML tools not previously cited, it would be nice for the authors to discuss other AutoML tools further in their main paper, as well as to acknowledge in the main paper which AutoML tools are most similar to MLme in overall design and capabilities. Based on my own review of AutoML tools the most similar tools would include STREAMLINE and MLIJAR-supervised.

5. I like the addition of Figure S10 that more clearly lays out the elements included in MLme, but I still think the paper and documentation lacks a clear and transparent walk through of exactly what happens to the data and how the analyses are conducted from start to finish when using the AutoML (at least by default). This is important to trusting what happens under the hood for reporting results, etc.

Other comments responding to author responses: * I still disagree with the authors that a dataset with up to 1500 samples or up to 5520 features could be considered large by today's standards across all research domains. Even within biomedical data, datasets up to 100K subjects are becoming common, and 'omics' datasets regularly reach hundreds of thousands to multiple millions of features. I am glad to see the authors adding a larger dataset, but i would still be cautions when making suggestions about how well MLme handles 'large' datasets without including specifics for context. However ultimately this is subjective, and not preventing me from endorsing publication. * I also disagree that MLme isn't introducing a new methodology. The steps comprising an AutoML tool can be considered in itself a new methodology, even if it is built on established components, because there are still innumerable ways to put a machine learning analysis pipeline together that adds bias, data leakage, or just yields poorer performance. Thus I also don't think it's fair to just 'assume' your method will work as well as other AutoML tools, especially when you've ran it on a limited number of datasets/problems.

Reviewer 1 Joe Greener Revision 1

The authors have adequately addressed my concerns and I believe that the manuscript is ready for publication.

** Reviewer 2 Ryan J. Urbanowicz ** Original Submission

In this paper the authors introduce MLme, a comprehensive toolkit for machine-learning driven analysis. The authors discuss the benefits and limitations of their toolkit and provide a demonstration evaluation on 6 datasets suggesting it's potential value. Overall MLme seems like a nice, easy to use tool with a good deal of potential and value. However as the developer of STREAMLINE, an AutoML toolkit with a number of very similar goals and design architecture to MLme it was very surprising to have it not referenced or compared to in this paper. My major concerns involve the limited details about what this specifically does/includes (e.g. what 16 ML algorithms are built in), as well as what seems like a limited and largely biased comparison of this toolkit's capabilities to other autoML tools (most specifically STREAMLINE which has a significant degree of similarity).- There are many other autoML tools out there that that authors have not considered in their Table S1 or referenced. Eg. MLBox, AutoWeka, H20, Devol, Auto-Keras, TransmorgriffAI, and most glaringly in for this reviewer, STREAMLINE (https://github.com/UrbsLab/STREAMLINE).- In particular, with respect to STREAMLINE (https://link.springer.com/chapter/10.1007/978-981-19-8460-0_9), there are a large number of pipeline similarities and a similar analysis mission/goals to MLme that make it extremely relevant to cite and contrast to in this manuscript as well as in Table S1. STREAMLINE has a similar focus on the end-to-end ML analysis pipeline including automated exploratory analysis, data processing, feature selection, ML modeling with 16 algorithms, evaluation, and results visualization generation, interactive visualizations, pickled output storage, etc. The first STREAMLINE paper was published March of 2023, and a preprint of that manuscript published June 2022, as well as a precursor implementation of this pipeline published as a preprint in Aug of 2020 (https://arxiv.org/abs/2008.12829). This in contrast with MLme's preprint published July of 2023. While MLme has a number of potentially nice features that STREAMLINE does not (i.e. a GUI interface, spider plots, easy color palate selection, inclusion of a dummy classifier, ability to handle multi-class classification [which is not yet available, but in development for STREAMLINE along with regression]), it lacks other potentially important features that STREAMLINE does have (i.e. automated hyperparameter optimization, basic data cleaning and feature engineering [in the newest release], collective feature selection, pickled models for later reuse, collective feature importance visualizations, a pdf analysis summary report, the ability to quickly evaluate models on new replication data, and potentially other capabilities that I can't highlight because of limited details on what MLme includes). The absence of hyperparameter optimization is a particularly problematic omission from MLme, as this a fairly critical element of any machine learning analysis pipeline.-Table S1 should be expanded to highlight a broader range of toolkit features to better highlight the strengths and weaknesses of a greater variety of methodologies. The present table seems a bit cherry picked to make MLme stand out as appearing to have more capabilities than other tools, but there are uncaptured advantages to these other approaches.-This manuscript includes no citations justifying their pipeline design choices. In particular, I'm most concerned with the author's justification of automatically including data resampling by default as it is well known that this can introduce bias in modeling. It's also not clear what determines if data resampling is required, and whether this only impacts training data or also testing data.- Its not clear that resampling is a good/reliable strategy for an automated machine learning framework since data resampling to create a more balanced dataset can also incorporate bias in to an ML model.- In the context of potential datasets from different domains (including biomedical data), the datasets identified in this paper as being "large" have only up to 1500 sample and only up to 5520 features, which would not be considered large by most data scientist standards.- There are largely limited details in this paper and the software's github documentation in terms of transparently indicating exactly what this pipeline does, and what options, algorithms, evaluation metrics, and visualizations it includes.- Since the authors do not benchmark MLme against any other autoML tool and they have a very limited set of benchmarked datasets (6 total, with limited diversity of data types, sizes, feature types), I don't think it's fair to claim that their methodology necessarily excels in it's core objective of addressing classification tasks. Ideally the authors would conduct benchmarking comparisons to STREAMLINE, as well as other autoML toolkits, however this might also understandably be outside the scope of this current paper. I do suggest the authors be more conservative in what assertions they make and conclusions they draw with respect to MLme. The authors might consider using established ML or AutoML benchmark benchmark datasets used by other algorithms and frameworks to compare or facilitate comparison of their pipeline toolkit to others.

This work has been published in GigaScience Journal under a CC-BY 4.0 license (https://doi.org/10.1093/gigascience/giad111), and has published the reviews under the same license. These are as follows.

** Reviewer 1 Joe Greener ** Original Submission

Akshay et al. present MLme, a toolkit for exploring data and automatically running machine learning (ML) models. This software could be useful for those with less experience in ML. I believe it is suitable for publication provided the following points are addressed.# Major1. The performance of models is consistently over 90% but without a reference point it is unclear how good this is. Are there results from previous studies on the same data that can be compared to, with a table comparing accuracy with MLme to previous work? Otherwise it is unclear whether MLme is supposed to be a quick way to have a first go at prediction on the data or can entirely replace manual model refinement.2. With any automated ML system it is important to impress upon users the risks of ML. For example, the splitting of data into training and test sets is done randomly, but there are cases where this is not appropriate as it will lead to data leakage between the training and test sets. This could be mentioned in the manuscript and somewhere on the GUI. There isn't really a replacement for domain knowledge, and users of MLme should have this in mind when using the software.# Minor3. More experienced ML users may want to use the software to have a first go at prediction on the data. For these users it may be useful to provide access to commands or scripts, or at least information on which functions were used, as additional options in the GUI. Users could then run these scripts themselves to tweak hyperparameters etc.4. The visualisation tab lacks an info button by the file upload to say what the file format should be.

** Reviewer 1 Liuyang Zhao ** R1 version

The manuscript presented by the authors provides a useful tool on the microbiome, which named "Vulture: Cloud-enabled scalable mining of microbial reads in public scRNA-seq data", using a large and valuable dataset. The study is important in deepening our understanding of "microbiome in public data". However, the author comments not fully address my concerned, there are some issues for improvement in the manuscript. Here are the requirements for new software that is good enough to be published: 1. A docker provided is better, however, most used install method conda is still missing. 2. The more microbial detect example is missing. Can you provide an example of using like Kraken2 full NCBI database (RefSeq) to check all the microbial is more useful. 3. Author still not promotion his software in social media. If no more people take part in use it, how can we know it's useful? The reviewers still have may work to do. Not have enough time to test this software. Just promote it in twitter and Chinese WeChat will help software better. 4. The software name should be unique, which is convenient to count the real users through all available resources (such as QIIME, ImageGP, and EasyAmplicon). However, the name vulture is unacceptable, due to millions of hits in Google scholar. Must be no hit is a unique name，OK? Otherwise, hardly to know the read number of users. 5. The source code to support the generation of individual figures in this paper will be available on the GigaDB after being published. Where to check by the reviewers?

** Reviewer 3 Liuyang Zhao ** Original submission

The authors aim to develop Cloud-enabled approaches for detecting viral reads in public single-cell RNA sequencing (scRNA-seq) data. This study makes a significant contribution to the identification of viruses and bacteria in public scRNA-seq data. Although the outcomes are satisfactory, the novelty of the proposed methods is limited. To date, no evidence has been provided to demonstrate their superiority over recently published methods (such as PathogenTrack and Venus, et al) when executed on a local machine. There are also several issues that need to be further addressed, as highlighted below: 1.The documentation available on the GitHub pipeline does not explain how to utilize the latest virus database or how to incorporate a user's custom database. Because the virus database is updated very quickly now. It might be more appropriate if the author updates the database promptly or if one can customize and create their own database. 2. Figure 2a only has an overall comparison graph, it can be improved by adding detailed comparison graphs with Cumulus, PathogenTrack and Venus. 3. Figure 2b. The persuasiveness is not enough, it would be better to compare several pipeline platforms with similar functionalities or compare some specific steps, such as the four steps in figure 2a. By the way, all of these comparisons use comparison software developed by other same researchers, so please provide a detailed description of why the author's method is faster? 4. Figure 3c can be created with microbial clustering and non-microbial clustering to highlight the impact of virus identification on classification results. 5. Fig. S1 It should be the "Quality control on read level".

** Reviewer 2 Jingzhe Jiang ** Original submission

In this study, Chen et al. introduce Vulture, a scalable cloud-based pipeline that performs microbial calling for single-cell RNA sequencing (scRNA-seq) data, enabling meta-analysis of host-microbial studies from the public domain. And they further applied Vulture to COVID-19, HCC, and gastric cancer human patient cohorts with public sequencing reads dataand discovered cell-type specific enrichment of SARSCoV2, hepatitis B virus (HBV), and H. pylori positive cells. Generally speaking, this study is innovative, has good application potential, and can better assist the work of single cell research from the point of view of infection. I only a few minor questions that need the author to reply: 1. Background: The first appearance of H. pylori should be replaced with its full name. 2. Methods-Downstream analysis of scRNA-seq samples: Why use different tools (SCANPY/Seurat, BBKNN/Harmony) to analyze different datasets instead of using the same tool to analyze different datasets? 3. Cell-type enrichment of microbial UMI: format error of formula. 4. Analyses-Page 11: "The statistical test identified that SARS-CoV-2 is enriched (p-value < 0.05) in epithelial cells, neutrophils, and plasma B cells (Fig. 3d and Table. 2)". It is best to highlight p < 0.05 data points in other colors rather than red squares. Why are there no p < 0.05 square in fig. 3e? 5. Fig. 2a and 2b: There are 8 colors in figure 2a, however only 4 figure legend were showed. What do the four light-colored bar mean? And the same to Fig 2b.

This work has been published in GigaScience Journal under a CC-BY 4.0 license (https://doi.org/10.1093/gigascience/giad117), and has published the reviews under the same license. These are as follows.

** Reviewer 1 Yongxin Liu** Original submission

The manuscript presented by the authors provides a useful tool on the virome, which named "Vulture: Cloud-enabled scalable mining of viral reads in public scRNA-seq data", using a large and valuable dataset. The study is important in deepening our understanding of "virome in public data". However, there are some issues for improvement in the manuscript. Here are the requirements for new software that is good enough to be published: Major comments: 1. The software, tested data and results are required to be uploaded on GitHub for peers to use, and conda and/or docker installation modes are recommended for software with complex dependencies. We will take software Star, Fork, and downloads of GitHub as one of the audience indicators. I found the GitHub links: https://github.com/holab-hku/Vulture. However, the readme.md show pipeline on AWS cloud. If I not have an AWS, how can I run it in my server. Now this project is only 2 stars. You need more people to take part in and interest in this project. 2. Software installation and User tutorial are required in Readme.md or Wiki in GitHub. Please provide step by step protocol to deploy it in the laptop or server. 3. A video of software download, installation, operation, and result display is required with a computer or server without any related software installed, to make sure that any new user can perform the whole process according to the tutorial. 4. The software is required to be posted on twitter and other social media, you can contact @ iMetaScience, @microbe_article etc. to get help in tweet or retweet. The number of Retweet, Like and View as one of the audience indicators. 5. Chinese is largest single langue science society. Provide the Chinese tutorial and video presentation of the software, contact meta-genome Official account for help to promote. The Number of readers, share and favorite also one of the audience indicators. 6. According to the feedback from users in all over the world, the author continuously maintains and optimizes the method to ensure its availability, ease of use and advancement. 7. The software name should be unique, which is convenient to count the real users through all available resources (such as QIIME, ImageGP, and EasyAmplicon). However, the name vulture is unacceptable, due to million of hits in Google scholar. 8. The figures in your papers are diversity. However, I cannot find enough visualization function in your pipeline. The pipeline for integrated software is easy, the specific and diversity visualization plan is difficult. All the authors want their analysis result is ready-to-published. 9. Why only focus on the virus? Can this pipeline to generated all the microbiome, which is more interest and overview of the microbes.

Reviewer 2--Julia Voelker

The manuscript about NLR-type resistance genes in two haplotypes of Melaleuca quinquenervia is a relevant contribution to the research of Myrtaceae genomes and other long-lived trees. The methods are well described and should be reproducible with the available information and raw data, provided the authors mentioned all non-default settings in the method section. The FindPlantNLRs pipeline seems to be well documented on github.

I believe that this manuscript is ready for publication after some small changes. Page and line numbers in the comments below refer to the PDF document: 1. The quality of some figures is not good (even upon download and zoom into the plot) and should be improved to higher resolution for publication. Especially in figure 3, all labels are too pixelated and hard to read. I would also recommend an increase in text size for this figure. In Figure 6 D & E, the authors should consider using consistent text sizes on the axes, and even though the quality is acceptable, a higher resolution of the labels would still be better.

1. p. 10, Table 2: Although it is a standard statistic for genome assemblies, it would be helpful for some readers to specify what N50 and L50 are.

2. p. 19, line 436: I believe the authors are referring to the wrong figure number.

Below are some additional comments regarding typos or other language issues. While the text is generally well written, I would appreciate commas in certain sentences to improve readability, and think that some nouns are missing articles. I hope the authors will read through their text again and add articles where required, I won't point them out individually.

p.4, line 33: wide range of p.7, line 130: 'a' instead of 8? p. 8, line 177: genome p.12, line 250: chromosome 2, add comma before 'while' in next line p.12, line 253: on all other chromosomes? p. 13, line 271: to occur? p.16, line 347: remove 'and' p.17, line 382, 384: orthologs? p.20, line 469: 'lead to the triggering of defence response' rephrase to make sense with the previous half of the sentence, also, defence response should have an article p.20, line 489/490: missing word?

Reviewer 1– Andrew Read – University of Minnesota

In the manuscript, A high-quality pseudo-phased genome for Melaleuca quinquenervia shows allelic diversity of NLR-type resistance genes, the authors assemble and analyze a phased genome of a long-lived tree species. In addition to providing a phased genomic resource for an important species, the authors analyze and compare the NLR gene complement in each of the two diploid genomes. I was surprised by the level of diversity of NLR genes in the two copies of the genome (this may be due to my biases based on working in highly homozygous species). This level of within-individual diversity has been largely overlooked by researchers owing to the difficulties of sequencing, assembly, and NLR identification. To address NLR identification, the authors publish a very nice pipeline that combines available tools into a framework that makes a lot of sense to me and will be valuable to anyone doing NLR gene work on new or existing genome assemblies. My main concern comes from not knowing how sequencing gaps and NLRs correlate across the two diploid genomes. Other than this, I think it’s a very nice paper that adds to the growing catalog of NLR gene diversity by tackling the challenge of NLRs in a heterozygous genome.

Many of the authors’ interesting observations are based on comparisons of NLRs on the two haploid genomes, however some things are not clear to me:
1.  Do any predicted NLR-genes overlap gaps in the alternative haploid genome? 
2.  If there is a predicted NLR-gene in one haploid genome and not the alternative genome, what is at the locus? Is it a structural variant indicating insertion/deletion of the NLR or is there ‘NLR-like’ sequence there that just didn’t pass the pipeline filters indicating an NLR fossil (or similar) – to me this is an important distinction.
3.  How many of the NLR-genes on the two haploid genomes cluster 1:1 with their homolog on the alternative haploid genome – I’m particularly interested in the 15 ‘mismatched’ N-term-NBARC examples. It would be nice to know if these have partners in the alternative haploid genome, and if the partner has the same mismatch (if not, it would support the proposed domain swapping story)
I believe each of these concerns will require whole genome alignment of the two haploid genomes.

Additional comments (by line where indicated) The authors introduce the idea that M. quinquenervia is invasive in Florida, but this thread is never followed up on in the discussion and makes it feel a bit awkward. It would help if the authors clarified how the genome could help with management in native and invasive ranges

Could the authors add some context for why ONT data was included and how it was used?

It would be helpful if the authors provided a weblink to the iTOL tree

164-166 – The observation of inversions potentially caused by assembly errors is nice!

206 – add reference: Bayer PE, Edwards D, Batley J (2018) Bias in resistance gene prediction due to repeat masking. Nat Plants 4: 762–765. pmid:30287950

240-246 – I’m not sure about excluding these incomplete NLRs – it would be interesting and potentially informative to see where they cluster (do they cluster with an NLR from the alternative haplotype? If so it may indicate truncation of one copy, etc) – however, if the author’s wish to remove these at this step I think they can add a statement like “we were interested in full-length NLRs, the filtered incomplete NLRs may represent….”

429-430 – The criteria used to define clusters is described in the methods, can you confirm (and mention) that this is the same as used in the analyses you’re comparing to for E. grandis, rice, and Arabidopsis.

435-437 – I’m interested to know if the four heterogenous clusters contain any of the N-term domain-swapped NLRs

479-480 – The zf-BED domain is also present in rice NLRs – include citation for Xa1/Xo1

523-524 – can you specify which base-call model was used on the ONT data?

I’m curious about the presence/absence of IDs in the analyzed NLRs and would be very curious to know if the authors observe syntenic homologs across the two haploid genomes with ID presence/absence or presence of different IDs polymorphisms.

Nanopublication: RAOk_Yih3v "Organism of Elaphe carinata (species) - observed nucleotide sequence - SRX20564100" https://w3id.org/np/RAOk_Yih3v2q9s4LMZsy1v-qEhZ5ZGceChnl5h-godB2M

Reviewer 2. Luca Ermini

This manuscript, which I had the pleasure of reading, is, simply put, a benchmark of five long read de novo assembly tools. Using 13 real and 72 simulated datasets, the manuscript evaluated the performance of five widely used long-read de novo assemblers: Canu, Flye, Miniasm, Raven, and Redbean.

Although I find the methodological approach of the manuscript to be solid and trustworthy, I do not think the research is particularly innovative. Long-read assemblers have already been benchmarked in the scientific literature, and similar findings have been made. The authors are aware of this limitation of the study and have added a novel feature: the impact of read length on assembly quality, which in my opinion is still lacking sufficient innovation. However, the manuscript as a whole is valid and worthy of consideration. In light of this, I would like to share some suggestions I made in an effort to make the manuscript unique and more novel.

Please see my comment below.

1) Evaluation of the assemblies The metrics used to evaluate an assembly are frequently a murky subject as we are still lacking a standard language. The authors assessed the assemblies using three types of metrics: compass analysis, assembly statistics, and the Busco assessment, in addition to computational metrics like runtime and RAM usage. This is not incorrect, but I would suggest making a clear distinction between the metrics using (in addition to the computational metrics) three widely recognised metrics, or in short, the 3C criterion. The assembly metrics can be broken down into three dimensions: correctness (your compass analysis), contiguity (NG50) and completeness (the BUSCO assessment). The authors should reconsider the text using the 3C criterion; this will provide a clear, understandable, and structured way of categorising metrics. The paragraph beginning at line 197, for example, causes some confusion for the reader. The NG50 metrics evaluate assembly contiguity, whereas the number of misassemblies (considered by the authors in terms of relocation, inversion, and translocation) evaluate assembly correctness. I must admit that the misassemblies and contiguity can overlap, but I would still recommend keeping the NG50 (within contiguity) and misassemblies (within correctness) metrics separate.

2) Novelty of the comparison The authors of the study had two main goals: to conduct a systematic comparison of five long-read assembly tools (Raven, Flye, Wtdbg2 or Redbean, Canu, and Miniasm) and to determine whether increased read length has a positive effect on overall assembly quality. The authors acknowledge the study's limitations and include an evaluation of the effect of read length on assembly quality as a novel feature of the manuscript (see line 70).

The manuscript that described the Raven assembler (Vaser, R., Sikic, M. Time- and memory-efficient genome assembly with Raven. Nat Comput Sci 1, 332-336 (2021)) compared the same assemblers' tools (Raven, Flye, Wtdbg2 or Redbean, Canu and Miniasm) evaluated in this manuscript plus two more (Ra and Shasta), used similar eukaryotes (A. thaliana, D. melanogaster, and Human), and reached a similar conclusion on Flye in terms of contiguity (NG50), and completeness (genome fraction) but overall there is not a best assembler in all of the evaluated categories. In this manuscript authors increased the number of eukaryotic genomes (including S. cerevisiae, C. elegans, T. rupribes, and P. falciparum) and reached similar conclusions: there is no assembler that performs the best in all the evaluation categories, but overall Flye is the best-performing assembler. This strengthens the manuscript, but the research is not entirely novel.

Given that the field of third-generation technologies is rapidly progressing toward the generation of high-quality reads (Pacbio HiFi technology and ONT Q20+ chemistry are achieving accuracy of 99% and higher), the manuscript should also include a HiFi assembler benchmark. This would add novelty to the manuscript and pique the scientific community's interest. The authors have already simulated HiFi reads from S. cerevisiae, P. falciparum, C. elegans, A. thaliana, D. melanogaster, T. rubripes in addition to reference reads (or real reads) from S. cerevisiae (SRR18210286). P. falciparum (SRR13050273) and A. thaliana (SRR14728885).

Furthermore, I am not sure what the benefit is of evaluating Canu on HiFi data instead of HiCanu, which was designed to deal with HiFi data. The authors already included some HiFi-enabled assemblers like Flye and Wtdbg2 but also HiFiasm should also be considered. I would strongly advise benchmarking the HiFi assemblers to complete the study and add a level of novelty. I would like to emphasise that the manuscript is solid and that I appreciate it; however, I believe that some novelty should be added.

3) C elegans genomics The now-discontinued RSII, which had a higher error rate and a shorter average read than Sequel I or Sequel II, was used to generate the genomic data from C elegans. I understand the authors' motivation for including it in the analysis, but the use of RSII may skew the comparisons, and I would suggest adding a few sentences to the discussion about it.

4) CPU time (h) and memory usage The authors claim the benchmark evaluation included runtime and RAM usage. However, I missed finding information about the runtime and RAM usage. Please provide CPU time (h) and memory usage (GB)

Minor comments:

1) Lines 64-65 "Here, we provide a comprehensive comparison on de novo assembly tools on all TGS technologies and 7 different eukaryotic genomes, to complement the study of Wick and Holt" I would modify "on all TGS technologies" as "at the present the two main TGS technologies"

2) Line 163 Real reads. The term "real reads" may cause confusion for readers, leading them to believe that the authors produced the sequencing reads for the manuscript. I would use the term "ref-reads" indicating "reads from the reference genomes"

3) Lines 218-219 Please provide full names (genus + species): S. cerevisiae, P. falciparum, A. thaliana, D. melanogaster, C. elegans, and T. rubripes

4) Supplementary Table S4 "Accession number SRR15720446 seems to belong to a sample sequenced with 1 PACBIO_SMRT (Sequel II) rather than ONT

5) Figures 2 and 3. Figures 2 and 3 give visual results of the performance of the five assemblers. I want to make a few points here: According to what I understand, the top-performing assembler is marked with a star and is plotted with a brighter colour than the others. However, this is not immediately apparent, and some readers might have trouble identifying the colour that has been highlighted. I would suggest either lessening the intensity of the other, lower-performance assemblers or giving the best assembler a graphically distinct outline. I also wonder if it would be useful to give the exact numbers as supplemental tables.

Re-Review:

Dear Cosma and colleagues, Thank you so much for addressing my comments in a satisfactory manner. The manuscript, in my opinion, has dramatically improved.

This work has been published in GigaScience Journal under a CC-BY 4.0 license (https://doi.org/10.1093/gigascience/giad100), and has published the reviews under the same license. These are as follows.

**Reviewer 1. Brandon Pickett **

Overall, this manuscript is well-written and understandable. There's a lot of good work here and I think the authors were thoughtful about how to compare the resulting assemblies. Scripts and models used have been made available for free via GitHub and could be mirrored on or moved to GigaDB if required. I'll include a several minor comments, including some line-item edits, but the bulk of my comments will focus on a few major items.

Major Comments: My primary concern here is that the comparison is outdated and doesn't address some of the most helpful questions. CLR-only assemblies are no longer state-of-the-art. There are still applications and situations where ONT (simplex, older-pore)-only assemblies are reasonable, but most projects that are serious about generating excellent assemblies as references are unlikely to take that approach.

Generating assemblies for non-reference situations, especially when the sequencing is done "in the field" (e.g., using a MinION with a laptop) or by a group with insufficient funding or other access to PromethIONs and Sequel/Revios, is an exception to this for ONT-only assemblies. Further, this work assumes a person wants to generate "squashed" assemblies instead of haplotype-resolved or pseudohaplotype assemblies. To be fair, sequencing technology in the TGS space has been advancing so rapidly that it is extremely difficult to keep up, and a sequencing run is often outdated by the time analyses are finished, not to mention by the time a manuscript is written, reviewed, and published.

Accordingly, in raising my concerns, I am not objecting to the analysis being published or suggesting that the work performed was poor, but I do believe clarifications and discussion are necessary to contextualize the comparison and specify what is missing.

1. This comparison seeks to address Third-generation sequencing technologies: namely PacBio vs. ONT. However, each company offers multiple kinds of long-read sequencing, and they are not all comparable in the same way. Just as long noisy reads (PacBio CLR & ONT simplex) are a whole new generation from "NGS" short reads like from Illumina, long-accurate reads are arguably a new generation beyond noisy long reads. If this paper wants to include PacBio HiFi reads in the comparison, significant changes are necessary to make the comparison meaningful. I think it's reasonable to drop HiFi reads from this paper altogether and focus on noisy long reads since the existing comparison isn't currently set up to tell us enough about HiFi reads and including them would be an ordeal. If including HiFi, consider the following:

1.a. Use assemblers designed for long-accurate reads. HiCanu (i.e., Canu with --pacbio-hifi option) is already used, as is a similar approach for Flye and wtdbg2. However, raven is not meant for HiFi data and miniasm is not either (though, it could be done with the correct minimap2 settings, but Hifiasm would be better). Assemblies of HiFi data with Raven and miniasm should be removed. Sidenote – Raven can be run with --weaken (or similar) for HiFi data, but it is only experimental and the parameter has since been removed. Including Hifiasm would be necessary, and it should have been included since Hifiasm was out when this analysis was done. Similarly, including MBG (released before your analysis was done) would be appropriate. Since you'd be redoing the analyses, it would be appropriate to include other assemblers that have since been released: namely LJA. Once could argue that Verkko should be included, but that opens another can of worms as a hybrid assembler (more on that later).

1b. Use a read simulator that is built for HiFi reads. Badreads is not built for HiFi data (though using custom parameters to make it work for HiFi reads wasn't a bad idea at the time), and new simulators (e.g., PBSIM3, DOI: 10.1093/nargab/lqac092) have since been released that consider the multi-pass process used to generate HiFi data.

1c. ONT Duplex data is likely not available for the species you've chosen as it is a very new technology. However, you should at least discuss its existence as something for readers to "keep an eye on" as something that is conceptually comparable to HiFi. 1d. Use the latest & greatest HiFi data if possible and at least discuss the evolution of HiFi data. Even better would be to compare HiFi data over time, but this data may not really be available and most people won't be using older HiFi data. Though, simulation of older data would conceivably be possible. While doing so would make this paper more complete, I would argue that it isn't worth the effort at this juncture. For reference, in my observation, older data has a median read length around 10-15 kb instead of 18-22 kb. 1e. Include real Hifi data for the species you are assembling. If none is available and you aren't in a position to generate it, then keep the hifi assembler comparison on real data separate from that of the CLR/ONT assembler comparisons on real data by using real HiFi data for other species. 2. Discuss in the intro and/or discussion that you are focusing on "squashed" assemblies. Without clever sample separation and/or trio-based approaches (e.g., DOI: 10.1038/nbt.4277), a single squashed haplotype is the only possible outcome for PacBio CLR and ONT-only approaches. For non-haploid genomes, other approaches (HiFi-only or hybrid approaches (e.g., HiFi + ONT or HiFi + Hi-C)) can generate pseudohaplotypes at worse and fully-resolved haplotypes at best. The latter is an objectively better option when possible, and it's important to note that this comparison wouldn't apply when planning a project with such goals. Similarly, it would probably be helpful to point out to the novice reader that this comparison doesn't apply to metagenome assembly either. 3. The title suggests to the reader that we'll be shown how long reads makes a difference in assembly compared to non-long read approaches. However, this is not the case, despite some mention of it in near line 318. Short read assemblies are not compared here and no discussion is provided to suggest how long read-based assemblies would improve outcomes in various situations relative to short reads. Unless such a comparison and/or discussion is added, I think the title should be changed. I've included this point in the "Major Comments" section because including such a comparison would be a big overhaul, but I don't expect this to be done. The core concern is that the analysis is portrayed correctly. 4. Sequencing technologies are often portrayed as static through time, but this is not accurate. This is a failing of the field generally. Part of the problem is the length of the publishing cycle (often >1yr from when a paper is written to when it's published, not to mention how long it takes to do the analysis before a paper is even written). Part of the problem is that current statistics are often cited in influential papers and then recited in more recent papers based on the influential paper despite changes having been made since that influential paper was released. Accordingly, the error rate in ONT reads has been misreported as being ~15% for many years even though their chemistry has improved over time and the machine learning models (especially for human samples) have also improved, dropping the error rate substantially. ONT has made improvements to their chemistry and changed nanopores over time and PacBio has tinkered with their polymerase and chemistry too. Accordingly, a better question for a person planning to perform an assembly would be "which assembler is best for my datatype (pacbio clr vs ont) and chemistry/etc.?" instead of just differentiating by company. Any comparison of those datatypes should at least address this as a factor in their discussion, if not directly in their analysis. I feel that this is missing from this comparison. In an ideal world, we'd have various CLR chemistries and ONT pores/etc. for each species in this analysis. That data likely doesn't exist for each of the chosen species though, and generating it would be non-trivial, especially retroactively. Using the most recent versions is a good option, but may also not exist for every species chosen. Since this analysis was started (circa Nov/Dec 2021 by my estimate based on the chosen assembler versions), ONT has released pore 10; in combination with the most recent release of Guppy, error rates <=3% are expected for a huge portion of the data. That type of data is likely to assemble very differently from R9.4, and starker differences would be expected for data older than R9.4. Even if all the data were the most recent (or from the same generation (e.g., R9.4)), library preps vary greatly, especially between UL (ultralong) libraries and non-UL libraries. Having reads >100kb, especially a large number of them, makes a big difference in assembly outcome in my observation. How does choice of assembler (and possibly different parameters) affect the assembly when UL data is included? How is that different from non-UL data? What about UL data at different percentages of the reads being considered UL? A paper focusing on long noisy reads would be much more impactful if it addresses these questions. Again, this may not be possible for this particular paper considering what's already been done and the available funding, and I think that's okay. However, these issues need to addressed in the discussion as open questions and suggested future work. The type of CLR and ONT data also needs to be specified in this work, e.g., in a supplemental table, and if the various datasets are not from the same types, these differences need to be acknowledged. At a minimum, I think the following data points should be included: chemistry/pore information (e.g., R9.4 for ONT or P2/C5 for PacBio), basecaller (e.g., guppy vX.Y.Z), and read length distribution info (e.g., mean, st. dev., median, %>100kb), ideally a plot of the distribution in addition to summary values. I also understand that these data were generated previously by others, and this information should theoretically be available from their original publications, which are hopefully accessible via the INSDC records associated with the provided accessions. The objective here is making the information easily accessible to the readers of this paper because those could be confounding variables in the analysis.

1. This comparison considered only a single coverage level (30x). That's not an unreasonable shortcut, but it certainly leaves a lot of room for differences between assemblers. If the objective the paper is to help future project planners decide what assembler to use, it would be most helpful if they had an idea of what coverage they can use and still succeed. That's especially true for projects that don't have a lot of funding or aren't planning to make a near-perfect reference genome (which would likely spend the money on high coverage of multiple datatypes). It would be helpful to include some discussion about how these results may be different at much lower (e.g., 2x or 10x coverage) or at higher coverage (e.g., 50x, 70x, etc.) and/or provide some justification from another study for why including that kind of comparison would be unlikely to be worthwhile for this study, even if project planners should consider those factors when developing their budget and objectives.
2. Figure 2 and 3 include a lot of information, and I generally like how they look and that they provide a quick overview. I believe two things are missing that will improve either the assessment or the presentation of the information, and I think one change will also improve things. 6a. I think metrics from Merqury (DOI: 10.1186/s13059-020-02134-9) should be included where possible. Specifically, the k-mer completeness (recovery rate) and reference-free QV estimate (#s 1 and 3 from https://github.com/marbl/merqury/wiki/2.-Overall-k-mer-evaluation). Generally these are meant to be done from data of the same individual. However, most of the species selected for this comparison are highly homozygous strains that should have Illumina data available, and thus having the data come from not the exact some individual will likely be okay. This can serve as another source of validation. If such a dataset is not available for 1 or more of these species, then specify in the text that it wasn't available, and thus such an evaluation wasn't possible. If it's not possible to add one or both of these metrics to the figures (2 & 3), that's fine, but having it as a separate figure would still be helpful. I find these values to be some of the most informative for the quality of an assembly. 6b. It's not strictly necessary, so this might be more of a minor comment, but I found that I wanted to view individual plots for each metric. Perhaps including such plots in the supplement would help (e.g., 6 sets of plots similar to figure 4 with color based on assembler, grouping based on species, and opacity based on datatype). The specifics aren't critical, I just found it hard to get more than a very general idea from the main figures and wanted something easy to digest for each metric. 6c. Using N50/NG50 for a measure of contiguity is an outdated and often misleading approach. Unfortunately, it's become such common practice that many people feel obligated to include it or use it. Instead, the auN (auNG) would be a better choice for contiguity: https://lh3.github.io/2020/04/08/a-new-metric-on-assembly-contiguity.
3. This paper focuses on assembly and intentionally does not consider polishing (line 176), which I think is a reasonable choice. It also does not consider scaffolding or hybrid assembly approaches (again, reasonable choices). In the case of hybrid assembly options, most weren't available when this analysis was done (short read + long read assemblers were available, but I think it's perfectly reasonable to not have included those). Given the frequency of scaffolding (especially with Hi-C data [DOIs:10.1371/journal.pcbi.1007273 & 10.1093/bioinformatics/btac808]) and the recent shift to hybrid assemblers (e.g., phasing HiFi-based string graphs using Hi-C data to get haplotype resolved diploid assemblies (albeit with some switch errors) [DOI: 10.1038/s41587-022-01261-x] or resolving HiFi-based minimizer de bruijn graphs using ONT data and parental Illumina data to get complete, T2T diploid assemblies [DOI: 10.1038/s41587-023-01662-6]), I think it would be appropriate to briefly mention these methods so the novice reader will know that this benchmark does not apply to hybrid approaches or post-assembly genome finishing. This is a minor change, but I included it in this section because it matches the general theme of ensuring the scope of this benchmark is clear.

Minor Comments: 1. line 25 in the abstract. Change Redbean to wtdbg2 for consistency with the rest of the manuscript.

1. "de novo" should be italicized. It is done correctly in some places but not in others.

2. line 64. "all TGS technologies": I would argue that this isn't quite true. ONT Duplex isn't included here even though Duplex likely didn't exist when you did this work. Also, see the major comments concerning whether TGS should include HiFi and Duplex.

3. Table 1. Read length distributions vary dramatically by technology and library prep. E.g., HiFi is often a very tight distribution about the mean because of size selection. Including the median in the table would be helpful, but more importantly, I would like to see read-length distribution plots in the supplement for (a) the real data used to generate the initial iteration models and (b) the real data from each species.

4. line 166 "fair comparison". I'm not sure that a fair comparison should be the goal, but having them at the same coverage level makes them more comparable which is helpful. Maybe rephrase to indicate that keeping them at the same coverage level reduces potentially confounding variables when comparing between the real and simulated datasets.

5. line 169. Citation 18 is used for Canu, which is appropriate but incomplete. The citation for HiCanu should also be included here: DOI: 10.1101/gr.263566.120.

6. line 169. State that these were the most current releases of the various assemblers at the time that this analysis was started. Presumably, that was Nov/Dec 2021. Since then, Raven has gone from v1.7.0->1.8.1 and Flye has gone from v2.9->2.9.1.

7. line 175. Table S6 is mentioned here, but S5 has not yet been mentioned. S5 is mentioned for the first time on line 196. These two supp tables' numbers should be swapped.

8. There is inconsistent use of the Oxford comma. I noticed is missing multiple times, e.g., lines 191, 208, 259, & 342.

9. line 193. The comma at the end of the line (after "tools") should be removed. Alternatively, keep the comma but add a subject to the next clause to make it an independent clause (e.g., "...assembly tools, and they were computed...").

10. line 237. The N50 of the reference is being used here. You provide accessions for the references used, but most people will not go look those up (which is reasonable). The sequences in a reference can vary greatly in their lengths, even within the same species, because which sequences are included in the reference are not standardized. Even the size difference between a homogametic and heterogametic reference can be non-trivial. Which are included in the reference, and more importantly included in your N50 value, can significantly change the outcome and may bias results if these are not done consistently between the included species. It would be helpful if here or somewhere (e.g., in some supplemental text or a table) the contents of these references was somehow summarized. In addition to 1 copy of each of the expected autosomes, were any of the following included: (a) one or two sex chromosomes if applicable, (b) mitochondrial, chloroplast, or other organelle sequences, (c) alternate sequences (i.e., another copy of an allele of some sequence included elsewhere), (d) unplaced sequence from the 1st copy, (e) unplaced sequence from subsequent copies, and (f) vectors (e.g., EBV used when transforming a cell line)?

11. Supplemental tables. Some cells are uncolored, and other cells are colored red or blue with varying shading. I didn't notice a legend or description of what the coloring and shading was supposed to mean. Please include this either with each table or at the beginning of the supplemental section that includes these tables and state that it applies to all tables #-#.

12. Supplemental table S3. It was not clear to me that you created your own model for the hifi data (pacbio_hifi_human2022). I was really confused when I couldn't find that model in the GitHub repo for Badreads. In the caption for this table or in the text somewhere, please make it more explicit that you created this yourself instead of using an existing model.

**Reviewer 2. Luke Carroll **

The paper applies machine learning to publicly available proteomics data sets and assesses the ability to transfer learning algorithms between projects. The primary aim of these algorithms appears to be an attempt to increase consistency of retention time prediction for data-dependent acquisition data sets, however this is not explicitly stated within the text. The application of machine learning to derive insight from previous performed proteomics experienced is a worthwhile exercise.

1. The authors report ΔRT to determine fitting for their models. It would be interesting to see whether the models had other metrics used to assess fitting, or could be used to increase number of identifications within sample sets, and whether this was successful. ALternatively, was there any conclusions able to be drawn about peptide structure and RT determination from these models?

2. Project specific libraries are well known to improve results compared with publicly available databases, and the discussion on this point should be developed further through comparison of this work with other papers - particularly with advances in machine learning and neural networks in the data independent analysis field.

3. Comparison of Q-Exactiv models vs Orbitraps appears to be somewhat redundant, and possible a result of poor meta-data as Q-Exactiv instruments are orbitrap mass spectrometers. A more interesting comparison to make here would be between orbitrap and TOF instruments, though as the datasets have all been processed through MaxQuant, it is likely the vast majority were acquired on orbitrap instruments.

4. The paper uses ΔRT as the readout for all models tested, however the only chromatography variable considered in testing the models is gradient length. However, chromatography is also dependent on column chemistry, column dimensions, composition of buffer, use of traps, temperature etc. These are also likely to be contributing the variance observed between the PT datasets where these variables will be consistent and publicly available datasets. These factors are also likely to play a role in higher uncertainty for early and late eluting peptides where these factors are likely to vary most between sample sets. The metadata may not be available to use to compare within the data sets selected, so the authors should at minimum make discussion around these points.

5. Sample preparation is likely to have similar effects, and as the PT datasets are generated synthetically using ideal peptides, publicly available datasets will be generated from complex sample mixtures, and have increased variance due to inefficiencies of digestion, sample clean up and matrix effects. Previous studies on variance have also described sample preparation as the highest cause of variance. This needs further discussion

6. While the isolation windows of the m/z will lead to unobserved space, search engines setting will also apply here. From the text, it appears that the only spectra that were considered were those already identified in a search program (due to having Andromeda cut-off scores always apply). Typical setting for a database search will have a cut off of peptide sequences of at least 7 residues, making peptide masses appearing lower than 350 m/z unlikely. There is also significant amount of noise below 350 m/z and this also likely contributes to poorer fitting.

7. The authors identify differences in MSMS spectral features, however, most of these points are well known in the field. The authors should develop the discussion on the causes of the differences in fragmentation, as CID low mass drop off is expected, and the change in profile is expected with increasing activation energies. A more developed analysis could exclude precursor masses from these plots and focus solely on fragment ions generated.

8. The authors highlight that internal fragmentation of peptides could be used as a valuable resource to implement in machine learning. There has already been some success using these fragmentation patterns for sequence identification within both top-down and bottom up proteomic searches that the authors should consider discussing. However, these data do not appear to be incorporated into the machine learning models in this paper - or at least seem not to play a significant role in prediction, and this section appears to be a bit out of place.

Re-Review The changes and additions to the discussion for the paper address the key points, and have addressed some of the limitations imposed by the availability and ability to extract certain data elements particularly around sample preparation and LC settings. I think this strengthens their manuscript, and provides a more wholistic discussion of factor in the experimental setup.

This work has been published in GigaScience Journal under a CC-BY 4.0 license (https://doi.org/10.1093/gigascience/giad096), and has published the reviews under the same license. These are as follows.

**Reviewer 1: Juntao Li **

This paper aimed to facilitate machine learning efforts in mass spectrometry data by conducting a systematic analysis of the potential sources of variance in public mass spectrometry repositories. This paper examined how these factors affect machine learning performance and performed a comprehensive transfer learning to evaluate the benefits of current best practice methods in the field for transfer learning. Although the experimental content is extensive and provides promising results, some major points need to be addressed as follows:

1.Please explain the rationality of the RT used for evaluating model performance. In addition, it is necessary to increase other evaluation metrics to provide a more powerful comparison of model performance.

2.The curves in Figures 6 and 8 should provide more explanations to help readers understand. In addition, all figures are somewhat blurry and clearer figures should be provided.

3.This paper does not provide specific implementation steps of variance. Please describe the variance analysis process in mathematical language and provide the corresponding mathematical formula.

4.There are some formatting issues: Keywords and the title 'Data Description' should only have the first letter capitalized. On pages 6, 17, and 18, the font size of the article is inconsistent.

5.There are some grammar issues: On pages 6 and 16, dataset should be added with 's'. On page 7, lines 9-10, the tense is not unified.

6.There are significant issues with the format of references. Inconsistent capitalization of initial letters in literature titles, such as [1] and [5]; Some literature lacks page numbers, such as [6] and [18]. Please re- organize the references according to the format required by the journal.

Re-Review:

I am glad to see that the authors have revised the manuscript based on the reviewer's comments and improved its quality. However, the responses to some comments did not fully convince me. I suggest the authors further revise or explain the following issues.

1. I agree the rationality of ΔRT as a performance measure, but does not agree with the author's viewpoint of 'However, as the model performance indicates metric variance, and there are no changes to the conclusions drawn from the model performance'. I suggest the authors truthfully provide other classic machine learning performance metrics on the test dataset and analyze the differences.

2. In order to avoid randomness caused by single data partitioning (training and testing data partitioning), multiple random data partitioning strategie (100 or 50 times) is usually adopted to evaluate the performance of learners using multiple average performance measures and variance. It is recommended that the authors consider this issue.

3. The structure and references of the papers that I have seen that have been officially published in GigaScience are very different from the manuscript (the author has claimed to have organized and written according to the requirements). I am not sure if it was my mistake or the authors' mistake. I suggest the authors confirm the issue again and improve the writing.

**Reviewer 2: Katharina Scherf ** General comments This paper is a very thorough report on large-scale proteomics mapping of ca. 4000 wheat samples and several challenges related to sample preparation, measurement and data analysis. It is the first paper reporting such an extensive dataset and tools for analysis. Overall, I think that the authors have done in-depth work and it is also described in a way that can be understood well. The descriptions of how the authors arrived at the final workflow will also be useful to other groups attempting to do proteomics of wheat or other grains. I have only few comments for improvement. Note: line numbers would have been helpful

Specific comments Abstract - Results: "LMA expression greatly impacted grain starch and other carbohydrates …" and then alpha-gliadins and LMW glutenin is mentioned. However, these are proteins and their relation to starch/carbohydrates is not clear.

Introduction overall: Please harmonize the use of alpha-amylase and a-amylase; alpha-amylase is recommended, or else the Greek letter.

p3, L1: "great source of protein": In terms of quantity, this is true. However, you should also include a brief statement about protein quality, which is not ideal, especially when considering gluten proteins

section 2.1: Please include if all samples were grown together at the same place in one year (or not); i.e. include the information from section 3.1.1 already here.

This work has been published in GigaScience Journal under a CC-BY 4.0 license (https://doi.org/10.1093/gigascience/giad084), and has published the reviews under the same license. These are as follows.

**Reviewer 1: Nobuaki Takemori **

The large proteome dataset for wheat, a representative grain, presented in this manuscript is valuable not only for agriculture science but also for basic plant science, but unfortunately, the manuscript is too wordy in its description and informative. Of course, a detailed description of the experimental methods and data generation process is an important component in obtaining reproducibility, but excessive information in the main text may have the unintended effect of hindering the reader's understanding of the manuscript. The volume of the main text in this manuscript should be reduced to 1/2 or even 1/3 of the original by referring to the following suggested revisions.

Title: It looks rather like the title of a review article and is not appropriate for the title of an original research paper. An abbreviation is also used, making it difficult to understand. It should be changed to a title that more specifically and pragmatically reflects the content of the paper.

Materials and Methods 2.3: The sample pretreatment used in this experiment has already been described in Ref. 41, so detailed description in this text is unnecessary. Also, Figure 1, which visualizes the experimental process, is too packed with information and is difficult to read in its small font. In addition, many extraneous photographs of LC-MS instruments and other common equipment are included. Sample pretreatment should be described very briefly in the text, and only those areas where there are differences from previous reports should be mentioned. If the author wishes to describe the details of the experiment to assure reproducibility, it is recommended to describe it in the form of an experimental protocol and include it in the Supplementary Information.

Materials and Methods 2.5: The 11 different paths the authors have set up for LC-MS/MS analysis are difficult to understand in text. Maybe they could be summarized in a table or visualized using a flowchart.

Materials and Methods 2.6 to 2.9: It is recommended that only the essentials be described in the text and the minute details be moved to the Supplementary Information.

Results 3.2.(p 26, line 11-20): The description should be moved to the introduction.

Results 3.1.3-3.1.4 Too detailed and too long. Only the main points should be mentioned. It would be effective to use concise Figures where possible.

Figure 6: Too much information; A, B, F, and G should be supplemental information.

Figure 8: Wheat cartoon is unnecessary. The font is too small. This information should be in a Table.

Editors Assessment: Antimicrobial resistance (AMR) is a global public health threat, and environmental microbial communities can act as reservoirs for resistance genes. There is a need for genomic surveillance could provide insights into how these reservoirs change and impact public health. With that goal in mind this study tested the ability of nanopore sequencing and adaptive sampling to enrich for AMR genes in a mock community of environmental origin. On average adaptive sampling resulting in a target composition 4x higher than without adaptive sampling, and increased target yield in most replicates. The methods and scripts for this approach were reviewed and curated together, although the scope of this study was limited in terms of communities tested and AMR genes targeted. And the authors improved their analysis by conducting an additional analysis of a diverse microbial community. Demonstrating the method is reusable and its results are promising for developing a flexible, portable, and cost-effective AMR surveillance tool.

*This evaluation refers to version 1 of the preprint *

This work has been published in GigaByte Journal under a CC-BY 4.0 license (https://doi.org/10.46471/gigabyte.103), and has published the reviews under the same license. These are as follows.

**Reviewer 1. Ned Peel. **

Is the source code available, and has an appropriate Open Source Initiative license been assigned to the code?

Yes. I do not think the authors have included a specific license and assume the code will be released under a Creative Commons CC0 waiver.

As Open Source Software are there guidelines on how to contribute, report issues or seek support on the code?

No. No guidelines on how to contribute, report issues or seek support on the code.

Is there a clearly-stated list of dependencies, and is the core functionality of the software documented to a satisfactory level?

Yes. A list of software used, along with version numbers, can be found in "dart_methods_notebook.md"

Additional Comments:

The authors describe each step of the analysis well and have provided code to reproduce the analysis and figures from the manuscript.

**Reviewer 2. Julian Sommer **

Is there a clear statement of need explaining what problems the software is designed to solve and who the target audience is?

No. Not applicable to this study, since no novel software is described.

Is the source code available, and has an appropriate Open Source Initiative license been assigned to the code?

Not applicable to this study, since no novel software is described.

As Open Source Software are there guidelines on how to contribute, report issues or seek support on the code?

No. Not applicable to this study, since no novel software is described.

Is the code executable?

Unable to test. The code and software used for analysis of the data is reported in the supplement data. However, the data used in this study in the SRA biobank is not available to download at the time of this review.

Is installation/deployment sufficiently outlined in the paper and documentation, and does it proceed as outlined?

Unable to test. See above.

Is the documentation provided clear and user friendly?

Yes. The analysis steps are clearly commented.

Is there enough clear information in the documentation to install, run and test this tool, including information on where to seek help if required?

No. The code provided for the data analysis is not usable without the raw sequencing data.

Have any claims of performance been sufficiently tested and compared to other commonly-used packages?

Not applicable.

Additional Comments.

The aim of this study was to test the ability of adapting sampling sequencing on the Oxford Nanopore sequencer to enrich for antibiotic resistance genes in a synthetic mixture of bacterial DNA. DNA from six environmental bacterial isolates with known antibiotic resistance genes were mixed at equal mass and used for metagenomic sequencing on an Oxford Nanopore MinION MK1B, comparing adaptive sampling with standard sequencing. By analysing 10 sequencing runs using low throughput, low cost flongle flow cells, the authors obtained sequencing data to compare adaptive sampling and standard sequencing approaches. Using a defined composition of sequenced sample and technical and biological replicates, the method is generally suitable. From their data, the authors conclude that adaptive sequencing significantly reduces throughput and increases gene target enrichment by analysing different parameters.

This result is important for the use of adaptive sampling in general, but has already been published in numerous publications, the author cites in his study. According to the author, the novel aspect of this work is the environmental origin of the bacteria used to generate the synthetic mock community. However, since the approach of adaptive sampling does not change regardless of the origin of the sequenced DNA, there are no significant new insights generated in this study. Also, the synthetic mock community of six members does not resemble an environmental metagenomic sample with incomparably more complex species diversity with different abundances. From the data presented in this study, no conclusions can be drawn regarding the performance of adaptive sampling sequencing of environmental metagenomic samples.

To improve the study, I suggest the following: Sequencing of DNA from environmental samples using nanopore sequencing without adaptive sampling and identification of antibiotic resistance genes. Subsequently, resequencing the sample using adaptive sampling based on the identified antibiotic resistance genes and comparing the results in terms of gene target enrichment as analysed in the study. This was partly suggested by the authors and should be carried out to gain new insights into the very interesting application of metagenomic sequencing for the One Health approach.

Additionally, there are some inconsistencies in the manuscript. For example, line 128 – 132 describes the sequencing process using different flowcells and technical replicates. However, it remains unclear, how the half of the channels of each flowcell were reserved for adaptive sampling sequencing since the adaptive sampling sequencing is always performed on the whole flowcell. Additionally, it is stated, that each flowcell was used twice for sequencing, however, no method on how to reuse the flongle flowcells is described and no protocol for this is available from oxford nanopore.

Nanopublication: RAyW5v4w76 "Article: The genome assembly and annotation of the Chinese cobra, Naja atra" https://w3id.org/np/RAyW5v4w76mcFJYDreFTuhc4Yu0sKwZQBccYfoB_Q-7_o

Nanopublication: RAt6pmOk9T "Organism of ?term=txid8656 - sequenced nucleotide sequence - PRJNA955401" https://w3id.org/np/RAt6pmOk9T4pCGTI5HTJ3hntFoIWRNv5zpGSNxX0JTYVk

Editors Assessment:

The hairy vetch Vicia villosa is an annual legume widely used as a cover crop due to its ability to withstand harsh winters. Here a new a 2.03GB reference-quality genome is presented, assembled from PacBio HiFi long-sequence reads and Hi-C scaffolding. After adding some more methodological details and long-terminal repeat (LTR) assembly index (LAI) analysis the assembly quality and metrics look quite convincing as a chromosome-scale assembly. This resource hopefully providing the foundation for a genetic improvement program for this important cover crop and forage species.

This evaluation refers to version 1 of the preprint

This work has been published in GigaByte Journal under a CC-BY 4.0 license (https://doi.org/10.46471/gigabyte.98), and has published the reviews under the same license. These are as follows.

Reviewer 1. Rong Liu

See reviewer comments document: https://gigabyte-review.rivervalleytechnologies.com/journal/gx/download-files?YXJ0aWNsZT0zODcmZmlsZT0xNTAmdHlwZT1nZW5lcmljJnZpZXc9ZmFsc2U~

Reiewer 2. Haifei Hu

Fuller et al. conducted an interesting work on the Vicia villosa genome study, which could be beneficial for the science community. However, there are some concerns about this work before it can be published.

1. Introduction The MS seems to indicate the V.villosa genome is important for breeding, and it is an ideal legume that can grow in winter. But the coming analysis and results are missing to address this. The authors should include additional analysis, at least in the gene annotation session, to indicate what genes are potentially associated with the improvement of genetic-based selection and the ability to grow in winter conditions. After reading the MS, it looks like it mainly focuses on the comparison of the V.vilsoa genome and the V.sativa genome. Please indicate why it is important to do so and provide more background on V.sativa in the introduction. Line 59. It is too sudden to start to describe high heterozygosity as still in the challenge without directly linking to V.villosa. The authors need to include the background that V.villosa is heterozygous first, then talk about how challenging it is to generate an assembly.

2. Methods Line 112: Why is the estimation based on K-mer size quite different from the generated assembly size? The authors’ explanation is weak and needs an in-depth and better explanation of these unexpected results. Did you see any similar observations in other studies? Please give examples(citations). Line 121: Any reason not to use the commonly used HiFi assembler HFi-asm? Line 142-143: Did you have a file to record which genome regions you have introduced the breaks and how this step was performed? Line 158: the unit bp changed into Mb for better comparison Line 160: Here, you should use contig N50 rather than scaffold N50 to indicate the quality of the gnome. And you need to compare the contig N50 with the V.sativa.

3. DATA VALIDATION AND QUALITY CONTROL Should perform BUSCO and LAI to assess the quality of the genome in the main text.

4 Phylogenetic tree construction Soybean is an important legume species, and it will make this result more useful and interesting for readers. You should include the Wm82 V4 genome for this analysis. And the version of other legume species’ genomes needs to be indicated.

5 Figures Figure 3 HiC alignment map shows near 600Mb genomes can not be scaffolded into a genome. Any reason? What is the green dot point in the figure? Figure 4 b, the BUSCO of Vvil1.0 is much higher than V.stativa. Any reason? And no description of how you perform the BUSCO analysis in the main text. Figure 6 Circle plot, would that possible to rename the scaffold as a chromosome based on the alignment between V.sativa and V.vil?

Editors Assessment: Aedes mosquito spread Arbovirus epidemics (e.g. Chikungunya, dengue, West Nile, Yellow Fever, and Zika), are a growing threat in Africa but a lack of vector data limits our ability to understand their propagation dynamics. This work describes the geographical distribution of Ae. aegypti and Ae. albopictus in Kinshasa, Democratic Republic of Congo between 2020 and 2022. Sharing 6,943 observations under a CC0 waiver as a Darwin Core archive in the University of Kinshasa GBIF database. Review improved the metadata by adding more accurate date information, and this data can provide important information for further basic and advanced studies on the ecology and phenology of these vectors in West Africa.

This evaluation refers to version 1 of the preprint

This work has been published in GigaByte Journal under a CC-BY 4.0 license (https://doi.org/10.46471/gigabyte.98), and has published the reviews under the same license. These are as follows.

**Reviewer 1. Luis Acuña-Cantillo **

Are the data and metadata consistent with relevant minimum information or reporting standards?

They must be review the standard Darwin core format for sampling events. https://www.gbif.org/darwin-core.

Is there sufficient detail in the methods and data-processing steps to allow reproduction?

No. They don't describe how the map of the study area was created, whether they used a GIS or not. Sampling points must be included on the map.

They don't mention how the identification of the larval stages was carried out and how they were differentiated from other genera of species of the Culicinae subfamily, such as Culex, Haemagogus, Mansonia, Sabethes or other species of the genus Aedes, since the two main species of this genus, were its objective.

In 5 reference, they mention is only for adult identification. They should include or cite the collection protocols and describe them as much as possible so that the study can be replicated in other African countries.

Is there sufficient data validation and statistical analyses of data quality?

Not my area of expertise. The data could be validated with biological collection of specimens

Is there sufficient information for others to reuse this dataset or integrate it with other data?

The scientific names must follow the same nomenclature, the first time the full name Aedes aegypti is mentioned and the second time Ae.aegypti, if there are two species within the same genus only one is mentioned the first time and the second time both abbreviated Ae.aegypti and Ae.albopictus.

Bibliographic references should be cited accordingly, for example: (1-4).

The names of the diseases must follow the same writing with a capital letter at the beginning or all in lower case Chikungunya or chikungunya.

Is there sufficient information for others to reuse this dataset or integrate it with other data?

From the description of the study and the collection times, I would believe that it fits more with Sampling Events, the data is well organized, however, it is suggested to review the Darwin Core template for this type of data and adjust to the corresponding model. , event_core review: https://www.gbif.org/darwin-core.

Additional Comments: The data paper can be published with suggestions for improvement. Congratulations, very good job!

**Reviewer 2. Mary Ann Tuli **

See the data audit file for more:

https://gigabyte-review.rivervalleytechnologies.com/journal/gx/download-files?YXJ0aWNsZT00NjQmZmlsZT0xNzYmdHlwZT1nZW5lcmljJnZpZXc9dHJ1ZQ~~

**Reviewer 3. Paul Taconet **

Is the language of sufficient quality?

Yes. Some minor changes that I recommend : "And the relative annual average humidity is 79%." may be changed to "The relative annual average humidity is 79%.". "Aedes albopictus is the most abundant species in the studied region" may be changed to "Aedes albopictus was the most abundant species in the studied region"

Are all data available and do they match the descriptions in the paper?

No.

1/The data available are of type 'occurrence' (only in 1 file - the "occurrence" file). For a better presentation of the data, I would suggest to transform them into "sampling event" data, which is more suited to this kind of data acquired from sampling events (see https://ipt.gbif.org/manual/en/ipt/latest/sampling-event-data), while keeping the occurrence dataset. This would enable the user to quickly understand the dates and locations of the sampling events.

2/ In the data, the only available date (column eventDate) is the first of January (eg. 2021-01-01T00:00:00). This does not enable to separte the data into seasons (Rainy et Dry) as presented in table 1 of the manuscript. I strongly suggest the authors to provide the specific date for each collected mosquito in the data.

Is the data acquisition clear, complete and methodologically sound?

No. 1/Larval collections : sampling strategy used ? 2/How many collection rounds in total ? please provide the dates of collection.

Is there sufficient data validation and statistical analyses of data quality?

No. 1/Human landing catch : was any quality control done during the collection of data (i.e. check that the collectors were at their place, etc.) ?

Is there sufficient information for others to reuse this dataset or integrate it with other data?

Yes. 1/comments for figure 1 (map) : - "legend" should be written in english (and not in french) - "harvesting sites" -> entomological collection points - the background layer is not very appropriate. Maybe better to put an Open Street Map background layer

2/What about ethical approval for the Human Landing Catches ? please provide the name of the institution who has approved the HLC and the approval number, if relevant

3/ in the dataset, for the species scientific name, I suggest to use the names as presented in : Harbach, R.E. 2013. Mosquito Taxonomic Inventory, https://mosquito-taxonomic-inventory.myspecies.info/ . Or at least, to provide the "nameAccordingTo" column.

4/ In the dataset, many columns seem totally empty. Please remove them if so.

Additional Comments: Thanks for this nice work and the effort put to publish your entomological data. I strongly suggest you to add the real dates of collection of the data in the GBIF dataset (see comments above).

**Reviewer 4. Angeliki Martinou **

Are all data available and do they match the descriptions in the paper?

Yes. It will be good for the authors the first time that they cite the two species to use the full names Aedes (Stegomyia) albopictus (Skuse) Aedes (Stegomyia) aegypti (Linnaeus, 1762)

In the methods section the title should be Human Landing Catches and not Human capture on landing

This work has been peer reviewed in GigaScience (see https://doi.org/10.1093/gigascience/giad092), which carries out open, named peer-review. This review is published under a CC-BY 4.0 license:

Reviewer Name: Ramil Mauleon

The paper titled "Developing best practices for genotyping-by-sequencing analysis using linkage maps as benchmarks" aims to present an end to end workflow uses GBS genotyping datasets to generate genetic linkage maps. This is a valuable tool for geneticists intending to generate a high confidence linkage map from a mapping population with GBS data as input.I got confused on reading the MS though, is this a workflow paper or is this a review of the component software for each step of genetic mapping and how parameter/use differences affect the output ? If it's a review, then the choice of software reviewed are not comprehensive enough, esp on SNP calling, and linkage mapping.There is no clear justification why each component software was used,example the use of GATK and freebayes for SNP calling I am familiar with using TASSEL GBS and STACKS for SNP calling using GBS data, why weren't they included in the SNP calling software. The MS would benefit greatly from including these SNP calling software in their benchmarking.Onemap and gusmap seems also pre-selected for linkage mapping, without reason for use, or maybe the reason(s) were not highlighted in the text. I've had experience in the venerable MAPMAKER and MSTMap, and would like to see more comparisons of the chosen genetic linkage mapping software with others, if this is the intent of the MS.The MS also clearly focuses on genetic linkage mapping using GBS, which should be more explicitly stated in the title. GBS is also extensively used in diversity collections and there is scant mention of this in the MS, and whether the workflow could be adapted to such populations.Versions of sofware used in the workflow are also not explicitly stated within the MS.The shiny app is also not demonstrated well in the MS, it could be presented better with screenshots of the interface , with one or two sample use cases.

This work has been peer reviewed in GigaScience (see https://doi.org/10.1093/gigascience/giad092), which carries out open, named peer-review. This review is published under a CC-BY 4.0 license:

Reviewer name: Peter M. Bourke

I read with interest the manuscript on Reads2Map, a really impressive amount of work went into this and I congratulate the authors on it. However, it is precisely this almost excessive amount of results that for me was the major drawback with this paper. I got lost in all the detail, and therefore I have suggested a Major Revision to reflect that I think the paper could be somehow made more stream lined with a clearer central message and fewer figures in the text. Line numbers would have been helpful, I have tried to give the best indication of page number and position, but in future @GigaScience please stick to line numbers for reviewers, it's a pain in the neck without them.

Overall I think this is an excellent manuscript of general interest to anyone working in genomics, and definitely worthy of publication.Here are my more detailed comments:

General comment: if a user would like to use GBS data for other population types than those amenable for linkage mapping (e.g. GWAS or genomic prediction, so a diversity panel or a breeding panel), how could your tool be useful for them?

Other general comment: the manuscript is long with an exhaustive amount of figures and supplementary materials. Does it really need to be this detailed? It appears like the authors lost the run of themselves a little bit and tried to cram everything in, and in doing so risk losing the point of the endeavour. What is the central message of this manuscript? Regarding the figures, the reader cannot refer to the figures easily as they are now mainly contained on another page. Do you really need Figures 16-18 for example?

Figures 13 and 14 could be combined perhaps? I am sure that at most 10 figures and maybe even less are needed in the main text, otherwise figures will always be on different pages and hence lose their impact in the text call-out.

Abstract and page 4: "global error rate of 0.05" - How do you motivate the use of a global error rate of 5%? Surely this is dataset-dependent?

Page 4 - how can a user estimate an error per marker per individual? The description of the create_probs function suggests there is an automatic methodology to do this, but I don't see it described. You could perhaps refer to Zheng et al's software polyOrigin, which actually locally optimises the error prior per datapoint. Maybe something for the discussion.

Page 6 "recombination fraction giving the genomic order" do you mean "given"?Page 10 section Effects of contaminant samples - if you look at Figure 9 you can see that the presence of contaminant samples seems to have an impact on the genotypes of other, non-contaminant samples, especially using GATK and 5% global error. With the contaminants present, the number of XO points decreases in many other samples. This is very odd behaviour I would have thought. Is it known whether this apparent suppresion of recombination breakpoints in non-contaminant individuals is likely to be "correct"? Perhaps the SNP caller was running under the assumption that all individuals were part of the same F1? If the SNP caller was run without this assumption (eg. specifying only HW equilibrium, or model-free) would we still see the same effect? This is for me a quite worrying result but something that you make no reference to as far as I can tell.

Page 12 "Effects of segregation distortion" In your study you only considered a single linkage group. One of the primary issues with segregation distortion in mapping is that it can lead to linkage disequilibrium between chromosomes, if selection has occurred on multiple loci. This can then lead to false linkages across linkage groups. Perhaps good to mention this.Page 12 "have difficulty missing linkage information" - missing word "with"

Page 17 I see no mention of the impact of errors in the multi-allelic markers on the efficiency, particularly of order_seq which seems to be very poorly-performing with only bi-allelics (Fig 20). If bi-allelic SNPs have errors then it is not obvious why multi-SNP haplotypes should not also have errors.

Page 3 Figure 1 - here the workflow shows multiple options for a number of the steps, which can lead to the creation of many map variants (e.g. 816 maps as mentioned on Page 4). Should all users produce 816 variants of their maps? With potentially millions of markers, this is going to take a huge amount of time (most users will want 100% of all chromosomes, not 37% of a single chromosome). Or should this be done for only a subset of markers? What if there is no reference sequence available to select a subset? As there are no clear recommendations, I suspect that the specific combination of pipeline choices will usually be datasetdependent. You actually mention this in the discussion

page 17. And with only 2 real datasets from 2 different species, there is also no way to tell if eg. GATK works best in rose, or updog should be used for monocots but not dicots etc. It would be helpful if the authors were more explicit about how their tool informs "best practices for GBS analysis" for ordinary users. Perhaps it is there, but for me this message gets lost.

Page 17 "updates in this version 3.0 to resolve issues with inflated genetic maps" - if I look at Figure 20, it seems that issues with inflated map length have not yet been fully resolved!

Page 17 "we provide users tools to select the best approaches" - similar comment as before - does this mean users should build > 800 maps with a subset of their dataset first, and then use this single approach for the whole dataset? It is not explicitly stated whether this is the guidance given. What is the eventual aim - to produce a good linkage map, or to use the linkage map to critically compare genotyping tools?

This work has been peer reviewed in GigaScience (see https://doi.org/10.1093/gigascience/giad092), which carries out open, named peer-review. This review is published under a CC-BY 4.0 license:

**Reviewer Name: Zhenbin Hu **

In this MS, the authors tried to develop a framework for using GBS data for downstream analysis and reduce the impact of sequence errors caused by GBS. However, sequence error is an issue not specific to GBS, it is also for whole genome sequences. Actually, I think the major issue for GBS is the missing data. However, in this MS, the authors did not test the impact of missing data on downstream analysis.The authors also mentioned that sequencing error may cause distortion segregation in linkage map construction, however, distortion segregation in linkage map construction can also happen for correct genotyping data. The distortion segregation can be caused by individual selection during the construction of the population. So I don't think it is correct to use distortion segregation to correct sequence errors.The authors need to clear the major question of this MS, in the abstract, the authors highlight the sequence errors, while in the introduction, the authors highlight the package for linkage map construction (the last paragraph). Actually, from the MS, authors were assembling a framework for genotyping-by-sequencing data.Two major reduced-represented sequencing approaches, GBS and RADseq, have specific tools for genotype calling, such as Tassel and Stack. However, the authors used the GATK and Freebayes pipeline for variant calling, authors need to present the reason they were not using TASSEL and Stack.In the genotyping-by-sequencing data, individuals were barcoded and mixed during sequencing, what package/code was used to split the individuals (demultiplex) from the fastq for GATK and Freebayes pipeline?The maximum missing data was allowed at 25% for markers data, how about for the individual missing rate?On page 6, the authors mentioned 'seuqnece size of 350', what that means?

This work has been peer reviewed in GigaScience (see https://doi.org/10.1093/gigascience/giad091 ), which carries out open, named peer-review. These review is published under a CC-BY 4.0 license:

**Reviewer name: Qianqian Song **

This paper offers an open-source tool, i.e., cellsnake, to perform single-cell data analysis. This cellsnake tool offers advanced features for standard users and facilitates downstream analyses in both R and Python environments. It is also designed for easy integration into existing workflows, allowing for rapid analyses of multiple samples. I like the incorporation design of the metagenome analysis in this tool, which makes it different with other available tools in single-cell analysis.

1) I looked through their tutorial, and have a specific question regarding the resolution parameter. I wonder if this resolution argument needs to be pre-selected? Or the cellsnake tool can automatically select a resolution parameter?

2) Is it possible to add color legends in the umap? Rather than label all cell types on the umap. It can be very hard to distinguish the cell types, especially when there are many cell types available.

3) If the single-cell data is profiled from human tissue, is it also possible to use cellsnake to perform microbiome analysis?

4) I recommend the authors to compare cellsnake with other existing tools. Pros and cons need to be highlighted.

This work has been peer reviewed in GigaScience (see https://doi.org/10.1093/gigascience/giad091 ), which carries out open, named peer-review. These review is published under a CC-BY 4.0 license:

Reviewer name: Tazro Ohta

The manuscript describes Cellsnake, a user-friendly tool for single-cell RNA sequencing analysis that targets non-expert users in the field of bioinformatics. Cellsnake operates as a command-line application, providing offline analysis capabilities for sensitive data. The integration of popular single-cell RNA-seq analysis software within Cellsnake, as described in Table 1, enhanced its utility as a comprehensive workflow. Cellsnake has different execution options (minimal, standard, and advanced) with varying outputs and execution times. The authors have provided well-structured online documentation, including helpful quick-start examples that facilitated easy understanding and usage of Cellsnake.

The tool was tested using the Docker appliance and the provided fetal brain dataset and performed as expected. The manuscript explains the functions well, with the results reproduced from existing research using publicly available datasets. The following issues need to be addressed by the authors.

1. The authors should include the citation for the Snakemake paper to acknowledge its contribution. https://doi.org/10.1093/bioinformatics/bts480

2. To support the claim of unique features in Cellsnake, a comparison with other similar methods, such as that on Galaxy (https://doi.org/10.1093/gigascience/giaa102), should be included.

3. It is recommended to host the Docker container image on both the GitHub Container Registry and the Docker Hub for better availability and redundancy. The authors should publish the Dockerfile to enable users to build a container image, if needed.

4. Online documentation is missing a link to the fetal-liver example dataset (https://cellsnake.readthedocs.io/en/latest/fetalliver.html), which needs to be addressed. The fetalbrain dataset shared via Dropbox should also be deposited in the Zenodo repository to improve accessibility and long-term preservation.

5. To assist users who want to use Cellsnake as a Snakemake workflow, the tool documentation should provide clear instructions on how to run Cellsnake as a single snakemake pipeline. This would be useful for users who utilize existing workflow platforms to accept snakemake requests.

6. The benchmarking of Cellsnake must provide more precise specifications than simply referring to "a standard laptop" for computing requirements. My trial of "cellsnake integrated standard" with the fetalbrain dataset took more than 17 h via Docker execution on my M1 Max MacBook Pro. This may be because the provided Docker image is AMD-based, which let my MacBook run the container on a VM, but the recommended computational specifications will help users. The GitHub issue of the Cellsnake repository also mentioned that the software is not tested on Windows Conda, which should be mentioned at least in the online documentation.

7. In the Data Availability section, please ensure that the correct formatting and consistent identifiers are used for public data, such as replacing SRP129388 with PRJNA429950 and E-MTAB-7407 with PRJEB34784, specifying that these IDs are from the Bioproject database. It is important to mention that EGA files are under controlled access, requiring user permission for retrieval.

8. The references in the manuscript need to be properly formatted to ensure the inclusion of publication years and DOIs where available.

9. The help message from the Cellsnake command indicates that its default values are set for human samples. The authors should mention in the manuscript that the pipeline is configured for human samples and requires further configuration for use with samples from other organisms. A step-by-step guide to configuring the setting for the other species, including the reference data download, would be helpful in obtaining more audiences.

This work has been peer reviewed in GigaScience (see https://doi.org/10.1093/gigascience/giad082 ), which carries out open, named peer-review. This review is published under a CC-BY 4.0 license:

**Reviewer Name: Francesco Pampaloni **

This study represents a significant contribution to the field of screening and analysis of threedimensional cell cultures. The demand for reliable and user-friendly image processing tools to extract quantitative data from a large number of spheroids or other types of three-dimensional tissue models is substantial. The authors of this manuscript have developed a tool that aims to address this need by providing a straightforward method to extract the projected area and intensity of individual cellular spheroids imaged with bright-field microscopy. The tool is compatible with "Incucyte" microscopes or any other automated microscope capable of imaging multiple specimens, typically found in high-density multiwell plates.An admirable aspect of this work is the authors' decision to make all the code and pipeline openly available on Github. This openness allows other scientists to test and validate the code, promoting transparency and collaboration in the scientific community. However, several improvements should be made to the manuscript prior to publication.One important aspect that the authors should address in the manuscript is the suitability, rationale, and extent of using a neural network-based segmentation approach for the specific analysis described in the manuscript—segmentation of single bright-field images of spheroids.

While neural networks are anticipated to play an increasingly important role in microscopy data segmentation in the coming years, they are not a universal solution. Although there may be segmentation tasks that are challenging to accomplish with traditional approaches, where neural networks can be highly effective, other segmentation tasks can be successfully performed using conventional strategies. For example, in our research group, we were able to reliably segment densely populated bright-field images containing numerous organoids in a single field of view using a pipeline based on the ImageJ plugin MorphoLibJ (see references: https://doi.org/10.1093/bioinformatics/btw413 and https://doi.org/10.1186/s12915-021-00958-w). Therefore, it would be informative and valuable for readers if the authors compared the results obtained from the neural network with those achieved by employing simple thresholding techniques (such as Otsu or Watershed) on the same dataset, as demonstrated in a similar study (reference: https://doi.org/10.1038/s41598-021-94217-1, Figure 5).

Furthermore, to address the limitations of the model, the authors should provide specific examples (preferably in the supplementary material due to space constraints) of incorrect segmentations or artifacts that arise from applying the neural network to the data. For instance, it would be beneficial to explore scenarios where spheroids are surrounded by cellular debris or when multiple spheroids are present in the field of view. These real-life situations are common and it is important to provide insights into potential challenges that may arise when the images of the spheroids are not pristine.

This work has been peer reviewed in GigaScience (see https://doi.org/10.1093/gigascience/giad082 ), which carries out open, named peer-review. This review is published under a CC-BY 4.0 license:

**Reviewer name: Kevin Tröndle **

The authors present a "Technical Note" about an open-source web tool called SpheroScan. As input users could upload (large batches of) spheroid images (brightfield, 2D). The tool delivers two outputs: (1) Prediction Module: creates a file with area and intensity of detected spheroids (CSV), (2) Visualization Module: plots of the corresponding parameters (PNG). Performance was tested on 480 Incucyte images and 423 microscope images with 336 (70 %) and 265 for training, 144 (30 %) and 117 for validation, and 50 images for testing, respectively. The framework is based on Mask R-CNN and Detectron2 library. The performance was tested in the range of 0.5 to 0.95 against manual annotation (VGG Annotator). As evaluation measure they used Intersection over union (IoU), determining the overlap between the predicted and ground truth regions and calculates values of Average Precision (AP) for masking: 0.937 and 0.972 (Test), 0.927 and 0.97 (Validation) as well as AP for bounding box: 0.899 and 0.977 (test) 0.89 and 0.944 (Validation). They show a linear runtime, proofed with different sized datasets (1 s / image) for masking on a 16 core CPU, 64 GB RAM machine. The tool is available on GitHub and claimed to be available as a web tool on spheroscan.onrender.com.General evaluation:The concept of the tool serves some important needs of 3D cell culture-based assays: automated, standardized, high-throughput image analysis. As such, it represents value added for the research field.

However, it remains open how high the impact, the reproducibility, and the chances of potential application by other researchers will be. This is due to some significant limitations in accessibility (i.e. non-permanent or non-functional web tool), as well as the (potential) restriction of input data (i.e. brightfield only, not validated with external data) and the limited options for analysis of the metadata (i.e. area and intensity only). The greatest value stems from the possibility to access a web interface, which is easy to use and will ideally be equipped with additional functionalities in the future.

Comment 1 (minor):The presented tool uses the Mask R-CNN deep-learning model in their image processing pipeline. Several tools, which perform image segmentation, are based on this or other models are well-established and already implemented in several commercial imaging devices and allow for segmentation of cell containing image areas, e.g. to determine confluency or cell migration in "wound healing assays", mainly optimized for 2D cultures, but also applicable for 2D images of 3D spheroids. The concept of automated image segmentation is thus not novel and only meets the journal's input criterion as "update or adaptation of existing" tools.The state-of-the-art and preliminary work are not sufficiently referenced. Several similar and alternative (open-source) tools are existent and should be mentioned in the manuscript, e.g. (Lacalle et al., 2021; Piccinini et al., 2023; Trossbach et al., 2023), to give only a few examples.

Comment 2 (major):The authors claim to present an user-friendly open-source web tool. The python project is available on Github, and on a demo-server (https://spheroscan.onrender.com/) where the web interface can be accessed. Unfortunately the mentioned web tool is not functional, i.e. it is stated on the website: "This is a demonstration server and the prediction module is not available for use. To utilize the prediction functionality, please run SpheroScan on your local machine.".This is significantly limiting the applicability of the presented tool to users who are able to execute python code on their local hardware. Therefore, the demo server should either present a functional user interface (recommended), or the statement should be removed from the manuscript, which would limit the impact of the submission significantly

.Comment 3 (major):The presented algorithm was trained exclusively on internal data of brightfield images from "Incucyte and microscope platforms". Furthermore, two distinct models were generated, working with either Incucyte or microscope images.It remains unclear how the algorithm will perform on external data of prospective users. Given the fact that two distinct models had to be trained for different image sources (i.e. from two different platforms) indicates a limited robustness of the models in this regard. This is clearly a general problem of image processing algorithms, but one that will stand in the way of applicability by external users with certainly other imaging techniques. Since the web tool interface is not functional at this point, the authors will also not be able to evaluate or improve on this after publication. At least one performance test with external data, obtained from an ideally blinded user should be performed, to further elaborate on this.

Comment 4 (major):Many assays nowadays use fluorescent labels, for example to calculate cell ratios within 3D arrangements, e.g. for cell viability or the expression of certain proteins. The authors do not state if the algorithm (or future iterations thereof) is or will be able to process multi-channel microscope images of spheroids.This is a significant limitation of the presented work and should at least be mentioned in the corresponding section, respectively. Furthermore, a proof-of-concept test run with fluorescent images could be performed to test the algorithm performance and derive potentially necessary adaptations in future versions.

Comment 5 (minor):The output of the tool is a list of detected spheroids with corresponding area (2D) and bright field average intensity within the area.The usability of these two parameters is limited to specific assays, such as the mentioned use case to investigate collagen gel contraction assays. Several other parameters of interest could easily be derived from the metadata, such as roundness, volume estimation (assuming a spheroid shape), or even cell count estimation. This should again be mentioned in the "limitations and considerations" section.

This work has been peer reviewed in GigaScience (see https://doi.org/10.1093/gigascience/giad085 ), which carries out open, named peer-review. The review is published under a CC-BY 4.0 license:

Reviewer name: Raphael Leman

Summary: In this work Barbosa et al., presented a benchmarking of several splicing predictors for human intronic variants. Overall, the results of this study shown that deep learning based tools such as SpliceAI outperformed the other splicing predictors to detect splicing disturbing variants and so pathogenic variants.

The authors also detailed the performances of these tools on several subsets of data according to the collection origins of variants and according to the genomic localization of variants. This work is one of the first large and independent studies about splicing prediction performances among intronic variants and in particular among deep intronic variants in a context of molecular diagnosis. This work also highlights the need to have reliable prediction tools for these variants and that the splicing impact of these variants are often underestimated. However, I estimated that major points should to be solved before considering the article to publication.

**Major points ** 1 The most important point is that authors shown results in the main text but in following paragraphs they claimed that these results were biased. In addition, the results, taking into account these biases, were only shown in supplementary data and the readers should make the correction themselves to get the "true" results. Indeed, the interpretation of biased results and "true" results changes drastically. The two main biases were: i) the use of ClinVar data already used for the training of CAPICE (see my following comment n°2-), ii) the intronic tags of variants and the relative distance to the nearest splice site were wrong (see my following comment n°5-). Consequently, the authors should remove these biased results and only show results after bias correction.

2 Importantly, several tools used ClinVar variants or published data to train and/or validate their models. Therefore, to perform a benchmark on true independent collection of variants, the authors should ensure the lack of overlapping between variants used for the tool development and this present study.

3 As authors shown by the comparison between the ClinVar classification (N = 54,117 variants) and impact on RNA from in vitro studies (N = 162 variants), there was discrepancies between this two information (N = 13/74 common variants, 18%). Consequently, using ClinVar classification to assay the performance of splicing prediction tools is not optimal. To partially fix this point, I think further studying (ex: get minor allele frequency, availability of in vitro RNA studies, …) the intronic variants with positive splicing predictions from two or more tools with a ClinVar classification benign or likely benign and inversely, the intronic variants with negative splicing predictions from two or more tools with a ClinVar classification pathogenic or likely pathogenic could be interesting.

4 The authors used pre-computed databases for 19 tools, but the most of these databases do not include small insdels and so add artificially missing data in disfavor of the tool although the same tool could score these indels variants in de novo way.

5 The authors said that "We hypothesized that variability in transcript structures could be the reason [increase in performance in the deepest intronic bins]: despite these variants being assigned as occurring very deep within introns (> 500bp from the splice site of the canonical isoform) in the reference isoform, they may be exonic or near-splice site variants of other isoforms of the associated gene". To solve this transcript structure variability, firstly the authors could use weighted relative distance as following: |(|Pos_(nearest splice site)-Pos_variant |)-Intron_Size |â•„(Intron_Size ). Secondly, the ClinVar data contains the RefSeq transcript ID on which the variant was annotated (except for large duplications/deletions), so the authors should make the correspondence between these RefSeq transcript IDs and the transcripts used to perform splicing predictions.

6 With respect to the six categories of splice-altering variants, it is unclear how the authors considered cases in which variants alter physiological splice motives (e.g., natural consensus sequences 3'SS/5'SS, branch point, or ESR) but, instead of exon skipping, the spliceosome recruits another distant splice site that is partially or not affected by the variant.

7 In the table 1 listing the tools considered for this study, please explicit for each tool on which collections of data (ClinVar or splicing altering variants) and for which genomic regions the benchmark was done. This information will facilitate the reading of the article.

8 Accordingly to my comment n°3-, all spliceogenic variants are not necessary pathogenic. The mutant allele could produce aberrant transcripts without a frame-shift and without impact the functional domains of the protein. In addition, the transcription could also lead to a mix between aberrant transcript and full-length transcript. As a result, the main goal of splicing prediction tools is to detect splicing altering varaints. Considering variants with positive splicing prediction as pathogenic is a dangerous shortcut and only an in vitro RNA study could confirm the pathogenicity of a variant. The discussion section should be update in this sense.

9 The authors claimed that: "The models [SQUIRLS and SPiP] were frequently able to correctly identify the type of splicing alteration, yet they still fail to propose higher-order mechanistic hypotheses for such predictions.". I think that the authors over-interpreted the results (see my comment n° 21-).

10 The authors recommended prioritizing intronic variants using CAPICE, It is still true once the bias was corrected (see my comment n°1-).

**Minor points **

11 In the introduction the authors could clearly define the canonical splice site regions (AG/GT dinucleotides in 3'SS: -1/-2 and 5'SS: +1/+2) to make the difference with the consensus splice sites commonly define as: 3'SS: -12 (or -18)/+2 and 5'SS: -3/+6. 12 In the introduction, please also add that splice site activation could be also due to disruption of silencer motif. 13 In the ref [17], the authors did not say that the enrichment of splicing related variants within splice site regions was linked to exons and splice sites sequencing. They proved that whole genome sequencing increased the diagnostic rate of rare genetic disease, actually they did not focus on splicing variants. This enrichment was more probably induced by the fact that geneticists mainly studied variants with positive splicing predictions. 14 In the paragraph 'The prediction tools studied are diverse in methodology and objectives', please add that most of prediction tools target consensus splice sites (ex: MES, SSF, SPiCE, HSF, Adaboost, …).

15 In the paragraph 'The prediction tools studied are diverse in methodology and objectives', the authors claimed that 'sequence-based deep learning models such as SpliceAI, which do not accept genetic variants as input.' but it is wrong as SpliceAI could accept VCF file as input. 16 In the paragraph 'Pathogenic splicing-affecting variants are captured well by deep learning based methods', this is further explained in the section method, but I think a sentence explaining that the 243 variants were from 81 variants described in ref [19] and 162 variants from a new collection will clarify the reading of article 17 In the paragraph 'Pathogenic splicing-affecting variants are captured well by deep learning based methods', among the 13 variants incorrectly classified, please detailed how many variants were classified as benign and VUS. 18 Due to the blue gradient, the Fig 1C is hard to analyze. 19 In the paragraph 'Branchpoint-associated variants', the variant rapported in the ref [79] were studied within tumoral context and so the observed impact could not be the same in healthy tissue. 20 In the paragraph 'Exonic-like variants', the authors changed the parameters of SpliceAI predictions, from the original prarameters used for the precomputed scores, to take into account variants located deep inside the pseudoexon. Please ensure whether other prediction tools have also user-defined optimizable parameters to take into account these variants. 21 In the paragraph 'Assessing interpretability', the authors observed that non-informative SPiP annotations presented a high score level. This could be explained by the fact of the tool report a positive prediction without annotation only because the model score was high without a relation to a particular splicing mechanism. 22 In the paragraph 'Assessing interpretability', the authors could compare the SpliceAI annotations regarding the abolition/creation of splice sites and their relative positions to the variants to the observed effect on RNA. 23 In the paragraph 'Predicting splicing changes across tissues', by my count the analysis of AbSpliceDNA predictions was done on 89 variants (154 - 65 = 89), if true please indicate clearly in the text. 24 In the method section, paragraph "ClinVar", the 13 variants with discordance between the classification and the observed splicing impact, how many did they have confidence stars. 25 In the method section, paragraph "Disease-causing intronic variants affecting RNA splicing", the authors filtered out variants within the 10 pb around the nearest splice site, please explicit why. 26 In the method section, paragraph "Disease-causing intronic variants affecting RNA splicing", the authors used gnomAD variants as control set, however their threshold of variant frequency is too low (1%). Indeed, some pathogenic variants involved in recessive genetic disorders have a high frequency in population. A threshold of 5% is more appropriate. 27 In the method section, paragraph "Variants that affect RNA splicing", the authors should describe how they considered variants leading to multiple aberrant transcripts and variants with partial effect (i.e., allele mutant still producing full length transcript). 28 In the method section, paragraph "Variants that affect RNA splicing", regarding the six categories defined by the authors: How the indels variants were annotated if they overlapped between several categories.

The new splice donor/acceptor categories included only variants creating new AG/GT or variants occurring within the consensus sequences of cryptic splice sites. Among the category Donor-downstream, please make the distinction between variants located between [+3; +6] bp (i.e. consensus sequence) and variant beyond +6 bp. The exonic-like variants could be variants that did not impact ESRs motives (see my comment n°6-). 29 In the method section, paragraph "Variants that affect RNA splicing", the authors select for the control datasets, variants generating the CAGGT and GGTAAG motives. However, this approach lead to an over-enrichment of false positives. Moreover, it could be also interesting if among the variants creating new splice sites or pseudoexons to identify the presence of GC donor motif or U12-minor spliceosome motif (AT/AC) and how the different splicing tools can detect them. 30 In Fig S3C, scale the gnomAD population frequency in -logₕ₀(P) to make the figure more readable. 31 I saw several times double spaces in the text please correct them. English is not my native language so I am not the best judge, but some sentences seem syntactically incorrect (ex: "The splicing tools with the smallest and largest performance drop between the splice site bin ("1-2") and the "11-40" bin were Pangolin and TraP, with weighted F1 scores decreasing by 0.334 and 0.793, respectively"). Please have the article proofread by someone who is fluent in English.

This work has been peer reviewed in GigaScience (see https://doi.org/10.1093/gigascience/giad085 ), which carries out open, named peer-review. The review is published under a CC-BY 4.0 license:

**Reviewer name: Jean-Madeleine de Sainte Agathe **

This manuscript presents an important and very exhaustive benchmark concerning intronic variant splicing predictors. The focus on deep-intronic variants is highly appreciated as it addresses a very crucial challenge of today's genetics. The authors present the different tools in a very clear and pedagogical way. I should add that this manuscript is pleasant to read. The authors use the average precision score, allowing a refined comparison between tools.

They give practical recommendations. They emphasize the use of SpliceAI and pangolin for intronic variants. For branchpoint regions, they recommend Pangolin and LabRanchoR. It should be noted that this study is to my knowledge the first independent benchmark of Pangolin, CISpliceAI, ConSpliceML, AbSplice-DNA, SQUIRLS, BPHunter, LaBranchoR and SPiP together. Overall, this study is important as it will be very helpful for the interpretation of intronic variants. I hence fully and strongly support its publication. I have several comments that (I think) should be addressed before publication, especially the first point:

1) I admit that the curation of such large datasets is challenging, however, I failed to find some of the Table S6 variants in the referenced work. Please, could you kindly point me to the referenced variation for the following variants? - The variant "1 hg38_156872925 C T NTRK1 ENST00000524377.1:c.851-708C>T pseudoexon_inclusion keegan_2022" is classified as 'affects_splicing'. However, I did not find it in Keegan 2022 (reference 20). In Keegan, the table S1 mentions NTRK1 variants but not c.851-708C>T. For these NTRK1 variants, keegan et al refers to another publication Geng et al 2018 (PMC6009080), where I can't find the ENST00000524377.1:c.851-708C>T variants neither. - Same for "COL4A3 ENST00000396578.3:c.4462+443A>G 2:g.228173078A>G" - Same for "ABCA4 ENST00000370225.3:c.1937+435C>G 1:g.94527698G>C" - Same for "FECH ENST00000382873.3:c.332+668A>C 18:g.55239810T>G" - Concerning "MYBPC3 ENST00000545968.1:c.1224-52G>A 11:g.47364865C>T" , I did not find it in pbarbosa as stated, but in another reference which, I think, should be mentioned in this manuscript: https://pubmed.ncbi.nlm.nih.gov/33657327/ - "BRCA2 ENST00000544455.1:c.8332-13T>G 13:g.32944526T>G" is classified as splicing neutral based on moles-fernández_2021, but it has previously been shown to alter splicing (https://pubmed.ncbi.nlm.nih.gov/31343793/), please clarify. If these variants were somehow erroneously included, the authors should reprocess their results with the corrected datasets.

2) Although it has been done before, the usage of gnomAD variants as a base of splicing-neutral variants is questionable. Indeed, it is theoretically possible that such variants truly alter splicing. For example, genuine splicing alterations can result in mild inframe consequences on the gene products. Or splicing alterations can damage non-essential genes. I suggest that the authors: -either select another gnomAD variants list located in disease-associated genes, where benign splicing alterations seem less plausible. -or discuss this putative limitation in their results.

3) Table S8: "Variants above 0.05, the optimized SpliceAI threshold for non-canonical intronic splicing variation" Is that a recommendation of this work? Or was it found elsewhere? Please clarify. More generally, this manuscript uses Average Precision scores, but the authors should explain to their non-statistician readers how it relates to the delta scores of each tool (Fig 3C). Indeed, any indication (or even recommendation, but not necessarily) concerning the use of cut-off values would be very appreciated by the geneticist community.

4) p.3 "If the model is run twice, once with the reference and once with the mutated sequence, it is possible to measure splice site alterations caused by genetic variants." This study makes only use of the delta scores, which have previously been shown to be misleading in some rare cases (PMID 36765386). The authors would be wise to mention this. For example, in Table S3, "ENST00000267622.4:c.5457+81T>A 14(hg19):g.92441435A>T" is predicted by SpliceAI DG=0.16, but as the reference prediction is already at 0.84, this 0.16 is the maximal delta score possible, yielding donor score = 1.

5) p.12 "Among the tools that predict across whole introns, SQUIRLS and SPiP are the only ones designed to provide some interpretation of the outcome." Concerning the nature of the mis-splicing event, I think the authors should mention SpliceVault, which has been specifically built for this task (pmid 36747048).

6) p.14: "SpliceAI and Pangolin […]. If usability is a concern and users do not have a large number of predictions to make, SpliceAI is preferred since the Broad Institute has made available a web app for the task" Now, the broad institute web app includes pangolin (at least for hg38 variants). Please, rephrase of delete this sentence.

7) Concerning complex delins, which are not annotated with the current version of SpliceAI, the authors should give recommendations. For example, the complex delins from tableS9 "hg19_chr7 5354081 GC AT" is correctly predicted by CI-SpliceAI and SpliceAI-visual, both tools allowing the annotation of complex delins with the SpliceAI model.

8) p.8 "Unfortunately, BPHunter only reported the variants predicted to disrupt the BP, rendering the Precision-Recall Curves (PR Curves) analysis impossible." I agree with the authors. However, I think it is sometimes assumed (wrongly?) that all variants unannotated by BPhunter have BPH_score=0. Maybe the authors could explicit this. For example, by saying that the lack of prediction cannot be safely equated with a negative prediction.

This work has been peer reviewed in GigaScience (see https://doi.org/10.1093/gigascience/giad086 ), which carries out open, named peer-review. This review is published under a CC-BY 4.0 license.

** Reviewer name: Doreen Ikhuva Lugano **

This paper gives a good introduction on bats as reservoirs of several viral infections, which studies have shown is due to the uniqueness of their immune system. They and others suggest that bats immune system is dampened exhibiting tolerance to various viruses. This gives the study a good rationale as to why study the bats immune system, compared to other mammals. They also give a good rationale as to why they used single-cell sequencing, to allow the identification of various cell types and the differences in these cell types. From their finding the main conclusions are that differences in the host species are more impactful; than those among the different stimuli. They also suggest that bats initiate an innate immune response after infection with DNA viruses through an alternative pathway. For example, the induction dynamics of PRRs seems to be different in their dataset. They also suggest this could be due to the presence of species-specific cellular subsets. 1. Interesting model system and a good comparison of bats with other mammals. 2. Good technique in using single-cell sequencing, with a clear rationale as to why it was chosen. This advances knowledge on what was already known about bats immune system, but the species-specific cellular subsets are new. 3. Interesting technique to go through the bulk transcriptomic data in four species and four conditions. This allowed findings of the most important genes/pathways. 4. Good rationale / flow of experiments from one to another 5. I liked that they investigated stimuli from different pathogens , including DNA, RNA virus and bacteria and still show that bats had a different immune system, in the different stimuli. Minor comments 1. Do they speculate this occurrence in is this just in Egyptian Fruit bats or all species of bats? 2. Mentioned in the introduction why they used the egyptian fruit bats - which are a model organism, but this could help people who are not in this field understand exactly why use these bats. Advantages? Location? Proximity to the various viruses based on the fact they are mostly found in endemic regions such as Africa etc. 3. Can they include viral load in each species? 4. It is not clear which scRNAseq tools were used for data analysis in identifying the types of cells. Or did they use already established database based on markers?

This work has been peer reviewed in GigaScience (see https://doi.org/10.1093/gigascience/giad086 ), which carries out open, named peer-review. This review is published under a CC-BY 4.0 license.

** Reviewer name: Urs Greber **

Hirofumi Aso and colleagues provide a manuscript entitled 'Single-cell transcriptome analysis illuminating the characteristics of species specific innate immune responses against viral infections'. The aim was to describe differences in innate immune responses of peripheral blood mononuclear cells (PBMCs) from different primates and bats against various pathogenic stimuli (different viruses and LPS). A major conclusion from the study is that differences in the immune response between primate and bat PBMCs are more pronounced than those between DNA, RNA viruses or LPS, or between the cell types. The topic is of interest as the immunological basis for how bats appear to be largely disease resistant to some viruses that cause severe infections in humans is not well understood. One notion by others has been that bats have a larger spectrum of interferon (IFN) type I related genes, some of which are expressed constitutively even in unstimulated tissue, and there, trigger the expression of IFN stimulated genes (ISGs). Alongside, enhanced ISG levels may need to be compensated for in bats. Accordingly, bats may exhibit reduced diversity of DNA sensing pathways, as well as absence of a range of proinflammatory cytokines triggered in humans upon encountering acute disease causing viruses. The study here uses single-cell RNA sequencing (scRNA-seq) analysis, and transcript clustering algorithms to explore the profile of different innate immune responses upon viral infections of PBMCs from H sapiens, Chimpanzee, Rhesus macaque, and Egyptian fruit bat. Most commonly referred to cell types were detected in all four species, although naïve CD8+ T cells were not detected in bat PBMCs, which led the authors to focus on B cells, naïve T cells, killer T/NK cells, monocytes, cDCs, and pDCs. The study used three pathogenic stimuli, Herpex simplex virus 1 (HSV1), Sendai virus (SeV), and lipopolysaccharide (LPS). Specific comments The text is well written, concise, and per se interesting, but I have a few questions for clarification.

1) Can the authors provide quality and purity control data for the virus inocula to document virus homogeneity? E.g., neither the methods, nor the indicated ref 26 specify if or how HSV1 was purified. Same is true for SeV where the provided ref 34 does not indicate if virus was purified or not. If virus inocula were not purified then it remains unclear to what extent the effects on the PBMCs described in the study here were due to virus or some other component in the inoculum. Conditions using inactivated inoculum might help to clarify this issue.

2) What was the infection period? Was it the same for all viruses?

3) Upon stimuli application, there was a noteable expansion of B cells and a compression of killer T / NK cells in the bat but not the human samples, as well as compression of monocytes, the latter observed in all four species. Can the authors comment on this observation?

4) Lines 78-79: I do not think that TLR9 ought to be classified as a cytosolic DNA sensor. Please clarify.

5) Line 117: please clarify that the upregulation of proinflammatory cytokines, ISGs and IFNB1 was measured at the level of transcripts not protein.

6) Line 244: DNA sensors. Authors report that bats responded well to DNA viruses, although some of their DNA sensing pathways (e.g., STING downstream of cGAS, AIM2 or IFI16) were attenuated compared to primates (H sapies, Chimpanzee, Macaque). And they elute to the dsRNA PRR TLR3. But I am not sure if TLR3 is the only PRR to compensate for attenuated DNA sensing pathways. The authors might want to explicitly discuss if other RNA sensors, such as RIG-I-like receptors (RIG-I, LGP2, MDA5) were upregulated similarly in bats as in primate cells upon inoculation with HSV1.

7) Is it known how much TLR3 protein is expressed in bat PBMCs under resting and stimulated conditions? Same question for the DNA and RNA sensor proteins, e.g., cGAS, AIM2 or IFI16, RIG-I, LGP2, MDA5, or effector proteins, such as STING.

8) Can authors clarify if cGAS is part of the attenuated DNA sensors in the bat samples under study here? And it would be nice to see the attenuated response of DNA sensing pathways in the bat samples, as suspected from the literature, including STING downstream of cGAS, or AIM2 and IFI16.

9) What are the expression levels of IFN-I and related genes in the bat cells among the different stimuli?

10) Technical point: where can the raw scRNA-seq data be found?

This work has been peer reviewed in GigaScience (see https://doi.org/10.1093/gigascience/giad073 ), which carries out open, named peer-review. These reviews are published under a CC-BY 4.0 license and were as follows:

Reviewer: Leopold Parts

Summary Fu et al. explore utilising low-throughput mutational fitness measurements to predict the results of high-throughput deep mutational scanning experiments. They demonstrate that adding alanine scanning results to predictive models improves performance, as long as the alanine scan used a sufficiently similar evaluation approach to a deeper experiment. The findings make intuitive sense, and will be useful for the community to internalize.

While we have several comments about the methods used, and requests to fortify the claims with more characterization, we do not expect addressing any of them will change the core findings. One can argue that direct application of AS boosted predictions is likely to be limited due to the number of scans available and the speed at which DMS experiments are now being performed, so it would also be useful to discuss the context of these results in the evolution of the field, and we make specific suggestions for this. Regardless, the presented results are a useful demonstration of a more general use case of low-throughput or partial mutagenesis data for improving fitness prediction and imputation.

Major Comments

There are many other computational variant effect predictors beyond Envision and DeMaSk. It would be very useful to see how their prediction results compare to some others, particularly the best performing and common models that are also straightforward to download and run (e.g. EVE, ESM1v, SIFT, PolyPhen2). This would be important context to see how impactful the addition of AS data is to DeMaSk/Envision. Please run additional prediction tools for reference of absolute performance; there is no need to incorporate AS data into them. Several proteins have a very small number of AS residues (Figure 2), and from our reading of the methods, other residue scores are imputed with the mean AS value for that protein. (As an aside, it would be good to clarify if this average is across studies or within study). If this reading is correct, the majority of residues for each proteins will have imputed AS results (e.g. in case of PTEN, over 90%), which can be problematic for training and prediction. Please clarify if our interpretation of the imputation approach is correct, and if so, please also provide results for a model trained without imputation, on many fewer residues. If the boosting model has already implemented this, please integrate the Supplementary methods into the main methods, and reference these and the results when describing the imputation approach to avoid such concerns. It is not clear how significant/impactful the increases in performance are in figures 4, 5, S4, S5 & S6. Please use a reasonable analytical test, or training data randomization to evaluate the improvement against a null model. There are quite a few proteins with repeated DMS/AS measurements. In our experience these correlate from moderately to very highly. Including multiple highly correlated studies could lead to pseudo-replication and biasing the model performance results. Please present a version of the results where the repeats are averaged first to test whether that bias exists. Minor Comments [suggestions only; no analyses required from us]

A short discussion about the number of available alanine scans, particularly for proteins without DMS results, would help put the work in context. For example, it would be good to know how many proteins would benefit from improved de-novo predictions (e.g. no DMS data) and how many could have improved imputation (incomplete DMS data). Similarly the rate and cost of DMS data generation is important to understand the utility of their results. I think a short discussion of how useful models of this sort are in practice now and in future would be helpful to the reader. This seems most natural as part of the end of the discussion, but could also fit in the introduction. Figure 2 is missing y axis label. We also softly suggest log scale axis, to not obscure the degree to which some proteins have more residues covered and the proportion of residues covered by AS. Figure 3 includes DMS/AS study pairs with at least three alanine substitutions to compare - we think this is a low cut-off, particularly with the regularisation applied. I think something like 10+ would be more informative. I think their cross-validation scheme leaves out an entire protein at a time, as opposed to one study each iteration. I agree this is the better way to do it. However, I initially read it as the latter, which would lead to leakage between train/validation data since the same residue would be included in both if a protein had multiple datasets. It might be useful to be more explicit to prevent other readers doing the same. L231 In the discussion they mention fitting a model only using studies with a minimum DMS/AS correlation. This occurred to me as well while reading the relevant part of the results. Is there a good reason not to do this? It doesn't seem like a large amount of work and conceptually seems a good way to assess a model that says what a DMS might look like is it had the same selection criteria as a given AS. L154 Similarly, a correlation cut-off as well as choosing the most corelated study seems like it would be a fairer comparison in figure 5. Just because an AS is the most correlated doesn't necessarily mean it is well correlated. It would be interesting to see if the improvement results in figure 7 correlate with substitution matrices (e.g. Blosum) or DMS variant fitness correlations (e.g. correlation between A and C, A and D, etc.). Intuitively it feels like they should. It would be nice to label panels in figure 7. It also seems notable that predicting alanine substitutions is not the most improved - a brief comment on why would be interesting. The AS model adds 2x20 parameters to the model for encoding, which is a lot if CCR5 is held out, as there are only a few hundred total independent residues evaluated. While the performance on held out proteins is a good standard, it would be interesting to evaluate the increase from model selection perspective (BIC/AIC or similar) if possible. L217 The statement doesn't seem logical to me - if such advanced imputation methods were available surely they would be better used to impute all substitutions than just model alanine then use linear regression to model the rest? L331-332 The formula used for regularising Spearman's rho makes sense, and can likely be interpreted as a regularizing prior, but we found it hard to understand its provenance and meaning from the reference. A sentence on its content (not just describing that it shrinks estimates) and a more specific reference would be useful for interested readers like ourselves. L364 It says correlation results were dropped when only one residue was available whereas in figure legends it says results with less than three residues were dropped. Notwithstanding thinking three is maybe too low a cutoff, these should be consistent or clarified slightly if I've misunderstood the meaning. It would be nice to have a bit more comment on the purpose of the final supplementary section (Replacing AS data with DMS scores of alanine substitutions) - if you have DMS alanine results it seems likely you will have the other measurements anyway.

This work has been peer reviewed in GigaScience (see https://doi.org/10.1093/gigascience/giad073 ), which carries out open, named peer-review. These reviews are published under a CC-BY 4.0 license and were as follows:

Reviewer: Joseph Ng

This manuscript explored whether low-throughput alanine scanning (AS) experimental data could complement deep mutational scanning (DMS) to classify the impact of amino acid substitutions in a range of protein systems. The analysis partially confirms this hypothesis in that it only applies when the functional readout being measured in the two assays are compatible with one another. In my opinion this is an insight that should be highlighted in a publication and therefore I believe this manuscript deserved to be published. I just wish the authors could clarify & further explore the points below better in their manuscript before recommending for acceptance:

In my opinion the most important bit of data curation is the classification of DMS/AS pairs as high/medium/low etc. compatible, and this is the key towards the authors' insight that assay compatibility is an important determinant of whether signals in the two datasets could be cross-matched for analysis. The criteria behind this classification are listed in Figure S2 but I feel the wording needs to be more specific. For example, in Figure S2, the authors wrote 'Both assays select for similar protein properties and under similar conditions' - what exactly does this mean? What does the authors consider to be 'similar protein properties'? I could not find more detailed explanation of this in the Methods section. The authors gave reasons in the spreadsheet in Supp. Table 1 for the labels they give to each pairs of assays, but I'm still not exactly sure what they consider to be 'similar'. Is there are more specific classification scheme which is more explicit in defining these 'similarities', e.g. by defining a scoring grid explicitly listing the different levels of 'similarities' of measurable properties, e.g. both thermal stability - score of 3; thermal stability vs protein abundance - 2; thermal stability vs cell survival - 1 (or equivalent, I think the key issue is to provide the reader with a clear guide so they can readily assess the compatibility of the datasets by themselves)? I would have thought discrepancy between the DMS and AS scores to be different across different structural regions of the protein, e.g. the discrepancy would be larger in ordered region compared to disorder as the protein fold would constrain the types of amino acids tolerable within the ordered segment of the protein. Is this the case in the authors' collection of datasets? If so, does the compatibility of assays modulate this discrepancy?

This work has been peer reviewed in GigaScience (see https://doi.org/10.1093/gigascience/giad073 ), which carries out open, named peer-review. These reviews are published under a CC-BY 4.0 license and were as follows:

Reviewer: Leopold Parts

Summary Fu et al. explore utilising low-throughput mutational fitness measurements to predict the results of high-throughput deep mutational scanning experiments. They demonstrate that adding alanine scanning results to predictive models improves performance, as long as the alanine scan used a sufficiently similar evaluation approach to a deeper experiment. The findings make intuitive sense, and will be useful for the community to internalize.

While we have several comments about the methods used, and requests to fortify the claims with more characterization, we do not expect addressing any of them will change the core findings. One can argue that direct application of AS boosted predictions is likely to be limited due to the number of scans available and the speed at which DMS experiments are now being performed, so it would also be useful to discuss the context of these results in the evolution of the field, and we make specific suggestions for this. Regardless, the presented results are a useful demonstration of a more general use case of low-throughput or partial mutagenesis data for improving fitness prediction and imputation.

Major Comments

There are many other computational variant effect predictors beyond Envision and DeMaSk. It would be very useful to see how their prediction results compare to some others, particularly the best performing and common models that are also straightforward to download and run (e.g. EVE, ESM1v, SIFT, PolyPhen2). This would be important context to see how impactful the addition of AS data is to DeMaSk/Envision. Please run additional prediction tools for reference of absolute performance; there is no need to incorporate AS data into them. Several proteins have a very small number of AS residues (Figure 2), and from our reading of the methods, other residue scores are imputed with the mean AS value for that protein. (As an aside, it would be good to clarify if this average is across studies or within study). If this reading is correct, the majority of residues for each proteins will have imputed AS results (e.g. in case of PTEN, over 90%), which can be problematic for training and prediction. Please clarify if our interpretation of the imputation approach is correct, and if so, please also provide results for a model trained without imputation, on many fewer residues. If the boosting model has already implemented this, please integrate the Supplementary methods into the main methods, and reference these and the results when describing the imputation approach to avoid such concerns. It is not clear how significant/impactful the increases in performance are in figures 4, 5, S4, S5 & S6. Please use a reasonable analytical test, or training data randomization to evaluate the improvement against a null model. There are quite a few proteins with repeated DMS/AS measurements. In our experience these correlate from moderately to very highly. Including multiple highly correlated studies could lead to pseudo-replication and biasing the model performance results. Please present a version of the results where the repeats are averaged first to test whether that bias exists. Minor Comments [suggestions only; no analyses required from us]

A short discussion about the number of available alanine scans, particularly for proteins without DMS results, would help put the work in context. For example, it would be good to know how many proteins would benefit from improved de-novo predictions (e.g. no DMS data) and how many could have improved imputation (incomplete DMS data). Similarly the rate and cost of DMS data generation is important to understand the utility of their results. I think a short discussion of how useful models of this sort are in practice now and in future would be helpful to the reader. This seems most natural as part of the end of the discussion, but could also fit in the introduction. Figure 2 is missing y axis label. We also softly suggest log scale axis, to not obscure the degree to which some proteins have more residues covered and the proportion of residues covered by AS. Figure 3 includes DMS/AS study pairs with at least three alanine substitutions to compare - we think this is a low cut-off, particularly with the regularisation applied. I think something like 10+ would be more informative. I think their cross-validation scheme leaves out an entire protein at a time, as opposed to one study each iteration. I agree this is the better way to do it. However, I initially read it as the latter, which would lead to leakage between train/validation data since the same residue would be included in both if a protein had multiple datasets. It might be useful to be more explicit to prevent other readers doing the same. L231 In the discussion they mention fitting a model only using studies with a minimum DMS/AS correlation. This occurred to me as well while reading the relevant part of the results. Is there a good reason not to do this? It doesn't seem like a large amount of work and conceptually seems a good way to assess a model that says what a DMS might look like is it had the same selection criteria as a given AS. L154 Similarly, a correlation cut-off as well as choosing the most corelated study seems like it would be a fairer comparison in figure 5. Just because an AS is the most correlated doesn't necessarily mean it is well correlated. It would be interesting to see if the improvement results in figure 7 correlate with substitution matrices (e.g. Blosum) or DMS variant fitness correlations (e.g. correlation between A and C, A and D, etc.). Intuitively it feels like they should. It would be nice to label panels in figure 7. It also seems notable that predicting alanine substitutions is not the most improved - a brief comment on why would be interesting. The AS model adds 2x20 parameters to the model for encoding, which is a lot if CCR5 is held out, as there are only a few hundred total independent residues evaluated. While the performance on held out proteins is a good standard, it would be interesting to evaluate the increase from model selection perspective (BIC/AIC or similar) if possible. L217 The statement doesn't seem logical to me - if such advanced imputation methods were available surely they would be better used to impute all substitutions than just model alanine then use linear regression to model the rest? L331-332 The formula used for regularising Spearman's rho makes sense, and can likely be interpreted as a regularizing prior, but we found it hard to understand its provenance and meaning from the reference. A sentence on its content (not just describing that it shrinks estimates) and a more specific reference would be useful for interested readers like ourselves. L364 It says correlation results were dropped when only one residue was available whereas in figure legends it says results with less than three residues were dropped. Notwithstanding thinking three is maybe too low a cutoff, these should be consistent or clarified slightly if I've misunderstood the meaning. It would be nice to have a bit more comment on the purpose of the final supplementary section (Replacing AS data with DMS scores of alanine substitutions) - if you have DMS alanine results it seems likely you will have the other measurements anyway.

This work has been peer reviewed in GigaScience (see https://doi.org/10.1093/gigascience/giad073 ), which carries out open, named peer-review. These reviews are published under a CC-BY 4.0 license and were as follows:

Reviewer: Joseph Ng

This manuscript explored whether low-throughput alanine scanning (AS) experimental data could complement deep mutational scanning (DMS) to classify the impact of amino acid substitutions in a range of protein systems. The analysis partially confirms this hypothesis in that it only applies when the functional readout being measured in the two assays are compatible with one another. In my opinion this is an insight that should be highlighted in a publication and therefore I believe this manuscript deserved to be published. I just wish the authors could clarify & further explore the points below better in their manuscript before recommending for acceptance:

In my opinion the most important bit of data curation is the classification of DMS/AS pairs as high/medium/low etc. compatible, and this is the key towards the authors' insight that assay compatibility is an important determinant of whether signals in the two datasets could be cross-matched for analysis. The criteria behind this classification are listed in Figure S2 but I feel the wording needs to be more specific. For example, in Figure S2, the authors wrote 'Both assays select for similar protein properties and under similar conditions' - what exactly does this mean? What does the authors consider to be 'similar protein properties'? I could not find more detailed explanation of this in the Methods section. The authors gave reasons in the spreadsheet in Supp. Table 1 for the labels they give to each pairs of assays, but I'm still not exactly sure what they consider to be 'similar'. Is there are more specific classification scheme which is more explicit in defining these 'similarities', e.g. by defining a scoring grid explicitly listing the different levels of 'similarities' of measurable properties, e.g. both thermal stability - score of 3; thermal stability vs protein abundance - 2; thermal stability vs cell survival - 1 (or equivalent, I think the key issue is to provide the reader with a clear guide so they can readily assess the compatibility of the datasets by themselves)? I would have thought discrepancy between the DMS and AS scores to be different across different structural regions of the protein, e.g. the discrepancy would be larger in ordered region compared to disorder as the protein fold would constrain the types of amino acids tolerable within the ordered segment of the protein. Is this the case in the authors' collection of datasets? If so, does the compatibility of assays modulate this discrepancy?

**Editors Assessment: **

Irises on top of being a popular and beautiful ornamental plant, have wider commercial interest due to the many interesting secondary metabolites present in their rhizomes that have value to the fragrance and pharmaceutical industries. Many of these have large and difficult to assemble genomes, and to fill that gap the Dalmatian Iris (Iris pallida Lam.) is sequenced here. Using PacBio long-read sequencing and bionano optical mapping to produce a giant 10Gbp assembly with a scaffold N50 of 14.34 Mbp. The authors didn’t manage to handle the haplotigs separately or to study the ploidy, but as all of the data is available for reuse others can explore these questions further. This reference genome should also allow researchers to study the biosynthesis of these secondary metabolites in much greater detail, opening new avenues of investigation for drug discovery and fragrance formulations.

This evaluation refers to version 1 of the preprint

This work has been published in GigaByte Journal under a CC-BY 4.0 license (https://doi.org/10.46471/gigabyte.94), and has published the reviews under the same license. These are as follows.

**Reviewer 1. Baocai Han **

Iris pallida Lam., an ornamental plant, produces secondary metabolites with potential value to the fragrance and pharmaceutical industries, while also possessing anti-oxidant, anti-inflammatory, and immunomodulatory effects. The genome assembly of this species could be more helpful in investigation for drug discovery and fragrance formulations.

I have a number of comments that follow:

1. Line 10 (page 2): “resulting in a 10.04 Gbp assembly with a scaffold N50 of 14.34 Mbp”. I found the genome size is 13.49 Gb in Table 2 and line 18 (page 7) due to differing haplotigs in the phased assembly. While I can not find how to deal with this problem. I suggest to purge the duplicates from the genome using the Purge_Dups pipeline. (Guan D, McCarthy SA, Wood J et al. Identifying and removing haplotypic duplication in primary genome assemblies. Bioinformatics, 2020; 36(9): 2896–2898.)

2. Line 5 (page 8): why is the gene number of the Complete and duplicated BUSCOs so high. Is it due to issues with genome assembly or the presence of a particularly high number of repetitive sequences in the species?

3. there is no reference or website for many softwares and pipelines, eg. HybridScaffolding pipeline (line 22, page 5), lima (line 2, page 6) and Exonerate (line 11, page 6)

4. I suggest upload the genome annotation file, given that genome annotation has already been performed.

**Reviewer 2. Kang Zhang **

Is the language of sufficient quality?

Yes. Though I found several sentences confusing: P2L8 (Is the DNA/RNA extraction particularly difficult for iries?), and P9L5 (wording).

Is there sufficient information for others to reuse this dataset or integrate it with other data?

Yes. With the following comments.

1. P7L20. The basic stats of the subreads should be introduced before the assembling process.

1. The authors should provide more methodological details about the BUSCO assessment, such as the database version, the mode (genome or protein), etc.
2. I am curious about the genome size enlargement introduced by the scaffolding. Were different haplotigs (from different haplotypes) were used for scaffolding, and why? I suppose that only the primary haplotigs should be used.
3. Considering the high proportion of duplicated BUSCO genes, I wonder whether the iris sequenced is a polyploid or not? Please clarify it in the Background.

Additional Comments: Dr. Wong and her colleagues reported a genome assembly of iris using the PacBio technology. Due to the huge genome size, the generated data volume is impressive. Although the quality of the assembly is not so satisfying, it is reasonable considering the genome size and the high heterozygosity, which is commonly found in many flowers. Overall, the methods used in this work are well described, and the data could be accessed. I only get several minor points regarding the details during the assembling process.

**Editors Assessment: **

While Bacterial Artificial Chromosomes libraries were once a key resource for building the human genome project over time they have been rendered relatively obsolete by long-read technologies. In the era of CRISPR-Cas systems pairing this data with one of the many guide-RNA libraries to find targets for manipulation with CRISPR tools is bringing back BACs advantages for genomics. With this in mind the authors have developed a BAC restriction map database containing the restriction maps for both uniquely placed and insert-sequenced BACs from 11 libraries covering the recognition sequences of available restriction enzymes. Alongside a set of Python functions to reconstruct the database and more easily access it (which were debugged and had improved documentation added during review). The presented data should be valuable for researchers simply using BACs, as well as those working with larger sections of the genome in terms of synthetic genes, large-scale editing, and mapping.

*This evaluation refers to version 1 of the preprint *

This work has been published in GigaByte Journal under a CC-BY 4.0 license (https://doi.org/10.46471/gigabyte.93), and has published the reviews under the same license. These are as follows.

**Reviewer 1. Po-Hsiang Hung **

Are all data available and do they match the descriptions in the paper?

No. The dataset in FTP includes all the Bac sequences and the restriction enzyme recognition sites in csv files. However, I could not find the database of pairs of BACs, which have overlaps generated by restriction enzymes that linearize the BACs. The makePairs function gave me an error when I tried running it locally, so I was not able to verify what is in these datasets. Personally, I find this function to be one of the most useful features described in this manuscript.

Are the data and metadata consistent with relevant minimum information or reporting standards? See GigaDB checklists for examples http://gigadb.org/site/guide

Yes. This manuscript contains the necessary minimal information (Submitting author, Author list, Dataset title, Dataset description, and Funding information)

Is there sufficient detail in the methods and data-processing steps to allow reproduction?

No. The authors provide their code in GitHub such that researchers can download the datasets and analyze the sequences locally. However, I felt that the descriptions in the readme.md file is often insufficient to reproduce the data presented in the manuscript, especially for researchers with little to no programming experience. Detailed information includes examples of how to use each function, the input format, and the location of the output folder/files. I also encountered software version issues during the installation of bacmapping. Please re-test the code in a new environment and describe all the versions of each software. For instance, I found Python version 3.11 is incompatible with this package while Python version 3.7 is compatible.

Is there sufficient data validation and statistical analyses of data quality?

No. The author used the BioRestriction class from Biopython to get the digestion site information. No extra validation is conducted in this manuscript. Due to the errors I encountered in re-running the code (see details in Any Additional Overall Comments to the Author), an independent method for checking several digestion sites in some Bac clones is suggested. The suggested independent method is to do enzyme digestion on some Bac clones or upload some Bac sequences to other software and compare the digestion sites.

In the output files that contain the digestions sites for each enzyme, some of the enzyme digestion sites are either NA or []. What is the difference between the two? If they mean the same thing (no cutting by the enzyme), bugs or other coding errors may cause this inconsistency. Please check the code again and also verify some of them using the independent methods suggested above. Examples of this issue are the files in maps>sequenced>CEPHB. Here I list two enzymes that show different results in each file: 3.csv : Ragl ([]), SchI (NA) 6.csv: EspEI (NA), AccII([]) 13.csv: EcoT22I ([]), Hsp92II (NA) X.csv: PacI ([]), AcIWI (NA)

Is the validation suitable for this type of data?

No. No validation in this manuscript. See the answer above.

Additional Comments: The authors make a database with enzyme digestion site information of Bac clones to help people to use the Bac clones for further usage. I think it is useful to have this information and also have the code to do further analysis locally. Thus, I think providing a very detailed user manual (or readme.md) is very important to help people use this dataset. Below I summarized the issues I encountered in running codes and also some suggestions. Major points: (1) I tested some bacmapping functions, and I discovered that some functions are not working as intended due to typos/bugs - The version of the software is required to help people properly install this package - Refining the code and also providing a better user manual is very helpful for people without a lot of coding experience to use it. The detailed information includes examples of how to use each function, the input format, and the location of the output folder/files. Descriptions for some functions in the readme file are not detailed enough and often do not describe what the input needs to be. For example, getCuts() require ‘row’ as input. But the author never gives a detailed description of what ‘row’ is in the readme file. I had to look in bacmapping.py to understand what ‘row’ is. If a function requires the variable ‘row’, show a few examples of how ‘row’ can be extracted from the proper input file. - mapPlacedClones() requires an input file (‘/home/eamon/BACPlay/longboys.csv’, line 335) that is located in the author’s local computer and is not available through github. - Typo in line 814 in getMap(). Should be: name = cloneLine[‘CloneName’] - Inconsistency in output variable type in getMap() (line 830 and 851). When local == ‘sequenced’, the output variable is a tuple, which causes issues in downstream functions such as getRestrictionMap() (line 869). (2) Add pairs of BACs into the dataset (3) The output file of digestion sites of each enzyme, some of the enzyme digestion sites showed NA or [ ]. Please double-check this and explain the differences (4) Validation of an independent method for the digestion map is suggested

Minor points: (1) Add a title to each column of sequencedStats.csv is useful for understanding the table easier

Re-review:

The authors have addressed majority of my points. The software installation works great after considering version control. The updated read.me provide detailed information for each function and their required input variables, and the examples in jupyter notebook are a great help for running the code. I did, however, encounter two minor errors when I tested the Ch19_bacmapping_example.ipynb on a Mac system. Please check this and update it.

(1)The .DS_store file that is automatically generated on a Mac system in the bacmapping/Examples/Ch19_example/maps/placed folder causes an error when running bmap.mapPlacedClones(cpustouse=cpus, chunk_size=chunksize). The same problem happened when I ran bmap.mapSequencedClones(cpustouse=cpus). After I deleted .DS_store in the folder, the code worked.

Here is the error message when I ran bmap.mapSequencedClones(cpustouse=cpus). NotADirectoryError: [Errno 20] Not a directory: '/Users/user_nsame/bacmapping/Examples/Ch19_example/maps/sequenced/.DS_Store'

(2) The second error is from running bmap.getRestrictionMap(name,enzyme). I got the error message, 'list' object has no attribute 'item'. I was able to run this function after changing maps[enzyme].item() to maps[enzyme] in line 779 of bacmapping.py. I encountered the same error with the drawMap function. I was able to run to run this function after changing line 847 of bacmapping.py from rmap = maps[nenzyme].item() to rmap = maps[nenzyme].item().

Here is the error message

AttributeError Traceback (most recent call last) Cell In[20], line 5 3 maps = bmap.getMaps(name) 4 #print(maps) #this is a big dataframe of all the maps, uncomment to check it out ----> 5 rmap = bmap.getRestrictionMap(name,enzyme) 6 print('Sites in ' + name + ' where ' + enzyme + ' cuts: '+ str(rmap)) 7 plt = bmap.drawMap(name, enzyme)

File ~/miniconda3/envs/bacmapping/lib/python3.11/site-packages/bacmapping/bacmapping.py:779, in getRestrictionMap(name, enzyme) 777 maps = getMaps(name) 778 nenzyme, r = getRightIsoschizomer(enzyme) --> 779 return(maps[nenzyme].item())

AttributeError: 'list' object has no attribute 'item'

**Reviewer 2. Wei Dong **

Is there sufficient data validation and statistical analyses of data quality? Not my area of expertise

Is the validation suitable for this type of data? I am not sure about this.This is not my specialty.

Overall comments: This is a great idea, fully exploring, integrating, and utilizing existing data for new research.

**Editors Assessment: **

This work presents a new standardized graphical approach for visualizing genetic associations across a wide range of allele frequencies. These proposed TrumpetPlots have a distinctive trumpet shape, hence the proposed name. With the majority of variants having low frequency and small effects, while a small number of variants have higher frequency and larger effects, this view can help to provide new and valuable insights into the genetic basis of traits and diseases, and also help prioritize efforts to discover new risk variants. The tool is provided as a novel R package and R Shiny application and to demonstrate its use the article illustrates the distribution of variant effect sizes across the allele frequency range for over 100 continuous traits available in the UK Biobank. After some problems in testing the package is now available and easy to deploy via CRAN.

*This assessment refers to version 1 of this preprint. *

This work has been published in GigaByte Journal under a CC-BY 4.0 license (https://doi.org/10.46471/gigabyte.89) and has published the reviews under the same license. These are as follows.

**Reviewer 1. Clara Albiñana **

As Open Source Software are there guidelines on how to contribute, report issues or seek support on the code?

No. Although there are no explicit guidelines for contribution in the manuscript or website, it is true that by placing the project on gitlab it is possible to contribute to the project / open issues.

Is the code executable?

No. Unfortunately, I wasn't able to install the R package. I have now opened an issue on the gitlab page so that it can hopefully get solved.

Is installation/deployment sufficiently outlined in the paper and documentation, and does it proceed as outlined?

Yes. It is very common for new R packages to just use devtools for installation.

Is the documentation provided clear and user friendly?

Yes. The requirements for generating a trumpet plot just involve providing a set of GWAS summary statistics with column-specific names, together with the GWAS sample size. This is very common for GWAS summary statistics-based tools. I think it is fine for the R package to require re-naming the columns to fit the format, as one already needs to upload the file into R. However, I find it inconvenient to have to re-save the summary statistics file with different name-columns for the shinyapp tool. Providing e.g. column indexes alone would be much more user-friendly.

Is there enough clear information in the documentation to install, run and test this tool, including information on where to seek help if required?

No. I cannot answer this question until I can install the tool.

Have any claims of performance been sufficiently tested and compared to other commonly-used packages?

Not applicable. There are no existing comparable tools.

Is automated testing used or are there manual steps described so that the functionality of the software can be verified?

Yes. I can see there is a toy dataset included with the R package.

Additional Comments:

I think the manuscript is very clear and good at making the point of the utility of the software. The proposed trumpet plots are very visually appealing and can be useful to characterise the genetic variation of diverse phenotypes. The novelty of the trumpet plots, as compared to previously proposed effect size vs. allele frequency plots, is the use of positive and negative effect sizes, making it look like a trumpet. I also appreciate the style decisions in the standard generated plots, with a nice visually-appealing color scheme and design.

On the use of the software, I have focused my testing on the R package, which I was not able to install. The shinyapp is very useful for visualising the existing, pre-computed trumpet plots, but I do not find it very useful for generating user-uploaded summary statistics for the reasons I mentioned above. Another comment on the ShinyApp is that I appreciate the possibility to download the plots but it would be very useful to include the name of the visualized phenotype as the plot title, for example, to avoid confusion when downloading multiple plots.

I also found an incorrect sentence in the abstract, which is think should be reversed: " The proposed plots have a distinctive trumpet shape, with the majority of variants having low frequency and small effects, while a small number of variants have higher frequency and larger effects".

**Reviewer 2. Wentian Li **

Is the documentation provided clear and user friendly?

No. Many aspects of Fig.1 are not explained.

Overall Comments: Plots with allele frequency as x axis and effect size (e.g. odds ratio) as y axis is a very common display of the contribution from both common and rare alleles to genetic association. A schematic form of this plot is practically on almost everybody's presentation slides when introducing this topic (to see an example, see, e.g. Science (23 Nov 2012), vol 338(6110), pp.1016-1017 ). Considering how many people have already been familiar with this type of plot, I feel that very little new is added in this paper: maybe only a new name ("trumpet"), and/or the power lines. The other methods contributions (log-x, one variant per LD, avoiding gene-level statistics) are rather straightforward. People without experience with "shiny" (R package) can still use ggplot2 or plot in R to get the same result. Generally speaking, I think the paper is weak, though OK as a program/package announcement.

Major comments: * I think the trumpet shape (increase of "effect size" for rare variant) is probably a direct consequence of using odds-ratio as a measure of effect size. If the allele frequency in normal population is p0, that in disease population is p1, [p1/(1-p1)]/[p0/(1-p0)] ~ p1/p0 tends to be large for small p0's, simply because the denominator is small. On the other hand, if population attributable risk (p0(RR-1)/(1+p0(RR-1))) is used as the y-axis, I am uncertain what the shape of the plot would be.

• A risk allele has these pieces of information:
• allele frequency,
• effect size (e.g. odds ratio),
• type-I error/p-value,
• type-II error/power. The plot in this paper show #1 vs #2 and #4 being added as extra. In another publication with a proposal to plot genetic association results (Comp Biol. and Chem. (2014), 48:77-83 doi: 10.1016/j.compbiolchem.2013.02.003), #2 is against #3 with #1 being an added extra. I'm sure using other combinations could lead to other types of plots. The authors should discussion/compare these possibilities.

Minor comments: In Fig.1, the size of the dots, the brown vs cyan color, the discontinuity of scatter dots around 0.01, are not explained.

Re-review:

I have read authors' response and I'm mostly satisfied. Only two minor comments: * Witte 2014 Nature Rev. Genet. article summarizes the point I tried to make well. I understand that rare variants should have a relatively higher effect from an evolutionary perspective, but since these are rare, their individual or even collective contribution to a disease in the population is still small. A casual reader may not realize this point and I think it would be helpful to cite Witte's article. * My minor comment on Fig.1 is still not addressed: there seem to be more points on the right side of p=0.01 line than the left side. Why this discontinuity? (the added text in Revision is about the color and size of the dots, not about this discontinuity)

Xupo Ding 1. The CDS and protein sequences could not extracted from the file of masked.fasta with gff3 file when verifying the accuracy of genes loci and related proteins. The extract software is gffread in cufflinks 2.1.1. Please confirm the final assembly file that would upload to GigaDB.2. Confirmed the accuracy of gene predication, especially for ks calculation.3. Before the repeat masked with the software of Repeatmasker, the final sequences were scanned with LTR_retriever and the LAI index have been generated in this folder. The LAI values were 20.55 and 18.06, which could be classified the haplogenome assembly as the reference or gold level, please describe the LAI values after busco completeness in the revised manuscript.4. The percentages of two largest subfamilies of LTR, Gypsy and Copia, were not presented in the supplementary TableS5.5. Two Eucalyptus genomes have been published (Nature 2014; Gigascience, 2020) and they were all not analysis the LTR insert time in detail. The insert times of all TE, Gypsy and Copia would highlighted this manuscript, especially the basic data have been presented with *.list in the LTR_harvest and LTR_retriever scan.6. Did the special genes of each haplogenome classify? Which pathways or Go terms they enriched in?7. Some SVs may be associated with the plant traits. The genes distributing in the regions of different SVs type should be furtherly identified and enriched with GO and KEGG.8. "Syntenic gene pairs between the E. grandis and E. urophylla haplogenomes were identified using a python version of MCScan, JCVI v1.1.18."Syntenic gene pairs in Figure 4 seemed only from JCVI，not using MCScan.9. The reference cite should be consistent, such as Candotti et al in the section of Genome scaffolding should be revised.10. Language should be improved and modified by academic editor.

Chao Bian: This study, entitled "Haplogenome assembly reveals interspecific structural variation in Eucalyptus hybrids", has reported two haplotypes from Eucalyptus grandis and E. urophylla.Both genomes are of high quality and high completeness. Nevertheless, why not directly and separately sequenced the Eucalyptus grandis and E. urophylla, and separately assembled each genome? In this way, the authors will not perform so much assembling steps to distinguish haplogenome.On the other hand, the authors have written a large paragraph to show the SV and SNP between both Eucalyptus species. However, the author only shown the number of SVs and SNPs, but did not show any relationship between the SV and biological characters. Could some SVs and SNPs involved in or impacted some genes can interpret some biological difference between Eucalyptus grandis and Eucalyptus grandis?In my view, only showing the number of SVs and SNPs is indeed fruitless for wide interests of this study. Some biological stories should be reported in a genome study.Please provide new figures with higher resolution. These figures are too much unclear.Please use the novel version of BUSCO V5.2.2, and indicate the used library.What's the QUAST assessment result in this study?The English language of this paper needs to be largely polished. Too much spelling and mistakes were appeared in the manuscript.Some minor suggestions:The decimal places should be uniform, such as "(567 Mb and 545 Mb) to 97.9% BUSCO completion" and "scaffold N50 of 43.82 Mb and 42.45 Mb for the E. grandis and E. urophylla haplogenomes, respectively".In 'All scripts used in this study is available on github.', 'is' should be 'are'.The language of this sentence should be revised "Illumina short-reads were used for k-mer based genome size estimation was performed using Jellyfish v2.2.6 (Jellyfish, RRID:SCR_005491) [25] for 21- mers and visualised with GenomeScope v2.0"For scaffolding step, why the authors removed all contigs smaller than 3kb?'The predicted gene space was' should be 'The predicted gene spaces were'.For "a contig N50 of 3.91 Mb 1." and 'was greater than 88.0% 2', what're meaning of the last '1' and '2' in these sentences.In this sentence 'Approximately 3.3 μg of HMW DNA from was used without', 'from' what?"a BUSCO completeness score of at least 95.3% was obtained for contigs anchored to one of the eleven chromosomes.", for one of the eleven chromosomes? Why contigs were only anchored to one chromosome?Revise 'markers each.,'."BUSCO completeness scores of 94.6% and 95.8% was obtained", 'was' should be 'were'."Although there is a greater number of local variants compared to SVs", 'there is' should be 'there are'."respectively, Supplementary Table S3)" revised to 'respectively, (Supplementary Table S3)'.'Mbp' revised to 'Mb'.'assemblies was' should be 'assemblies were'.

Ilan Gronau: This manuscript describes updates made to GADMA, which was published two years ago. GADMA uses likelihood-based demography inference methods as likelihood-computation engines, and replaces their generic optimization technique with a more sophisticated technique based on a genetic algorithm. The version of GADMA described in this manuscript has several important added features. It supports two additional inference engines, more flexible models, additional input and output formats, and it provides better values for the hyper-parameters used by the genetic algorithm. This is indeed a substantial improvement over the original version of GADMA. The manuscript clearly describes the different added features to GADMA, and then demonstrates them with a series of analyses. These analyses establish three main things: (1) they show that the new hyper-parameters improve performance; (2) they show how GADMA can be used to compare performance of different approaches to calculate data likelihood for demography inference; (3) showcase new features of GADMA (supporting model structure and inbreeding inference). Overall, the presentation is very clear and the results are interesting and compelling. Thus, despite being a publication about a method update, it shows substantial improvement, provides interesting new insights, and will likely lead to expansion of the user base for GADMA.The only major comment I have is about the part of the study that optimizes the hyperparameters. The hyper-parameter optimization is a very important improvement in GADMA2. The setup for this analysis is very good, with three inference engines, four data sets used for training and six diverse data sets used for testing. However, because of complications with SMAC for discrete hyperparameters, the analysis ends up considering six separate attempts. The comparison between the hyper-parameters produced by these six attempts is mostly done manually across data sets and inference engines. This somewhat beats the purpose of the well-designed set up. Eventually, it is very difficult for the reader to asses the expected improvement of the final suggested values of hyperparameters (attempt 2) to the default ones. I have two comments/suggestions about this part.First, I'm wondering if there is a formal way to compare the eventual parameters of the six attempts across the four training sets. I can see why you would need to run SMAC six separate times to deal with the discrete parameters. However, why do you not use the SMAC score to compare the final settings produced by these six runs?Second, as a reader, I would like to see a single table/figure summarizing the improvement you get using whatever hyper-parameters you end up suggesting in the end compared to the default setting used in GADMA1. This should cover all the inference engines and all the data sets somehow in one coherent table/figure. Using such a table/figure, you could report improvement statistics, such as the average increase in log-likelihood, or average decrease in convergence times. These important results get lost in the many improved figures and tables.These are my main suggestions for revisions of the current version. I also have some more minor comments that the authors may wish to consider in their revised version, which I list below.Introduction:===========para 2: the survey of demography inference methods focuses on likelihood-based methods, but there is a substantial family of Bayesian inference methods, such as MPP, Ima, and G-PhoCS. Bayesian methods solve the parameter estimation problem by Bayesian sampling. I admit that this is somewhat tangential to what GAMDA is doing, but this distinction between likelihood-based methods and Bayesian methods probably deserves a brief mention in the introduction.para 2,3: you mention a result from the original GADMA paper showing that GADMA improves on the optimization methods implemented by current demography inference methods. Readers of this paper might benefit of a brief summary of the improvement you were able to achieve using the original version of GADMA. Can you add 2-3 sentences providing the highlights of the improvement you were able to show in the first paper?para 3: The statement "GADMA separates two regular components" is not very clear. Can you rephrase to clarify?Materials and methods - Hyper-parameter optimization:==============================================I didn't fully understand what you use for the cost function in SMAC here. Seems to me like there are two criteria: accuracy and speed. You wish the final model to be as accurate as possible (high log likelihood), but you want to obtain this result with few optimization iterations. Can you briefly describe how these two objectives are addressed in your use of SMAC? It's also not completely clear how results from different engines and different data sets are incorporated into the SMAC cost. Can you provide more details about this in the supplement?para 2: "That eliminate three combinations" should be "This eliminates three combinations".para 3: "Each attempt is running" should be "Each attempt ran"para 3: "We take 200×number of parameters as the stop criteria". Can you clarify? Does this mean that you set the number of GADMA iterations to 200 times the number of demographic model parameters? Why should it be a linear function of the number of parameters? The following text explains the justification, butTable 1: I would merge Table S2 with this one (by adding the ranges of all hyper-parametres as a first column). It's important to see the ranges when examining the different selections.Materials and methods - Performance test of GADMA2 engines:=====================================================para 2: "ROS-STRUCT-NOMIG" should be "DROS-STRUCT-NOMIG" Also, "This notation could be read" - maybe replace by "This notation means" to signal that you're explaining the structure notation.Para 4 (describing comparisons for momi on Orangutan data): "ORAN-NOMIG model is compared with three …". You also consider ORAN-STRUCTNOMIG in the momi analysis, right?Results - Performance test of GADMA2 engines:========================================Inference for the Drosophila data set under model with migration: you mention that the models with migration obtain lower likelihoods than the models without migration. You cannot directly compare likelihoods in these two models, since the likelihood surface is not identical. So, I'm not sure that the fact that you get higher likelihoods in the models without migration is a clear enough indication for model fit. The fact that the inferred migration rates are low is a good indication for that. It also seems like despite converging to models with very low migration rates, the other parameters are inferred with higher noise. For example, the size of the European bottleneck is significantly increased in these inferences compared to that of the NOMIG. So, potentially the problem here is that more time is required for these complex models to converge.Inference for the Drosophila data set under structured model (2,1): the values inferred by moments and momentsLD appear to neatly fit the true values. However, it is not straightforward to compare an exponential increase in population size to an instantaneous increase. Maybe this can be done by some time-averaged population size, or the average time until coalescence in the two models? This will allow you to quantify how good the two exponential models fit the true model with instantaneous increase.Inference for the Orangutan data set under structured model (2,1) without migration: you argue that a constant population size is inferred for Bor by moments and momi because of the restriction on population sizes after the split. You base this claim on a comparison between the log-likelihoods obtained in this model (STRUCT-NOMIG) and the standard model (NOMIG) in which you add this restriction. I didn't fully understand how you can conclude from this comparison that the constant size inferred for Bor is due to the restriction on the initial population size after the split. I think what you need to do to establish this is run the STRUCT model without this restriction and see that you get exponential decrease. Can you elaborate more on your rationale? A detailed explanation should appear in the supplement and a brief summary in the main text.Inference for the Orangutan data set with models with pulse migration: This is a nice result showing that the more pulses you include, the better the estimates become. However, your main example in the main text uses the inferred migration rates. This is a poor example, because migration rates in a pulse model cannot be compared to rates in a continuous model. If migration is spread along a longer time range, then you expect the rates to decrease. So, there is no expectation of getting the same rates. You do expect, however, to get other parameters reasonably accurate. It seems like this is done with 7 pulses, but not so much with one pulse. This should be the main the focus of the discussion of these results.Results - inference of inbreeding coefficients:======================================When you describe the results you obtained for the cabbage data set, you say "the population size for the most recent epoch in our results is underestimated (6 vs 592 individuals) for model 1 without inbreeding and overestimated (174,960,000 vs. 215,000 individuals) for model 2 with inbreeding". The usage of under/overestimated is not ideal here, because it would imply that the original dadi estimates are more correct. You should probably simply say that they are lower/higher than estimates originally obtained by dadi. Or maybe even suggest that the original estimates were over/underestimated?Supplementary materials:=====================Page 4, para2: "Figure ??" should be "Figure S1"Page 4, para 4: Can you clarify what you mean by "unsupervised demographic history with structure (2, 1)"?Page 22, para 2: "Compared to dadi and moments engines momentsLD provide slightly worse approximations for migration rates". I don't really see this in Supplementary Table S16. Estimates seem to be very similar in all methods. Am I missing anything? You make the same statement again in the STRUCT-MIG model (page 23).Page 22, para 4: "The best history for the ORAN-NOMIG model with restriction on population sizes is -175,106 compared to 174,309 obtained for the ORAN-STRUCT-NOMIG mod". There is a missing minus sign before the second log likelihood. You should also specify that this refers to the moments engine. Also see comment above about this result.

Ryan Gutenkunst: In this paper, the authors present GADMA 2, an update of their population genomic inference software GADMA. The author's software serves as a driver for other population genomics software, enabling a consistent user interface and a different parameter optimization approach. GADMA 2 extends GADMA by adding two new inference engines: momi2 and momentsLD, hyperparameter optimization for the genetic algorithm, demes visualization, selection, dominance, and inbreeding modeling, and a new method for specifying model structures. In this paper, the authors show that their optimized genetic algorithm is somewhat more effective than the original hyperparameter settings. They also compare among inference engines, finding some differences in behavior. Lastly they compare with dadi itself in two models with inbreeding, finding better likelihood parameter sets than those previously published.GADMA has already found some use in the population genomics community, and GADMA 2 is a substantial update. The manuscript describes the updates in good detail and demonstrates the effectiveness of GADMA 2 on two real-world data sets. Overall, this is a strong contribution, and we have few major concerns.Major Technical Concerns:1) The authors claim to now support inference of selection and dominance. But what they support is very limited and not very biological. In particular, they currently support inferences which assume a single selection and dominance coefficient for the entire data set (as in Williamson et al. (2005) PNAS). In reality, any AFS will include sites with a variety of selection coefficients, usually summarized by a distribution of fitness effects. Since Keightley and EyreWalker (2007) Genetics, this has been the standard for inferring selection from the AFS. The authors should be clear about the limitations of what they have implemented.2) Figure 4 shows that optimization runs using GADMA 2 tend to find better likelihoods than bare dadi optimization runs. But the advice for using dadi or moments is to run multiple optimizations and take the best likelihood found, with some heuristic for assessing convergence. So most users would not (or at least should not) stop with the result of a single dadi optimization run. It does seem that GADMA 2 reduces the complexity of assessing convergence between multiple dadi optimization runs. But another important consideration is computational cost. (At an extreme, if each dadi run was 100 times faster than a single GADMA 2 run, then the correct comparison would be between the best of 100 dadi runs and a single GADMA 2 run.) It is not clear from the paper how the 100 GADMA 2 runs compare to the 100 dadi runs in terms of computational cost. It would be very helpful to have a table or some text describing the average computational cost (in CPU hours) of those runs.Major Writing / Presentation Concerns:1) Bottom of page 5: The authors are sharing the results of their hyperparameter optimizations from their own server, with uncertain lifetime. These results should be moved to an archival service such as Dryad.Minor Technical Concerns:1) The authors note that the DROS-MIG models had worse likelihoods than the DROS-NOMIG models. Since these are nested models, the DROS-MIG model must mathematically have a better global optimum likelihood. It would be worth pointing out that the likelihoods they found indicate a failure of the optimization algorithms. The authors should also present the DROS-MIG model results in a supplementary table.2) The Godambe parameter uncertainties in Tables S20 and S21 are pretty extreme, sometimes 10^-13 to 10^12. This may be due to instability of the Godambe approximation versus step size. In Blischak et al. (2020) Mol Biol Evol, the authors tried several step sizes and sought consistent results between them (Tables S1-S4). We suggest the authors take that approach here.Minor Writing / Presentation Concerns:1) The author claims that "GADMA does not require model specification". However, it seems that GADMA "structure model" rather describes a different and perhaps broader way to specify demographic models rather than completely eliminates model specification.2) The authors use the term "inference engine" for the four tools GADMA 2 builds upon. But to us, the act of inference includes parameter optimization. In this case, these tools are not being used for the inference itself, but rather to calculate the (composite) likelihood of the data. Perhaps a better term would be "likelihood calculator"?3) The authors suggest engine-specific hyperparameter optimization as a future goal. But the optimal hyperparameters are also likely to be model specific. (For example, 2- versus 4-population models might benefit from different optimization regimes.) Can the authors comment on this?Writing Nitpicks1) Abstract: "optimization algorithms used to find model parameters sometimes turn out to be inefficient" → vague: more details on why/how they are inefficient would be helpful2) Introduction: "Inference of complex demographic histories… in the population's past." needs citation.3) Page 2: "parameter to infer, for example, all migration" is a comma splice and should be split into two sentences.4) Supplement page 4: Figure ?? reference is broken.

Michel Dumontier: This paper describes KGML-xDTD, a knowledge graph-based ML framework to predict and explain potential applications of drugs. The main approach is the use of graph reinforcement learning to predict drug-disease pairs and provide a knowledge-based path as a potential mechanism of action. The method is evaluated against other approaches, various data partitioning strategies, comparison to a manually curated database of mechanisms of actions, and two use cases. The paper is well written, easy to read, and makes a contribution to the scientific literature. Accurate prediction of drug uses remains an important and challenging problem in biomedical informatics. The novelty of the approach is to use graph reinforcement learning to achieve state of the art performance for the problem, and it also is able to generate plausible paths within a knowledge graph to serve as mechanistic explanations. There are some limitation to the work that should be addressed. These include: 1) The baseline models (GAT & GraphSAGE+SVM) only use a small subset of drug-disease replacements. The authors indicate that the smaller subset is necessary owing to time performance constraints. However, there is no discussion as to the possible impact the reduced subset any aspects in relation to their method. 2) The approach only evaluate 3-hop KG paths, which is 1/7 of what is available in DrugMechDB. What is the quality/performance impact of choosing longer paths? Wouldn't the the number of biologically reasonable paths to explain a predict be substantially reduced? I worry that this is cherry picking the dataset to show good performance for the only case (3-hop) that it is capable of (While critizing other methods as not being performant) 3) The authors use RepoDB as one of their sources, and specifically use the "withdrawn" set as true negatives. However, most withdrawn tags are linked to reasons other than safety or efficacy of the clinical trial. As such it is not clear that this set is a good true negative set. 4) The authors use MyChem as a resource for drug indications/contraindications. However, MyChem is not an original source - it aggregates other resources. The authors should properly identify the source of "human curated annotations". 5) I commend the authors for their evaluation, which uses a number of different train/test strategies and against different methods. However, as far as i can see the train/test strategy does not adequately remove similar true drugs-disease pairs from the training/test set. That is to say there are many drugs that are approved for very similar conditions, and therefore it becomes somewhat trivial to predict these (this problem is highlighted in the 2011 PREDICT paper by assaf gottlieb). More work should be done here to report an accuracy based on more stringent evaluation criteria. 6) It's unclear to me that the 124k diseases are real (diagnosable) diseases that could be prescribed for. Inflating the number of possible (but implausible) diseases might augment the performance, but contribute nothing to medicine. Elaborate. 7) Figures 5, 6 are difficult to read 8) It's nice to see the 2 use cases in the paper. However, the extracted subgraphs are quite different than the DrugMechDB MOA paths. So there's something to be said about the succinctness of the DrugMechDB MOA paths, which might prove to be a better training set for some explanation algorithm, rather that one that is independently generated. Overall, this is a nice paper with an interesting approach.

Yuansheng Liu: The paper entitled "KGML-xDTD: A Knowledge Graph-based Machine Learning Framework for Drug Treatment Prediction and Mechanism Description" proposes KGML-xDTD, a two-module, knowledge graph-based machine learning framework . Author constructs a large knowledge graph for the training of the model. The model is divided into two modules, one for drug repurposing prediction and the other for Mechansim Of Action Prediction. Both modules have achieved good results compared with the existing baseline. Here are my specific points: (1) It is mentioned on page 6 that the data are classified into three categories, while other data are classified into two categories. How did you exclude the "unknown" category and adjusted result? (2) Drug Repurposing Prediction model and Mechanism of Action Prediction model seems to be two separate training model. I can not find evidence of multitasking training from the content. If the model is trained separately, which model is the evaluation metrics according to? If training together, the model section should be written more clearly. (3) The introduction part only mentioned about Drug Repurposing Prediction Model, but it didn't describe existing Mechanism Of Action Prediction model. (4) Baseline seems to be Drug Repurposing Prediction SOTA model. But the best performance of the work is about Mechanism Of Action Prediction. (5) The data set appears to selectively chose drug-disease pairs with intermediate paths. But if the drug or disease in the network do not connect, that how dose Drug Repurposing Prediction model perform?

**Reviewer 2. Po-Yu Liu **

The Metaphor is a workflow with high completeness for short-read-based metagenomic analysis. I look forward to its compatibility with long-read platforms (ONT and PacBio). This work is worth publishing. However, it is still a bioinformatic knowledge and skill-required toolkit. If the Metaphor can be integrated into a web-based platform, such as Galaxy or Kbase, it would be more user-friendly for much more users.

This work has been published in GigaByte Journal under a CC-BY 4.0 license (https://doi.org/10.1093/gigascience/giad055) and has published the reviews under the same license. These are as follows.

**Reviewer 1. Thomas Brüls **

The authors present a snakemake-based workflow to automate and chain the main computational ingredients (assembly and binning) of genome-centric metagenomics; the authors developed a technically sound tool for this purpose, and by itself it is certainly valuable to the research community and worth of publication. however, even if the article is casted as a technical note -hence with an emphasis on the design, implementation and assessment of the tool-, I feel that a more thorough discussion of both its abilities and inabilities (e.g. strain resolution, detection of low abundance organisms, identification of virus bins, etc) would be worth for a more general audience. On the same token, a more deep discussion of some of the results obtained with their tool (see below) would be of interest and would also illustrate useful use cases. I would suggest the following modifications/additions:-the experiments with the strain madness dataset suggest that the genomes (or fragments thereof, i.e. the bins) resolved should be viewed as "species" genomes, or composite genomes possibly originating from multiple strains. if so, do the authors think this represents a hard limit to the assembly + binning approach, or could further existing tools (e.g. performing variant detection on top of cross-assembly before the binning step) be integrated or developed in the future for strain-resolution (i.e. to identify strains not dominant in any sample)? -related, a simple summary of the number of individual strains recovered in individual bins for the strain madness experiment would be interesting.-another issue that would be worth discussing in my opinion is the impact of genome abundance on the recovery of corresponding bins and their quality. the platform developed by the authors appears to be well suited for such kind of analyses and the results would be of both theoretical and practical interest. to put it simply, what is the minimal initial coverage of genomes required in order for them to be recovered in bins of a given size and quality?-rem: theses two issues (strain-level diversity and individual strain genome abundances) likely interact to limit bin resolution, and this could be mentioned by the authors.-the data presented by the authors suggest that the metabat binning engine significantly outperforms the other two tools (concoct and vamb, which are both widely used), see e.g Figure 2; what would account for that, and do the authors think this is a general observation (i.e. beyond the specific CACB setting or marine metagenome shown in Fig 2)? -a bin refinement step (based on the DAS tool and dereplication) is frequently mentioned but should be more detailed (including a precise definition of the bin quality metric used).

further rather minor comments: -in the abstract, when mentioning "technical challenges associated with...", it would be worth mentioning that algorithmic challenges are present as well. -in the introduction, "It is hypothesised that pooled assembly and binning may lead to improved results when analysing communities with high genetic diversity, and to poorer results when there is a high level of intraspecies/strain-level diversity". I would assume there are many instances in the real world that are both, i.e. that present both high inter-species and intra-species genetic diversity, what then?-in the future directions, the authors mention the identification of eukaryotic and viral contigs and bins, and could shortly elaborate how this could be done properly. -the sentence "In summary, our assessment of ..." at the end of the ms appears to have a syntactic problem.

**Reviewer 2. Paolo Provero **

In this work Himmelstein and collaborators introduce a statistically controlled way of extracting significant node pairs in heterogeneous networks (hetnets) without relying on a ground truth and related training. The method "explains" why two nodes are significantly connected by extracting the metapaths most responsible for the enrichment. This is based on computing a null distribution of the DWPC, which allows assigning a P-value to each metapath joining two nodes, and then to visualize the individual paths responsible for the enrichment. The method is novel and significant, and can be in principle be applied to many hetnets, in life sciences and beyond, when a ground truth is not available or not desirable as it would introduce bias. The software tools developed appear to be readily available to other researchers.

Major comment: If I understand correctly, given two nodes (say "Alzheimer disease" and "Circadian rhythm") the method extracts, in a statistically controlled way, the most significant metapaths joining the two nodes, and then the individual paths responsible for the enrichment. But this is not the most obvious question a life scientist would ask the network, which would be instead something like "Which are the pathways most significantly connected to "Alzheimer disease"? Indeed this type of question would be the one to ask when aiming for drug repurposing (possibly replacing "pathways" with "compounds" or "pharmacologic classes"). Based on Fig. 4A, the pathways are presented, or "suggested," in decreasing order of number of metapaths, but this is hardly a ranking by significance. Would it be possible to summarize the results in such a way as to rank the pathway nodes connected to a given disease node by significance (or more generally to rank the nodes of a certain type by the significance of their connection to a given node of another type)? This should be discussed.

I also have several minor concerns. (1) The authors introduce and compute a null distribution of the DWPC which takes into account node degree in a statistically controlled way when evaluating the connectivity between two nodes. However, the DWPC itself does take into account node degree, as the name implies, and contains a tunable parameter that can be optimized, at least when a ground truth is available (as in Ref 39 by the same first author). I understand such tuning is not possible when, as in the present case, no ground truth is available, but the authors should make this point more clearly. (2) I find Fig. 1B a bit confusing: according to the legend, the top rows are known treatments, which should have higher than expected connectivity. However, based on the colors as explained by the legend, the bottom treatment/disease pairs seem to have higher connectivity (3) The acronym DWPC is defined after it has been used several times (4) The legend of Figure 2 should specify that these results apply to the nodes "Alzheimer disease" and "Circadian rhythm", although this becomes clear in Fig. 4 (5) I don't think Figure 3, representing the home page of the web site, is especially useful (6) I found Fig. 4 confusing: the sum of the path counts for the selected metapaths in panel B is way larger than the 425 results shown in Panel C. As far as I understand no path can belong to more than one metapaths, so is there some further selection here? (7) The "Frontend" section of the Methods seems a bit too detailed for the Gigascience audience.

Re-review: The authors have addressed all my comments in a satisfactory way.

This work has been published in GigaByte Journal under a CC-BY 4.0 license (https://doi.org/10.1093/gigascience/giad047) and has published the reviews under the same license. These are as follows.

**Reviewer 1. Karthik Raman **

The paper is very well-written and addresses an important problem. The database appears easy to use and contains a lot of pre-computed data, which will be useful for researchers to query and generate useful insights. I only have a few minor comments, which if addressed, could further strengthen this manuscript.

Minor comments: Without line and page numbers, it was a bit tricky to point out the issues.

1. "One such application" in the introduction does not read well - just "one application"2. It is nice to see that DWPCs that are not retained by the database can be generated on the fly. The para goes on to mention "while still allowing on-demand access to the full metrics for all metapaths with length ≤ 3" --- is it also possible to generate metrics for longer paths if needed?

2. Below Fig 2, there is a point about the adjusted p-value. I see that the discussion about FDR is presented later in the manuscript (and well justified), but there could be a pointer here to that section.

3. Is there a possibility to include other computations like betweenness centrality and motifs also? This kind of data looks really ripe for an excellent analysis of repeated motifs etc.

4. I found the Methods extremely long and may be a bit distracting for readers of this manuscript --- I was wondering if some of these can be moved to Supplementary.

5. In the section on "Details of matrix DWPC implementation", it is stated that "our matrix methods were validated". It is not clear where these validations have been discussed.

Supplementary? 7. In the section on "Permuted hetnets", it is not fully clear what the parameters for XSwap algorithm was. What were the parameters, e.g. number of swaps, etc.?

1. In the section on "Details of the gamma-hurdle distribution", there is perhaps a missing equation below the second statement of "The probability of a draw from the distribution is"

2. The validation here which points to an ipynb, could be put in Supplement.

3. In the section on "Prioritizing enriched metapaths for database storage", what is the logic underlying the choice of parameters? "For metapaths with length ≥ 2, we chose an adjusted pvalue threshold of 5 × (nsource × ntarget)^−0.3".

4. Under "Visual Design", are the colours chosen "colour-blind friendly"?

**Reviewer 2. Philippe Rocca-Serra **

The reviewer thanks the authors for their efforts in producing the submitted manuscript. The authors describe a django based web application designed to support data management. The tool is built to support experimental metadata capture using the ISA format in its tsv form. The tool relies on irods to manage data files associated with the experimental metadata. The tool offers programmatic access via an API and clear front end.

Main comments: The title: "SODAR: enabling, modeling, and managing multi-omics integration studies" could be clearer.Being more concise "SODAR: standard compliant management of multi-omics studies " would deliver a better message. Page 1 , Abstract: it would benefit from further refinement as there are several repetitions. Check 3rd sentence for English. "ranging from....to..." , s/whereas/to/"Scientists from diverse backgrounds also have different demands for interfacing with the data, ranging from computational users that need programmatic or command line access whereas non-computational users need graphical interfaces. "to:"Scientists, with different backgrounds, ranging from computational scientists to wet-lab scientists, have different needs when it comes to data access, with programmatic interfaces being favoured by the former and graphical ones by the latter". Instead of saying "under a permissive licence", be more explicit and plainly state "under MIT licence. "Page 2, Introduction:what is the difference between " data analysis and integration of data"? Repetition/redundancy in "An example of such complex study is (Esterhuyse et al., 2015) in infection biology, which will be used as an example below. "Suggestion:Use of term "modeling": using "plan" or "planning" may be better to remove any ambiguity about the nature of the modelling (statistical modeling, data modeling). Alternating, perfer 'representation' or 'representing'. (the term model is repeated many times in the following sentences) The statement "The most comprehensive standard for describing study metadata is the ISA-Tab format ..." is probably too strong. There are more formal (UML) models such as FUGE-OM (https://doi.org/10.1038/nbt1347 ) or CDISC SDM & SDTM.A more understated assessment such as "a popular standard, owing to its simplicity, is the ISA-Tab format""Alternatives include..." possibly cite other options for managing such complex datasets as seen with BIDS in neuroscience (Gorgolewski, K., Auer, T., Calhoun, V. et al. The brain imaging data structure, a format for organizing and describing outputs of neuroimaging experiments. Sci Data 3, 160044 (2016). https://doi.org/10.1038/sdata.2016.44) or why not mention HDF5 specification. This section could be improved by refining the transitions between the different ideas presented or organising the flow. For example, by layout out the challenges of 1/ dealing with experimental metadata and 2/ dealing with digital objects produced by instruments, which have the characteristics outlined by the authors (volume, depth). Then review the technical solutions and then present the choices made by this implementation and possibly identify the selection criteria which led to choosing one specification over another. Results:Page 4: " Non-computational users can interface with SODAR using the graphical UI, whereas computational users can use command line interfaces and REST APIs from scripts and other external software. "Repeat from the abstract. I would suggest rephrasing to 'humanise' 'computational users' vs 'non-computation users', and identifying the function and roles in actual labs (bioinformaticians, data analysts, aka dry lab scientists) vs (experimentalists, wet-lab biologists). Figure 1: same comment (in fact confirming by the choice of characters).a question about the diagram: Is it the case that the Web UI does not talk to server via the API as done in some modern development. Probably highlight there the reliance on the Django framework. Section 2.1The first sentence needs attention, check the English. "for both serving for modeling experiments..."Also, there are systems (EBI Metabolights tools on their github repo, DataVerse, FAIRdom SEEK, Zendro...).So the story telling should probably first talk about the survey of the existing and then only bring to arguments justifying new development. Table 1.It is odd to lump blanket statements for tools such as LIMS, ELN or 'Study Databases' without clearly stating which ones specifically have been evaluated. It seems that one could formulate a table with very different results.

Question: How was selection bias controlled for? Page 5:This section should be reorganised and each explanatory statement refined to add clarity. Case in point:"Arbitrary Experiments": Does experiment equate 'ISA.Assay'? is it akin to a Workflow or process Sequence ? Question: among the key feature that such a system should have to support the work of dry/wet lab scientists, surely, deposition to public repositories should be high on the list. Why is this absent? Page 6:typo: s/bioinfsormaticians/bioinformaticians/punctuation: to be checked: missing commas make for a difficult read.suggestion: simplify the role of 'experimentalists' in the context of SOBAR."They use the templates provided by the Data Stewards to instantiate a wet lab track and track its metadata." Question: How are data stewards trained in ISA-Tab? Access to the demo tool gives the opportunity to use and test the component. While the UI is simple and intuitive, a number of limitations in the editing functionality make usage more difficult that it needs to be.Page 7:"of course, using the REST-API of SODAR, it is possible to automate these tasks" Could the author produce a jupyter notebook showing how to do so? It would be a nice addition and possibly a good resource that could facilitate uptake. Section 2-3:page 8-9-10: this section could be streamlined and condensed to really focus on the interaction between shaping a sample processing & data acquisition workflow into a template which can be used by a wet lab scientists. All this while allowing a markup with ontology terms. Note: the ontology terms on the demo server do not resolve properly. Question: Why choosing Bioportal over other services, e.g. EBI OLS? Question: How can value-sets be constrained in SODAR? Question: ontology browser: it is unclear if the ontologies need to be loaded locally or if they are accessed via an API call to the relevant services ? Can the authors clarify this point? the demo server did not seem to allow it or I wasn't able. may be a figure showing the functionality would help? Page 11: Internal Usage Statistics Question: it seems that the mean size of an experiment stored in SODAR is ~60 samples and about 10 files per sample. These are relatively small sized studies. Can the authors provide insights about the performance of the platform with large studies (several thousands of samples and above)?

Methods: Question: Installation and deployment of SODAR.Why the authors omit to mention that SODAR can be deployed via Docker? It seems useful information. Question: AltamISAChecking the library, it seems that development has stalled. It is a concern? Have the authors tested swapping AltamISA with ISA-API ? Is it at all possible ? could it be made via an adaptor of some sort? Can Altam ISA convert to ISA-JSON or other public repository compatible format to provide a capability to assist users disseminate their results? Comment: figure 3 should not be a supplementary material but a proper content as it is useful as showcasing SODAR UI and customization.

Re-review: The reviewer thank the authors for their efforts and extensive rework of the manuscripts, and for delivering this software stack. minor corrections:

page 4, 2nd paragraph, first sentence: typo -> s/approaching itusing/approaching it using/page 7, 2nd paragraph, suggested edit:change from: "For publication, raw and processed data and metadata are deposited in scientific catalogues, study databases and registries. An example is the BioSamples database for metadata [22].""to:For publication, metadata and raw or processed data are deposited in scientific catalogues, study databases and registries. Examples are the BioSamples database for metadata [22] and Short Read Archive for raw sequencing data [citation needed]."

"important clarifications: 1. this sentence makes a disservice to the manuscript: "Our work isrepresentative of the work typically done by core units in clinics. Clinical settings often deal with humans as their primary sample source. This implies controlled access of data, or not being allowed to share confidential data. Thus, developing support for hosting data in a public repository is not our aim. Likewise, uploading data to other public repositories has not been a priority. "Two reasons:- the first one is opening the can of worms of data governance and oversight of patient related information. I would steer clear of that in this piece.- the second one is because i would flip the argument around. "While deposition to public repositories was not necessarily the priority, the development of an (almost, see below ) ISA compliant system provides such a capability should the data owner need it" 2. in the result section, or in the documentation, a welcome addition would be example of templates for non-sequencing based assays. For instance, since the authors mentioned their need to support proteomics and mass-spectrometry users, it would make sense to highlight the templates available. In other words, it would help the target audience of the manuscript locate 'metadata profile definitions' (somewhat akin to ISA configurations) for specific assay types. If I have missed it from the manuscript or the github repo, please ignore. 3. "dialectic" ISA format:Several examples are available from the GitHub repository generally follow the ISA-Tab specifications but also introduce a local field: "Library Name". While such value would make sense in the official ISA specification, it is currently not supported. This leads to the creation of a diverging format.It would be sensible to keep the "Library Name" as an presentation label (for display in the UI) and substitute it to "Labeled Extract Name" when exporting outside the database to the tab format, in order to retain compatibility with other ISA parser and the official specifications. It could be added as an output option to the Altam-ISA parser in case deposition to public repositories is needed (e.g. EMBL-Metabolights). This would go some way in helping 'Interoperability' and would not be too onerous a change. Worth of note, I was recently made aware that ENA repository would be accepting submission in ISA-Tab and ISA-JSON format, hence raising this point to the authors. Suggestion: clarify this in the Methods section. Also, it seems the following example is missing 'Assay Name' and 'Raw Data File' fields:https://raw.githubusercontent.com/bihealth/sodar- paper/main/GSE96583_PBMC_Single-Cell_Demo_Project/a_PBMC_test_scRNAseq_nucleotide_sequencing.txt

This work has been published in GigaByte Journal under a CC-BY 4.0 license (https://doi.org/10.1093/gigascience/giad052) and has published the reviews under the same license. These are as follows.

**Reviewer 1. Xiaotao Shen **

The authors developed the SODAR tool, which supports multi-omics integration studies. This is a great tool that has a user-friendly interface and supports multi-omics integration. However, I have several concerns that need to be addressed before this manuscript can be considered to be published. How does the SODAR handle the multi-omics data that are from different samples? For example, the gut microbiome data from stool samples and proteomics data from blood samples, which may be from the same person but collected at different dates. Since SPDAR supports cell editing, so how does it make the metadata and expression data consistent automatically? The authors claim that the SODAR can support multi-omics integration studies. However, I didn't find out how SODAR can do that. Could the authors give more descriptions about that?

Re-review: The authors have addressed all my comments and concerns.

**Reviewer 2. Jianxin Wang **

In this manuscript, the authors present MuLan-Methyl, a deep-learning framework for predicting 6mA, 4mC, and 5hmC sites. They use DNA sequence and taxonomic identity as features, and implement five popular transformer-based language models in MuLan-Methyl. MuLan-Methyl is open-sourced, and a web server is also provided for users to access it. Overall, I think the methodology of MuLan-Methyl is clear and innovative, and the experiments seem comprehensive. However, I do have several concerns that I believe should be addressed before the paper is accepted by GigaScience.

Major 1. One major concern is that, in my opinion, DNA methylation is dynamic. Cytosines in the same position of the DNA sequence may have different methylation status in different samples, different cells, or even in different development stages of a cell. So, how can we predict the methylation status of a site based on only its sequence (and taxonomic identity)? -- The authors should clarify that in what cases, MuLan-Methyl (as well as other methods that use only DNA sequence) can be used to study DNA methylation, in Introduction or Discussion section. -- The authors discuss motifs in Fig. 3, but only for positive samples. How about the motif distribution in the negative samples? Can I understand that this method is actually for discovering motifs (or sequence structures) that are highly correlated with methylation? -- How is the performance of MuLan-Methyl without taxonomic identity? 2. The authors compared MuLan-Methyl against iDNA-ABF and iDNA-ABT, especially on the independent test set (Fig. 2E). I think the authors should clarify that whether they trained the models of the three methods using the same training datasets. If not, the authors should clarify the reason. 3. I'm curious about the computational efficiency of MuLan-Methyl. How many parameters in its model? Does MuLan-Methyl have advantages over other methods in terms of computational efficiency?

Minor 1. I don't understand why the references were not ordered from 1 in the main text. 2. I suggest that the authors re-organize the Introduction section. There are too many small paragraphs in this section. 3. At the end of Page 2, "The type 4mC type is present in 4 species" should be corrected.

Re-review:

The authors have addressed most of my concerns. However, I still have one minor concern about the computational efficiency. The response of the authors is not convincing by only saying "The number of models that MuLan-Methyl need to train and test on is less than the others, thus it has better computational efficiency than other models to some extent". If possible, I strongly suggest that the authors show some data to compare how much time and resources (GPU/CPU/RAM) needed by each method. The authors have addressed most of my concerns. However, I still have one minor concern about the computational efficiency. The response of the authors is not convincing by only saying "The number of models that MuLan-Methyl need to train and test on is less than the others, thus it has better computational efficiency than other models to some extent". If possible, I strongly suggest that the authors show some data to compare how much time and resources (GPU/CPU/RAM) needed by each method.

This work has been published in GigaByte Journal under a CC-BY 4.0 license (https://doi.org/10.1093/gigascience/giad054) and has published the reviews under the same license. These are as follows.

**Reviewer 1. Yupeng Cun **

Zeng et al. proposed an ensemble framework for identifying three type DNA-methylation sites, and performed a benchmark comparison in multiple species' genomic data. This paper give a valuable study on how ensemble transfer learners works and the predictability in different species. My suggestion is the manuscript acceptable with following minor revision: 1. Calculated a consensus ranking using Kendall's tau rank distance method for each method in Figure 2-C. 2. the multi-head self- attention and self-attention head formula should redescribed by following this preprint: https://arxiv.org/pdf/1706.03762.pdf 3. MLM and MuLan-Methyl mixed in some cases, which need be used in a consensus way.

**Reviewer 2. Luohao Xu **

This manuscript by Barros et al. presents a high-quality dipoid turkey genome assembly which shows significant improvement relative to the previous one. This new assembly is timely and will likely be used as the reference turkey genome, but the authors should acknowledge that the W chromosome is absent (because the F1 individual was a male?). This manuscript fits more with "Data Note" than "Research" as I see most results are descriptive and confirmatory. While the chromosomal assembly is relatively complete, I am concerned whether it still contains assembly errors (because of not being polished by long reads?) which led to fewer genes annotated. This assembly metric needs to be taken into accounts if this assembly were to be used as a reference. The authors need to provide the QV value (see the VGP standard), and evaluate indel errors in coding regions. Some of the results are very brief without showing details or a figure, so difficult for assessment, for instance those SVs affecting genes. Page 4, "two most important avian agricultural species", I think duck should be the second most important poultry species? Page 5, I believe the "F1 assembly" refers to the primary assembly or collapsed assembly - please define it more clearly. Page 6, it's unclear how the 36 chromosome models are defined, particularly for small microchromosomes (29-35). According to the karyotype of turkey (2n=80), a few chromosomal models are missing. Page 6, "This captures the chromosome arms in a single contig" does it apply to all chromosomes? This is unlikely, and data is not shown. Page 6, any idea why the coverage of two parents differs (110X vs. 137X)? Page 6, "anchored the assemblies to the F1 assembly using RagTag". This suggests and chromosomal assembly of the two haplotypes was not independent, and replied on the F1 assembly. This can potentially lead to missing structural variations between two haplotypes (inversions, translocations). Page 7, please show more data to support the correct assembly of the chrZ inversion, including Hi-C heatmap, and long-read alignment spanning the inversion breakpoints. Note the Z chromosome inversion has been reported in Zhang et al. 2011 (BMC genomics), which is not cited until in the Discussion. Page 8, it's possible some genes were not annotated because of the presence of indels in coding regions. The genome assembly QV value can be calculated to measure the error frequency (Rhie et al, 2021 Nature). Page 8, please provide a statistical result for gene density comparison. Page 8, at the bottom, please cite the sources of these bird genomes. Page 9, "Gene family contractions and expansions". These analyses were a bit crude. " Orthologous groups" is not equivalent to "gene family". Page 10, the phrase "F1 and parent assemblies" is confusing. Both haploid assemblies are derived from the diploid F1. Consider changing to "paternal and maternal genomes". Also, as I commented above, both parental chromosomal assemblies are based on the same reference (Mgal_WU_HG_1.0), so the contigs were ordered and placed in the same way. This process could mask the potential non-co-linear segments. For a more appreciated way to independently assemble two chromosome-level assemblies, see the marmoset diploid genome paper (Yang et al., 2021 Nature). Page 10, please use a figure to show the SV over the BLB2 gene. Page 11, again, please visualize the result on the MAN2B2, GEMIN8, RIMKLB and RALYL cases. Page 11, "Loss of function variation", I am wondering whether variations mentioned in this part are fixed in the corresponding populations? Page 11, "Knockouts of this gene lead.." reference is needed. Page 12, "Avian genomes are known to…" references are missing. Page 12, "Distinct genomic landscapes of turkey micro and macrochromosomes", some patterns have been described in the literature, for instance, 10.1111/nyas.13295. Please also perform some statistical analyses to support the claims, not just a figure. Page 13, "Conserved synteny within the Galliformes clade", please cite 10.1159/000078570 and 10.1007/s00412-018-0685-6 Page 13, "it is evident that especially the Z chromosome" also observed in 10.1038/s41559-019-0850-1 Page 13, "inversion of around 19 Mbp on the turkey Z" also reported in 10.1186/1471-2164-12-447 Page 14, "tail of the chicken Z chromosome lacks synteny" also reported in 10.1038/nature09172. This means figure S11 does not provide a novel finding. Page 14, "Combining long reads and genome-wide chromatin interaction data (Hi-C) enables the capture of chromosome arms in a single contig", again, is that correct, chromosome arms in a single contig? Page 18, it's known wtdgb2 assembly tends to contain errors, but it looks the authors did not use long reads for polishing, but only used short reads? Page 20, "The corrected reads from TrioCanu were mapped to the Triocanu assembly with Minimap2 v2.17-r941 (Minimap2, RRID:SCR_018550) [45], options -x map-pb", what was is used for? Page 20, "Duplicated sequences were removed." How was this done?

Re-review The manuscript has been improved. After reading the revised manuscript, I have a few more concerns.

Chromosome models. I suggest the chromosome naming should follow chicken's, e.g., chr6 can be chr2a, and the microchromosomes should be named according to chicken homology. I then noticed chr32 and chr35 do not have chicken homology which is very concerning. It is either due to novel. chromosomes (very unlikely), or the sequences could be an unlinked contigs. In either scenario, the chromosome models must be clarified. The authors should provide strong evidence to support the chromosome model assembly for chr32 and chr35, e.g. FISH images, Hi-C zoom-in view (Fig. S1 shows the whole genomes where the microchromosome models are not visible), synteny with chicken (note there is a new chicken assembly ASM2420605v1) or zebra finch chromosomes; otherwise, chi32 and chr35 can not be identified as a chromosome. Centromere and telomere. To support complete chromosome assembly, I suggest the authors provide information about the assembly of telomere and centromere sequences, e.g. the presence/absence of TTAGGG at chromosomal ends. Most galliformes microchromosome centromeres are known to contain a 41-bp satellite (10.1139/gen-2022-0012). The authors should investigate whether such centromere satellites are present in the assembly. Data availability. It appears the Hi-C data is not available in NCBI. The raw reads must be provided. In the abstract, there is not such term as "complete scaffold", please remove "complete". Again, I do not see the support for two chromosome models: chr32 and chr35. The chrZ inversion is highlighted in the abstract, but this is not a novel finding - the writing is thus misleading. Instead, the new genome assembly only CONFIRMS this inversion. The subtitle "Lineage specific expansion and contraction of protein-coding gene families" is unrelated to the following text. "a 1.47 Mbp inversion on chromosome 1" I am wondering if this is the centromere? According to chicken chr1 centromere position, it looks like so. In the Table 5, the Parent2 has a much large size of gained copy. Please show more details, e.g. chromosomal distribution "BLB2", is this gene associated with parent2-specific trait? Similarly, what about TRIM36, GRIA2 and MAN2B2, and LRRC41? "The inversion was supported by a normal alignment at the approximate breakpoints (Supplementary File 1: Table S7 - Figure S16) and by the HiC contact map". The writing here is unclear. Hi-c data does not show signal for inversion, instead, it only supports that the assembly is correct. Bellott et al 2020 should be Bellott et al 2017. "Centromeres, however, are too long to traverse reliably in most cases". I do not see any analyses on centromeres. PRJEB42643 does not contain Hi-C data

Re-re-review A new chicken genome has been published during the revision: https://www.pnas.org/doi/10.1073/pnas.2216641120, I suggest the authors revise some parts of the manuscript: e.g. L66, L78, L83-85 L103, please make it clear only the F1 was sequenced with long-read. L117-142, those results are very interesting, but perhaps the language can be more concise. L231-236, this paragraph is not important, please either move them to supplementary material or remove them. In general, this manuscript can be much more streamlined. L310-315, this part has also been reported by Huang et al. 2023 PNAS, so this is not a novel finding. Please either streamline or remove it. L327, ref 36 is not a "recent" finding.

This work has been published in GigaByte Journal under a CC-BY 4.0 license (https://doi.org/10.1093/gigascience/giad051) and has published the reviews under the same license. These are as follows.

**Reviewer 1. Yunyun Lv **

Reviewer Comments to Author: The turkey has importance for agriculture as it is the second contributor to word poultry meat production. This study completes a chromosome-scale genome assembly with long reads sequencing and use trio-binning approach to generate a haplotype-resolved turkey genome, which give scientific significance to further genetic studies within this species. However, I feel the content within this article need improvement. Some parts were unclear and hard to follow, I list some of them as below. After substantial revisions, I will suggest the publication.

In abstract: The sentence "These assemblies cover 35 chromosomes in a single scaffold and show improved genome completeness and continuity" seems weird and hard to understand directly. Please revise it and make it clear. "The three assemblies are of higher quality than the previous draft quality assembly and comparable to the current chicken assemblies (GRCg6a and GRCg7)." Please indicate the parameters used for comparison clearly and how prove them with a higher quality. "Structural variation between the parent haplotypes were identified in genes involved in growth providing new target genes for breeding." The theoretical context of this sentence is not clear, so I suggest more information added to make it clear.

Considering no statistic in the conclusion, I suggest the conclusion sentence can be revised as "we contribute a new high-quality turkey genome at chromosome-level, benefiting turkey genetics and other avian genomics research as well as turkey breeding industry."

In the introduction: "Most of the chromosomes are small microchromosomes, while only a few macrochromosomes are present in the karyotype." Please clearly indicate how many microchormosomes in turkeys and chicken. "most of" is uninformative for readers. "and by current standards would be considered of draft quality". What is the current standards? Please indicate it clearly. "Ongoing efforts in producing high quality assemblies of the microchromosomes in avian genomes have been unsuccessful due to multiple causes" what the multiple causes represent for? Or the features of microchromsomes leads to the unsuccessful assembly as mentioned above? "For instance, improved annotation of (non)-coding genes benefits the functional interpretation of genome wide association studies (GWAS), and aids in identifying targets for gene editing", why are non-coding genes (I understand the non-coding genes are referred as regulatory regions, but actually, they are not real genes.) benefits …? Why protein-coding genes (structural genes) can not undertake the roles? "The genome assemblies of turkey (this paper) and chicken, however, are of considerably higher quality compared to other Galliforme species. This provides opportunities for an in-depth comparison between the two most important avian agricultural species." I cannot follow the logic of why the placement of this sentence is here. Obviously, it should be part of discussion after the comparison of turkey genome with other avian genomes. "In this study we use a relatively new technique, the trio-binning approach, to construct high quality haplotype-resolved turkey assemblies." I feel it is necessary to give an explanation of the term "trio-binning approach" as many readers do not understand what is standard for? And the long-reads sequencing technology within it also connect the former theoretical context closely.

In results: Have you used other assemblers to complete the genome assembly? Such as flye, or nextdenovo, or mecat2 that may have better performance. Have you ever tried 3D-dna for chromosome-scale assembly? which may be better as my experience. The gene annotation should be assessed by BUSCOs.

In discussion: "The quality of the assemblies presented in this study confirms the value of this method in not only providing a quality assembly but also in uncovering structural genomic variation." Please indicate which quality index that reflect your genomic assembly. "Thanks to these recent sequencing technologies, we are able to correct a number of wrongly oriented contigs in Turkey_5.1, a phenomenon often observed in short-read based assemblies." I feel this sentence is not formal in writing.

Re-review: The author has carefully amended the work in response to my prior concerns, and the quality of the new version has greatly improved, hence it is suggested that the manuscript be accepted.

Editor’s Assessment

This work has generated metabolic models for the human pathogens Mycobacterium leprae and Mycobacteroides abscessus, alongside a new computational tool that can be used to identify potential drug targets. The standardised genomic scale metabolic models have been developed using the systems biology community standards for quality control and evaluation of models. After providing more detail on reproducibility, comparative performance of the models, and reuse, these resources are now published and are available for reuse by the global scientific community via the GigaDB, Biomodels, and PatMeDB repositories.

This assessment refers to version 1 of this preprint.

This work has been peer reviewed in GigaScience (see https://doi.org/10.1093/gigascience/giad048), which carries out open, named peer-review. These reviews are published under a CC-BY 4.0 license and were as follows:

Elizabeth Ryder

This technical note is an informative explanation of Training-Infrastructure-as-a-Service, which is a free service available to facilitate Galaxy training sessions. The service provides an easy way for instructors to set up infrastructure for trainings, enables learners to make progress through the training without long waiting times, and includes a dashboard through which instructors can easily monitor progress of learners. The article provides data showing the large number of events and locations that have benefited from using TIaaS. Because of the utility and general applicability of TIaaS, the article will be of interest to the readers of GigaScience.Minor suggestions:In the Development section: As a practical matter, it would be useful to know the typical timeline for approval of a training session. Also, can anyone who uses Galaxy become an instructor and request this service?In the Usage section, there is a sentence that reads, 'Class sizes have ranged considerably, from the median of 25 participants (std. dev 121) to a maximum of 1500 registrants for afully asynchronous (self-paced) course.' It's a little unusual to talk about a median and standard deviation, since medians are non-parametric measures and SDs are parametric and measured with respect to the mean. I'd suggest using the median and interquartile range instead. I think a histogram of class size distribution would be informative, similar to the event distributions in Fig. 4.Grammatical / spelling errors:I'm not sure why 'Findings' appears before 'Background' - perhaps an editing error?p. 2'a limiting factor for events with large number of participants, 'should read'with a large number of participants''by it's design'should read'by its design''which to to preference'should read'which to preference'p.4'univeristy'should read'university'p.5This sentence is hard to scan as written; I think it needs a semi-colon after 'cluster' to make sense. Galaxy Europe uses it with HTCondor, and job rules that allow spill over to the main cluster, new machines are brought up in an OpenStack cluster specifically for training events and destroyed afterwards.

This work has been peer reviewed in GigaScience (see https://doi.org/10.1093/gigascience/giad048), which carries out open, named peer-review. These reviews are published under a CC-BY 4.0 license and were as follows:

**Azza Ahmed **

The paper is well-written and neatly reports on the development of Training-Infrastructure-as-a-Service (TIaaS), a free infrastructure resource originally developed by Galaxy Europe and the Gallantries project together with the Galaxy community. TIaaS is a step towards democratizing bioinformatics training, where infrastructure can be a major barrier- even in advanced and well-developed countries.I specially appreciate the value of this resource for instructors and students in low and middle income countries where infrastructure limitations may be exacerbated by the availability of well-trained system administrators able to cater specific training needs. It was indeed gratifying to see training events using TIaaS in such countries in the figure 3 map- especially that it is not clear TIaaS is deployed in such counties. The utility of the resource is self-evident: 438 training events in 48 months targeting > 19000 students. Thus, overall, I congratulate the authors for the success of their project, and the community for having such a great free resource at their disposal.

This work has been peer reviewed in GigaScience (see https://doi.org/10.1093/gigascience/giad043), which carries out open, named peer-review. These reviews are published under a CC-BY 4.0 license and were as follows:

Samuel Lambert (revision 2)

I commend the authors for doing these extra analyses focused on more real-world applications of the method and adding them to the paper. I think the discussion is better contextualised and my final recommendation is that these warnings/caveats are placed in the software documentation as well (https://choishingwan.gitlab.io/EraSOR/).

This work has been peer reviewed in GigaScience (see https://doi.org/10.1093/gigascience/giad043), which carries out open, named peer-review. These reviews are published under a CC-BY 4.0 license and were as follows:

Samuel Lambert (revision 1)

The revised manuscript is much clearer and better illustrates when and how to use the EraSOR method. However, I still think important analyses reflecting more common use cases are missing:- Use of EraSOR with multi-ancestry summary statistics- Use of EraSOR corrected sumstats with other PGS-derivation methods (e.g. LDpred or PRS-CS).- Providing results of a real sensitivity analysis for sample overlap. I understand that you won't know the true overlap in UKB but the difference in the adjusted and unadjusted SumStats performance in the presence of known overlap would be illustrative. Adding these analyses to the real UKB section would greatly benefit the manuscript and utility of the method. Apart from that I note that related to line 19, the impact of sample overlap was also outlined as a pitfall by Wray et al Nat Genet (2013, PMID:23774735).

This work has been peer reviewed in GigaScience (see https://doi.org/10.1093/gigascience/giad043), which carries out open, named peer-review. These reviews are published under a CC-BY 4.0 license and were as follows:

Samuel Lambert

In this paper Choi et al. describe EraSOR, a new tool to remove the effects sample overlap between a set of summary statistics and a target dataset. EraSOR works by running a GWAS in the target dataset and then using LD-score regression techniques to estimate the heritability, genetic correlations of the phenotypes, and number of overlapping samples to decorrelate the effect sizes. The method is thoroughly described, and the simulation scenarios are relevant and well-motivated. However, the manuscript could better describe the inputs and characteristics of the decorrelated summary statistics, focusing more on the degree of bias in effect sizes rather than p-value inflation, and the practicalities of how the tool may be used.Specific Comments: The results of Figure 1/Supp Figure 1 are highly motivating, but the p-value of the association doesn't seem like the perfect measure of inflation. Plots of the effect size of the PRS compared to its expected effect (0, based on heritability) would better illustrate this. The paper proposes a method to remove the effects of sample overlap on summary statistics, but instead mostly focuses on how overlap biases the results of PRS prediction. Additional exploration of the decorrelated summary statistics themselves is needed to illustrate the validity of the method. Specifically, how different are the EraSOR adjusted summary statistics from the true summary statistics measured without sample overlap (e.g. distribution of effect sizes differences); what types of variants does EraSOR fail for or overcorrect (e.g. MAF differences between the summary statistics and the target cohort)? Are the results used as-is in other analyses, or do they have to be filtered in some way? The PRS analyses in the paper all use PRSice to perform clumping+thresholding, selecting the best p-value and LD thresholds on the target datasets. This could be considered overfitting to the target data, and other derivation methods that do not require a sample to optimize hyperparameters (e.g. PRS-cs, LDpred-auto) could be used. It would be good to provide some additional analyses showing that EraSOR outputs also work with other methods of PRS derivation, and that the results are not sensitive to overfitting through hyperparameter optimization. The PRS analysis of the real phenotype data in UKB should be expanded. Currently the analysis uses summary statistics derived in UKB with varying levels of overlap; however, this does not match the real scenario that EraSOR will likely be used in (applying EraSOR to an externally-sourced GWAS and applied to UK Biobank). The authors should perform a descriptive analysis to show that EraSOR is useful in this real-world scenario by downloading summary statistics from the GWAS Catalog (with and without inclusion of UK Biobank), applying EraSOR, and quantifying the difference in accuracy (r2) and effect size. On a related note: does the ancestry of the summary statistics have to perfectly match the target cohort? How well does EraSOR work with multiancestry summary statistics where the LD-panel might be mismatched? The point about insufficient adjustment the authors raise on lines 336-42 is quite important. Proper signposting about the limits of the decorrelation is needed in the software description and the discussion. From this passage that the authors suggest that known sample overlap should be avoided and EraSOR should only be used as a sensitivity analysis to ensure that overlap does not exist? It would be useful to get the authors perspective on whether the evaluation of a PRS in a cohort derived using EraSOR-adjusted summary statistics can be seen as truly external to the source GWAS. The paper should be accompanied by a more detailed user guide and some test data for the EraSOR tool. Are there any diagnostic plots that are produced that could be used to inspect the data quality?

This work has been peer reviewed in GigaScience (see https://doi.org/10.1093/gigascience/giad043), which carries out open, named peer-review. These reviews are published under a CC-BY 4.0 license and were as follows: ** Jack Pattee **(revision 1)

Thank you for your detailed responses; I have no further comments.

This work has been peer reviewed in GigaScience (see Description), which carries out open, named peer-review. These reviews are published under a CC-BY 4.0 license and were as follows:

** Jack Pattee**

Overall, I think that this manuscript is strong and describes a well-formulated method to address a relevant problem. There are a few outstanding questions about the performance of the EraSOR method from my perspective, which I'll detail as follows.My understanding of reference [16] indicates that equation (3) of this manuscript only holds for null SNPs, i.e. if SNP g is not associated with the outcome Y. If this is the case, then this should be discussed in the manuscript. I wonder if this can partially explain the 'under-estimation' behavior we see in the application to real data in Supplementary Figure 3. In particular, I am referencing the behavior where the EraSOR correction will under-estimate the predictive accuracy of the PRS in the target data, i.e. where delta-R^2 is negative. This behavior is not seen in the simulation and warrants further investigation and discussion. While the bias appears small, for some cases delta-R^2 approaches -.025, which corresponds to an under-estimation of Pearson's r by roughly .15; this is substantial. Could it be the case that, for highly polygenic traits such as height and BMI, the null-SNP assumption is unreliable and the performance of EraSOR is degraded? Does a fundamental assumption of sparse genetic association underlie EraSOR?I recommend that the real data application play a larger role in the manuscript narrative and be moved out of the supplementary. The simulations are appreciated and helpful, but there is nuance in the analysis of real data that cannot be replicated in simulation.I believe the reference to "Supplementary Figure 2" on line 346 should actually be "Supplementary Figure 3". I believe that the axis labels in Supp Figure 3 are flipped.Lines 82 and 83 reference genetic stratification and subpopulations; I think the relevance of these concepts should be introduced more clearly and they should be defined in this context. EraSOR concerns the overestimation of predictive accuracy and association incurred by sample overlap between the base and target GWASs; to this reader, it's not clear what this central issue has to do with population stratification. I realize that the derivation of the LD score method is motivated heavily by correcting for stratification; however, these concepts should be introduced more clearly in this manuscript.Line 88: consider defining LD score l_j.Lines 94-96: consider outlining the mathematical consequence of the assumption that "the two outcomes and cohorts are identical." It's the case that N_1 = N_2 = N_c = N, correct?Line 109 / equation (11): My understanding is that the relevant quantity of this derivation is N_c / sqrt(N_1 N_2), which allows us to define the correct matrix C in expression (4). If this is the case, perhaps the quantity of interest should be moved to the LHS of the equation in the final line of the expression, for clarity.As discussed in the manuscript, the estimated heritability is in the denominator of the expression for N_c / sqrt(N_1 N_2). The authors correctly discuss that the method should not be applied when there is doubt as to whether the heritability is different from zero. I would take this a step further; in cases where the heritability is zero, we cannot meaningfully apply the EraSOR correction, and thus I am not sure of the utility of the 'type I error' simulations in the manuscript. Perhaps an explicit test for h^2 > 0 should be worked into the EraSOR workflow?Line 148 / expression (12): If beta has a normal distribution here, it is the case that all SNPs in the simulation are associated with the outcome Y. This is a somewhat unusual choice for the distribution of SNP effects in a simulation; other applications such as LDPred (Vilhjalmsson et al, AJHG 2015) and LassoSum (TSH Mak et al, Genetic Epi 2017) use a point-normal distribution for simulated SNP effects, which effectively simulates the sparsity frequently observed in nature. Is there a reference or justification for the non-sparse simulation structure here?Line 215: there may be a typo in the expression for the variance of the residual term. Is it the case that the variance of the residual depends on the variance of a covariance term? If so, I am confused as to the derivation.Line 241: 'triat' should be 'trait'.The simulation results in this paper are based on clumping and thresholding for PRS, which does not estimate joint SNP effects i.e. account for LD. Methods such as LDPred and LassoSum do so. Is there any reason to believe the results would be different for a method such as LassoSum?I am confused by the very low Fst between the simulated Finnish and Yoruban samples in simulation. As detailed on line 385: the reported Fst is > .1, but the simulated Fst is essentially zero. This seems likely to be an undesirable simulation artefact, and potentially invalidates the simulation study (or, at least, doesn't provide evidence that EraSOR functions correctly when Fst is large, which was the ostensible motivation for this simulation). Is there no way to effectively simulate populations with a larger Fst?

This work has been peer reviewed in GigaScience (see https://doi.org/10.1093/gigascience/giad043), which carries out open, named peer-review. These reviews are published under a CC-BY 4.0 license and were as follows:

Christopher C. Chang Reviewer Comments to Author: This paper addresses a significant need that has arisen in the interaction between privacy rules and ever-larger genomic datasets, and I find the results to be very promising and clearly worth publishing. I just have a few comments on some methodological details:line 130: Have you compared the effectiveness of this algorithm with plink2 --king-cutoff?lines 145-155: If I understand this correctly, these simulated quantitative traits are still normally distributed, they just aren't standardized to mean 0 variance 1. If the intent is to "simulate phenotypes that [do] not follow the standard normal distribution", I'd expect it to be more valuable to look at e.g. the log-normal case, where an alert user might transform the phenotype to normal, but some users may fail to do so. A mixture distribution may also be worth looking at.lines 238-239: Have you considered using the "cc-residualize" option of plink2 -glm, which removes most of the computational cost of including PCs in your binary trait analysis?lines 383-387: This is interesting; there is some room for follow-up investigation here. Thanks for posting all the scripts needed for another researcher to easily reproduce this Fst=0.00639 value; this could help facilitate development of a better genotype-simulation tool.Also, some minor copyedits:line 84: "subpopulation" -> "subpopulations"line 342: "overlaps" -> "overlap"line 363: "ErasOR" -> "EraSOR"line 376: "different level of environmental stratifications" -> "different levels of environmental stratification"line 384: "population" -> "populations"line 402: "capture" -> "captured"

Editor’s Assessment

Like other mollusc species, the freshwater pearl mussel (Margaritifera margaritifera) has a challenging genome to assemble owing to the large size of their genomes, heterozygosity, and repetitive sequence. The first published M. margaritifera genome was highly fragmented, but here an improved reference genome assembly was generated using PacBio CLR long reads to reduce fragmentation levels, missing and truncated genes, and chimerically assembled regions. The number of gene models predicted is a bit higher compared than other molluscan genomes, but after clarification and double checking these seem in line with some Mollusca and Bivalvia with similar and higher numbers of gene predictions. This new genome represents a new resource to start exploring the many biological, ecological, and evolutionary features of this threatened and commercially important group of organisms.

This assessment refers to version 1 of this preprint.

Editor’s Assessment

Hybrid genomes are tricky to assemble, and few genomic resources are available for hybrid grapevines such as ‘Chambourcin’, a French-American interspecific hybrid grape grown in the eastern and midwestern United States. Here is an attempt to assemble Chambourcin’ using a combination of PacBio HiFi long-reads, Bionano optical maps, and Illumina short-read sequencing technologies. Producing an assembly with 26 scaffolds, an N50 length 23.3 Mb and an estimated BUSCO completeness of 97.9% that can be used for genome comparisons, functional genomic analyses, and genome-assisted breeding research. Error correction and pilon polishing was a challenge with this hybrid assembly, but after trying a few different approaches in the review process have improved it, and as they have documented what they did and are clear about the final metrics, users can assess the quality themselves.

This assessment refers to version 2 of this preprint.

This work has been published in GigaByte Journal under a CC-BY 4.0 license (https://doi.org/10.46471/gigabyte.84) and has published the reviews under the same license. These are as follows.

**Reviewer 1. Lingfei Shangguan ** Reviewers Comments: Grapevine is one of the most important fruit crops in the world, and ‘Chambourcin’ is a hybrid wine grape variety in the world, which represented the cross species between North American and European Vitis species. The authors have sequenced the genome sequence of ‘Chambourcin’, and obtained the repeat sequences and gene annotation information. However, the sequence depth was too low for the grape genome, especially the high heterozygosity. They also not applied the illumine sequencing for sequence correction.

Re-review: Since the authors have made some correction and improvement, the genome quality was still low, and the manuscript has not improvement significantly. Authors should provide the haplotype sequences, and describe the genome assembly and correction steps more clearly. Moreover, the innovation of the article is insufficient. I suggest reject.

**Reviewer 2. Pablo Carbonell-Bejerano **

Are all data available and do they match the descriptions in the paper? No. Access to the raw data for the RNA-seq dataset that was used for gene predictions is not indicated

Are the data and metadata consistent with relevant minimum information or reporting standards?

No. Any description of the RNA-seq dataset and its origin or features is fully missing. I could not find other data that would be required according to guidelines in http://gigadb.org/site/guide: - Full (not summary) BUSCO results output files (text) - readme.txt including all file names with a brief description of each - sample metadata that complies with the Genomic Standards Consortium.

Is the data acquisition clear, complete and methodologically sound?

Yes. Sequencing and bioinformatic methods followed are generally sound.

Is there sufficient detail in the methods and data-processing steps to allow reproduction? No. 1. Availability for the scripts used in bioinformatic analyses and data plotting is generally missing.

1. L124. Authors describe that minimap2 was used to obtain the dotplot. However, minimap2 alone does not produce dotplots.

2. L131. It is unclear how ‘PN40024’ 12X.v2, VCost.v3 protein annotations were used as input of BRAKER2. Do authors mean protein sequences instead? Where were these protein data retrieved from? How are proteins aligned to the assembly? Was BRAKER run from masked or unmasked assembly?

Is there sufficient data validation and statistical analyses of data quality? No. 1. Validation of the original material for its true-to-typeness as 'Chambourcin' cultivar genotype is not mentioned, neither the number of different plants used for DNA extraction. While post-assembly validation of the Chambourcin genome assembly genotype from the mapped Chambourcin rhAmpSeq markers may be possible, such genotype validation is not mentioned either in the text.

1. In general, the quality and the genome variation represented in the Chambourcin genome assembly produced here could have been further tested. For instance, from 2% BUSCO duplication and 501.5 Mb of primary assembly size as compared to the 481.5 Mb haploid genome size that can be inferred from the k-mer analysis presented by the authors indicates, it seems that further duplication purging of the primary assembly is likely needed. This issue could be addressed by looking for assembly regions with reduced alignment depth when all HiFi reads are mapped to the primary assembly. Duplicated regions to be purged could also be supported by co-linear assembly segments sharing BUSCO duplicated genes. For assembly reliability assessment, 10X, rhAmpSeq, or Illumina WGS data that is available for Chambourcin could also be used to validate genome variants represented in this Chambourcin assembly when comparing the inter-haplotype variants detected between primary and haplotig assemblies or the haplotypes with genome assemblies from other genotypes.

Is the validation suitable for this type of data? Yes. The validation is suitable, although it might not suffice in all cases.

Is there sufficient information for others to reuse this dataset or integrate it with other data? No. As described before, there is missing information at several instances, like for the origin of the RNA-seq.

Additional Comments: 1. L171. Is it correct that total length of Bionano maps was as small as 962,964 bp? Or do authors mean kb instead of bp in that sentence?

1. The mapping of Chambourcin rhAmpSeq markers could have been further exploited to phase contig haplotypes before purging haplotypes and assembly scaffolding?

2. For the Conclusion in L254, it might be arguable whether the presented Chambourcin genome assembly is the first genome assembly of a complex interspecific hybrid or not. For instance 'Shine Muscat' might also be considered a complex inter-specific hybrid grape cultivar and its genome assembly was published: https://academic.oup.com/dnaresearch/article/29/6/dsac040/6808674 It might even be arguable whether the one presented in this publication is the first Chambourcin genome assembly as there is a 10X Genomics-based assembly available for Chambourcin: https://www.nature.com/articles/s41467-019-14280-1

Re-review: Efforts to improve the accuracy of the MS and the availability of data are clear in the revised version. Authors have included descriptions of M&M procedures and information about the origin of several datasets that were missing. They also included files with commands and original results to the FTP server. In addition, they did further de-duplication of the assembly, added Illumina sequencing for assembly polishing, and included further QC stats and comparisons to another recently published hybrid grapevine genome assembly.

Most revision actions were successful. However, it is not recommended to polish HiFi assemblies with Illumina reads as in most cases it harms the consensus quality more than it improves it, which is particularly true for repetitive and highly heterozygous genomes like the one of Chambourcin grapevine cultivar. In fact, the BUSCO Completeness of 97.9% after Pilon short-read polishing compared to 98.2% in the former version indicates that polishing with Illumina short-reads is indeed harming in this revised version. I indeed agree with authors that 28x depth of PacBio HiFi reads should suffice to produce a quality genome assembly without using more depth or another sequencing technologies as they indicate in their response. I would recommend to remove the Pilon polishing from the final assembly version, which is only recommended in error-prone PacBio CLR or Nanopore assemblies. Instead, authors could use the Illumina reads for k-mer analysis of assembly consensus quality and completeness.

**Editorial Board Member adjudication: **

Comment 1. How many times did you do the polishing with Pilon? This is not clear in the documents provided. It could be 1 round or many. Many would be a concern. When we run error correction on genomes, we monitor BUSCO and when it drops, roll back one iteration. Comment 2. How many sites were corrected in the polishing of the primary and haplotig assembly? Comment 3. Can you run KAT (KAT: A K-Mer Analysis Toolkit to Quality Control NGS Datasets and Genome Assemblies.” Bioinformatics 33 (4): 574–76) to check the diploid, primary and haplotig assemblies? Comment 4. Can you align the mRNAseq and whole genome shotgun reads to diploid, primary and haplotig assemblies and report the percent mapping including the properly paired?

This work has been peer reviewed in GigaScience (see https://doi.org/10.1093/gigascience/giad035 ), which carries out open, named peer-review. These reviews are published under a CC-BY 4.0 license and were as follows:

**Reviewer Yimin Wang **

This work (FriendlyClearMap) attempts to combine several tools such as ClearMap 1/2, BrainRender, etc., and integrate certain functions into a Docker image for the ease of use. The authors then demonstrated the use of FriendlyClearMap by analysing PV+, SST+ and VIP+ neurons. Some details comments are as below:

1/ P4, second paragraph, line 3, "vs." -> "versus".

2/ P9, third paragraph, line 8, conflict between "lastly" and "finally"

3/ P9, third paragraph, line 8, "our tool allows …".

4/ This work can be regarded as a reengineering effort based on several previous toolkits in order to facilitate the workflow of registration, segmentation, analysis, and visualization. Essentially, no new technology involved is involved in this work and no new application is enabled by FriendlyClearMap. Therefore, in order to emphasize the unique contribution of this work, the author could elaborate how this tool makes biologists' work easier.

5/ The results for Figure 2g are somewhat trivial. The authors might consider replace it with some more impressive analysis.

6/ The majority of the results are related to cell segmentation and counting. Quantitative plots/tables could be provided for more information. In addition, the accuracy of the results could also be discussed.

7/ Last but not least, as there is no substantial novelty in the software, the authors actually could consider change the focus of the manuscript from a tool paper to a resource/results paper, emphasizing new biological findings which is obtained by using FriendlyClearMap.

This work has been peer reviewed in GigaScience (see https://doi.org/10.1093/gigascience/giad035 ), which carries out open, named peer-review. These reviews are published under a CC-BY 4.0 license and were as follows:

Reviewer Chris Armit

This Technical Note paper describes "FriendlyClearMap: An optimized toolkit for mouse brain mapping and analysis".

Whereas the core concept of a data analysis tool to assist in spatial mapping of cleared mouse tissues is perfectly reasonable, there are multiple issues with the documentation that renders this toolkit very difficult to use. I detail below some of the issues I have encountered.

1. GitHub repositoryThe installation instructions are missing from the following GitHub repository: https://github.com/MoritzNegwer/FriendlyClearMap-scriptsThe closest reference I could find to installation instructions is the following: "Please see the Appendices 1-3 of our <X_upcoming> publication for detailed instructions on how to use the pipelines. <X_protocols.io goes here>"Step-bystep installation instructions should be included in the GitHub repository. In addition, the authors should add the protocols.io links to their GitHub repository.

2. Protocols.ioThe installation instructions are missing from the following protocols.io links:Run Clearmap 1 docker dx.doi.org/10.17504/protocols.io.eq2lynnkrvx9/v1Run Clearmap 2 docker dx.doi.org/10.17504/protocols.io.yxmvmn9pbg3p/v1Both of these protocols include the following instruction:* "Then, download the docker container from our repository: XXX docker container goes here"In the documentation, the authors need to unambiguously refer to the specific Docker container that a user needs to install for each software tool.

3. Test Data I could not find the test data in the form of image stacks that would be needed to test the FriendlyClearMap protocols. Figure 1 refers to 16-bit TIFF image stacks, and I presume these to be the input data that is needed for the image analysis pipelines described in the manuscript. The authors should provide details of the test imaging dataset, including links if necessary to where the image stacks data can be downloaded, in the 'Data Availability' section of the manuscript.

4. Platform / Operating SystemsIn the 'Data Availability' section of the manuscript, the authors specify that the Operating Systems are "platform-independent". However, the protocols.io documents lists a set of requirements for Windows and LINUX, but not for MacOS. The authors should provide installation instructions and system requirements for MacOS.I reject this manuscript on the grounds that, due to lack of appropriate documentation and installation instructions, the software tool is too difficult to use and therefore has extremely low reuse potential.

This work has been peer reviewed in GigaScience (see https://doi.org/10.1093/gigascience/giad031 ) , which carries out open, named peer-review. These reviews are published under a CC-BY 4.0 license and were as follows:

**Reviewer Stian Soiland-Reyes ** Hi, I am Stian Soiland-Reyes https://orcid.org/0000-0001-9842-9718 and have pledged the Open Peer Review Oath https://doi.org/10.12688/f1000research.5686.2: *

Principle 1: I will sign my name to my review Principle 2: I will review with integrity Principle 3: I will treat the review as a discourse with you; in particular, I will provide constructive criticism Principle 4: I will be an ambassador for the practice of open science. This review is licensed under a Creative Commons Attribution 4.0 International License

. --- This article presents a method for comparing reproducibility of computational workflow runs captured as RO-Crates, by calculating a set of genomics metrics ("features") and adding these to the crate's metadata. Overall I find this a valuable contribution and worthy of publication with GigaScience, primarily as a way for users of workflow systems CWL, Nextflow, Cromwell or Snakemake to ensure reproducibility, but also for workflow engine developers who may want to build on this methodology to improve their provenance support. In general the method proposed is sound, however it does have some limitations and inherent assumptions that are not highlighted sufficiently in the current manuscript, particularly concerning the selection of features and the reproducibility of the metrics calculation itself. I have detailed this with some points below that I would like the authors to clarify in a minor revision.

--- Note - the below questions from GigaScience Reviewer Guidelines mainly relate to data, but I also here interpret them for the software described.

Q1: Is the rationale for collecting and analyzing the data well defined? The author's workflow executions https://doi.org/10.5281/zenodo.7098337 are based on three 3rdparty bioinformatics workflows. Although they are not particularly "large-scale", they are representative best-practice pipelines in this field (data sizes from 200 MB to 6 GB) and also fairly representative for scalable workflow systems (Nextflow, CWL and WDL) used by bioinformaticians.

Q2: Is it clear how data was collected and curated? It is not explicit in the text why these particular workflows were selected, beyond being realistic pipelines used in research. I would suggest something like "these workflows have been selected as fairly representative and mature current best-practice for sequencing pipelines, implemented in different but typical workflow systems, and have similar set of genomics features that we can assess for provenance comparison." The workflows have each been cited, but I would appreciate some consistency so that each workflow is cited both by its closest journal article and as their original download sources (e.g. GitHub).

Q3: Is it clear - and was a statement provided - on how data and analyses tools used in the study can be accessed? Yes, full availability statements have been provided both for data and software, archived on Zenodo for longevity.

Q4: Are accession numbers given or links provided for data that, as a standard, should be submitted to a community approved public repository? Yes, the tools have been added to https://bio.tools/ -- I don't think it's necessary to further register the data outputs with accession numbers. RRIDs for tools can be considered at a later stage, perhaps only for Sapporo.

Q5: Is the data and software available in the public domain under a Creative Commons license? Yes, the software and dataset is open source under Apache License, version 2.0. The dataset https://doi.org/10.5281/zenodo.7098337 embeds existing workflows and data, however this is OK as included resources such as the rnaseq Nextflow workflow have compatible licenses (MIT) or are also Apache-licensed. The manuscript has software citations for two of the workflows, but this is missing for the CWL workflow, which is only cited by manuscript (33) (also missing DOI). It is unclear if any of the workflows are registered in https://workflowhub.eu/ but that should primarily be done by their upstream authors. The RO-Crates in https://doi.org/10.5281/zenodo.7098337 don't include any licensing and attribution for the embedded workflows, and its metadata file is misleadingly declaring the crate license as CC0 public domain. While CC0 is appropriate for examples and metadata file itself, the embedded MIT/Apache workflows from third parties can't legally be relicensed in this way and should have their original licenses declared. See https://www.researchobject.org/ro-crate/1.1/contextualentities.html#licensing-access-control-and-copyright I understand these RO-Crates are generated automatically by Sapporo, which does not directly understand licensing, and for documenting the test runs with Sapporo, I think these should not be modified post-execution. Pending further license support by Sapporo, perhaps a manual outer RO-Crate that aggregate these (e.g. adding a direct top-level ro-crate-metadata.json to the Zenodo entry) can provide more correct metadata as well as workflow citations. The authors could add to Discussion some consideration on (lack of) propagation of such metadata for auto-generated crates as part of workflow run provenance. For instance, if a workflow run was initiated from a Workflow Crate https://w3id.org/workflowhub/workflow-ro-crate/ at WorkflowHub, its license, attributions and descriptions could be carried forward to the final Workflow Run Crate provenance together with the Sapporo-calculated features.

Q6: Are the data sound and well controlled? Yes, the data is sound. The testing on Mac gives null-results, but the authors explain the workflows failed to execute there due to archicectural differences, which is flagged as a valid concern for reproducibility. It may be worth further investigating if this is due to misconfiguration on that particular test machine in which case these columns should be removed.

Q7: Is the interpretation (Analysis and Discussion) well balanced and supported by the data? The authors' discussion have some implicit assumptions that should be made more clear, together with implications: The Tonkaz tool assumes the workflow execution has already extracted the features and added them to the RO-Crate This assumes the right features have been correctly extracted by each execution Feature extraction also depend on bioinformatics tools that are subject to change/updates Newer versions of Sapporo-service, and in particular any non-Sapporo executors also making Workflow run Crates, may have a different feature selection Being able to fairly compare two workflow runs therefore depends on careful control of the Sapporo executor versions so that they have consistent feature selection This means the reproducibility metrics proposed has a potential reproducibility challenge itself This is not to say that the approach is bad, as the feature extraction is using predictable measures such as counting sequences, rather than heuristics. This means Future Work should point out the need for guidelines on what kind of features should be selected, to ensure they are consistent and reproducible. The set of features also depend on the type of data and class of analysis. As a minimum, the RO-Crate should therefore include provenance of that feature extraction, noting the Sapporo version, and ideally the version of the tools used for that. The authors may want to consider if feature extraction should be a separate workflow (e.g. in CWL), that itself can be subject to the same reproducibility preservation measures, and therefore also can be performed post-execution as part of Tonkaz' comparison or as a curation activity when storing Workflow Run Crates.

Q8: Are the methods appropriate, well described, and include sufficient details and supporting information to allow others to evaluate and replicate the work? Yes, it was very easy to replicate the Tonkaz analysis of the workflow run crate that is already provided, as it is provided also as a Docker container. The Docker container is provided as part of GitHub releases, and so is not at risk of Docker Hub's automatic deletion. I have not tried installing my own Sapporo service to re-execute the workflow, but detailed installation and run details are provided in the README of both Tonkaz https://github.com/sapporowes/tonkaz#readme and sapporo-service https://github.com/sapporowes/sapporo/blob/main/docs/GettingStarted.md

Q9: What are the strengths and weaknesses of the methods? The method provided is strong compared to naive checksum-based comparison of workflow outputs, which has been pointed out as a challenge by previous work. The advantage of the feature extraction is that the statistics can be compared directly and any disreprancies can be displayed to the user at a digestible high-level. The disadvantage is that this depends wholy on the selection of features, which must be done carefully to cover the purpose of the particular workflow and its type of data. For instance, a workflow that generates diagrams of sequence alignments could not be sufficiently tested in the suggested approach, as analyzing the diagram for correctness would require tools that may not even exist. Perhaps feature extraction should be a part of the workflow itself, so it can self-determine what is important for its analysis? The current approach also is quite sensitive to output data filenames, so changes in filename would mean features are not compared, even where such files are equivalent. This should be made more explicit in the manuscript, for instance workflows should ensure they don't include timestamps or random identifiers in their filenames. Further work could have a deeper understanding of the workflow structure to compare outputs based on their corresponding FormalParameter in the RO-Crate.

Q10: Have the authors followed best-practices in reporting standards? Yes, the details provided are at a sufficient detail level, and the authors have re-used the RO-Crate data packaging. The RO-Crates created by Sapporo-service adds several terms for the metrics, which are declared on the @context according to RO-Crate specs https://www.researchobject.org/rocrate/1.1/appendix/jsonld.html#extending-ro-crate However the terms point to GitHub "raw" pages, which are not particularly stable, and may change depending on sapporo versions and GitHub's repository behaviour. I recommend changing the ad-hoc terms to PIDs such as a namespace under https://w3id.org/ or https://purl.org/ so that these terms can be stable semantic artefacts, e.g. submitting them to https://github.com/ResearchObject/ro-terms to register https://w3id.org/ro/terms/sapporo#WorkflowAttachment that can be used instead of https://raw.githubusercontent.com/sapporo-wes/sapporo-service/main/sapporo/roterms.csv#WorkflowAttachment or alternatively https://w3id.org/sapporo#WorkflowAttachment could be set up to redirect to the ro-terms.csv on GitHub. (discussed with the authors at ELIXIR Biohackathon) In doing so you should separate into two namespaces, the general Sapporo terms like "sha512", and the particular genomics feature sets including "totalReads" (e.g. https://w3id.org/datafeatures/genomics#WorkflowAttachment) as the second are a) Not sapporo-specific b) domainspecific. RO-Crate is developing Workflow Run profiles https://www.researchobject.org/workflow-runcrate/profiles/, although these have not been released at time of my review they are now stable, so the authors may want to check https://www.researchobject.org/workflow-runcrate/profiles/workflow_run_crate to ensure "FormalParameter" are declared correctly in the generated RO-Crate as separate entities, linked from the "File" using "exampleOfWork".

Q11: Can the writing, organization, tables and figures be improved? The language and readability of this article is generally very good. Light copy-editing may improve some of the sentences, e.g. reducing the use of "Thus" phrases.

Q12: When revisions are requested. See suggestions from above for minor revisions: Make explicit why these 3 workflows where selected (see Q2) Make pipeline software citations consistent in manuscript (see Q2, Q5) Avoid declaring CC0 within generated RO-Crate -- move this to only apply to the ro-cratemetadata.json Add an outer RO-Crate metadata file to Zenodo deposit to carry the correct licenses and pipeline licenses for each of rnaseq_1st.zip, trimming.zip etc. Improve discussion to better reflect limitations of the features and its own reproducibility issues (see Q7, Q9) Consider improvements to the RO-Crate context (see Q10) - this may just be noted as Future Work in the manuscript rather than regenerating the crates In addition: p2: Add citation for claim on file checksums different depending on software versions etc., for instance https://doi.org/10.1145/3186266 p3. "We converted Sapporo's provenance into RO-Crate" -- re-cite (20) as this is the paragraph explaining what it is. p10. Citations 7, 8 are missing authors p10. Citation 15 is now published, replace with https://doi.org/10.1145/3486897 p0. Citations 28, 33 is missing DOI

Q13: Are there any ethical or competing interests issues you would like to raise? No, the third-party pipelines selected for reproducibility testing are already published and are here represented fairly, and only used as executable methods (as intended by their original authors), which I would say do not need ethical approval.

This work has been peer reviewed in GigaScience (see https://doi.org/10.1093/gigascience/giad031 ) , which carries out open, named peer-review. These reviews are published under a CC-BY 4.0 license and were as follows:

Reviewer Stephen R Piccolo:

This manuscript describes a methodology for automating evaluation of the reproducibility of datascience workflows for genomics analyses. The authors explain that reproducibility should be evaluated on a scale rather than on a binary basis. They explain concepts related to these issues and apply their methodology to real-world data. The manuscript was well written and addresses an important issue. I believe this manuscript provides new insights. I have a few minor concerns that I would appreciate being addressed:

• The manuscript indicates that it's not feasible to compare images automatically. However, this is actually pretty easy. For example, using the Pillow package in Python, you can calculate a percentage similarity between two image files. I'm not suggesting that the authors should do this in their study. But the text should not preclude this as a possibility.

• The authors describe scenarios where the outputs might be different but these differences would be immaterial to the overall conclusions. They also describe a few scenarios where the outputs differ for biological features but that the differences are relatively small and could be considered to be acceptable. Examples include when BAM files are sorted differently. I think it would be helpful to add a bit more discussion of scenarios where differences in biological features could occur and what would cause those differences.

• Although a person checking the outputs can change the numeric threshold, it would be difficult to know what that threshold should be. Perhaps the authors could describe additional situation(s) where having relatively large differences would be acceptable and other situation(s) where they would not. For example, you could have a single difference in the biological feature outputs and perhaps that would make a huge difference in the interpretation in some cases. Additional discussion would be helpful.

• This paper focuses on automating the verification process. I think the big picture could be explained more. Who might perform this verification process in a scientific context? In what context would they do it? - Please add brief discussion about generalizing this methodology beyond Tonkaz.

This work has been peer reviewed in GigaScience (see https://doi.org/10.1093/gigascience/giad030 ), which carries out open, named peer-review. These reviews are published under a CC-BY 4.0 license and were as follows:

Reviewer Janna Hastings

The manuscript describes a toolkit for the automated semantic enrichment and quality control of electronic health data using ontologies. This is a much needed utility that will add value to electronic data sharing and re-use for many different purposes including the development of machine learning for medical applications and personalised medicine. Overall the manuscript is well written and the functionality offered by the toolkit is well thought out and motivated. The internal consistency checks and the use of ontology-based information content to semantically aggregate variables into more informative meta-variables are particularly welcome functions.

However, I recommend that the description of the tool functionality be clarified in some points, and the evaluation could be strengthened.page 6-7, internal consistency:

1. How should the user specify semantic dependencies between variable pairs? Would it not be helpful to use a standard format for this specification to enable interoperability and re-use of such specifications?

2. Should the specification of semantic relationships between variables not be linked to the knowledge from the ontologies? Ontologies are able to represent many different types of logical relationships between classes, which make them ideal for then serving as a standard and interoperable format for specifying this type of constraint. Rules are another promising standard approach for logic-based knowledge representation.

Page 11, figure 4 a: I think it would be informative for evaluating the operation of the tool if the heatmap of variable missingness after application of the tool could also be illustrated beside the current Fig 4a.

Page 13, ontology preparation: The paragraph describes what the authors have done to prepare ontologies for use with the tool. Is this preparation procedure also necessary for users to follow when they use the eHDPrep tool? How can alternative ontologies be incorporated (which may be useful for other domains)?Evaluation: The biggest shortcoming of the presented manuscript is that the evaluation is limited to the application of the tool to one dataset and subsequent manual evaluation of the outcome by one group, the study authors.

The results as presented are positive, but there is a significant risk that the tool performs well on this task, as assessed by these study authors, but then fails to generalise to other tasks and datasets that future users might wish to use it with. To mitigate against this challenge, it would be optimal if somewhat more independent methods could be found for evaluating the performance of the different aspects of the tool. One approach could a rigorous comparison of this tool's performance against the performance of other tools that have similar functionality, e.g. comparison of the semantic aggregation function with other tools that find and recommend MICAs. An alternative approach might be to apply the tool to an additional dataset for which a group outside of the study authors would be prepared to provide an independent evaluation.

This work has been peer reviewed in GigaScience (see https://doi.org/10.1093/gigascience/giad030 ), which carries out open, named peer-review. These reviews are published under a CC-BY 4.0 license and were as follows:

Reviewer Hugo Leroux

This well-written paper describes techniques for semantically-enriching clinical data pertaining to colorectal cancer diagnosis.It describes an R-based tool, eHDPrep, to extract the data, which is subsequently cleaned, actioned for missing and erroneous values, encoded and enriched semantically using SNOMED CT and the GO, and ultimately exported after having undergone some QC.The paper is well-written and the methods really well-explained, for which the authors should be commended.I only have a few comments for the authors:

1. It is not clear to me how, in the discussion on page 14, the authors have dealt with the issue of representing negative findings and missing values, as described within their enrichment outcomes section.

2. In the "Ontology Preparation" section, the authors describe how they have taken both the SNOMED CT terminology and performed some transformations to OWL and conversion to CSV format before mapping the Colo-661 variables to it. They don't however discuss the challenges that such an approach entails. The authors might consider perusing through this article (https://doi.org/10.1186/s13326-018-0191-z), which addresses many of the challenges relating to ontology matching

3. Please insert an additional ")" when stating the "Equations", e.g. page 6: "... zero entropy [27] (Equation (1)) ...", also , page 13

This work has been peer reviewed in GigaScience (see https://doi.org/10.1093/gigascience/giad026 ), which carries out open, named peer-review. These reviews are published under a CC-BY 4.0 license and were as follows:

Reviewer Kaixuan Luo

This paper develops a novel pipeline TF-Prioritizer to prioritize condition-specific TFs thorough integrative analysis of histone modification (HM) ChIP-seq and RNA-seq data. The pipeline integrates multiple computational tools: calculate TF binding site affinities and link candidate binding sites to genes using the TRAP and TEPIC. It uses DYNAMITE, a sparse logistic regression classifier, to infer TFs related to differential gene expression between conditions. It computes an aggregated score "TF-TG score" to score TFs from multiple types of evidence, and obtains a prioritized list of TFs from all histone modifications using a discounted cumulative gain ranking approach. It also provides additional functionality and web interface to visualize the results.

Overall, the pipeline could be very useful for biologists with a user-friendly web application to automate the entire process from data preprocessing to statistical analysis and obtain interactive reports to gain novel biological insights. However, more systematic evaluations are needed to demonstrate the benefits of this pipeline.

Major comments:

1. In the computation of an aggregated score "TF-TG score", it uses a multiplicative function to combine differential expression (absolute log2FC), TF-Gene scores computed from TEPIC, and the total coefficients computed from DYNAMITE. One concern about this approach is that it may miss some TFs with support from only one or two types of evidence. In Fig 5, we see diffTF identifies a lot more TFs than diffTF. I don't think we can conclude that diffTF is less specific than TF-Prioritizer simply based on the number of TFs prioritized. Some of the TFs identified only by diffTF may be important but missed by TF-Prioritizer? I would like to see more detailed analysis comparing the lists of TFs identified by diffTF and TF-Prioritizer. Other evidence or metrics in addition to the number of prioritized TFs would be helpful to evaluate the plausibility of the prioritized lists of TFs.

2. It is hard to interpret and evaluate the contribution of the evidence for prioritized TFs. Figure 6b is helpful, but it is unclear how the users would be able to evaluate the contribution of the components. Does the software run each of the combination separately and outputs a list of prioritized TFs under each combination?

3. The TEPIC2 paper has already developed a very comprehensive pipeline, including TF affinity calculation by TRAP and computation of TF gene scores by TEPIC, as well as logistic regression to identify TFs between conditions by DYNAMITE, and it is already well paralyzed. The authors should clearly list the novel contributions from this work. It would be helpful to have a table comparing the functionalities and technical features between TF-Prioritizer and TEPIC2.

4. The software takes histone modification ChIPseq and RNA-seq data as input. It will significantly improve the usage of the software if it supports DNase-seq and/or ATAC-seq, which are widely used. If this software could take ATAC-seq or DNase-seq data as input, it is important to include those data types and provide some examples to illustrate the usage and performance.

5. The software combines multiple histone modification ChIP-seq datasets using a discounted cumulative gain ranking approach. However, different types of histone modifications have different epigenomic functions and different combinations indicate different chromatin states. Some TFs may be only enriched in a small subset of histone modifications (already discussed by the authors) and may be missed by the simple discounted cumulative gain ranking approach. The authors should provide prioritized TFs from each histone modification ChIP-seq dataset, and evaluate which TFs were prioritized by all the combined datasets, and which TFs by only one dataset. Also, some ChIP-seq datasets may be of poor quality. Does the software provide other options to rank the TFs from different epigenomic datasets? e.g. set different weights for different epigenomic datasets, etc.

6. The authors conducted cooccurrence analysis based on the overlapping of peaks. It is unclear if the method would calculate some statistical measure (e.g. p-value) for the significance of co-occurrence. Also, since the TRAP model generates quantitative measure of TF binding affinity, I am curious to see if the quantitative TF binding affinity are also correlated for those co-occurred binding sites.

Minor comments: 1. In Figure 1, it would be helpful to highlight which steps were already implemented in existing tools (and label the tools used), and which steps are novel in this study. 2. H3K4me3 data seems to be missing in the L10 time point. How does the method handle missing data? 3. It is unclear how the Pol2 ChIP-seq data was used in this study? Was it included in the model or only in the downstream analysis? 4. It is hard to interpret the browser tracks of the TF predictions ("Predicted xxx") in Figure 3 and 4. Please add more details about those tracks .5. Figure 6, the authors should provide more details to help understand this figure, especially panel b. The figure legend is too short.

This work has been peer reviewed in GigaScience (see https://doi.org/10.1093/gigascience/giad026), which carries out open, named peer-review. These reviews are published under a CC-BY 4.0 license and were as follows:

Reviewer: Roza Berhanu Lemma

In this manuscript, Hoffmann and Trummer et al. reported a new automated pipeline that utilizes existing methods, namely (1) DESeq2 to perform differential gene expression between sample groups, (2) TEPIC, a method that links CREs to genes using a biophysical model TRAP and (3) DYNAMITE, which provides an aggregate score for TF-target genes that determine the contribution of TFs to condition specific changes between sample groups. Finally, the pipeline utilizes Mann-Whitney U test to prioritize TFs among a background distribution and a ChIP-seq specific TF distribution, which allows the identification of TFs with roles in condition-specific gene regulation. Their pipeline allows large-scale processing of data and returns a feature-rich and user-friendly interactive report.

The authors demonstrated how to use TF-prioritizer using public datasets for mouse mammary gland development study and performed independent validation using datasets from ChIP-Atlas. They were able to capture both known TFs with previously reported roles in mammary gland development/lactation and new TFs that may have a role in these processes. The work is very well thought and executed but to keep the quality of the work even higher, the authors should address the following points.

Major:

1. Although their validation nicely portrays the potential application of their pipeline in answering biological questions, my fear is for this not to be an isolated case. Therefore, the authors should test their pipeline using another example dataset and convince their readers. A suggestion could be, to run TF-Prioritizer on one of deeply profiled cell lines (e.g. K562, MCF-7, etc) to investigate TF prioritizations for e.g during differentiation (change of cell fate) and see if lineage determining TFs are prioritized in such cases. This may potentially highlight the versatility and robustness of TF-prioritizer. This is also important as your readers are not (certainly not all of them) from the mammary gland development field. As such, dedicating a large portion of your discussion about this process is too much. If you manage to highlight the versatility of your pipeline by capturing more than one specific developmental process will do the paper a great favor by highlighting the different ways TF-Prioritizer can be used, which in turn may attract more users to utilize your pipeline.

2. I have an issue on how the 'Results and Discussion' section is organized. The authors dedicated separate subtopics for each TFs they prioritized and made literature review of their role in mammary gland development and lactation. My recommendation is to instead have one subtopic and discuss these TFs paragraph by paragraph in a concise manner. A more concrete way to reorganize this will be to separate these into two subtopics, (1) Known TFs with role in mammary gland development/lactation (2) Novel TFs with predicted role in mammary gland development/lactation. To make these reorganization easier/smooth, cutdown details of what you observe in the figures (e.g. p16, line 22-27 and p17, line 1-3), discuss the main message and put the detailed text about the figures in the Figure captions

.3. All figures and tables should have more information in the caption including those in 'supplementary Material'Minor:1. p7 line 9, how often do one find these combinations of data types (modalities) in different conditions, cell types or models being studied. Could some of the HMs be replaced with other data modalities e.g ATAC-seq, DHS data or data from other chromosome profiling methods? Could the pipeline be adapted to incorporate Cut and tag/cut and run or is it specific to only ChIP-seq data. Authors should try to discuss whether this is possible or not.2. P13 line 3, the authors discuss that "ChIP-Atlas provides more than 362,121 datasets for six model organisms…". Could TF-Priotitizer be easily adapted to other databases/resources, which ChIP-Atlas do not cover (e.g. for other organisms) that the community might be interested in?3. p14 line 2 "... expressed gene for this analysis but focus on affinities only". Why this is the case is not argued/discussed. This and other choice of parameters would be nice if they are discussed under a separate subtopic to easily inform future readers/users of TF-Priotitizer

1. Figures should be cited in chronological order. Adjust the text or reorder the figures

2. When the authors discuss the evaluation of the prioritized TFs in separate sections, they often start with "In Figure Xa) …" and "Figure Yc) shows that …", etc, such kind of texts best fit as Figure captions instead of in the 'Results and Discussion'.

3. p21 line 16, "We predicted that several Rho GTPase-associated genes are regulated by the predicted TFs" This sentence sounds a bit circular, you may rephrase as follows 'We propose that our predicted TFs regulate several Rho GTPase-associated genes

'7. Figure 3 and 4 have the same general message/purpose and look redundant. This is reflected in the phrase '...(black arrows) as they are already known to be crucial in either mammary gland development or lactation.' and 'In the heatmaps, we can observe a clear separation of these target genes between the time points X and Y…'. I suggest the authors choose one of them as a main figure and place the other in Supplementary Material.

1. On Fig.3,4 captions the authors should indicate what the black boxes represent. One can guess what they are from your main text but the captions could profit from a bit more detailed explanation. You should at-least describe some of the things that needs to be highlighted from the figures to easily guide your readers

This work has been peer reviewed in GigaScience (see https://doi.org/10.1093/gigascience/giad026 ), which carries out open, named peer-review. These reviews are published under a CC-BY 4.0 license and were as follows:

Reviewer Xiaowo Wang : Markus et al. developed a new pipeline TF-Prioritizer to discover potential cell or tissue-specific transcription factors (TF) with ChIP-seq data of histone modification and RNA-seq data. TF-Prioritizer is mainly based on the framework of the state-of-art method TEPIC to model TFs regulating the gene. The authors extend TEPIC by integrating more information like differential gene expression using DEseq and linking the TF binding in cis-regulatory element to the gene expression using DYNAMITE. They also designed a new statistical method to rank the TFs across different cell types or in the time-serious cells. The authors also provide some cases to validate the pipeline. The pipeline is useful in biomedical research. The manuscript is well-written and provides enough details. The authors addressing or further considering the following issues may benefit readers.1. TF-Prioritizer requires ChIP-seq of histone modification (HM) as the input. It may support different types of HM. Users may want to know how to choose a proper set of HMs? Authors should evaluate some cases to show TF-Prioritizer's performance when inputting different HMs.2. ATAC-seq is more widespread for different kinds of cells or tissues. It seems TF-Prioritizer can also apply to ATAC-seq peaks. Why TF-Prioritizer does not support ATAC-seq now?3. On page 11, there may be some mistakes in the definition of BG(m) and FG(t,m). t \in TF(m) of BG(m) should be moved to FG(t,m)?4. The software is hard to install without sudo/root account. It would be better to provide a docker image that is ready for the users to run the software.

This work has been published in GigaByte Journal under a CC-BY 4.0 license (https://doi.org/10.46471/gigabyte.81), and has published the reviews under the same license. These are as follows.

**Reviewer 1. Jin Sun **

Gomes-dos-Santos et al., have upgraded the freshwater mussel Margaritifera margaritifera genome with the usage of long-read sequencing. Overall, this version has been dramatically improved compared to the former one, with the increased N50 value and BUSCO score and decreased No. of contigs. Considering the important economic value of M. margaritifera and the high quality of assembly, I must congratulate the authors on this. However, in contrast to the high-quality assembly, I am a bit aware of the genome annotation part. To me, the number of gene models predicted is a bit higher compared with other molluscan genomes. This can also be reflected by the low proportion of gene models that can be annotated by Swissprot or GO etc. I suspect that the high number of gene models could be the consequence that only the ab initio evidence was applied in the current study. More sophisticated ways, such as EVM or maker, shall be used to see whether the number of gene models can be reduced without sacrificing the BUSCO scores on the gene models.

Line 76, The official name shall be “Oxford Nanopore Technology (ONT)”.

Fig. 1, it is interesting to see the wide distribution of M. margaritifera. I am a bit interested to know whether there are any genetic differentiations between the European population and the North American population.

**Reviewer 2. Rebekah L. Rogers **

Is there sufficient detail in the methods and data-processing steps to allow reproduction?

Y. All methods seem standard and high quality for a genome release.

If the authors could add a table comparing with other Unio genomes, that might be helpful. Gene numbers, BUSCO scores, N50s, and other relevant stats. It will help readers see the value of this more contiguous genome -V. ellipsiforma (Renaut et al.) -M nervosa -P. streckersonii

This work has been published in GigaByte Journal under a CC-BY 4.0 license (https://doi.org/10.46471/gigabyte.80), and has published the reviews under the same license. These are as follows.

**Reviewer 1. Grace Mugumbate **

Please add additional comments on language quality to clarify if needed

Yes. First person reporting has been used with the word "We' used extensively.

Are the data and metadata consistent with relevant minimum information or reporting standards?

No. There is need to specify the type, size, standardisation and curation of the data that was used, especially when additional data was obtained from different databases.

Is the data acquisition clear, complete and methodologically sound?

Yes. Sources of data are indicated in the paper, however the size of the data sets and type of data is not clear.

Is there sufficient detail in the methods and data-processing steps to allow reproduction?

No. There is need to give more detail in the methods for reproducibility.

Is there sufficient data validation and statistical analyses of data quality?

No. Validation was performed, however no statistical analyses was mentioned.

Is there sufficient information for others to reuse this dataset or integrate it with other data?

No. More detail is needed on data retrieval to allow reuse of the dataset.

Additional Comments:

The Authors presented their work entitled 'Mycobacterial Metabolic Model Development for Drug Target Identification'. This is very innovative work that led to generation of M. laprae and M. abscessus models, important tools for drug target identification. Target identification for a number of infectious diseases provides information for structure-based molecular modification of new and alternative diseases. The target specific compounds will help reduce side effects among other things. Generation of the models by the authors is commendable.

There are a few corrections: 1) Under Abstract: Line 4: Please note that Mycobacterium tuberculosis is not a disease but the bacterium that causes the diseases tuberculosis. 2) Mehtods, GEM reconstruction, curation and simulation (i) Line two: Name the "other organism specific databases" (ii) Give a brief description of the COBRApy and the GLPK even if the source had been given. 3) The Method section need to be more informative to allow for reproducibility.

**Reviewer 2. Nagasuma Chandra **

Is there sufficient detail in the methods and data-processing steps to allow reproduction?

Yes. It would be useful if the authors could comment on how the models vary between the two species and with respect to M. tuberculosis. Specifically, a note on how the authors deal with alternate enzymes and whether they included enzymes specific to each species, would be helpful.

Is the validation suitable for this type of data?

Yes. A figure depicting the overall capability of the models would be useful

Additional Comments:

Genome-scale metabolic models are useful to the community as they can be used to address a variety of questions. It would be useful if the authors could include a section on the comparative performance of the models and link it to the known metabolic capability of these microbes.

This work has been peer reviewed in GigaScience (see paper https://doi.org/10.1093/gigascience/giad028), which carries out open, named peer-review. These reviews are published under a CC-BY 4.0 license and were as follows:

Reviewer 1: Philippe Boileau

This manuscript introduces the new docker-based JupyterLab framework in Galaxy, describing its core components and demonstrating its use in the reproduction of two analyses. The proposed framework is also thoroughly compared to competitors, like Google’s Colab and Amazon’s SageMaker. This tool is bound to have an impact on the life sciences: it democratizes computational analyses and facilitates reproducibility. I thank the authors for their important work. However, I think that this technical note should be reviewed for grammatical errors and faulty punctuation. I’ve identified some such issues in the comments below but wasn’t able to address all of them. Included in the comments are other remarks which, if addressed, could strengthen some key takeaways. • The first sentence of the abstract states that AI programs require “powerful compute infrastructure” when applied to large datasets. I think readers would like to know how you qualify an infrastructure as “powerful”. A brief definition could be included in the second sentence instead of repeating “. . . hosted on a powerful infrastructure . . . ”. • Is it “JupyterLab” or “jupyterlab notebook”? The Project Jupyter site seems to use the former. Based on the documentation, JupyterLab is a web-based user interface that can open Jupyter notebooks (.ipynb files). • The statement “Artificial intelligence (AI) approaches such as machine learning (ML) and deep learning (DL) . . . ” implies that ML and DL are distinct aspects of AI. This distinction is insinuated throughout the rest of text. Isn’t DL a subset of ML? I suggest replacing “ML and DL algorithms” by “ML algorithms” and specifying “DL algorithms” only as needed. • I believe there’s a missing comma between “ecosystems” and “enabling” in the first sentence of the Docker container section. • Consider reformatting “A container runs . . . of the running software.” to “A container runs an isolated environment with minimal interactions between it and the host OS. Running software in a container is more secure.” • Related to the suggestion above: Can you explain why this increased security is necessary? An example might help emphasize the importance of a secure container. • I think “Docker container inherits . . . ” should be “The Docker container inherits . . . ”. Same goes for “Docker container is decoupled . . . ”. • Consider reformatting “Moreover, it can easily be extended by installing suitable packages only by adding their appropriate package names in its dockerfile.” to “Moreover, the Docker container is easily extended: additional software packages can be installed by adding their names to the dockerfile.” • Consider replacing “some of the popular ones are” by “including” • I believe there’s an unneeded comma between “. . . platform for both” and “rapid prototyping. . . ”. • I believe that there’s missing a word in the last sentence of the Features of jupyterlab and notebook infrastructure section: “. . . an H5 file.” • “google” and “amazon” should be capitalized. • Consider removing “and non-ideal” from Related infrastructure section. • I believe the comma in “. . . but they come at a price, . . . ” should be replace by a colon. • I believe there’s missing a comma between “. . . free of charge” and “similar to colab . . . ”. • Why is sharing a sessions’s resources across multiple notebooks more useful than operating each notebook in a separate session? Isn’t the latter preferable when a notebook causes a session to crash? • “deep learning” in the Implementation section should be replaced by “DL” for consistency. • I think that readers would find a link to your tool on Galaxy Europe useful: https://usegalaxy.eu/root?tool_id=interactive_tool_ml_jupyter_notebook. The same is true for your tutorial: I think readers would find a URL in the text more easily than in the references. However, the tool failed to execute on usegalaxy.edu with the following error message: “This tool is restricted to authorized users”. I was unable to follow the tutorial. Was this a one-off issue with the Galaxy servers?

Reviewer 2: Milot Mirdita

Kumar et al. present a Docker-based integration of Jupyter Notebooks in the Galaxy workflow system that can utilize GPUs. This notebook is also available in the Galaxy Europe instance.

I was able to create a Galaxy Europe account, find the newly introduced Galaxy tool and submit a job. However, it remained stuck with the message "This job is waiting to run" and the job info "Stopped" for multiple hours. I was able to download the docker image and run it on a local server with multiple Nvidia GPUs. This resulted in a running Jupyter Lab, however running the GPU based examples resulted in driver mismatch errors/warnings (pynvml.nvml.NVMLError_LibRmVersionMismatch: RM has detected an NVML/RM version mismatch; kernel version 470.141.3 does not match DSO version 515.65.1 -- cannot find working devices in this configuration). Thus, the examples ran on CPU only. I did not try to resolve this issue and only repeated some examples.

The authors show two use-cases for the GPU Jupyter Docker and provide a step-by-step tutorial for usage on Galaxy Europe. Shipping machine learning applications that utilize GPUs as Jupyter Notebooks has become popular recently and supporting these through well-known and freely accessible Galaxy servers, such as Galaxy Europe, would be of clear benefit to users. Additionally, it would be very valuable for method developers like me to easily deploy GPU-based methods to Galaxy servers.

Major: - As mentioned before, I had issues getting a running Jupyter Lab on the Galaxy Europe server. Is this due to a limited number of GPUs or was this due to an error? - Our ColabFold Multiple Sequence Alignment server currently processes about 10-20k MSAs per day. We do not know how many of these are running on Google Colab or on users' local machines. However, a substantial number of predictions are running inside Google Colab. The authors claim that Google Colab's and Kaggle's resources are scarce. However, generally, users (with either free or pro accounts) are given an instance nearly immediately on Colab. I recognize that it is extremely difficult to compete with these commercial platform providers. However, providing a long-term, freely available and securely funded, platform with ML accelerators would be extremely beneficial for the whole community. I would like to see a discussion on what GPU resources are currently available to users of Galaxy Europe (and the whole Galaxy Project) and what plans exist to expand these in the future. - The size of the docker container (compressed ~10GB, uncompressed ~22GB) seems difficult to sustain. Both keeping up an up-to-date Docker image and ensuring the availability of older images for reproducibility looks difficult to me, especially with such fast moving dependencies such as machine learning frameworks. How do the authors plan to deal with this issue?

Minor: - Please highlight the tutorial (https://training.galaxyproject.org/training-material/topics/statistics/tutorials/gpu_jupyter_lab/tutorial.html) on GitHub and inside the container readme (home_page.ipynb). It is very easy to overlook. I also nearly overlooked the example notebook repository (https://github.com/anuprulez/gpu_jupyterlab_ct_image_segmentation). I found it confusing, that I could not find the two shown example use-cases inside the Docker container. I only later figured out that I have to clone the example repository into the running container. - The manuscript highlights various workflow methods (elyra, kubeflow, airflow), however it needs clarification on how the Galaxy workflow integration works. I saw that it is possible to give input of another Galaxy output to the tool. I would appreciate a tutorial on how to make the GPU Jupyter Docker into part of a Galaxy workflow with multiple tools running. I think the above mentioned tutorials can be expanded to show how the output can be given to the next tool. - Docker Hub has introduced many business-model changes such as deleting container images that are rarely used, which poses a challenge for reproducibility. I know that Dr Grüning is involved in the Biocontainers project. I would recommend investigating if it is possible to combine these efforts to make this GPU container and derived containers long term available. - The Docker container is explicitly running as a root user, while the manuscript highlights the security benefits of Docker. The cited report by Baset et al. highlights the security benefits and the many security challenges that Docker containers pose. I suggest checking what security best practices for Docker containers are possible to implement, while still allowing GPUs to be exposed to users. - I recommend revising the manuscript for conciseness, with an additional focus on capitalization of words.

This work has been peer reviewed in GigaScience (see paper https://doi.org/10.1093/gigascience/giad025), which carries out open, named peer-review. These reviews are published under a CC-BY 4.0 license and were as follows:

Reviewer 1: Weilong Guo, PhD

Patrick König and colleagues have built a web application for the interactive query, visualization and analysis of genomic diversity data, supportting population structure analysis on specific genetic elements, and data export. The application can also be easily used as a plugin for existing web application. According to its documentation, this application can be easily installed form pip, Docker and conda, which would be useful for population genomic studies. There are still several concerns about this manuscript.

Major concerns:

1. As for the SNP visualization function, there are only very limited numbers of SNPs can be read on the webpage, without function such as "zoom in" or "zoom out"(it is suggested to add such functions or similar functions). Although the application can export almost all the SNP sites of a whole VCF file, it is far from user-friendly.It is suggested to add a track of chromosomes showing the genomic windows under querying, allowing the cursor to select or adjust the genomic regions (UCSC-browser style), which is necessary for an intuitive user experience.

2. The BLAST function could serve as a useful entry point. But what is the starting position of the query sequence when mapped on minus strand? The authors should make it more clearly explained on the website.

3. TThe authors mentioned that their application would convert the inputted VCF file into Zarr format. Thus, more performance evaluation should be declared to show the advantages of this strategy (rather than using the VCF file directly).

4. The authors should also compared the their applications with other similar existing web applications, such as CanvasDB, Gigwa, SNiPlay and SnpHub, to highlight their advantages and improvemences.

Minor concerns:

1. The analysis functions are still insufficient. Commonly used analysis tools or methods, such as haplotype analysis, STRUCTURE analysis, distribution of nucleotide diversity and selection sweep analysis, are also suggested to be supported.

2. Ref. 22 is not completed.

Reviewer 2: Armin Scheben

The authors present the web app DivBrowse for visualizing genomic variant data. Their code is publicly available, and their web app is well-documented and provides several demonstration implementations for human, mouse and barley. The manuscript is well-written and concisely covers the key features of DivBrowse and summarizes the implementation of the software.

I was able to test the demonstration website and was impressed with how smoothly everything ran and was set up. Due to time constraints, I was not able to test the installation and set up of DivBrowse but the documentation looks sufficient to allow easy set up by experts. Overall, I think this is a useful contribution to the community. One key issue I believe the authors should address, however, is that the manuscripts presents DivBrowse in a vaccum, not providing much mention of or comparison with existing software with overlapping functionality. Below I provide some further details illustrate my point and how it might be addressed, as well as listing several other minor comments.

Main comment

The authors rightly indicate in their introduction that the growing amounts of genomic data generated require robust solutions for visualization and exploration that does not require use of the command-line. But the authors fail to mention that there exists a considerable ecosystem of software that already does this. Moreover, some of the software available offers substantially expanded features compared to DivBrowse.

To help readers better decide when DivBrowse might be the right choice for their needs compared to other options, the authors could cite existing software and provide some comparison. My knowledge of all available software is not exhaustive, but Wang et al. 2020 (https://doi.org/10.1093/gigascience/giaa060) in their publication of SnpHub provide a comparison table including SnpHub itself and Jbrowse. I would consider both of these tools for exploration and visualization of SNPs and additional data, similar to DivBrowse. Jbrowse is relatively widely used and considerably more feature-rich. The standalone offline tool TASSEL (https://academic.oup.com/bioinformatics/article/23/19/2633/185151) also offers many options for visualisation and exploration and analysis of VCF data offline. There may also be other tools I am not aware of, and readers would likely benefit from some brief overview of the landscape and the pros and cons of each piece of software and what differentiates DivBrowse.

Minor comments

The authors can consider the minor comments below as 'take it or leave it' comments. I do not think it is essential to address these, but in my view they may enhance the manuscript.

1) In the discussion, the authors point out the efficiency and low latency of DivBrowse, however this is not quantified in the manuscript. If it were technically feasible without substantial effort, it might be useful to quantify in some way just how efficient DivBrowse can be, especially if this could be one of the stand-out features of DivBrowse.

2) The authors use divergence Bezier curves to increase the amount of variant calls that can be visualized. This is helpful and a useful default. However, invariant sites can also be of considerable evolutionary and breeding/medicinal interest. When collapsing invariant sites, they become indistinguishable from unmapped regions. This is a fundamental issue and many VCF files may not encode information on invariant sites, so it may not be possible to develop robust functionality that allows users to also show invariant sites optionally. Still, this point may be worth briefly mentioning in the discussion, if the authors agree it is noteworthy.

3) One advantage of visualization of relatively raw data like SNPs is that it can reveal patterns that are less obvious in other types of data exploration. To fully take advantage of this tools like Jbrowse allow export of the browser window in SVG format, allowing users to incorporate images into high-resolution figures. I don't expect the authors to necessarily implement this feature for this review, but it may be worth adding it to the list of potential enhancements that could be implemented based on user demand.

Reviewer 2: Mulin Jun Li

In this manuscript, the authors updated their previous ReMM to the GRCh38 human genome build, supported convenient and fast data source. Then, the authors take some examples to demonstrate the usability of the resource. It's original to point that the difference in prioritized tools between different genome build. However, we have following concerns and comments:

Major: 1. How to deal with missing value variants in test datasets when compare new ReMM with other tools, the author mentioned that ExPecto annotated only half of the million negative variants. 2. Although the CADD used the same negative training dataset, it's not suitable to compare it in the ReMM training dataset. How those tools performance in the independent test datasets. 3. The author presumes that new genome build will get better performance, is there some evidence can support this perspective, like the distribution of feature or training data in different genome build. 4. Other existing similar tools can prioritization disease-causal noncoding variant, such as regBase-PAT, NCBoost, ncER, etc. can the authors compare new version of ReMM with these tools.

Reviewer 3: Wyeth Wasserman

SYNOPSIS The manuscript describes an updated release of the ReMM regulatory variant mutation scoring system. The paper presents the performance of an updated version of the system and describes how it was applied to the most current release of the reference human genome.

OVERALL PERSPECTIVE This is a valuable resource for the community of researchers and clinicians working on the interpretation of genetic variants in the human genome. The work appears to be thoughtfully done and appropriate assessments have been provided. The use of the random forest models to weigh the contributions of features was particularly noted for the insights it provided into how features contribute to prediction. My biggest concerns are stylistic, which falls outside the scientific quality of the work. I provide these comments for the authors to consider and do not expect that my stylistic preferences will be uniformly accepted. A fair amount of justification of the manuscript focuses on the value of having a release for version 38 of the human genome, pointing to the field as not having done so broadly. I think this is misguided, as by the time people are reading the manuscript such points will have lost relevance. I suggest a focus on the science be given, as there is no need to justify things based on where other resources have progressed in releasing their version 38 updates. Points below include language/text clarifications that can be assessed by the authors. Writing styles differ, so stylistic comments should be optional.

MAJOR POINTS None. Well done and clearly presented.

MINOR POINTS 1. The word "various" is vague and often shows up when people are too busy to provide an accurate statement. Starting the manuscript with it makes a bad impression on this reader. You do not have to change it, but I thought you might appreciate knowing this impression. You could delete it with no harm to the sentence. (Not to get carried away, but the next sentence starting with "some" heightens the impression of 'hand waving'.) 2. I think I understand ", we apply cytogenic band-aware cross-validation using ten folds" but I encourage the authors to provide clearer wording for this point. 3. I would allow the reader to make their own judgement of performance. So please remove "excellent" from "we achieve an excellent performance" 4. "Rather than using ReMM scores for ranking, some users need to specify score thresholds" is confusing. I would change 'need to' to 'choose to' 5. "with lots of false positives" is a bit informal. I suggest "with a high false positive rate" 6. I am confused by "from three genomic regions (genic content and not overlapping with assembly gap changes) " as the brackets include two items, not three. 7. "maybe due to better mapping" - "maybe" should be "may be" 8. I think the language like "seems to be the only tool directly trained on training data and features derived from GRCh38." Is not particularly valuable long term. This is a useful contribution, but many tools are being updated to 38 and by the time this appears and is read, such statements decline in relevance. I would focus on providing this valuable resource, and not try to justify it based on a transient perception of where the field stands in updating versions. 9. "It is worth noting that in the context of extremely unbalanced data…" - you do note it. So I would change the wording to "In the context of extremely unbalanced data…"

Reviewer 1： Yan Guo

1. In the abstract "Some methods and annotations are not available for the current human genome build (GRCh38), for which the adoption in databases, software and pipelines was slow." Not sure what the author is referring to by some methods, this could be a grammar problem.
2. "Restricting variants to non-coding only removes a small proportion of variants", what is the proportion? Also, I don't understand the need to remove coding variants, shouldn't your model works also with coding variants?
3. The method the author used is based on a previous publication. However, there is still the need to give the detail of the method in this manuscript. There is a lot of missing information. For example, what is the outcome, whether a position is deleterious? How is the probability for deleteriousness calculated?
4. by a few specific variants. Thus, the overall Mendelian disease-related variants should be low. I am guessing that's why 406 hand-curated variants were used in the previous version of ReMM. If my assumption is correct, there shouldn't be a lot variants for Mendelian disease. How many variants are found to be positive in the entire genome?
5. In the online application, the results are limited to 500, the rest cannot be seen or downloaded. I would be better to allow the user to download the entire results.
6. The authors performed comparison with other tools and generated ROC curve which is dependent on knowing the true positives. There is no description of the dataset that was used for the comparison. Did the authors make sure that the training variants is not used for the comparison?

This work has been peer reviewed in GigaScience (see paper https://doi.org/10.1093/gigascience/giad024), which carries out open, named peer-review. These reviews are published under a CC-BY 4.0 license and were as follows:

Reviewer 2: Ben Woodcroft

Cornet et al have generated a collection of NextFlow pipelines which provide a pipeline to analyse data associated with genome or raw sequencing data of microbial organisms and protists. The methodology appears sound and reproducible. My main concern with the manuscript is that it is not well described in the abstract, introduction or GitHub repository. It isn't clear whether the analyses are specific for genomics questions arising from culture collections, or if it is more broadly applicable. There is also no discussion about other pipelines which achieve similar things e.g. ATLAS https://metagenome-atlas.github.io/

I also had a number of minor concerns, detailed below.

A number of grammatical errors detected, these should be fixed. Parts of the manuscript are also slightly too informal e.g. "This confirms the interest of 221using ORPER to spot interesting SSU rRNA sequences" It would be helpful if the GitHub front page could provide a concise description of what the software aims to achieve, to make its use more understandable. 106: "as it happened" grammatical error "Assembly.nf" Commonly assembly is a separate process to binning, but here binning has been included. Perhaps a clearer name might be Genome-recovery.nf ? 124: "Researchers interested in a better understanding of these tools can read the recent review on the detection of genomic contamination made by Cornet et al. [15]." While not inappropriate, this is perhaps too much self-citation. Why is contamination assessed but not completeness? 129: "annotation of bacterial proteins is automatic" Automatic in what sense? Annotation also refers to describing the function of the protein usually, but here the meaning appears to be restricted to ORF calling. I found this somewhat confusing. Also "in the different GEN-ERA workflows" is unclear - does this mean that prodigal is run as part of the Assembly.nf workflow for instance? 143: "Orthology.nf automatically provides the core genes, shared by all the organisms in unicopy" what is meant by "all organisms" here? 145: "The OGs of proteins 145 can be further enriched" what does "enriched" mean? 163: GTDB.nf is described in the "Other workflows" section, when it is phylogeny-related. 172: "it was 173 technically not possible to include Mantis in a container" I am curious as to why this was the case? I do not have any specific insight or ability to judge the accuracy of this statement, just curious. Inclusion of a sentence describing the difficulties might help other workflow developers and/or the Mantis developers. 190: "Gloeobacterales are the most basal order of the 191 Cyanobacteria phylum" This statement is somewhat controversial, because the GTDB has defined the Melainobacteria as being a part of the Cyanobacteria phylum based on RED values. I would suggest removing "the most basal" or making it clear that cyanobacteria refers to photosynthetic cyanobacteria rather than the phylum. 189: The methods for this section are not described in the methods section. They are only briefly described in the Findings section. A clearer link to these methods should be made from the maintext and methods. 212: Showed -> show. 215: "estimate the sequencing level of the order" it isn't clear what meaning this has. 224: Our results demonstrate the absence of one metabolic 225pathway" There are many metabolic pathways, presumably it is missing more than one. 233: "examples of the practical usage of the GEN-ERA toolbox are available in Supplemental 234File 1." this does not make it clear that this refers to the methods for this specific example.

This work has been peer reviewed in GigaScience (see paper https://doi.org/10.1093/gigascience/giad022), which carries out open, named peer-review. These reviews are published under a CC-BY 4.0 license and were as follows:

Reviewer 1: Shakuntala Baichoo

Paper Title: The GEN-ERA toolbox: unified and reproducible workflows for research in microbial genomics The GEN-ERA toolbox provides a number of containerized workflows to researchers (without any specific training in bioinformatics) to study the diversity of well-characterized strains for fundamental and applied research. More specifically It facilitates all steps from genome downloading and quality assessment, including genomic contamination estimation, to tree phylogenetic reconstruction. It additionally provides workflows for average nucleotide identity comparisons and metabolic modeling. The supplementary file provides details of how to run the whole workflow (through 10 steps), found in the GEN-ERA toolbox on basal, for an empirical dataset of early emerging cyanobacteria. It provides an up-to-date phylogenomic analysis of the Gloeobacteralesorder, the first group to diverge in the evolutionary tree of Cyanobacteria. The github repo located at https://github.com/Lcornet/GENERA also provides more details about the GEN-ERA toolssuite. Though in the manuscript it is mentioned that the call to Mantis could not be included in the Singularity call, on the github repo they have indicated that Mantis is now installed in a singularity container for the Metabolic workflow (install is no longer necessary). The tool has been tested on an empirical dataset of 18 (meta)genomes of early-branching Cyanobacteria and the time taken as well as the results of the run are documented in the supplementary file. The authors claim that the toolsuite can be used to study the diversity of microorganisms, including bacteria and fungi. From the github repo, it is clear that a number of publications in high-impact journal papers have already resulted from the development of the GEN-ERA.

1) Are the methods appropriate to the aims of the study, are they well described, and are necessary controls included? This study aims at describing a toolbox, named GEN-ERA, and the methods section defines the various steps of the toolsuite. Looking at the supplementary file and the github, it is easy to follow the manuscript. The versions of the programs used in the case study are provided in the forms of nextflow scripts.

2) Are the conclusions adequately supported by the data shown? The results of running the toolsuite on an empirical dataset of 18 (meta)genomes of early-branching Cyanobacteria, at each step, as well as the time taken to download the files and the running each step, are convincing that it works fine, at least for Cyanobateria. But this is found in the Supplementary Material. There should be section on Discussion and Conclusion in the main text.

3) Please indicate the quality of language in the manuscript. Does it require a heavy editing for language and clarity? But t The use of English language is adequate and concise and can be understood clearly, by researchers interested in studying diversity of micro-organisms.

4) Are you able to assess all statistics in the manuscript, including the appropriateness of statistical tests used? The statistics involved in the phylogenetic analyses are integrated in the existing programs. Hence I am not able to assess the statistics.

5) Final Comments The proposed toolbox/toolsuite described in this manuscript is very relevant and worth a read for researchers interested in studying the diversity of microorganisms, including bacteria and fungi, especially as it helps to facilitate their life through the use of well-defined containerized NextFlow workflows.

I strongly believe that there should be a section on the Discussion of the results of running the toolbox for the case study and a Conclusion in the main manuscript. This will help readers in understanding the importance of the toolbox better.

See the announcement https://doi.org/10.1093/gigascience/gix046

Reviewer2-Raphael Eisenhofer

Piro and Renard introduce GRIMER, a tool that automates microbiome-related analyses and creates rich, offline-supported report that can be shared with collaborators or hosted online. I think that they gave a great summary of the problem of contamination in the microbiome field, and clearly explain the gap that their software fills. They exhibit GRIMER on previously published datasets, which are available to view online. Overall, I'm very impressed with the dashboard—it looks great, is easy to explore datasets, and highly portable. I can certainly see myself using GRIMER on some of my future datasets, and I have no doubt that it can be a valuable tool for others in the field. I do however think that the documentation and usability of the tool can be improved, and I give some suggestions below. Addressing these issues will, in my opinion, lead to a wider adoption of the tool by researchers in the field.Usability:I managed to test GRIMER on a 16S amplicon dataset, but given the sparsity of the documentation, this took me a little longer than expected (in addition to quite a few steps), and I think that there are improvements that could be made to make it easier for people to use GRIMER from formats that people commonly generate.For example, QIIME2 is perhaps the most used 16S amplicon analysis pipeline, so the ability to import directly from .qza files (e.g. table.qza, taxonomy.qza) would give GRIMER much greater reach. If this is beyond the scope to incorporate within the GRIMER codebase, at least provide the exact code needed in the documentation for people to export their .qza files to files compatible with GRIMER.Likewise from phyloseq, a commonly used R package for microbiome analyses. Could some documentation/code be added about how best to export phyloseq objects to a format that GRIMER can handle?I mostly analyse shotgun metagenomic datasets (genome-resolved), and I foresee more users using these types of data in the future. Therefore, the ability to parse gtdb-tk outputs directly would be very helpful. Perhaps have a flag --gtdb that parses the 'gtdbtk.bac120.summary.tsv' and 'gtdbtk.ar53.summary.tsv' files.Following on from this, CoverM (https://github.com/wwood/CoverM) is quite commonly used for generating final MAG count tables (.tsv), so the ability to import them directly would be a really nice quality-of-life addition, and something that would not require much coding to accomplish.I believe that these adjustments will make the tool far more accessible for everyday users and increase the adoption of GRIMER by the wider community.For the actual report, if possible, I would like the ability to export ASVs/features/MAGs from the report that the user thinks are contaminants. This could be a list that the user could copy/paste, or the direct export of a .txt/.tsv. Perhaps the user could tick a box next to the ASVs/features/MAGs to save them to a list/viewer? The reason for this is that the logical next step I see after using GRIMER is to go back to your dataset and filter out the putative contaminant ASVs/features/MAGs. Being able to produce such a list will make subsequent filtering by the user easier.I couldn't get decontam to work with my dataset, here was the error:raise KeyError(f"None of [{key}] are in the [{axis_name}]")KeyError: "None of [Float64Index([nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,\n nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,\n nan, nan, nan, nan],\n dtype='float64')] are in the [index]"I can post this as an issue on the repo if you'd like.Regarding the specification of negative and positive controls in the config.yaml, would it be possible for this to be implemented from the executable? For example, there could be a flag '--control-column' that specifies the column in the user's metadata file. '--control-column control' would parse the 'control' metadata column, and for cases where are values 'negative', 'positive' assign them automatically. This is just a suggestion that could make it a bit easier for users to set control samples, rather than having to create a new .txt file and change the config.yml.Dependencies:When installing via conda, I ran into the following error:ImportError: cannot import name 'PearsonRConstantInputWarning' from 'scipy.stats'It seems that this can't be imported from later versions of scipy, but I managed to fix it by forcing scipy=1.8.1. You should be able to force this version in the conda recipe.Minor grammar:Line 16: replace 'perform' with 'performs'Line 50: 'found in the [9]'Line 56: replace 'as technicians body' with 'microbes from laboratory technicians'Line 60: I would remove the 'environmental' adjective here, as contamination affects all low-biomass samples.Line 63: I would use 'samples' in place of 'environments' here. You may also consider suggesting that some samples may even contain no microbial DNA. E.g. replace 'low amounts of' with 'little to no'.Line 64: Replace 'ideal scenario for an exogenous contaminants' with 'an ideal scenario for exogenous contaminants'.Line 72: perhaps consider referencing decontam here.Line 79: replace 'due to increase in costs' with 'due to the increase in cost associated with their inclusion'.Line 81: Consider referencing first author's last name, e.g. 'Moreover, XXX et al. [45] reported…'Line 88: remove 'outcomes'

Reviewer1-Gavin M Douglas

Piro and Renard present GRIMER, which is a bioinformatics tool for summarizing microbiome taxonomic data in various ways, with the main purpose of identifying putatively contaminant taxa. The authors convincingly argue that there is great value in looking at several different aspects of a dataset when determining which taxa are potential contaminants. I think this tool could potentially be very useful for the field, but I think at the moment there are several places where users might be confused and perhaps be overwhelmed without more documentation.The main point of confusion I'm concerned about is regarding the "common contaminants". It's not convincing that you can just classify a taxon as a contaminant regardless of what environment is being profiled. Also, under this approach, if a taxon is identified once as a contaminant in an earlier study, would it then be classified as a contaminant in all datasets processed by GRIMER? This would mean that a lot of high-abundance taxa in certain environments would be wrongly thrown out. For instance, you can imagine high-abundance taxa on the human skin might be more likely to be contaminants during sequencing preparation, but of course many researchers are very interested in profiling the skin microbiome. I think the authors realize this, but I'm concerned that typical users may not appreciate this point. I think explicit discussion of this point in the discussion is needed and also an example of how this might look in practice (e.g., if skin microbiome samples were input to GRIMER, as part of a larger tutorial that could be online [see next point], would help avoid this mistake).The authors do a great job of walking through some results in the text, but more documentation is needed for the reports. The authors should include a basic tutorial that provides example input files and then walks through each individual tab. This could done all through text with screenshots of the GRIMER, or perhaps with a video tutorial. In addition, for someone just opening the example reports, I'm sure they will be wondering what data was produced by GRIMER (e.g., they might wrongly think GRIMER did the taxonomic classiciation) and what data was needed as input.The authors should expand on how the correlation step is used to identify contaminants. There is great interest in identifying clusters of co-occurring taxa, so identifying a cluster of 9 genera in Figure 5 doesn't seem like evidence of contamination to me. Perhaps it is when considered with other lines of evidence though, but this should be made clearer. Currently this legend implies that it alone points to reagent-derived contaminationThe figure text needs to be increased in size. Using more panels split across additional rows and removing unnecessary info (e.g., not all control categories need to be shown in Figure 1) would make these figures easier to interpret. I realize that you were hoping to use the raw GRIMER figures, but based on the current display items it does not seem like they are publication ready.The acronym WGS generally refers to "whole genome sequencing" (i.e., for single isolate organisms) not "whole metagenome sequencing". The standard acronym for the latter case would be "MGS", for "metagenomics". Also, the term "shotgun metagenomics sequencing" is mostly commonly used in this context, I've never come across "whole metagenome sequencing" before. Either way, "WGS" will mislead casual readers with the current usage, so this should be changed on your website and in the manuscript.The taxa parsing capabilities sound like they will save a lot of tedious, manual data mapping! Just checking - how does it perform with new taxa names / typos?Text editsL11 - "are challenging task" should be "is challenging"L12 - can remove "by design"L12 - "helping to" should be "to help"L13 - "can potentially be a source" I think should be "that could reflect"L14 - "evidences" should be "evidence"L13 + L14 - Unclear what is meant by "external evidences, aggregation of methods and data and common contaminant" - should be clarifiedL15 - "that perform" should be "that performs"L17 - "towards contamination detection" should be something like "to help detect contamination"L41 - "hypothesis" should be "hypotheses"L42/43 - "analysis can hardly be fully" should be something like "the required analysis is difficult to fully…"L56 - "technicians body" should be "a technician's body"L60 - "strongly affects environmental" should be "especially environmental," (note comma)L64 - "ideal scenario for an" should be "an ideal scenario for"L67 - "not to bias measurements and not to" should be reworded, possibly as: "to not bias measurements and to ensure that bias is not propagated into databases"L75 - "were proposed. They are " should be "have been proposed. These are"L77 - "among others" should be ", and others" (note comma)L79 - "increase in costs" should be "the required increase in costs"L88 - add "a" before focusL90, L196, L265, and elsewhere - "evidences" should be "evidence"L99, L104, L117, and possibly elsewhere - "analysis" should be "analyses" (when plural)L106 - "each samples/compositions" should be "each sample/composition"L110 - add "a" before taxonomy database and "the" before "DNA concentration"L132 - "specially" should be "especially"L134 - remove "a" before "the"L151 - add "of" after "thousands"L182 - "is" should be "are"L196 - "evidences" should be "evidence". And rather than "Evidences towards" it would be correct to say "Evidence for" or "Evidence supporting"L208 - add "the" before "overall"L246/247 - "generated several studies and investigations" should be something like "motivated several investigations"L248 - should be something like "from the maternal and fetal sides"L279 - remove "a"L280 - Add "the" before "Jet"L284 - capitalize "Qiita" and re-word "Pick closedreference OTUs with 97% annotated with greengenes taxonomy"L293 - Should be "Furthermore" rather than "Further"L295 - I think it should be "with low and high human exposure, respectively"? Or do you mean they both have highly variable exposure?L297 - "could be a also an" should be "could be driven by an"L300 - "against" should be "and"L304 - "correlated genus" should be "correlated genera" (and in other cases, such as in the Fig 5 and 6 legends, where "genus" should be plural version, i.e., "genera")L305 - "Such pattern" should be "Such a pattern"L307 - Should be "groups" rather than "organisms groups", or just "genera" as I believe each is a genusL313 - Remove "a"Fig 5 legend: "point" should be "points"Fig 6 legend: "taxa is abundant" should be "This taxon is abundant" and "inversely correlate" should be "inversely correlated". "a contamination evidence" should be "potential contamination"

Reviewer4-Madeleine Geiger

This well written study integrates different approaches and methodologies to tackle the still obscure nature and origin of the dingo and its sub-populations by thoroughly characterising and comparing an "archetype" dingo specimen. I have read and commented on the abstract and the introduction, as well as the morphology related parts of the methods, the results and the discussion. The methods of morphological comparison, as well as their description and the reporting of the results are sound. However, in some sections it is difficult to comprehend the results and their interpretations, as well as the significance and nature of the suggested "archetype" specimen Cooinda. I therefore made some suggestions for additions and edits to the text and the figures, which hopefully help to increase comprehensibility and consistence of the text (see my comments below). I could not check and comment on the raw data because the links to the supplement given in the manuscript (figshare) do not work. Sorry if I'm stating the obvious here, but to be able to access the raw data is particularly important if the described dingo should act as a reference archetype. L. 74: Add «of the dingo» after "ecotypes": "[…] compare the Alpine and Desert ecotypes of the dingo […]". Otherwise it's not really clear what this is about. L. 91: It's unclear to me what you mean by "this female". I would suggest to exchange this expression with the previously used name of the animal. L. 94 ff.: The conclusions do not really fit to the rest of the abstract, specifically the aims as stated in the beginning. What I read from the "Background" section is that this work is about defining a "dingo archetype" via different approaches (genetic and morphological). The conclusion, however, is centred around the individual Cooinda. I would suggest to open up this section, to also make conclusions concerning the previously stated aims of the paper. L. 105 ff. and L. 369 and L. 508: A very nice opening! However, I feel that there is a somewhat misleading interpretation of the domestication process as a discrete trichotomy: wild > tamed > domesticated, when in fact domestication is a continuum with various stages in between the two extremes of the "wild" and the "intensively bred". There are various forms - even today - of "half-domesticated" populations, such as e.g., many of the Asian domestic bovids, or the reindeer. Thus, I would strongly argue that the dingo - although special due to the almost complete lack of human influence on its evolution in the last millennia - is not the only link between the "wild" and the "domesticated". See e.g.: Vigne, Jean-Denis. "The origins of animal domestication and husbandry: a major change in the history of humanity and the biosphere." Comptes rendus biologies 334.3 (2011): 171-181 L. 117: How do you define "large carnivore"? And: Are dogs more numerous than cats? I don't know the tallies overall, but in many parts of the world domestic cats are more frequent than dogs. L. 120 - 121: I think this sentence does not contribute to the manuscript and I would suggest to delete it. I also think that these are not the usual characteristics to discern the wolf from other canids. 123 - 125: I do not understand this distinction. In my opinion, the dingo could well be both, a tamed intermediate between wolf and domestic dog AND a feral canid. If I understand the current view of dingo evolution correctly, the dingo most probably constitutes an early domestic stage of the dog, which became feral. L. 150: I do not understand the reference to Figure 1 at this point. If you want to keep the figure reference at this place, I would recommend to extend the legend in order to be more descriptive about the significance of this individual dingo. Also: Is the question mark on purpose? Intro and Results in general: Cooinda is central for the research question and the paper. However, I do not really understand her position and significance right away from the text. Maybe this is just a matter of sequence of the paragraphs (some information is given at the beginning of the methods section at the end of the manuscript), but I think it would be crucial to introduce and explain Cooinda and her role (as kind of a reference "archetype") for the aims thoroughly already early on, preferably in the Introduction. This would e.g. also include: why of all the dingoes in Australia is Cooinda an appropriate choice to function as the "archetype". Further, it would be helpful to maybe have a figure showing the geographical distribution of the compared populations (alpine and desert, as well as Cooindas origin) to better understand the setting. L. 320 ff. and Figure 5: Would it be possible to add a visualisation of the shape changes described in the text into the figure? It is otherwise impossible to evaluate these shape changes. L. 328 - 345: It would be interesting to pursue the variation along PC2 further: Do you maybe have information from the raw-data if specimens of both the alpine and the desert group that were found to have particularly low or high values for PC2 are especially young and female, or old and male? In other words, do you find evidence in the dataset that there is an actual age and/or sex gradient along PC2? And what age was Cooinda when she died? L. 347: As also pointed out below, it would be important to note somewhere if these two specimens died at about the same time and/or were similarly treated (because of brain shrinkage in specimens that were frozen or otherwise fixed for a long time). L. 472: I would suggest to rewrite as: "Cooinda's brain was 20% larger than that of a similarity sized domestic dog […]". Further, I do not agree with the rest of the statement in this sentence. One of the hallmark characteristics of domestication is brain size reduction, which might be the result of selection for tameness (which you also describe later on). However, selection for tameness (an evolutionary process within a population) is not the same as taming (on the level of the individual). I would therefore suggest to re-write this sentence. Further and in general concerning the brain size part of this study: It would greatly increase the significance of this part of the work if you would compare the dingo brain size not only to one domestic dog, but set it into a larger context. There are plenty of published references for wolf, domestic dog, and dingo brain size estimates and it would be enlightening to compare your findings with those. Of course, there are methodological issues, but maybe a meaningful comparison is possible for some of them. For this I could recommend this review article: Balcarcel, A. M., et al. "The mammalian brain under domestication: Discovering patterns after a century of old and new analyses." Journal of Experimental Zoology Part B: Molecular and Developmental Evolution (2021). L. 483: Many of the surviving populations of re-introduced (i.e., feral) domestics were part of a fauna that did not correspond to the one of their wild relatives, but was somehow characterised by reduced predation or competition. This was certainly the case for the dingo (few other large predators in Australia) and for some island populations. Maybe you should double-check if this is really the case for the provided examples, but maybe it would be better to write that brain size reduction persists in feral populations at least under certain circumstances. L. 527: Why is it important that the reference dingo is a female? Please explain. L. 535 ff.: Please explain the significance of these special characteristics. Why and how are they special and important for the current study? Also: I'm not a native speaker, but I have the impression that some of the sentences in this section are a bit unusual. Please double-check the grammar. L. 739: What do you mean by "below" in the brackets? L. 741: Is this the right figure reference? I do not find this figure. Do you mean supplementary Figure 9a? 744 - 745: Could you briefly explain in one sentence the nature and number etc. of landmarks used in this reference study? (For those who cannot check the referenced work.) This would be quite important to be able to interpret the results. L. 744: Delete "earlier". L. 755: Could you briefly explain here if these were freshly dead specimens, or if they were already older (e.g. frozen, stored in a liquid etc.) please? This has some implications on brain morphology and size. L. 784 ff.: The figshare-links don't work. L. 884: I would suggest to re-write the sentence like this: "This was required because the brain was removed immediately after death, which caused some damage to the braincase." Supplementary Figure 9c: It's hard to match the reds of the convex hulls with the reds of the legend. Would it be possible to write down the names right next to the corresponding convex hulls? L. 895: Position remains the same relative to which other analysis? Maybe make a reference to text and/or figure (I guess Fig. 5) here.

Reviewer3-Sven Winter

The manuscript " The Australasian dingo archetype: De novo chromosome-length genome assembly, DNA methylome, and cranial morphology" does describe a de novo genome assembly of the Alpine dingo based on PacBio, ONT, 10X Genomics Chromium, BioNano, and Hi-C. Furthermore, it describes cranial morphometrics and methylation patterns to describe an Alpine dingo "archetype". The methods used seemed overall sound, yet, the writing was often confusing and unclear, so it was difficult to understand what was done and why. The writing, in general, is my biggest criticism of the manuscript, so much so that I was wondering if the authors, by accident, uploaded an earlier version of it. Throughout the manuscript, multiple writing styles and skill levels are evident, and I am sorry to say that it seemed as if the manuscript was copied together from different sources written by the different coauthors rather than a coherent manuscript. Some of the Figure Captions have superscript numbers to highlight individuals and at the same time proper labels that make them obsolete. I first thought they were remnants of footnotes in a previous version. Unfortunately, the methods section, for me one of the most important sections, needs serious improvements. I am not a big fan of having the methods at the end of the manuscript (I know that is how GIGAScience likes it), especially when some methodology is mentioned in the results in a way that you must look up the details in the methods section to understand it. Unfortunately, that is the case with this manuscript. As there are so many paragraphs in this manuscript that need some improvements, I can only focus on some of them in this review but encourage the authors to have a careful look at the whole manuscript before resubmission, as in the current state, I would not recommend it for publication. Detailed Comments: Abstract: The abstract is overall too long and needs to be much more concise, e.g., the discussion on taxonomic designation (L75-78) should be part of the discussion section but not the background paragraph of the abstract. L91 "this female" which female? Introduction: I am missing a short review of the taxonomy of the dingo, mostly with respect to the dog or wolf. I know you do not want to draw taxonomic conclusions from this study, but a short review of what others proposed would be helpful. Also, even though everybody knows what a dog is, scientific taxon names are a requirement in scientific writing and should be added at the first mention of any taxon (e.g., dog, gray wolf, dingo, etc.). L113: intermediate in what sense? Morphological, behavioral, ecologically? L123: Please explain in more detail why a type specimen is needed, especially, as I would argue, that a population-level genomic, morphological and behavioral study would be better to answer if dingoes are feral dogs or an intermediate form instead of a single individual type specimen. L139: reevaluation L142: this can be more concise, e.g., "Zhang et al. (17) found evidence for a separation of Australian dingoes into a northwestern group and a southeastern group, clustering with New Guinea Singing dogs" L150: remove "?" L151-154: This belongs in the discussion. L153: "… being characterized. However, we suggest …." Results: L161: Please consider changing it to chromosome-scale or chromosome-level genome assembly, which is much more common. L162-170: This whole paragraph is a short summary of the methods and does not include a single result. I know it sometimes is nice to recap the methods but in this case I do not see the need for it. Or at least it can be shortened even more to something like: "The final assembly after hybrid long-read assembly, polishing, and scaffolding has a total length of 2,398,209,015 bp …." L163: I'll highlight it again in the methods, but Supplementary Figure 1 shows 18 pacbio SMRT cells were used, but the methods say 15. L164: please be more precise were they pacbio CCS or CLR reads? L167: Please provide Supplementary Figure 2 with a better contrast allowing us to see the high and low contact density on the centre of the scaffold squares. L172: ungapped is not a term I would use; instead, I prefer to refer to the assembly as scaffolded or scaffold-level if it is in scaffolds with gaps and contig-level if the scaffolds are split up into contigs for statistics or analyses. In this case, I would only state the total scaffolded length and maybe the amount of N's or gaps. Also, the second sentence would be better combined with the first e.g., "The final assembly had a total length of 2,398,209,015 bp in 477 scaffolds and a scaffold and contig N50 of 64.8 Mb and 23.1 Mb, respectively." L174: What does full-length mean? L175: please reference the dog genome properly with the accession number and reference if available. L176: Please rewrite. There is something not quite right with the bracket and the following remaining sentence. L178: Carnivora_odb10 L182: Please check the manuscript and the supplementary data for consistent spelling of Cooinda (or Cooindah). L184: what does "were full-length by BUSCOMP" mean? Please give more details here on which basis this is determined and what it means that the two other genomes hat a few more. Also, I am not sure if you have to repeat the list with canine assemblies if you have them properly listed in the methods. Again, that's why I prefer to have the methods before the results. Table1: Again "ungapped" sounds not right. Please consider changing it to be clearer. As a general side note, when you want to compare two assemblies of different assembly length, it would be better to compare NG50 instead of N50. I doubt that in this case, with only a 40-50Mb difference, it would change the results much but consider adding NG50 values. Number of gaps is also not very clear, as the gaps can be of different sizes and can be of a determined length or a standard number of N's as a placeholder for a gap of unknown length. L198: "to align Alpine dingo long reads to the Desert dingo assembly" seems not to fit here. Please check the sentence structure and rephrase. L199: "These plots show low variation on the X chromosome" More context is needed. low compared to? Why are the results only so briefly mentioned after multiple lines of "methods". This is an issue I see throughout the results. There are barely any results and mostly method summaries. Figure 2: Explain what the plot shows. I am, in general, not a big fan of these multi-layer circus plots as each individual plot is way too small to show much. However in this case the lower amount of SVs on the X chr is visible enough, but the caption needs more details. L211: Why list a reference for something that is a results of this study? Supplementary Fig. 5: Each chromosome is too small and the resolution too low to see details of the SVs. L217: So why is that important to mention? If there is no further reason I would remove it, it does not add to the story. L226: "In addition, however, we also found …" ïƒ change to "In addition, we found… " or "We also found …" L227: Consider joining the two sentences: "We also found two structural events on Chromosome 26 (SFig. 6) containing mostly short genes…" L227: What does perfectly conserved mean? L228-229: Why not show it? L230-232: This can be more concise and easier to read for example: "The Alpine and Desert dingo both have a single copy pancreatic amylase gene (AMY2B) on Chr 6. However, only the copy in the Desert dingo includes a 6.4kb long LINE." I am not sure why reference 10 is cited twice here in the results. Is this a result already known before? If so, this belongs in the discussion. L233: Again, the whole section is a short methodological summary, and there are absolutely no results. Figure 3: The figure caption needs to be rephrased completely. Not sure how this ended up here, but "NOTE: A and C as well as B and D are similar plots. However, A and B use SNVs while C and D use indels." really does not belong in a proper caption, especially as each plot is listed before stating if it is based on SNVs or indels. Bootstrapping usually does not need to be explained in a figure caption. Instead, it would be more important to mention what type of phylogenetic tree it is and on how many SNVs it is based. There is also no scale on the trees, does that mean these are pure cladograms? For B and D, please explain what an ordination analysis is and change the axis labels to something meaningful. Labeling the x and y axis "Axis 1" and "Axis 2" is absolutely pointless. I am quite surprised that this passed the final ok from all co-authors. L260: please use Desert dingo and not Sandy. L263-265: Not sure why this is important here if it is not discussed later. Figure 4: Again, the figure caption needs a complete rewrite. For example, L281 "dingo Sandy is in this clade" is very unclear and confusing; "In this figure," ïƒ remove!; What are the superscript numbers for? Please remove them, they look like they belong to some footnotes from an earlier manuscript version, which are now missing. Instead, important info such as the type of network, the meaning of the small lines, and the scale are missing. Methylome: L295: I would remove "the" before MethylSeek and a period is missing before UMRs. L297-299: I think here it would be very nice to not just mention that there are other studies but give some examples and comparisons. "These analyses" could either refer to the MethylSeek analyses or the analyses of reference 55, please rephrase to be clearer. Also, it is unclear what previously reported numbers mean, again give more details ("… in line with previously reported numbers of promoters and enhancers in, e.g., humans (promoters xxxx, enhancers xxxx), mouse (xxxx), and rat (xxxx).") L301: what does "we lifted over the former UMRs to the latter genome" Please rephrase, it is very unclear to me what you mean. L302: why was average DNA methylation calculated for UMRs. Should they not be unmethylated by definition? L306: Why have a sentence about that a gene is highly conserved but not perfect and then give the percentage of identity instead of just stating that it is 99.8% identical? I have now mentioned quite a few examples where the manuscript could be much more concise. I cannot list them all but would encourage you to read through the manuscript again and make it more concise. Morphology: My knowledge of morphological analyses is limited, but, despite the unfamiliar terminology, this section of the manuscript is easy to read and focuses in a more concise way on the actual results. My only suggestion would be to label Supplementary figure 9a with the different morphological features mentioned or adding an additional schematic to the supplementary, so non-morphologists can easier follow. L353-354: I would suggest adding the sizes after you mention the individual to avoid repeating dingo and dog brain. For example: "… the dingo brain (75.25cm3) was 20% larger than the dog brain (59.53 cm3) (Figure 5B)." Figure 5: Please add an explanation of what the polygons in 5A represent. Also, consider changing the labels in 5B. I assume LHS and RHS are short for the left-hand side and the right-hand side. This, for me, is usually used to describe positions in unlabeled figures. I would suggest changing it to Cooinda dingo (CD) and domestic dog (DD). Discussion L375: It is not clear why Cooinda should be considered the archetype at the beginning of the discussion. I would place this in the conclusions and base it on the results and the discussion. L394-395: Please rephrase and shorten, e.g.,: "There is a single copy of AMY2B in both dingo genomes; however, they differ by a 6.3 kb retrotransposon insertion present in the Desert dingo." L394-405: I would like to see a more in-depth discussion on the differences between wolf, dingo, and dog. If there is no LINE in the wolf but both in the dingo and the dogs, when did the transposition happen? Could be two independent events in the dog and dingo lineages or one in the ancestral lineage. Are the LINEs in dog and dingo at the same position in the gene region? Could it be the same insertion that was reduced in length in the dog lineage, and what does that mean for the evolution of dogs and dingoes? L431/432: please use Alpine and Desert dingo instead of the individuals' names. L471-473: Not sure if a single sample (Cooinda) is sufficient to come to this conclusion, also how does it compare to the wolf? She could just have been a dingo with an exceptionally large brain. I think a more in-depth discussion is needed. Methods: Overall, the methods need to be more concise but at the same time clear and complete. L530-531: Why is solving the puzzle-box experiment important? Does that not suggest an exceptionally intelligent dingo if she was the only one, and could that not potentially explain the large brain size? How does brain size and intelligence or the potential to solve the puzzle-box correlate? L532: her brothers L533: What is the importance of the ginger color? As it is stated here, it is a bit out of context. Why is it important? L535-542: Why is this detailed report on her appearance of importance? I am often missing logical connections in the manuscript. L541: I would usually not expect to read such a statement with an emotional connotation in a scientific manuscript. L545ff: When were the samples taken? What type of samples were taken? How were they preserved? As it is stated that fresh blood was used, I assume Cooinda was still alive at that point. Are there any sampling and ethics permits to be mentioned? L552-56. This whole section about the pulse-field electrophoresis can be much shorter without losing any information, e.g., "Molecular integrity was assessed by pulse-field gel-electrophoresis using the PippinPulse (Sage Science) with a 0.75% KBB gel, Invitrogen 1kb Extension DNA ladder (cat ….) and 150 ng of DNA on the 9hr 10-48kb (80V) program." L556: What libraries? You have not explained how the libraries were prepared. CLR or CCS? L557: Which Sequel platform was used? Sequel I, II or, IIe? L558: remove the hours of movies, that does not matter unless you used a custom sequencing program. L559: AS 15 SMRT cells were used, I assume the sequencing was performed on the Sequel I. L561: I usually avoid starting a section or paragraph with "for". Please consider rephrasing as you start most paragraphs that way. L564: 119 ng of library, especially for long DNA-molecules, seems very low for a decent ONT run. I have mostly used the MinION, and I would usually only load a library with so little DNA if I only needed a few reads. I am just curious how well that worked on the larger PromethION flow cell. L573: In some sections, this manuscript reads like an early draft that was accidentally submitted. "User Guide, manual part number CG00043 Rev B." Please rephrase. L575: For me personally, it does not matter where Qubit measurements were taken, but if you include that info, please try not to repeat it as you did in L578. L576-577: Please shorten the two sentences about sequencing to one, e.g., "Sequencing was performed in 150bp paired-end sequencing mode on a single lane on the Illumina HiSeq X Ten platform with a version 2 patterned flowcell." L581-582: Does reference 8 use the same protocol version? If so, I would remove the brackets. If not, Is there no version number of the protocol available? L594-601: Why is this a mixture of insufficiently described methods and results? Please give additional information about trimming and assembly using canu. Why mention the number of sequences, bubbles, and unassembled sequences in the methods? L597: How were the reads aligned to the assembly? I have not used Arrow but if the pipeline uses mapping tools, please mention them. L598-601: These are results and should not be part of the methods. L614: what does finishing mean? In the literature, it is more common to write "manually curation of" or scaffolds were "manually curated" or "manually edited". L615ff: Again, these are results. I would not place them in the methods. L621: Was gap-closing performed only once? Were pacbio and ONT reads combined on one iteration of gap-closing? Maybe PBJelly suggests only using it once, but in my experience, gap-closing can be performed iteratively to further improve the contiguity. L622-623: Again, results in the methods. L634: Why is it important when the chromosome mapping was completed? You did not specify when the sequencing was performed or when the samples were taken. L635: Please add accession number and, if available the reference. L644: Circos is a tool for plotting data in a circular plot. How were the SNV, and indels identified? L647: X chromosome L652: I usually use GeMoMo for homology-based gene prediction. I would like to see a short description of the method rather than just linking to a previous publication. L658: "processes that produce differences" is not very precise, please give some more info here. I would usually remove Indels from phylogenetic datasets due to the uncertainty of their mutational history. How were they coded and how were they analysed? L659: What is WA distance? Reference? L660: The Glazko et al reference is quite out of context here. Better phylogenetic properties than what? Why use distance-based phylogeny? How many SNVs were used? How ere they filtered L662: Maximum parsimony is not frequently used anymore for phylogenetic reconstruction. Why not use a Maximum-likelihood or Bayesian approach? L664: It is mentioned that the wolf should be the outgroup, but the dataset itself is not mentioned in the methods. List all samples that were included in the phylogeny. If the dingo is assumed to be an intermediate between wolf and dog why did you not use a different canine as outgroup to avoid bias? L665: include version and the URL of the tool if there is no paper to cite. I cannot judge the methods for Methylation and Morphology, as this is not my expertise, but these method sections read very well and seem clear to me. Availability of supporting data: There are quite some broken sentences and misplaced periods in this section. Please check the text again and make sure that the links to your datasets are functioning. Overall, the presented data are interesting and a valuable resource, but the manuscript itself needs some major improvements to make the interesting results available to the reader in an easier-to-follow and more understandable form. It is obvious that multiple authors with different expertise worked on different sections of the manuscript. The challenge during the revision is to bring it together into a concise and easy-to-read manuscript with a consistent writing style. I hope that my comments, questions, and suggestions can help in that process. Please take my writing suggestions as such, feel free to adjust and change it in a different way as long as the result is more reader-friendly and more concise.

Reviewer2-Jack Tseng

My evaluation of the manuscript was restricted to the geometric morphometrics (GM) section. The authors seem to have followed a standard GM procedure in their analysis of cranial shape differences among dingo skull samples. My only suggestion is that additional detail be provided in the landmark data collection for the GM analyses: -Reference 58 was cited as the source of the landmarks used in this study, but no other details are provided. A list of landmarks that forms the basis of the geometric morphometric analyses should be presented in order for the reader to fully interpret the PCA plots.

Reviewer1-Andreas Chavez

The article "The Australasian dingo archetype: De novo chromosome-length genome assembly, DNA methylome, and cranial morphology" is well written and interesting study examining the evolutionary relationships between the focal species, the dingo, and related canids (both domestic and wild). This study uses an impressive amount of state-of-the-art genomic data and resources to produce a (chromosome-length) de novo assembly of the dingo genome. The approach for assembling the genome are all adequate. The comparisons of chromosomal structural variation and methylation patterns with another dingo ecotype and other canids show interesting patterns of divergence that are potentially important regions of adaptive differences. I have two major concerns and some minor concerns for this paper. Major concerns: I believe this paper would be stronger if it contained analytical methods that addressed admixture between dingos and domestic dogs more explicitly. The authors state that admixture between dingos and domestic dogs (Line 453) is one of a few hypotheses that may explain phenotypic differences between the two dingo ecotypes. To evaluate this hypothesis with the genomic data, they rely primarily on phylogenetic analyses to explore the evolutionary relationships between the dingo, wolf, and domestic dog lineages. They show that the dingo lineages are outside of the domestic dog clade and that wolves are outside of the dog/dingo clade. Although it is probably true that dingos are a unique evolutionary lineage, phylogenetic analyses are not the strongest tool for assessing admixture and the contribution of genomic variation from different ancestral source populations. I would recommend using methods that would test admixture hypothesis more explicitly. D-statistic tests (ABBA BABA test) and related tests would seem appropriate for this kind of data and sampling scheme. I also have concerns about the interpretations of brain size differences between dogs, dingos, and wolves. Although I am intrigued by the idea that domestication may have driven reductions in brain size and shape variation, I find it hard to not consider natural selection pressures in the case of dingos and wolves. The best scenario for testing the domestication-driven hypothesis would be if dogs, dingos, and wolves evolved in a common environment and domestication practices were the most notable differences between them. However, given that wolves and dingos in Australia evolved with different prey and habitats on different continents, it seems hard to me to not consider environmental adaptations as another important factor in the evolution of brain-size and shape variation. Minor concerns: Introduction: It took me awhile reading deeper into the manuscript to understand what was meant by the name Cooinda. For awhile, I thought it was the name for a dingo subspecies or ecotype. I would suggest including a brief section in the introduction stating that the genomic and morphological data in this study is based off of a single individual named Cooinda and that there are questions about it's ancestry and placement as one of the dingo ecotypes. Line 172: ")," should be replaced with ")." Line 371: I don't think it is necessary to say "The passing of Cooinda the dingo" Line 463: more is needed to finish the point of "will illuminate" Line 494-497: The role of venomous animals as barriers to gene flow is conceptually not clear and is not supported by the citations from what I can readily tell. Line 540: dewclaws? Line 541: "Regrettably" isn't necessary to include

Reviewer3-: Borbala Mifsud

Gigascience - Cell type-specific interpretation of noncoding variants using deep learning-based methods Sindeeva et al. have developed DeepCT, a convolutional neural network-based model that predicts sequence and cell type-specific epigenetic profiles from available epigenetic data. The novelty of the approach is that the model can learn unmeasured epigenetic profiles in a given cell type, if there is another cell type that has the target feature measured and shares one or more other epigenetic data types with the cell type it aims to predict in. The authors demonstrated that the framework works well and that the model learns both sequence context and cell type-specificity and used the model to predict which de novo variants, identified in the Simon Simplex Collection, have the highest effect in any of the cell types they studied. Focusing on one variant with high predicted effect in glial cells they suggested a mechanism, whereby the variant in a putative enhancer element within the SMG6 gene reduces FOS binding, which might affect SMG6 expression in these cells. I have a few minor comments to clarify the applicability of this model. Minor comments: 1. In Figure 2D the authors showed that adult heart and fetal heart cell state representations cluster together even though they did not share the measured epigenetic features. This is an interesting observation, however one of them had ATAC-seq data while the other had DNase-seq data which are highly correlated. It would be good to know how much this can be generalized to other cases. What is the level of correlation between two epigenetic features that is required for correct clustering of the cell states between two cell types that do not share epigenetic features? 2. In both Figure 2C and in Supplementary Figure 1, the 2D visualization of the cell state representations, show that some cell types cluster well together while others do not cluster at all. Even those that cluster well, like "Digestive", "Kidney" or "Muscle" cells have many cell types that do not cluster with the others. Apart from biological differences, could this be also reflective of cell types with lower quality epigenetic tracks? How much does the quality of the tracks effect the model? 3. Figure 3E shows that there are some points where the accuracy of the model is much higher when leaving out certain epigenetic tracks from the training of the model. Is that also related to quality of those data or is there a specific epigenetic feature where the model consistently shows higher accuracy when the feature is left out? 4. The authors used 1000bp for representation of the sequence, but the target sequence that is checked for overlap of the epigenetic features is only 200bp. Does the model learn from the additional 800bp? 5. For the cell state tail the chosen emb_length was 32. Based on Supplementary Figure 1, I assume this is due to the number of cell type groups expected, but it would be good to include the rationale in the methods. 6. For the GO term enrichment what background was used? I would expect that the nearest genes of all de novo variants found in autism cases would show enrichment for similar GO terms. 7. Pg.11 last line should be "FOS transcription factor binding" instead of "grinding".

Reviewer2-Yuwen Liu

The manuscript entitled "Cell type-specific interpretation of noncoding variants using deep learningbased methods" interpreted the non-coding genomic variants by integrating the single-cell epigenetic profiles with the convolution neural network. The author found the CNN can capture the cell typespecific properties and generate a biologically meaningful cell state representation by embedding the cell to the latent space. In general, the architecture of the convolution neural network is novel, and, to a certain extent, the model may be helpful for improving our understanding of genomic non-coding variant effects at single-cell level. Major comments: 1. In Figure1C the author intended to quantify how often unmeasured epigenetic marks can be inferred from available profiles. Although, in fact, the modification of the epigenetic marks is correlated and sometimes colocated in the genome (Ernst and Kellis 2015). However, the connected graph is not a piece of strong and solid evidence or data for quantify the predictive ability of the epigenetic marks. They should provide other compelling evidences or undertake more analysis. 2. The author used an empirical p-value threshold to detect the peak position along the genome. The definition of the peaks for epigenetic mark is crucial for the whole study. At least they should plot the distribution of p-value and explain why they choose the empirical threshold of p-value as 4.4 in detail. Furthermore, the false positive outcome of the test should be corrected. 3. Some epigenetic marks present broad modified regions of the genome, the 150 bp DNA sequence may not contain all the sequence determinants for that broad peak. That is may the prediction performance is poor for most of epigenetic marks. 4. In Figure3D and Supplementary Figure 2, the majority of epigenetic marks presented very poor prediction performance. The author should discuss the potential biological reasons that lead to this result and perform some analyses to preclude these confounding factors. 5.The author should scrutinize their data because they also use some epigenetic profiles form heterogeneous tissues which are composed of different cell types. And these heterogeneous profiles may weaken the predictive power of the convolution neural network model and impair the interpretability of the model. 6. The authors only used SSC data to showcase their predictive power in pinpointing potential causal non-coding variants of ASD. I suggest use GWAS data from a wide varieties of complex traits and diseases to generate a more thorough evaluation of the specificity of their prediction. Furthermore, the authors used prediction leveraging signals from 794 cell types in predicting non-coding causal variants for ASD. Including a large number of ASD-irrelevant cell types would likely bring strong noise and make the results hard to interpret. I suggest the authors mask the epigenetic marks of ASD-relevant cell types (treating these cells as if they do not have available epigenetic data), and then use epigenetic marks from other cell types to predict non-coding variants with high impact on epigenetic marks in ASDrelevant cells. Then use this new prediction to rerun Fig 4A and 4B. Achieving good performance with this new analysis would better demonstrate the core advantage of their new model, i.e., predicting celltype specific non-coding effects of cell using epigenetic information from other cell types. Minor comments: 1. The author defined peaks as 150 bp genomic intervals, however, they use 200 bp DNA sequence as the center when preparing the data for the CNN input. 2. The resolution of the figure should be greatly improved.

Reviewer1-: Fangfang Yan

In this manuscript, Sindeeva and colleagues describe a novel neural network-based algorithm, DeepCT, to cluster epigenetically similar cell types and infer unmeasured epigenetic features, which then can be used to interpret non-coding variants. The manuscript is well structured and well written, it is potentially interesting to a broad readership. Yet, the algorithm itself in the manuscript lacks rigor and thoroughness. Major points: 1. Lack of comparison with competing methods 2. As the authors state themselves in the results and discussion, the performance of DeepCT among some features is very low, such as H3K9 and H4K20 monomethylation. Could authors add more discussion and explanations of this almost zero average precision? 3. The authors said "statistically higher" or "outperforms" in a lot of statements but no statical test results. For example, on page 8, the authors write: "This analysis confirmed that average cosine similarity for embeddings representing cell types from the same tissue was significantly higher than for embeddings of randomly selected cell types". On page 9, "we note that this baseline has performance metrics substantially higher than expected in random (baseline AP=0.417)." 4. On page 8, the authors write "we show co-localization of muscle cells, as well as co-localization of digestive cells (Fig. 2C)". However, Figure 2C looks not quite convincing. Minor points: 1. Providing high-resolution vector-friendly figures will help a lot. I can barely see the content of the figure in the current version. 2. A jupyter notebook tutorial on the Github repo would be helpful for users to apply DeepCT quickly.

Reviewer2-Fadi G Alnaji

In this work, Sotcheff et al provide a comprehensive and nicely-written report about using the algorithm Virus Recombination Mapper (ViReMa) to identify and characterize different kinds of recombination events in different viruses. ViReMA was first reported - by the same group - in a separate paper (Routh et al, NAR, 2013) as a python-based algorithm that, by accounting for the high-diversity nature of virus populations, can efficiently detect a wide range of virus recombination junctions within virus-derived Next Generations Sequencing (NGS) datasets. In this paper, the authors described a couple of important updates on the original algorithm that enables ViReMa to cope with the new technological advances in NGS, including the read length and the significant increase in NGS library size and NGS-based experiments. Notably, the authors implemented a powerful validation approach by challenging the algorithm with a different type of NGS-based data containing various types of junctions from different viruses to highlight the contextual computational and biological connotations. Overall, the paper used a robust analysis method and sufficient controls to clearly demonstrate the capacity of ViReMa to detect different types of recombinant molecules in different NGS datasets and viruses with high sensitivity and specificity. I only have very few minor comments.Minor comments1) Since Fig 2E is showing the gradual effect of the permissibility imposed by the error-density values, transforming the tables into figures e.g. bar or scatter plots can render the effect more observable visually.2) At lines 500-501, the author found that the majority of reads mapped directly to the virus genome. Looking at the aligned read number, this dataset seems fairly large, I was wondering if using the newly added function --Chunk can come into play at this scenario to speed up the analysis? If it is the case, then maybe mentioning it would be valuable.3) At line 478, the authors stated: "The 'Reads' columns describe the number of reads at each particular nucleotide position", is this the average read number?4) Typos at line 206 "red", and at line 397 "(NL4-3)"

Reviewer1-Diogo Pratas

This article describes a pipeline (coded in Python) to detect and analyze recombination events of viral genomes using short-read FASTQ data. The paper presents some level of work accomplished by the authors. Usually, these types of articles hide numerous hours of coding and experimentation. Moreover, the authors present actual accomplishments that typically are unique architectural designs and important alternative ways to the area, including several results. However, many points require attention, namely:1) This pipeline expects exactly a specific virus. Hence, it uses a specific reference. However, this reference might not be the most representative because of the recombination events. Although it may be appropriate for smaller recombination events, detecting large-scale recombinations may face substantial difficulties. Moreover, since it is not prepared to deal with more significant variations (without de-novo support), it is exclusively for targeted support. Therefore, the article could be more descriptive about this specificity.2) The article states that the improvement is also inspired with the read length increase that NGS is bringing. Also, the reported depth coverages are very high. So, why not use de-novo assembly? For example, the de-novo assembly can be used to create scaffolds that can generate a reference sequence to be used after by the aligners. Please, comment on this.3) About the use of artificial poly-(A)tales to allow the mapper to align the reads, what happens when the read size is smaller than the k-mer hash of the aligner? Usually, repetitive A-sequence content appears in almost all samples because they have lower entropy and a higher probability of being generated. Wouldn't this create ambiguity, especially when there are very high-depth coverages? Please, comment on this matter.4) What is the minimum read size allowed to be considered a valid read for downstream analyses? Are the reads collapsed (in the case of Paired-ends) or considered split? Although less probable, the trimming is fundamental for excluding "events" generated at the tips of the reads that very rarely overlap, depending on the nucleotide distribution.5) Are the reads clipped above a particular depth coverage? This feature is especially critical in repetitive viral content, such as hairpins or poly- (A)tales - removing mountains that become the most significant factor in sequence depth coverage.6) Have some of these viruses been enriched for targeted capture? Please, provide this information in the manuscript. In some parts of the article, the coverage depth is very high: 300'000 - is this 300000? The simulated data used this coverage which may not be entirely similar to reality. Also, allowing lower depth coverage helps to understand how the pipeline behaves. Moreover, some aligners may have problems in older versions with these depth values.7) It was unclear which types of duplications were flagged and if the pipeline covers them.8) How does the pipeline deal with contaminants?9) This article states that the pipeline works for viral sequences. However, the tests used do not include large genomes. What about larger genomes? Some larger genomes contain repetitive content that provides additional reconstruction challenges. Therefore, the benchmark could have an example of this nature.10) While looking for recombination events, specially fusions with the host, what are the differences between sequenced viral integrations and fusion events at the analysis level? How do we distinguish both using this pipeline? Please, comment on this.11) The authors state that the pipeline provides accurate results. Regarding the calculation of accuracy values, several good practices and recommended by many experts in the field:a)https://www.sciencedirect.com/science/article/pii/S1386653220304339b)https://www.sciencedi rect.com/science/article/pii/S138665322100079212) Augmentation of existing pipelines in the area could guide the reader to other solutions and sometimes complementary. See, for example:a) ASPIRE: https://bmcgenomics.biomedcentral.com/articles/10.1186/s12864-022-08649-8b) TRACESPipe: https://academic.oup.com/gigascience/article/9/8/giaa086/5894824c) V-pipe: https://academic.oup.com/bioinformatics/article/37/12/1673/610481613) Line 113: "in range a of plant" - please correct;14) Line 120-121: Please, rephrase.15) There are several acronyms; perhaps an abbreviation list would improve the reading of the article.16) Line 394: ART is defined as "antiretroviral treated," but this acronym overlaps the ART simulator. Perhaps, in this case, adding another letter or changing it would remove the ambiguity.17) Line 753-754: Reference 27 is missing at least the title, journal, and year.18) Please, consider to add ViReMa to Bioconda.19) I've tried to clone the repository from sourceforge, and it came out empty. I had to download the package manually. I faced some problems, perhaps because it was not easy to follow. Possibly, users may face the same difficulties, which may be an obstacle to using the software. Please, consider having an elementary example for running ViReMa (already including some tiny read sample and reference along with the code and command description - including how to run the GUI). Please, consider using Github in the following times.

Reviewer2-Hadrien GourlÃ

Thank you for a great piece of software and great article.A few minor points:l166-168: the clause about structural variant is unclear to me, and perhaps to the reader. Please consider rephrasing.l209-210: I understand that the number of mapped reads is unadequate for abundance estimation of ONT data, but k-mers should not suffer from the same problem, shouldn't they? the number of k-mers matching to a genomic region (or a genome) will scale appropriately with read length. I have therefore a hard time understanding why k-mers are presented as problematic in the first sentence of the Abundance Estimation paragraph.l379: What do you mean by "pronounced". Please consider rephrasing.l438: geomes -> genomesfigure 2: panel 2 should have the same theme as the other subplots of the figuresoftware comments:- I'd like to be able the examples present in the documentation out of the box: please add a link and instructions on how to download and unpack the zymo community- Speaking of the zymo community, why do Campylobacter and S. cerevisiae have a different path than the other genomes in the examples?- If you plan to not update the pre-trained error models to a more recent version of scipy, please pin scipy 0.22.1 to the bioconda recipe, so that users can use pre-trained models out-of-the-box- Please make a new release of the software including pull request !67- In a future version of Nanosim, I urge you to consider gathering all scripts into subcommands (i.e. nanosim simulate [-- params] instead of simulate.py [--params]. I realise this a big breaking change but it is good practise, and avoids polluting a user's PATH with many scripts. This change is in my opinion not required for the paper to be published, but something I'd like you to consider for a future release

Reviewer1-Andre Rodrigues-Soares

This manuscript is of very good quality - as such, my review is quite limited. I would like to congratulate the authors for the development of the software and its comprehensive and extensive benchmarking.On l. 64 - Given the range of samples that can currently be sequenced using Nanopore sequencing and the recent focus on short reads as opposed to the previous highlight given to long reads, this statement is out-of-date. I would recommend describing instead the current range of sizes that can be sequenced via Nanopore.After reading the manuscript, I am however, left to wonder how the authors would approach the different error rates namely of different types of flowcell (R9 vs R10). It can be assumed that error rates of R9 sequences might indeed average 10% as stated in the manuscript, but with the advent of R10 flowcells (more recently R10.4.1) and respective updated chemistries, the error rates have decreased significantly. One specific error rate of 10% as stated in the manuscript can't be assumed for the sequencing technology as a whole anymore. While I don't think this should be central to the development of the tool, I think this should be addressed in the manuscript in some way.I would also have liked to see distributions of PHRED quality scores in the simulated reads in the analyses conducted in the manuscript. Although the assembly and genome recovery statistics namely in Figure 4 indicate these should have the expected distributions, I would have liked to understand how quality scores are distributed in the generated reads. If the two issues above are addressed in the manuscript, I will be happy to recommend its publication.I have no further reviews to add as the manuscript covers all other factors I would think could be worrying regarding a tool simulating Nanopore metagenomic reads.

Reviewer2-Sveinung Gundersen

The paper describes the FAIR Data Station, which is a lightweight application written in Java that facilitates FAIR-by-design by allowing the collection of structured metadata from the first phase of a project. To this end, the authors have applied and extended the ISA metadata framework to form a core data structure wherein attributes from a library of 40 frequently used minimal information checklists can be placed. The FAIR Data Station contains tools for generating and validating Excel metadata files, as well as conversion to RDF format as well as to a European Nucleotide Archive(ENA) compatible XML metadata file for submission.General comments:The FAIR Data Station (FAIR-DS) seems to be a useful application to help life science researchers to collect and structure metadata according to the FAIR principles. The software is based on core community standards, ontologies and checklists. As for deposition databases, the software currently seems to only integrate with ENA, which, on the other hand, is a central deposition database.The three main contributions of FAIR-DS is to my mind A) the metadata schema that has been carefully constructed by the authors, B) the validation functionality of metadata against said schema, and C) functionality for conversion of validated metadata into RDF and deposition formats There are, however, some architectural choices and technical limitations in the implementation that I have issues with and which makes me uncertain whether the software shows enough "innovation in the approach, implementation, or have added benefits", as mentioned in the "Instructions for Authors"(https://academic.oup.com/gigascience/pages/technical_note). I would therefore invite the authors to address the following issues:1. The authors state that "the FAIR-DS uses an extended version of the original three-tier Investigation, Study, Assay (ISA) metadata framework [https://isa-tools.org]". This leads the reader to think that the software applies the full ISA Abstract Model (https://isa-specs.readthedocs.io/en/latest/isamodel.html), which is not correct. Only the top level objects and a few attributes are retained. It is also not clear why the authors have found it necessary to add additional, custom object types, such as "Observation unit", explained as "the "object" from which the measurements are taken". The ISA model includes an attribute "source material" which seems to overlap. The authors have also added "sample" as a top-level object, even though there is already a "sample" attribute in the ISA model. It is unclear to me what is improved by adding new object types and whether any such improvements will outweigh the obvious drawbacks that comes with not following a community standard for the metadata schema.2. The FAIR-DS makes use of Excel files as an intermediate format for collection of user metadata. While the feature set of Excel and its familiarity for most users are good arguments its adoption, I miss a discussion on the fact that a commercial product is included in the core architecture of the system. FAIR principle I1 promote that: "(Meta)data use a formal, accessible, shared, and broadly applicable language for knowledge representation". As Excel is only an intermediate metadata format, while RDF is used for the final output, the FAIR-DS does not directly break principle I1, however I think the choice of a commercial file format is not following the "spirit" of FAIR. I see no reason why CSV could not be included as an alternative to Excel and that the authors could recommend an Open Source application as alternative for users that wish their entire software suite to remain in the Open Source domain.3. The metadata schema is not represented in a standard schema format, such as JSON Schema, Frictionless table schema, or similar. Using a shared format for representing the metadata schema makes it possible to make use of general validation libraries (such as the ELIXIR Biovalidator: https://doi.org/10.1093/bioinformatics/btac195). Shared schema formats also allows for reuse of the schema in other contexts/software. In FAIR-DS, the metadata schema seems to be primarily represented in an implicit way in the Java source code that generates the Excel files as a secondary representation of the schema. Even though the FAIR principles might not directly include a recommendation to share of the metadata schema in a FAIR way, one can argue that this falls under R1.3: "(Meta)data meet domain-relevant community standards". It would in any case be in "the spirit of FAIR".4. As a consequence of issue 3, the validation functionality is also specified implicitly in the Java source code and does not seem to reuse much external validation functionality. I particularly miss validation of ontology terms against the relevant ontologies, as well as more stringent validation of PMIDs, DOIs etc, preferable using CURIEs instead of URLs. All of these data types only seem to be validated as general strings, which is of limited use. Users might for instance introduce spelling variants for ontology term labels without this being detected by the validator.5. Due to the hard-coded nature of the metadata schema, the validator and the conversion functionality, I suspect the authors might not have designed the system flexibly enough to allow for easy updates based on updates in the external dependencies, i.e. the minimal information checklists, ontologies, or deposition schemas. For instance, EMBL-EBI, who are hosting ENA, are moving towards requiring the submission of sample data/metadata to BioSamples, prior to submitting the metadata to ENA, which might have consequences for the checklist requirements. Also, ontologies in particular are known to be updated regularly.6. I am not convinced that the authors have done a careful enough search of the literature to list relevant software solutions for comparison. For instance, the FAIRDOM Seek solution (https://doi.org/10.1186/s12918-015-0174-y) is not cited directly, although the functionality seems to be highly overlapping.7. The manuscript would benefit from careful proofreading of the language and grammar.When addressing these issues, I would urge the authors to better demonstrate "innovation in the approach, implementation, or ... added benefits",

Reviewer1-Dominique Batista: An overall a strong paper that creates a new bridge between the ISA model and the FAIR principles.A few points should be addressed:- page 2: "As one Investigation can have several research lines, each Study layer has a unique identifier ...": how do you generate these identifiers and control their uniqueness, persistency and stability? Are these identifiers resolvable ? "As an extension to the original three-tier ISA-model in between Study and Assay two additional layers of information were added Observation unit and Sample": would you clarify what problems were addressed ? More generally speaking, does the FAIR-DS integrate with existing implementation of the ISA model ? Did you consider a conversion and submission to external systems such as the ones mentioned in the conclusion ? The text for figure 1 is good, but the corresponding text in the core of the document is hard to read and understand. "Model specific attributes are optionally selected by the user": Does this mean users can add extra fields on top of the provided packages or that they have to select fields within the given package ?-page 3: "In addition, we included regular expressions obtained from the ENA checklist, such as "(0|((0Ë™)|([1-9][0-9]Ë™?))[0-9]*)([Ee][+-]?[0-9]+)? (g|mL|mg|ng)" for sample volume or weight for DNA extraction": good point. Is their a mechanism for users to add new regex ?

This work has been published in GigaByte Journal under a CC-BY 4.0 license (https://doi.org/10.46471/gigabyte.78), and has published the reviews under the same license. These are as follows.

Reviewer 1. Takeshi Takeuchi

Is there sufficient data validation and statistical analyses of data quality?

Scaffolding with the Chicago and Hi-C libraries did not significantly improve the assembly. In general, Hi-C scaffolding can produce a chromosome-scale assembly. I would suggest that the authors describe the quality of the Chicago and Hi-C sequence data. For example, the mapping rates of the Chicago/Hi-C reads to the assembly should be informative.

**Reviewer 2. Yang Zhou **

This is a fascinating study on the assembly of the first deep-sea scleractinian coral, Lophelia pertusa. The manuscript is well-written and easy to follow. I have gone through your manuscript and would like you to address the following concerns/comments before publication.

Line 47: 1.2 454 pyrosequencing reads means 1.2Gb 454 pyrosequencing reads? Line 51-52: Please add some references. Line 72: As far as I know, the DNA extraction process of stony corals is affected by calcium carbonate skeletons. How did you deal with this problem during the DNA extraction process? References: Please double-check the references for errors. Italics for species names, capitalization of journal titles, and so on.

This work has been published in GigaByte Journal under a CC-BY 4.0 license (https://doi.org/10.46471/gigabyte.79), and has published the reviews under the same license. These are as follows.

**Reviewer 1. Xuanmin Guang **

Han et al. had carried out genome assembly of Aplectana chamaeleonis, analysised the genome’s repeat content and annotated the genome. They descripted the geneset’s function and done a PSMC analysis. The genome is a key source for research, but there are so many mistakes in the manuscript, I suggest the author to revies the manuscript carefully and the grama and content should be re-organized. Some suggestions have been listed below:

1. In the context part, the first two sentence lacks continuity in logic, please change them.
2. The author didn’t mention which sequence platform they had used in the context, I think this should be added.
3. The average sequence length in the table is 496kbp, but the author it as 496Mbp , this is a mistake.
4. In table 1, why there aren’t any gaps in the scaffold genome?
5. The author said that “This suggests that the significant expansion of repeating elements is an important manifestation of species differences”. Its unreasonable to get this conclusion only based your genome repeat analysis.
6. In the text they claim that 12887 function gene had annotated, I want to know how much gene they have annotated? Please add this in the manuscript.
7. Too many decimal places have been used in the Table2.

Re-review: The author revised the paper as I concerned in the report and the paper could be accepted now.

Reviewer 2. Jianbin Wang

In this manuscript, Hou et al. present a genome assembly for Aplectana chamaeleonis, a parasitic nematode that infects amphibians. They report a genome of ~1 Gb, most of which is composed of repetitive elements. This genome draft is significant as it is the first assembled for this or any Cosmocercidae species. It may provide insights into the evolution of the nematodes – if it is thoroughly compared to other nematode genomes. It may also allow for better species identification than previous morphological methods. While the conclusions on genome size and composition described in the paper appear sound, there are many questions that go unanswered. The reasoning behind why this research was undertaken is not clear. What is the ecological or agricultural and economic impact of the species? How would the genome provide a better understanding of this species? More specific information is also needed to better understand the genome. How many chromosomes does this species have? Is there any cytology to help answer this question? Any notion of sex chromosome vs. autosome? This genome is much bigger than most of the assembled parasitic nematodes. The author did not make any efforts to explain what might contribute to this. Could the big size due to contamination in the samples used? Judging from the images, it does not look very convincing to me how clean the sample was for the genomic DNA extraction. Overall, there is a lack of in-depth data analysis and comparison between this genome and many other available nematode genomes. About the overall presentation and organization of the manuscript, the context is often lacking from results. How do these results compare to related species? How does figure 4/the demographic history fit in to this story? A round of general proofreading needs to be done for grammar, punctuation, capitalization, italics, etc – see below for some specific examples. In the Abstract, the repeat content in the Ascaris genome is 72.45%, and the total length is more than 742 Mb. The math does not add up (1.1 Gb x 72.45% = 797 Mb). Or do you mean the Aplectana genome? Should say total length of repeats. Why is this “Ascaris” genome? Ascaris is a parasite that infects pigs and human. Some sentences need addressing/clarification: Page 1. “and their diversity is also very high, many of which are above the national second-level protected animals” – what is the significance of this/how are these ideas related? Page 2. “Through the characteristics of the genome sequence, it shows that the genome is a highly continuous genome” – need to be more specific with metric and data. Page 4. “In addition, the enrichment of A. chamaeleonis genes in all metabolic pathways was found in twelve metabolic pathways.” – not sure what you are trying to say about the all or 12 pathways. Figure 1. - Images need scalebars. In A, what is the mat of material? For A, crop out area around the worm and enlarge the worm image. In B the worm is dark/shows little contrast or detail. In C, label which image is the head and which is the tail (or specify left vs. right in the legend text). The images in B and C look like they were taken using a cell phone pointed at a computer monitor – are there higher quality images? Table 1. – Why is the data in all four columns the exact same? What is the difference between each column? This appear to be a mistake when preparing the table. Very sloppy and unfortunate! Table 2 – Significant figures on the %s?. Is the “other” category needed (same for Fig2C)? Table 3 – Check text spacing (e.g. % in genome). Figure 3 – Recommend to redo the spacing of figures, increase size of text in each part of this figure. Need to refer to parts of figures in the body/text (Fig 3a vs. 3b vs. 3c). Can 3b be sorted from most number of genes to least? Figure 4 is not referenced in the body text. Consider merging Fig 4 with Fig 3. Figure 4 is lacking a description in the legend – what are the grey lines, definition of LGM? The x-axis scale and orientation are unintuitive – is the present on the left and the past on the right? Past should be on the left. Methods Genomic DNA was purification for Long-reads libraries preparation – should say purified What is the meaning of “The generation we used was 0.17” – what generation is this? and “the mutation rate was 9×10-9” needs units. The sentence “we used the pairwise sequentially Markovian coalescent (PSMC) model to estimate the effective population size of A. chamaeleonis within last million years.” should be moved to the section immediately after its current location.

Re-review: Overall, the writing has been improved in several places and is somewhat clearer than in the previous draft. These changes are mostly related to the minor concerns raised. However, many questions related to the broader impact of this research and how the new genome compares to other nematode species remain unanswered. The following comments were largely ignored. 1. The reasoning behind why this research was undertaken is not clear. 2. What is the ecological or agricultural and economic impact of the species? How would the genome provide a better understanding of this species? 3. More specific information is also needed to better understand the genome. How many chromosomes does this species have? Is there any cytology to help answer this question? Any notion of sex chromosome vs. autosome? 4. This genome is much bigger than most of the assembled parasitic nematodes. The author did not make an effort to explain what might contribute to this. 5. Overall, there is a lack of in-depth data analysis and comparison between this genome and many other available nematode genomes. How do these results compare to related species? 6. About the overall presentation and organization of the manuscript, the context is often lacking from results. Another round of general proofreading needs to be done for grammar, punctuation, capitalization, italics, etc. – see below for additional specific examples. The authors, not the reviewers, need to make a concerted effort to read and proofread their own manuscript.

In addition to the big picture points raised above, several other issues that were either brought up last time or are new and need to be addressed: 1. Not sure Table 1 is present the right way. The columns and rows should be reversed, I think. If so, there will be only one column - do you still need a table? 2. “Through the characteristics of the genome sequence, it shows that the genome is a highly continuous genome.” Unclear. The authors mentioned that they have fixed this in their response to the reviewers, but no change was seen in the updated manuscript. 3. “The generation we used was 0.17, and the mutation rate was 9×10-9 [8].” These numbers need units after them. Again, this was addressed in the response but not written out or clarified in the revised text. 4. “In addition, the enrichment of A. chamaeleonis genes in all metabolic pathways was found in twelve metabolic pathways.” Not sure what the authors were trying to say about the all or 12 pathways. Still confusing. 5. Photographs of the worms are still lacking scale bars. 6. Make sure that all genus and species names are italicized (in body text and in Fig.3). 7. Make section heading format is consistent (check capitalization). 8. “The results showed that 91 % of the sequences were compared to Arthropoda (1898/2088) and 7 % were compared to Arthropoda (122/2088).” Both of these say Arthropoda - is that a mistake? Also "compared to" is not the correct word, maybe "similar to"? 9. LGM acronym is defined after the second use of "last glacial period", should appear after the first use. Also, LGM stands for last glacial maximum, not period. This should be corrected.

Reviewer1: Lim Heo

In this manuscript, authors described a new platform, 3D-Beacons, which is an interface for accessing multiple sources of computational protein models (e.g., AlphaFold DB, SWISS-MODEL) and experimentally determined structures. As the number of protein sequences increases much faster than the growth of experimental structure database (e.g., PDB), computational protein structure models are great alternatives for proteins that do not have experimentally determined structures. Nowadays, many accurate protein models have become available thanks to the progress in template-based modeling techniques for decades and recent advances in de novo protein structure prediction methods using machine-learning approaches. However, those model sources were scattered at their own databases, so there has been difficulties in accessing these models. Thus, in my opinion, the development of a new database or platform, 3D-Beacons, for accessing various computational models is a great movement in the structural biology field. The manuscript well described the description of the platform and some technical details. I have a few minor comments on this work.1. I recently noticed that RCSB PDB also made it possible to search computational protein models by extending its web interface. The database included ~1 million models from AlphaFold DB and ~1,100 models from ModelArchive, which are main sources of this work as well and are maintained by some of the authors of this work. Even though the number of models and the diversity of the sources accessible via the RCSB PDB interface are fewer than this work, I think the purpose of both works are similar. As there are some overlaps between this work and the RCSB PDB interface in terms of data providers (and authors), what is the significance of this work compared to the RCSB PDB interface?2. Most computational models rely on a few data providers, AlphaFold DB, SWISS-MODEL Repository, and AlphaFill (for ligands). In my opinion, it would be better to make the platform richer by recruiting more diverse data providers with different points of view (e.g., conformational ensembles) or different modeling approaches (e.g., machine learning-based approaches with pre-trained protein language models such as OmegaFold). Is there any plan for such progress or promotion of the platform?3. It would be better to have a guide of model selection if there are multiple searched models for an Uniprot ID. Alternatively, providing universal quality assessment scores for models would be an option (by additional data provider). Currently, pLDDT scores are provided, but they are difficult to compare between modeling methods as they were trained independently for each method.4. I was able to search on the 3D-Beacons web page a few days ago. However, I could not at the moment of writing these review comments (Sept. 13, 6 p.m. in EDT).

Reviewer2: Carlos Rodrigues

This manuscript describes in detail the 3D-Beacons platform/initiative, which aims to facilitate access to 3D data as well as meta-information about experimentally determined and computationally predicted protein structure. This resource is very valuable for the broader scientific community in a time where the number of protein structure data available rapidly increases an many structures may be available for the same protein.A minor correction is required on page 7, where the authors describe 4 different types of protein structures: Experimentally determined, Template-based, Ab-initio anc Conformational Ensembles. On many examples available on the website (e.g. https://www.ebi.ac.uk/pdbe/pdbekb/3dbeacons/search/P15056), there is one extra category which is structures derived from "Deep learning" methods. I am assuming this comprises a sub-set of Ab-initio structures, which the authors decided to keep as a separate category after submitting this study for publication. The main text should be updated to reflect this change as well as Figure 4.

Reviewer1: J. Harry Caufield

This manuscript by Charbonneau et al. details efforts to address challenges in enhancing the value of metadata among projects in the NIH's Common Fund Data Ecosystem. They specifically detail how a new metadata model was developed and deployed to unify data properties across projects. Assembling such a model is a major accomplishment and a necessary step in promoting data reuse. Applying the model is another commendable achievement. The manuscript text undersells the value of these efforts. How has the value of data in the CFDE improved due to implementation of a unified metadata model and new infrastructure? The authors clearly delineate the challenges in searching CFDE data; these issues frequently appear in efforts toward improving biomedical data FAIRness and are directly relevant to the core challenges identified by Wilkinson et al. (2016) in their FAIR guiding principles. Much more emphasis could be placed on the overall impact of a consistent metadata model, whether within the CFDE alone or in the broader realm of bio-data management. Major issues: 1. As noted on page 11, "All C2M2 controlled vocabulary annotations are optional". Data producers will use terms outside the controlled vocabulary as needed, and are unlikely to consult any CFDE working groups in every instance. Is there some automated system for term normalization in place? How will data producers be encouraged to preferentially use controlled terms? Are they warned during submission, as noted on page 22 regarding data contents? Minor comments: 2. The first example of the mismatch between user expectations and actual results of searching for Common Fund program data is very illustrative. I appreciate how it notes that even instances like matching Dr. Phil Blood's name in a search can complicate Findability. 3. The abstract could include some brief description of the broader relevance and impact of the metadata model, including its potential for use outside the CFDE. 4. On page 5, the sentence "Thus, a researcher interested in combining data across CF programs is faced with not only a huge volume, richness, and complexity of data, but also a wide variety, richness, and complexity of data access systems with their own vocabularies, file types, and data structures" feels somewhat redundant and could benefit from some editing. 5. The structure of Figure 1 (or should this be Table 1?) is confusing. The general idea is clear - metadata types, properties, and formats are inconsistent across projects - but the two-column format presents issues with direct comparison. 6. It is interesting that, among all values presented in Fig. 1, just one includes a CURIE (HMP's ENVO:02000020). This may be worth further comment as it is striking that few of these projects have adopted unique identifiers within their metadata schemata. 7. Slightly more detail regarding the interviews with Common Fund programs would be helpful for understanding how these interactions contributed to the process. Were interviews primarily with PIs? Were several prominent issues repeatedly discussed in the context of multiple projects? 8. Is the C2M2 master JSON schema publicly accessible? 9. Some redundancy is present between the first and second paragraphs under the heading "Entities and associations are key structural features of the C2M2" - e.g., core entities and container entities are both described twice. 10. In Figure 2, some lines connecting tables are very close to the edge of the figure borders and are difficult to see as a result. 11. Is there a mechanism for dealing with obsolete terms as the ontologies contributing to the controlled vocabulary change? In the even that the NCBI Taxonomy renames a genus, for example, how will CFDE metadata change (if at all)?

Reviewer2: Carole Goble

The article is a very useful contribution to the growing number of metadata models and data catalogues in the life science data ecosystem. The recent NIH mandates in data sharing emphasize the need for findability of datasets, and the need to operate within a federation and ecosystem recognises the reality of independent data centers and legacy data collections. The paper states the context of the CFDE well, setting up the need for a centralized portal capable of ingesting, indexing, search and supporting cross dataset comparisons of dataset from different, independent data centers without the need for those centers to move, reformat or rehost their data. This is a common pattern that many data infrastructure providers will recognise. The incremental approach that supports minimal uploads and respects local DOI implementation is a pragmatic approach that has made onboarding the data centers feasible, I suspect. The insight that mapping to common ontologies does not actually lead to harmonised dataset and nor does it support search is a useful lesson that resonates and is useful to reiterate (although it is already well known). Given the approach is tabular, Frictionless data makes sense. The process of working with the Centers is interesting as is the choice of three core entities. Some more discussion on why these three and only these three would be appreciated. The ingest pipeline and process is not so clear. - It seems that each Center is required to map its datasets to the current C2M2 model in 48 TSV files, in a data package that is then uploaded to the catalogue and ingested into the portal's database. Is the data package a complete reupload each time or is the data package additive? There are hints in the text that it is a replacement each time. - What is the cost and complexity of this mapping and upload borne by the Centers? Any insights would be valuable. Is and tooling provide to help beyond the documentation? - Figure 5 could be improved to include the data that flows between the steps, and the actors. Could Figure 3 and 5 be merged? - If the datasets are reuploaded afresh each cycle, how are between-release analytics managed? By the use of the PIDs? Are there any restrictions on what cannot be changed between releases? - As the datasets can be incrementally improved with each release, are there any trends between releases that indicate changes in metadata enrichment - On page 18 you state that "DCCs get better at using the C2M2" - The data package needs clearer description: relationship between the TSV files, the Frictionless Data JSON and BDBag is of interest to many in the community and warrant a more thorough discussion. The portal - Why were these three basic kinds of search chosen? Were there user stories collected from the listening tour? - It would be helpful if there were some indications of the use of the catalogue by users rather than just the ingest and publishing pipeline. Page 5 the arguments are made that reusing common fund data for cross-cutting analysis is challenging and requires the hiring of dedicated bioinformaticians ("at considerable cost of NIH"). How does making the datasets available through a catalogue relieve the burden of skilled bioinformaticians to analyse data? The data still needs to be processed. Hasn't the burden just shifted to the Centers to prepare the TSV files for the ingest pipeline? Page 5 claims that the sociotechnical framework of the CFDE is a self-sustaining community. How? Working groups have been established but to what extent are these managed by the community and not the dedicated action of developing the portal? What is the sustainability of the portal? The easy expansion of the C2M2 seems to depend on two things: the incorporation of domain specific vocabularies and the cycle of ingest-releases at time points. Does this latter point constitute expansion that is easy? This would require each data center to adapt to the new table templates. Page 9 Containers are mentioned but it is not clear what the difference is between a container and a collection. Containers do not seem to appear again in the browsing. Page 18 the visibility of Biosamples changing over time in Figure 4 wasn't so clear to me

Reviewer1: Jianyi Yang

The authors present the predicted structures for the proteome of the reef-building corals. 8382 protein sequences were obtained by experiments, which are fed into ColabFold for structure modeling, generating 8166 structure models. Overall, this is a valuable study toward the understanding of the reefbuilding coral. Here are a few comments for possible improvement. 1. It becomes trivial for proteome-wide structure predictions nowadays with AlphaFold2 and other methods. I think the major contribution of the current study is the determination of the proteome sequences rather than the structure prediction. Thus, I would encourage the authors to spend more effort in analyzing the sequences, for example, how the sequences cover the Pfam families, how redundant the sequences are, how much they overlap with the sequences in UniProt, etc. 2. It may be meaningful to compare the predicted structure models to the SCOP or CATH database to see the fold distribution and if there is any new fold. 3. What happened to the ~200 proteins that ColabFold failed to work? 4. I suggest adding a browse function to the server for browsing the data.

Reviewer2: Brendan Robert E. Ansell

Zhu and colleagues report the generation of predicted protein structures via alpha-fold, for three coral species: A muricate, M foliosa and P verrucosa. Mass-spec analysis of the proteome of the three species is also performed. The authors describe a handful of structures that appear to be orthologues across the species and may have functions as pore-forming toxins, in calcium deposition and host-symbiont interactions. The generated protein structures will be of use to the scientific community and the web server is quite good. Major comments: Please ensure that the entire structure repository is available for unrestricted download as per http://corals.bmeonline.cn/prot/release.php Incorrect use of 'co-expression'. Assume the authors mean protein orthologues (i.e., homologues across species). Please replace with 'homologous proteins' throughout including in http://corals.bmeonline.cn/prot/release.php The link from 'CoralBioinfo' gives a 404 error: http://corals.bmeonline.cn/index.php In http://corals.bmeonline.cn/blast/, please include a link back to http://corals.bmeonline.cn/prot/ Although the manuscript lacks bioinformatic analysis of the structural proteome, this is not required for the data note category but would enhance the value of the publication if provided. In terms of validation, there is a technical control for the alphafold instance that this project used, which the authors should include. Specifically, please report the RMSD between structures predicted in this work with the published alphafold structures for the same proteins Acropora muricata ( 20 proteins), Montipora foliosa (8 proteins) and Pocillopora verrucosa (70 proteins), available at e.g. https://alphafold.ebi.ac.uk/search/text/Montipora%20foliosa%20?organismScientificName=Montipora %20foliosa Please detail in methods how the mass spec data relates to improving the genome or proteome annotation of each species. How was the mass spec data used? I presume it was used to identify 3-way orthologues between the species, and producing the "8,382 co-expressed proteins" that were selected for structural prediction. The data dump would be stronger if the mass spec proteomics data was also made available. What proportion of the structural proteome has mass-spectral support? Please include a supplementary text file containing the key features of each predicted protein e.g. % high confidence structure, gene id, interpro domain annotations , and top blast homologues. The long proteins could be split by domain to provide some structural information. To boost the value of this data, the authors might also consider predicting the coral symbiont proteomes followed by integrative analysis of host and symbiont proteomes to predict interacting partners. What are the domain and sequence features of the low and very-low confidence predictions? Is the reference genome available for any species? What is the completeness and content. How does the mass spec and structural data improve the genome annotation and vice versa? At present large parts of the discussion are irrelevant. Comments about covid-19 and the role of bioinformaticians are outside the scope of a research report. Minor comments: Comment on whether toxicity is reported for these coral species. Use full genus names on first use Proofreading of grammar required throughout, and elimination of non-scientific phrasing. Drop irrelevant arguments regarding COVID 19 and the call to arms for bioinformaticians.

Reviewer1: Satoshi Hiraoka

In this manuscript, the authors developed a new tool, What the Phage (WtP), for comparison of the output from multiple bioinformatics tools to predict phage sequences from genomic or metagenomic datasets. The purpose of this study is some or less meaningful. As the authors described in the Introduction section, currently it is difficult to predict reliable viral genomes, especially from cultureindependent metagenomic datasets precisely because of the lack of knowledge about viral genomes in current protein/genome databases. There are many bioinformatics tools already proposed and some of them are widely used in microbiology, however, the outputs from these tools are frequently varied and conflicted among them. However, there is no good integrative platform to compare the outputs. Here, the proposed tool easily generates well-summarized output derived from multiple tools, and thus, the tool might be facilitated the analysis of phage prediction in the field of microbiology. Indeed, the authors conducted (only but) one case study using real phage genomes and reported reasonable performance. I feel the tool has some potential to contribute to the wide fields of viral genomics. However, the user of this tool should keep in mind the fact that the tool just summarizes the output of multiple phage-prediction tools, meaning does not evaluate the reliability of the output, as described in the Discussion section. I feel thus the tool sometimes may lead to misunderstandings or make the users confuse rather than help them. It should emphasize that the majority decision among the multiple tools does not always bring the best result. The users may need further detailed analysis for the precise prediction of viral genome from metagenomes. Also, I feel that, because the development of bioinformatics tools is quite rapid, integrated platforms like WtP will be outdated very soon without continuous effort for maintenance and upgrade to assimilate future novel tools. I understand the 'sustainability' of the tool is out of the journal scope, but the perspective on this point will be better to be described in the manuscript or GitHub page. I have some suggestions that would increase the clarity and impact of this manuscript if addressed. [Background] Some tools (e.g., Virsorter2) can be used to predict viruses out from common bacteriophages, e.g., NCLDV and virophage (See the original article of VirSorter2). Those kinds of viruses should be described briefly in this section as well as common dsDNA phages. Assembly-free long read is described here, but I think this is a bit far from the scope of this manuscript. Indeed, the dataset used in this study (ERR575692) is derived from Illumina HiSeq and the performance of assembly-free long-read dataset was not analyzed in this study. I think the descriptions could be moved to the Discussion section rather than the Introduction. Rather than that, it would be better to add more attractive descriptions about studies of phage genomes identified from short-read metagenomes to emphasize the importance of phage prediction and the value of the proposed tool, WtP. e.g., History of viral genomics using metagenomic dataset, recent technical improvement of metagenomics, phylogenetic diversity of phages, discovery of novel phage lineages from environmental metagenome, etc. Only 5 out of 11 tools that used in WtP were introduced here. The remaining 6 tools would be better to also cite here with a brief explanation of those strategies for virus prediction. Also, MARVEL was cited here but not used in WtP. [Design and Implementation] Figure 1 is different from the one on the GitHub page ( https://mult1fractal.github.io/wtpdocumentation/figures/wtp-flowchart-simple.png ), which seem to be better than the Figure 1. What 'DAG' means? [Prediction and Visualization] 'a metagenome assembly' could rephrase like 'metagenomic assembled contigs' Metaphinder and Seeker are here with 'no release version'. I understand the situation but I feel this description is not good for reproducing the analysis. To specify the version of tools even if lack the official release version, mention the last commit date (For Metaphinder, Aug 10, 2021) or GitHub commit ID ( bebc447d00ec9ff9f4960f38b627d8651262ff72 ) is likely a good way. [Functional annotation & Taxonomy] In this manuscript, Prodigal was used for gene prediction. However, accurate gene prediction from phage genome is still difficult (see https://academic.oup.com/bioinformatics/article/35/22/4537/5480131). This fact have been affect both the phage prediction and functional gene annotation in the field of virology. I think the difficulty of gene prediction from phage genome and potential room for improvement should be noted in the discussion section. [Result report] The sentence ' ~ IMG/VR, iVirus, or VERVE-NET' here should be with appropriate citations or URLs. I found a paper of iVirus: https://www.nature.com/articles/s43705-021-00083-3 [Other features] WTP -> WtP [Analysis] Figure 3. X-axis title of left-bottom bar plot and Y-axis title of top-right bar plot. viral -> phage What 'prediction values' mean? Are these scores generated by each prediction tool? Figure 4. X-axis texts. Unify the format to either NodeID:assignment (e.g., NODE_5:unknown) or assignment:NodeID (T3:NODE_14). ' The sequences matched with 100% identity to Salmonella enterica (Salmonella enterica strain FDAARGOS_768 chromosome, complete genome), but not to prophage sequences. ' here. Does the sentence mean that the contig NODE_5 and NODE_8 were mis-predicted as prophage by CheckV? Table 1. completeness -> completeness (%) [Discussion & potential implications] Add citation in the line ' At least one multitool approach was implemented on a smaller scale by Ann C. Gregory et al. (comprising only VirFinder and VirSorter). ' [References] 16. Lack doi. 18. Lack doi. 19. Lack doi.

Reviewer2: : Huaiqiu Zhu

In this manuscript, the authors developed an integrated workflow WtP for identification, annotation and taxonomy of phage sequences. Based on Docker and Nextflow, WtP integrates 11 phage sequence identification tools (including 14 approaches), two functional annotation and taxonomy tools (Prodigal and HMMER), and a visualizing tool (chromoMap). When using WtP, it is convenient that users do not need to install each tool and can avoid the conflict between each installation package and between operating systems. Also, the WtP tool was applied to the artificial microbiome. The threshold of each phage sequence prediction tool can be manually adjusted and outputted. Annotation and taxonomy results of phage sequences can be further visualized by CheckV and by chromeMap tool. However, there are some limitations in this manuscript. For the annotation and taxonomy stage, only the Prodigal tool was used for gene prediction, and no other gene prediction tools (especially the phage-specific tools). It is necessary for an integrated workflow to include other similar tools. WtP needs at least 4 GB of memory and 75 GB of storage, so the author should develop a web version or at least a graphical interface version of WtP for its prevalence. Major comment: 1. Except for sequence identification, host prediction (e.g., HoPhage, PHP, and VirHost Matcher-Net) and lifestyle prediction (e.g., DeepPhage, PhagePred) of phage sequences are also important in microbial communities. However, WtP did not involve those functions. 2. In addition to the web version or graphical interface version of WtP, the author can also consider a video demo or usage illustration. To clarify the purpose of this study, I think it would be better to add the phrase 'a web server of ...' or 'a GUI platform of ...' into the title. 3. In 'Analysis' Section (Page 12), only four contigs of phage sequences can be annotated in artificial data: P22 (NODE_12), T3 (NODE_14), T7 (NODE_13) and phiX174 (NODE_30). The 'predicted_organism_name' of the remaining 102 phage contigs are 'no match found'. Can WtP improve or add more databases to annotate more contigs? 4. In 'Analysis' Section (Page 14), the author mentions 'No specialized phage assembly strategy or any cleanup step was included during the assembly step'. I think it is unreasonable, and the downstream analysis will inevitably be affected by the impurity sequences. 5. In Figure 2, it is possible to export results in the form of 'csv', 'pdf' or 'excel'. Can WtP export all the predicted phage sequences in the form of 'fasta'. The author should describe how to change or add the database during the annotation and classification phases. Minor comment: 1. In 'Functional annotation & Taxonomy' Section (Page 8), 'Figure 3' in the sentence 'All annotations are summarized in an interactive HTML file via chromoMap (see Figure 3)' should be 'Figure 4'. 2. The column of 'Computeness' in Table 1 missed the unit, and the author could add an outer border to Table 1. 3. Figure 2 and Figure 3 need to be clearer. 4. Page 5. 'approach to gain' should be 'approach to gaining'. 5. Page 13. 'In addition to' should be 'In addition to'.

Reviewer 1: Milton Pividori

In this manuscript, the authors analyzed different characteristics that are potentially related to the expression of human genes under IFN-a stimulation. A classification model is built to predict ISG (genes that are upregulated following IFN-a stimulation) from the human fibroblast cell. The model also performs feature selection, and the authors used different test sets (on different types of IFN) to validate their model. The authors provide a web server that implemented this machine learning model. I liked the introduction, the background and motivation were clear. However, the Results section was a bit hard to follow, in particular the implementation of the machine learning models, with different classifiers applied inconsistently across distinct features sets. At the beginning of this section, the authors perform extensive manual feature analyses across different feature types (related to alternative splicing, duplication, and mutation) to build a refined dataset. These analyses basically correlate each individual feature with the expression of genes in the presence of IFN-a. I have several concerns here, related mainly to the correlation between features, that I describe below. General comments: * Regarding reproducibility, the authors provide a Github repository with source code, the model trained and data. From the documentation and notes in the manuscript (lines 1015-1023), looks like this can only be run on mac OS, which makes it very hard for me to test (I'm a Linux user). I recommend the authors to read and follow the article "Reproducibility standards for machine learning in the life sciences" (https://doi.org/10.1038/s41592-021-01256-7). Having, for instance, a Docker image to download and run your analyses would be fantastic. * The authors perform a comprehensive analysis of features that differentiate different gene classes. I wonder why didn't they use first a machine learning model to automatically find these important features, and then try to analyze which features were selected (instead of the other way around as done in the study). I think there is perhaps too much manual feature engineering in the previous steps of training an ML model. * Related to the previous point, in my comments below one of my concerns is about feature correlation. The authors compare individual features regarding their ability to separate different gene classes (ISG vs background vs non-ISG). But one can imagine that some features are highly correlated. Some features might not be useful to separate gene classes from a single-feature analysis (as the authors do at the beginning), but they could be useful in combination with other features. Unless I'm missing an important point, I would leave the machine learning model to learn this and then analyze each feature individually after the model identifies them. * Authors are concerned that including too many features in the support vector machine (SVM) model would complicate the prediction task. To remedy this, they manually select the features according to, in my opinion, a more subjective criterion. Why didn't the authors use a feature selection algorithm here? I know that they propose a model including feature selection, but I guess I don't understand well all the previous manual feature analyses. Using a known feature selection method here would provide a more data-driven approach to improve classification, in addition to their manual expert curation (which is also valid). * They run several classification models, but not consistently across the same set of features. For example, only SVM is run across genetic, parametric, all features, etc, but not the other models. Why is that? * The manuscript would really benefit from a figure with the main steps of the analyses performed, models tested, datasets employed, etc. It's hard to get the big picture as it is now. Results/Evolutionary characteristics of ISGs: Paragraph between lines 131-148: * I think the window size used (mentioned in the text) should be added to the Figure 2 caption * What's the vertical dashed line? In the text, you say that those at the left of this line are IRGs, but I don't understand the meaning of that vertical line (-0.9 log fold change). This explanation, which I didn't see, should be added to the figure caption also. * From the text, I understand that in the subfigures in Figure 2 you have IRGs, non-ISGs and ISGs. Would it be possible, or meaningful for the reader, to add an extra vertical line to separate them? Results/Differences in the coding region of the canonical transcripts: Paragraph between lines 193-208: * If GC-content is underrepresented in ISGs more than non-ISGs, the ApT and TpA should be expected to be more enriched in ISGs, right? Sounds like a redundant analysis. I would expect these two sequencederived features to be correlated. If this is the case, maybe it would be better to highlight other features instead of a correlated/expected one? * Figure 4: here the authors divided the parametric set of features into four categories and compared their representations among ISGs, non-ISGs and background genes. The figure shows p-values of the tests on the y-axis, and the four categories of features on the x-axis. I think it's important to run a negative control: could you please run these tests again, say, 100 times, with gene IDs/names shuffled, and check whether some of these results also appear in these null simulations? Maybe you can keep the same figure, but remove those also found in the null simulations. Paragraph between lines 209-227: * Is it possible that the comparison of codons frequencies (third category of features) is correlated with previous findings (like GC content or ApT/TpA enrichment)? If so, would it be possible that maybe the analysis is also expected or redundant? For example, in ISGs there is an underrepresentation of GCcontent, and you also found that ISGs there is an underrepresentation of "CAG" codons. I might be missing something, but aren't these expected to be correlated? Results / Differences in the protein sequence: Paragraph between lines 302-323: * Figure 6: I would suggest adding the same negative control suggested before. Results / Differences in network profiles * I think it's important to define what are all those eight features in the network analyses (closeness, betweenness, etc), otherwise it's hard to follow what comes next. Results / Features highly associated with the level of IFN stimulations * Figures 9 and 10: it would be good to add the sign of the correlation in the figure, in addition to mentioning it in the caption (as it is now). Results / Difference in feature representation of interferon-repressed genes and genes with low levels of expression * Given the unique patterns or differences between non-ISG class and IRG class, wouldn't it be better to perform different analyses excluding IRG genes? The authors also acknowledge these risks in lines 539- 541. Results / Implementation with machine learning framework * It was hard for me to understand the workflow in this section: you used different machine learning models applied to distinct features sets, for example. Why don't you apply the same set of models to the same set of features? I think this section needs an initial paragraph with a global description of what you are trying to do. * For example, I don't think I understand very well the concept of "disruptive feature". What does it mean? * Table 3: I don't understand the threshold selection here. I guess you refer to classification or decision threshold from a model that outputs a probability of a gene to be ISG or non-ISG. First, I think there should be a line separating each performance measure to clearly show those that are "Thresholddependent" and "Threshold independent" * I also understand that, during cross-validation, you selected for each model/feature set combination, the threshold that maximized the MCC (this is explained in Table 3 as a footnote, but it should be more explicitly mentioned in the text). * Table 3: What is the "Optimum" set of features? Why is this "Optimium set" only used with SVM? * How does the "AUC-driven subtractive iteration algorithm (ASI)" compare with other feature selection algorithms. * Table 5: you mention this in the text, but it would be good to have an extra column indicating which datasets were used for training and which are for testing. * Figure 13: it would be good to have the AUROC in the figure, not only the curves. Web-server: * I think, in general, that the web application needs to be more intuitive and have more documentation. For example, the main interface says "Predict your human genes of interest", what does that mean? What does it predict?

Reviewer2: Muthukumaran Venkatachalapathy

First of all, this manuscript is well-written after a thorough research investigation. I enjoyed reading about interferons, interferon stimulating genes (ISGs), mechanisms and signalling pathways. In the introduction, the authors have highlighted the different methods (including other bioinformatics databases) available to identify ISGs and their potential pitfalls. This unmet need is addressed using in silico approaches which were used to classify interferon stimulating genes from non-stimulating ones in human fibroblast cells. Here, the authors have applied a combination of expression data and sequential/compositional features and designed a machine learning model for the prediction of ISGs from non-ISGs. Apart from features like duplication, alternative splicing, mutation and presence of multiple ORFs, the authors extracted various sequential features and found them to be correlated well with ISG prediction. For example, ISGs are prone to GC depletion and a significant difference in the codon usage among ISGs was found. In that context, the authors claim that ISGs are evolutionarily less conserved, codon usage features, genetic composition features, proteomic composition features and sequence patterns (especially like SLNPs and SLAAPs) are optimal parameters that can cumulatively help in differentiating ISGs from non-ISGs. When it comes to building a machine learning model, the authors faced challenges due to similarities between ISGs and IRGs. They have experimented using different algorithms for model building ranging from the decision tree, and random forest and found decent results with support vector machine. Limitation: Model Prediction accuracy was close to 70% for type I and III IFN and it performed below par when it comes to predicting ISGs activated by type II IFN system. There is scope to improvise the model prediction accuracy and extend its usage to type II IFN systems. If the authors could briefly add few points on how to improve the model accuracy and also highlight the application/impact of this work in their discussion, that would help scientists from other background to resonate with this manuscript. Relevance: I believe there are inherent attributes (genetic, compositional, expression) with ISGs which may facilitate or even elevate their expression after IFN stimulation. On the other end, I think these properties may also be leveraged by the viruses to escape or evolve from IFN mediated antiviral response. This study is relevant during the on-going pandemic, this bioinformatics tool can help design better drug target and may indirectly aid in developing novel antiviral compounds. I recommend this work for publication without any changes.

Reviewer1: Mahesh Neupane

Nicely written paper on selection signatures for Cashmere goats with detailed analysis and possible deletion. Here are some of the suggestions to improve the paper:

How was the optimal size of K determined in admixture? Please review the formatting on the manuscript, for example page 5 and page 6 figure have some formatting errors. Was sample size enough for all the comparison? What was the power of study design. How the results from mouse and human cell line justified for comparison with goat? Very good job of supplying all the codes used in the programs. Perhaps this codes or parameters can be combined together as supplemental material or GitHub repository.

Reviewer2: Yixue Li

The authors raised an interesting question, hoping to discover the genetic mechanisms associated with cashmere traits for breed improvement. The authors sequenced 120 native Chinese goats, including 2 cashmere goat breeds and 6 common goat breeds. Through analysis, the authors found and believe they confirmed that a 582 bp deletion at 367 kb upstream of LHX2 is involved in regulating cashmere yield and cashmere diameter. The results are very interesting, and if they convincingly answer the questions below, acceptance for publication is recommended. 120 goats, are they inbred, and are there enough to get a statistically significant result? What is the statistical basis? The article begins to describe: 582del and 504del are both correlated with cashmere yield, but only 582del is also significantly correlated with fiber diameter, while 504del has no significant correlation with fiber diameter. Then he added: The interaction effect between 582del and 504del was significantly correlated with cashmere fiber diameter, indicating an interaction between the two genes. What is the mechanism of this interaction? How they are significantly related to cashmere fiber diameter, further elaboration is needed. On the one hand, the text mentions that the deletion sequence 582del in the upstream region of the LHX2 gene may act as an insulator, preventing the function of the LHX2 enhancer. Later, it is mentioned that the deletion of the LHX2 insulator 582del increases the expression of LHX2 and promotes the growth of cashmere fibers during the growth period. There seems to be a contradiction here, please explain. To confirm the insulator function of the 582del sequence, the authors synthesized a 551 bp DNA fragment and inserted it into the pGL3 plasmid downstream and upstream of the SV40 promoter, and then co-transfected human 293T cells and mouse 3T3 cells, and thus confirmed the insulator function of the 582del sequence. Here: (1) the two sequences are not identical in length and identity, (2) using mouse and human cell lines respectively, how do we conclude that the 582del sequence will have the same function in goat? What is the experimental logic and biological logic here? Hopefully a convincing explanation can be given.

Reviewer3: Yu Jiang

This manuscript resequenced 42 cashmere goats and 78 ordinary goats, performing Genome-Wide Selective Sweeps and then a 582 bp deletion in the thirteenth intron region of DENND1A upstream of LHX2 was found to increase cashmere yield. This discovery provides resources for the development of the wool industry and the enrichment of animal genetic resources. However, the description in some parts of the manuscript is very rough, and many attachments are missing. Major concerns: 1.In the result "Plausible Causative Mutation near LHX2", you selected a lot of cashmere and fiber diameter data for association analysis and get Fig 5e, but your results may have high false positives. The author should demonstrate that the deletion is significantly associated with phenotype after excluding factors such as gender, age and so on. 2.It is found from Fig.2 a that MT and JNG contain a large number of cashmere goat pedigrees. When they are selected and analyzed with cashmere goat samples, the results may be affected by this mixing. The author should consider the particularity of these samples when using them. Perform subsequent analysis. 3.In Fig 3, are strongly selected loci linked to surrounding loci? Do these sites have an effect on gene expression? 4.Fig 3c & d, the horizontal and vertical coordinates of your two graphs are the same, but the trends of the graphs are different. Please mark clearly what the two graphs describe in the legend and the paper. Minor concerns: 5.In the abstract, "Luciferase assay shows that the deletion, which acts as an insulator, restrains the expression of LHX2 by interfering its upstream enhancers", but in the result, "Therefore, the deletion of the LHX2 insulator increases the expression of LHX2 and promotes cashmere fiber growth at the anagen stage, while deletion of the FGF5 enhancer reduces the expression of FGF5, inhibiting the regression". The conclusion is inconsistent, please clarify the logic of the paper and draw the correct conclusion. 6."Luciferase assay shows that the deletion, which acts as an insulator, restrains the expression of LHX2 by interfering its upstream enhancers. Our study discovers a novel insulator of the LHX2 involved in regulating cashmere production and diameter." These two sentences are easy to confuse us, and it will make people understand that two insulators are found on LHX2, one to suppress expression and one to regulate cashmere production and diameter, and it is recommended to modify. 7.There are inconsistencies in the sample names in the paper. For example,you use IRWG in the front of the sentence and IRW in the back,please unify the name. At the same time, the paper contains many spelling and symbol errors, such as "mddle", "goatswith", symbol repetition and so on. (1)In the STRUCTURE analysis, When K = 4, we observed five separate clusters: IRWG and ANG in west Asia, YNBB, GZB, JTB and CDB in southwest China; cashmere goat in north China; MT and JNG in mddle east China; and Korean goats in south Korea. At K = 6, goats in the southwest China further split into two geographic subgroups: the Yunnan-Kweichow Plateau group including YNBB and GZB goats, and the Chengdu Plain group including CDM and JTB goats. Two west Asian goats (IRW and ANG) were also separated . (2)More interestingly, we found that the 582del has a high frequency in the IRWG population (80.9 %), while the 504del was absent, which is also consistent with previous research. (3)10 JTB from Jintang County of Sichuan Province; 12 CDB from Chengdu City of Sichuan Province".." To further evaluate whether these two deletion variants were related to cashmere (4)traits, we selected 235 CDMC goatswith cashmere yield (Supplementary Fig. 22, Supplementary Table s8) and 581 CDMC goats with fiber diameter records (Supplementary Fig. 23, Supplementary Table s9) for association analysis. (5)Fig 2b, "KOG". 8."We inspected all variants within exons to identify the potential causal mutation around the DENND1A-LHX2 locus; however, no coding variants were found. " Please put the information of the relevant sites in the attachment, only one sentence will make the paper unconvincing. 9."Analysis of the 582del deletion region using the BLAST program revealed that it is not a highly conserved element but was found in the genomes of primate and ungulate species. " "582del deletion" is a repetition.

Reviewer name: Francesco Asnicar (revision 1)

This reviewer thanks the authors for their revision. However, the quality of the figures and the main goal the authors would like to reach with the tool named TAMPA is noThe main goal of TAMPA is to allow to compare taxonomic profiling tools, but it is evident from the supplementary figures that the software cannot allow such comparison when the taxonomic tree is large enough, as the circles added to the branches become unreadable. This I believe is a major flaw of the tool that aims to do that specifically and for such cases a smarter way that allow comparing taxonomic profilers should be found. For instance, a legend to each figure created by TAMPA should be added to make immediately clear what the colors represent. Also, for such taxonomic trees that the visualization fails in allowing comparing the taxonomic profilers a different and complementary data should be provided, for instance a table listing all branches and the numbers the depicted circles represent. In addition, such table should allow to overcome the limitation of just 3 tools allowed in the comparison.

Reviewer name: Alessio Milanese (revision 1)

Many thanks to the authors for their detailed responses to my comments.The edits have improved the manuscript and I have only few minor comments.COMMENT 1:In Figure 4b I can see that "Tenericutes" and "Planctomycetes" are both in orange, meaning that they bothhave been measured only by mOTUs. But in the main text I read "mOTUs failed to detect theTenericutes group, while MetaPhlAn failed to detect Planctomycetes", which is wrong.COMMENT 2:I would improve the figure legends. In particular, the description of 4b is the same as in 2a and 3a and 1:"The size of the discs represents the total amount of relative abundance at the corresponding clade in theground truth, or the tool prediction if that clade is not in the ground truth. If the tool predictions agree,a disc is colored half orange and half teal. The proportion of teal to orange changes with respect to thedisagreement in the prediction of that clade's relative abundance between the two tools being compared. Highlighted blue text represents clades where the difference between the relative abundances of the prediction and ground truth exceeds 30%".I would suggest to have this description only for figure 1, and then have a shorter description for thefollowing figures.COMMENT 3:The second color is described sometimes as "green" and sometimes as "teal". For clarity, I would suggestusing just one of the two.

Reviewer name: Francesco Asnicar

The manuscript by Sarwal et al. presents a novel tool for a standardized visualization of metagenomic taxonomic profiler tools, named TAMPA, that also enables a more general assessments of performances of taxonomic profiler tools by providing an extensive of different metrics.It would be interesting to see (if possible) the comparison of three (or more) taxonomic profiles at the same time. The evaluations shown are always binary, but in a real-case scenario where a user would like to evaluate 3 or 4 different taxonomic profiling tools on his community, it would be great to be able to do it.Other than the evaluation on the agreement between two (or more) taxonomic profiling tools, it is not clear how TAMPA can drive improvement over biologically-relevant question. Although it is clear, as the authors stated in the introduction, that different taxonomic profilers (with different parameters settings) can produce very different taxonomic representations, to support this statement it will be important to be able to show, at least one case, where TAMPA can suggest a different taxonomic interpretation of a microbial community that is also biologically relevant.Figures in general appear to be of low-quality and stretched, please consider improving them as they are the main point of TAMPA.

Reviewer name: Raul Guantes (Revision 1)

In the revised version and the response letter, the authors have clarified all the questions and addressed the comments raised in my previous report, and I think the manuscript is now suitable for publica

Reviewer name: De-Shuang Huang (Revision 1)

I think the paper can be accepted.

Reviewer name: Thomas Schlitt

The manuscript "contrast subgraphs allow comparing homogeneous and hetereogeneous networks derived from omics data" introduces and illustrates the application of contrast subgraph analysis to gene expression, protein expression and protein-protein interaction data. The method can be applied to weighted networks. The authors give a good description of the method and the context of other available methods.The authors apply the contrast subgraph analysis to three different omics data sets - overall these analysis are not very detailed and do not yield surprising results but they provide a nice illustration of the potential usefulnes of the contrast subgraph analysis in the context of omics data. To my opinion this is really where the merit of the paper is: to promote and make accessible the method to a wider audience of researchers in the field of bioinformatics/molecular biology. The authors have also applied their method to brain imaging derived networks, but that work is not part of this publication.The contrast subgraph analysis is particularly interesting, for data that is collected under different conditions but for the same set of nodes (i.e. genes, proteins, ...), i.e. where the nodes present do not change (much), but their interaction strengths differes between conditions. It remains to be seen where this method can deliver unique value that is not achievable by other means, but the approach is very intuitive. Its rationale can be readily understood, reducing the temptation to use it as a "black box" without critically questioning the results as might be the case for more complex methods. One of the downsides of the presented approach is that it does not provide any measures of confidence in the results - while there is a parameter >alpha< that allows some tuning, little information is given on how to choose a suitable value for this parameter (which obviously depends on the data). Another issue that might come a little too short is how to derive graph representations from experimental omics data in the first place. Usually these methods do not yield yes/no answers, but rather we obtain a matrix of pairwise measurements (e.g. correlation of coexpression) and to obtain a graph a threshold on these numbers is applied to obtain an edge or not. Various methods have been proposed to choose thresholds, but in the end, moving from a full matrix to graph representation means loosing some information - it would be interesting to see a deeper analysis on how much this thresholding influences the outcomes of the proposed method - this question is obviously linked to obtaining some confidence information on the results.Overall, the method described here is very interesting, it shares downsides with other graph based methods (thresholding), the biological examples given are brief, but illustrative for the use of the method, the manuscript is well readable. The manuscripts stimulates to add this method to your own toolbox and to apply it to interesting data sets to see if it yields results that were not obvious from other approaches.Minor comments:-figure captions esp 1-3 - please provide more information in the figure captions to make the figures "readable" on their own without a need for the reader to refer back to the text; figure captions for Fig 1-3 are almost identical, yet very different data is shown - a clear indication that important information is missing in the figure caption - such as what is the underlying data?Please explain all terms used in the figure in its caption: here what is "GeneRatio"? Figs A/B what is the x-axis showing for the violin plots?-figure 3c and para on Protein vs mRNA coexpression (p2-5) - are the differences really that striking - in 3C, the box plots do not look that different, super low p-values are probably due to very large number of data points, but not sure it is really that meaningful here (effect size?)-figure 4 is too small, nodes are barely visible, colours cannot be distinguished-algorithm 1 and description in text - I would probably move the description of the algorithm from the text to a "figure caption" for the algorithm box, to make it easier for the reader to find the definitions of the terms.

Reviewer name: Raul Guantes

In this manuscript the authors apply the method of contrast subgraphs (developed among others by some of the authors), that identifies salient structural differences between two networks with the same nodes, to several biological co-expression and PPI networks. This method adds to the extensive toolkit of network analyses that have been used in the last two decades to extract useful biological information from omics data. In particular, the authors identify subgraphs containing maximum differences in connectivity between two networks, and basically use functional annotations to assign biological meaning to these differences. Of note, contrast subgraphs is not the only method that provides 'node identity awareness' when comparing networks. For instance, identification of network modules or community partitions are common methods to identify groups of nodes that highlight potentially relevant structural differences between two networks, and have been applied to many biological and other types of networks.I find the manuscript well motivated and clearly written in general, but lacking detailed information on part of the Methods. The discussion connecting their findings on structural differences between networks to potential biological functions is also a bit vague and could be worked out in more detail. I feel that the paper is potentially acceptable in GigaScience after a revision to provide more details on the methods and on their findings. Here are my comments:Methods:1.- Coexpression networks for luminal and basal cancer subtypes:1a.- The authors don't give enough information about the data they are using to build these networks. How many samples/points are they using to calculate correlations? Do they correspond to different patients, expression dynamics after some treatment…? Is there any preprocessing in the data (e.g. differential expression with respect to healthy tissue) or they just take all quantified transcripts and proteins with minimal filtering (they only specified that filter out genes with FPKM < 1 in more than 50 samples in transcriptomic data)? How many nodes and links have the final coexpression networks?.1b.- To determine links between genes/proteins they calculate Spearman rho and transform it to (0.5(1+rho)^12 to give a 'signed' network. But since Spearman correlation ranges between +1 and -1, this transformed quantity lies between 0 and 1, so I don't see the sign. Moreover, why the exponent 12 in the transformation??. Please clarify because I don't know if they are analyzing just weighted networks, unweighted networks or signed networks in the end because somehow they 'keep track' of the sign of rho. They spend some space in Methods discussing the extension of the contrast subgraph method to sign networks, but I don't know if they finally apply it, since coexpression networks built in this way and PPI networks are not signed.1c.- Do they keep all links or use some cutoff in rho by magnitude/significance? Presumably yes, because otherwise the final network would be a clique and unmanageable, but they don't give any info on that. Again, which is the final size (node/links) of the coexpression networks?1d.- As for coexpression networks based on relative abundance data as those from transcriptomic/proteomic experiments, it is well known that correlations may be misleading due to the possible large number of spurious correlations (see for instance Lovell at al., PLoS Computational Biology 11(3) (2015) e1004075). The use of correlations requires some justification, and at least to acknowledge the potential pitfalls of this measure.1e.- How many nodes/links are in the first contrast subgraphs shown in Figures 1-2? Is the degree calculated within the whole network or just within the extracted subgraph?1f.- Page 4, last paragraph before 'Protein vs mRNA coexpression in breast cancer' section: 'the results obtained with the two independent breast cancer cohorts show good agreement, with the top differential subgraphs significantly overlapping for both the basal-like and the luminal-A subtypes (Fisher test p < 2.2 · 10-16)'. I guess the overlapping is in terms of functional annotations, how is this overlapping and the corresponding statistical test calculated?.2.- Protein versus mRNA coexpression:2a.- Please provide again information about the number of samples, how the 'subset of breast cancer patients included in the TCGA' is chosen and if transcriptome and proteome are quantified in the same conditions (relevant if one is directly to compare both networks). Provide also details about the number of link/nodes of each subnetwork and corresponding subgraph. Since transcriptomic data are provided usually in FPKM and proteomic in counts (sum of normalized intensities of each ion channel), are data further normalized to facilitate their comparison?3.- PPI networks:3a.- Since they are going to compare PPIs about different 'contexts', a brief explanation about the tissue origin and peculiarities of the three cell lines investigated is in order.3b.- Please provide details about number of proteins/interactions in the contrast subgraphs obtained from the comparisons of the three cell lines. Since these subgraphs are going to be compared to RNA expression data from a different dataset, please specify if these data are obtained from the same cell lines. Why PPI data are compared only to upregulated genes? (and not to up-down regulated). Also, concerning the criterion for 'upregulation' (logFC>1), is this log base 2?. How do they quantify the overlap between proteins in PPI and upregulated genes? They just state that 'did indeed significantly overlap the corresponding up-regulated genes'. How much is the overlap and what does 'significantly' mean?3c.- Discussion of the results shown in Figure 4 is not clear to me. First, the authors state 'We thus analyzed in more depth the first contrast subgraphs obtained from the comparison of the HEK293T PPI network with those obtained from the other two cell lines'. Does this mean that they analyze four subgraphs (2 for HEK vs. HUVEC and 2 for HEK vs. Jurkat?. When they say that the 'top contrasts subgraphs were identical', do they mean that the four subgraphs contained exactly the same nodes?. Also, in main text Figure 4 seems to contain the subnetwork of these subgraphs with only the nodes annotated as 'ribosome biogenesis' and 'signal transduction through p53', and the links would be the PPIs. But in the caption to Figure 4 they state that 'green edges join proteins involved in the two biological processes' (probably a subset of the PPIs). Please clarify. Why do they give only the comparison between HEK and HUVEC, and not between HEK and Jurkat if the same nodes are present?Interpretation of results:1.- Coexpression networks in two cancer subtypes: they find that the subgraph with the stronger connections in the basal subtype is enriched in 'immune response' and the subgraph denser in the luminal subtype is enriched in categories related to microenvironment regulation. If they identify clearly enriched genes they should discuss in more depth their known roles in connection to these two functions in their biological context. This would enrich and support their findings. It is tempting to speculate that, since the basal type is less aggressive, cancer cells are challenged by the immune system of the organism but, once they developed mechanisms to evade the immune system (becoming more aggressive as in the luminal subtype) they are committed to manipulate their microenvironment to proliferate. Are there any evidences for this in these subtypes of cells?2.- Comparison of transcriptomic and proteomic networks: From their analyses in Figure 3 they claim in the Discussion that 'adaptive immune system genes are more connected at the transcriptional level, while innate immune systems are more connected at the proteomic level'. This is a rather vague statement based on the functional enrichment analysis. First, they should identify and discuss in more detail the genes/proteins responsible for this enrichment, to see if their documented function supports their speculations (and since the data they use are from breast cancer, I don't know how general could be this observation of if it is specific of this type of tumor). Moreover, caution should be exerted when interpreting these coexpression networks: the most connected transcripts are not necessarily those who are being simultaneously translated. Also, since apparently the network is not signed the abundance of connected transcripts may be anticorrelated. Finally, Figure 3 is not clear: which panel corresponds to the transcriptomic subgraph and which one to the proteomic one? This should be specified in the caption or with titles in the panel.Minor comments:- The distinction between 'heterogeneous' and 'homogeneous' networks in the Introduction is a bit confusing, as they classify mRNA and protein coexpression networks as 'heterogeneous'. Why is that? Is that because they are built from many different samples/individuals or time course data?.- Although I have nothing against how the authors display differences between the first contrast subgraphs in panels A-B of Figures 1 and 2, it may be more eye-catching to display these differences as usual boxplots or violin plots, with perhaps the test for significant differences between the means of both degree distributions.

This work has been peer reviewed in GigaScience (see paper https://doi.org/10.1093/gigascience/giad010), which carries out open, named peer-review. These reviews are published under a CC-BY 4.0 license and were as follows:

Reviewer name: De-Shuang Huang

The authors proposed an algorithm based on contrasting subgraphs to characterize the biological networks, so as to analyze the specificity and conservation between different samples. It is interesting and I think there are some problems that need to be clarified.1, Sub-graphs are generated by dividing the whole graph in a certain way, and the similarity and difference of the samples are described by the comparison between the sub-graphs. The authors should discuss the advantages of the proposed approach in a non-heuristically way compared with the previous methods. Besides that, I wonder why subgraphs need to be non-overlapping.2, For TCGA or other databases, I think the authors should state the details of the samples, such as the number of samples, sequencing technology, batch effects, etc. In addition, the authors should describe the relationship between the subgraphs and GO modules to explain the results and draw some biological conclusions.3, The authors performed a similar analysis on protein networks and compared the results with RNA-seq, and get some conclusions. I'm a little confused whether the GO enrichment analysis of proteomics is to map the protein ID to the gene ID. If so, the authors can easily combine transcript co-expression and protein co-expression networks through ID-to-ID mapping, and I look forward to the results of such an analysis.4, I would like to know how the proposed method handles heterogeneous graphs by treating heterogeneous graphs as Homogeneous graph to generate subgraphs? I didn't figure out which dataset is the heterogeneous graph scenario.5, In addition to the elaboration of results such as degree and density differences between subgraphs, I would like to see the relationships between these results and the biological problems.6, Authors may consider citing the following articles on networks in molecular biologyBarabasi A L, Oltvai Z N. Network biology: understanding the cell's functional organization[J]. Nature reviews genetics, 2004, 5(2): 101- 113.Zhang, Q., He, Y., Wang, S., Chen, Z., Guo, Z., Cui, Z., ... & Huang, D. S. (2022). Base-resolution prediction of transcription factor binding signals by a deep learning framework[J]. PLoS computational biology, 2022, 18(3): e1009941.Hu J X, Thomas C E, Brunak S. Network biology concepts in complex disease comorbidities[J]. Nature Reviews Genetics, 2016, 17(10): 615-629.Z.-H. Guo, Z.-H. You, Y.-B. Wang, D.-S. Huang, H.-C. Yi, and Z.-H. Chen, "Bioentity2vec: Attribute-and behavior-driven representation for predicting multi-type relationships between bioentities." GigaScience 9.6 (2020): giaa032.Z.-H. Guo, Z.-H. You, D.-S. Huang, H.-C. Yi, K. Zheng, Z.-H. Chen, Y.-B. Wang, MeSHHeading2vec: a new method for representing MeSH headings as vectors based on graph embedding algorithm[J]. Briefings in bioinformatics, 2021, 22(2): 2085-2095.

Reviewer names: Alban Gaignard (Report on revision 1)

The reading of the revised paper would have been easier by providing updates in a different color but thank you for taking into account the comments and remarks, and clearly answering the raised issues. I also appreciated the extension of the discussion. However, I still have some concerns regarding the proposed approach. The proposed platform targets both workflow sharing and testing. It is explicitly stated in the abstract: "the validation and test are based on the requirements we defined for a workflow being reusable with confidence". It is clear in the paper that tests are realized through the GitHub CI infrastructure, possibly delegated to a WES workflow execution engine. Although I inspected Figure 3 as well as the wf_params.json and wf_params.yml provided in the demo website. It doesn't seem to be enough to answer questions such as: how are specified tests ? How can a user inspect what has been done during the testing process ? What is evaluated by the system to assess that a test is successful ? I tried to understand what was done during the testing process but the test logs are not available anymore (Add workflow: human-reseq: fastqSE2bam · ddbj/workflow-registry@19b7516 · GitHub) Regarding the findability of the workflows, in line with FAIR principles, the discussion mentions a possible solution which would consists in hosting and curating metadata in another database. To tackle workflow discoverability between multiple systems, accessible on the web, we could expect that the Yevis registry exposes semantic annotations, leveraging Schema.org (or any other controlled vocabulary) for instance. This would also make sense since EDAM ontology classes are referred to in the Yevis metadata file (https://ddbj.github.io/workflow-registry-browser/#/workflows/65bc3bd4-81d1-4f2a8886-1fbe19011d81/versions/1.0.0).

This work has been peer reviewed in GigaScience (see paper https://doi.org/10.1093/gigascience/giad006), which carries out open, named peer-review. These reviews are published under a CC-BY 4.0 license and were as follows:

Reviewer name: Kyle Hernandez

Suetake et. al designed and developed a system to publish, validate, and test public workflows utilizing existing standards and integration with modern CI/CD tools. Their design wasn't myopic, they relied heavily on their own experiences, work from GA4GH, and interacting with the large workflow development communities. They were inspired by the important work from Goble et. al that applies the FAIR standards to workflows. As someone who had a long history of workflow engine development, workflow development, and workflow reusability/sharing experience I greatly appreciate this work. There are still unsolved problems, like guidelines on how to approach writing tests for workflows for example, but their system is one level above this and focuses on ways to automate the validation, testing, reviewing/governance, and publishing into a repository to greatly reduce unexpected errors from users. I looked through the source code of their rust-based client, which was extremely readable and developed with industry-level standards. I followed the read me to setup my own repositories, configure the keys, and deploy the services successfully on the first walk through. That speaks to the level of skill, testing, and effort in developing this system and is great news for users interested in using this. At some level it can seem like a "proof of concept", but it is one that is also usable in production with some caveats. The concept is important and implementing this will hopefully inspire more folks to care about this side of workflow "provenance" and reproducibility. There are so many tools out there for CI/CD that is often poorly utilized by academia and I appreciate the author's showing how powerful they can be in this space. The current manuscript is fine and will be of great interest to a wide ranging set of readers, I only have some non-binding suggestions/thoughts that could improve the paper for readers: 1. Based on your survey of existing systems, could you possibly make a figure or table that showcases the features supported/not supported by these different systems, including yours? 2. Thoughts on security/cost safeguards? Perhaps beyond the scope, but it does seem like a governing group needs to define some limits to the testing resources and be able to enforce them. If I am a bad actor and programmatically open up 1000 PRs of expensive jobs, I'm not sure what would happen. Actions and artifact storage aren't necessarily free after some limit. 3. What is the flow for simply updating to a new version of an existing workflow? (perhaps this could be in your docs, not necessarily this manuscript). 4. CWL is an example of a workflow language that developers can extend to create custom "hints" or "requirements". For example, seven bridges does this in cavatica where a user can define aws spot instance configs etc. WDL has properties to config GCP images. It seems like in these cases, tests should only be defined to work when running "locally" (not with some scheduler/specific cloud env). But the author's do mention that tests will first run locally on the user's environment, so that does kind of get around this. 5. For the "findable" part of FAIR, how possible is it to have "tags" of sort associated with a wf record so things can be more findable? I imagine when there is a large repository of many workflows, being able to easily narrow down to the specific domain interest you have could be helpful.

Reviewer names: Alban Gaignard

The reading of the revised paper would have been easier by providing updates in a different color but thank you for taking into account the comments and remarks, and clearly answering the raised issues. I also appreciated the extension of the discussion. However, I still have some concerns regarding the proposed approach. The proposed platform targets both workflow sharing and testing. It is explicitly stated in the abstract: "the validation and test are based on the requirements we defined for a workflow being reusable with confidence". It is clear in the paper that tests are realized through the GitHub CI infrastructure, possibly delegated to a WES workflow execution engine. Although I inspected Figure 3 as well as the wf_params.json and wf_params.yml provided in the demo website. It doesn't seem to be enough to answer questions such as: how are specified tests ? How can a user inspect what has been done during the testing process ? What is evaluated by the system to assess that a test is successful ? I tried to understand what was done during the testing process but the test logs are not available anymore (Add workflow: human-reseq: fastqSE2bam · ddbj/workflow-registry@19b7516 · GitHub) Regarding the findability of the workflows, in line with FAIR principles, the discussion mentions a possible solution which would consists in hosting and curating metadata in another database. To tackle workflow discoverability between multiple systems, accessible on the web, we could expect that the Yevis registry exposes semantic annotations, leveraging Schema.org (or any other controlled vocabulary) for instance. This would also make sense since EDAM ontology classes are referred to in the Yevis metadata file (https://ddbj.github.io/workflow-registry-browser/#/workflows/65bc3bd4-81d1-4f2a8886-1fbe19011d81/versions/1.0.0).

Reviewer name: Samuel Lampa

The Yevis manuscript makes a good case for the need to be able to easily set up self-hosted workflow registries, and the work is a laudable effort. From the manuscript, the implementation decisions seem to be done in a very thoughtful way, using standardized APIs and formats where applicable (Such as WES). The manuscript itself is very well written, with a good structure, close to flawless language (see minor comment below) and clear descriptions and figures.

Main concern

I have one major gripe though, blocking acceptance: The choice to only support GitHub for hosting. There is a growing problem in the research world that more and more research is being dependent on the single commercial actor GitHub, for seemingly no other reason than convenience. Although GitHub to date can be said to have been a somewhat trustworthy player, there is no guarantee for the future, and ultimately this leaves a lot of research in an unhealthy dependenc on this single platform. As a small note of a recent change, is the proposed removal of the promise to not track its users (see https://github.com/github/site-policy/pull/582). A such a central infrastructure component for research as a workflow registry has an enormous responsibility here, as it may greatly influence the choices of researchers in the future to come, because of encouragement of what is "easier" or more convenient to do with the tools and infrastructure available. With this in mind, I find it unacceptable for a workflow registry supporting open science and open source work to only support one commercial provider. The authors mention that technically they are able to support any vendor, and also on-premise setups, which sounds excellent. I ask the authors to kindly implement this functionality. Especially the ability to run on-premises registries is key to encourage research to stay free and independent from commercial concerns.

Minor concerns

1. I think the manuscript is a missing citation to this key workflow review, as a recen overview of the bioinformatics workflows field, for example together with the current citation [6] in the manuscript: Wratten, L., Wilm, A., & Göke, J. (2021). Reproducible, scalable, and shareable analysis pipelines with bioinformatics workflow managers. Nature methods, 18(10), 1161-1168. https://www.nature.com/articles/s41592-021-01254-9
2. Although it might not have been the intention of the authors, the following sentence sounds unneccessarily subjective and appraising, without data to back this up (rather this would be something for the users to evaluate):

   The Yevis system is a great solution for research communities that aim to share their workflows and wish to establish their own registry as described. I would rather expect wording similar to: "The Yevis system provides a [well-needed] solution for ..." ... which I think might have been closer to what the authors intended as well. Wishing the authors best of luck with this promising work!

This work has been peer reviewed in GigaScience (see https://doi.org/10.1093/gigascience/giad002), which carries out open, named peer-review. These reviews are published under a CC-BY 4.0 license and were as follows:

Reviewer 1: Jonathan Coddington

This paper presents the first uloborid spider genome--and it is a chromosome level assembly. Genomes of this family are important because the orb web is supposedly independently and convergently evolved in this group. Although my expertise is not in the technology and informatics of genome sequencing, it appears to be well done.

Figure 1 A. geniculate -- spelling N. clavipes = T. clavipes Table S1 Number of Componenet Sequences-- typo Text single exon We found a -- typo can be ascribed by -- can be inferred by? an Araneid orb-weaver-- araneid usually not capitalized ♂X1X2/♀X1X1X2X2.[48] should be ♂X1X2/♀X1X1X2X2 [48]. You might want to be careful about citing Purcell & Pruitt, see https://purcelllab.ucr.edu/blog6.html and other questions about Pruitt's work.

Re methods, it would be of interest to know what HMW DNA fragment sizes were (expressed as kb, or mb), although Tape Stations are not very accurate. For people who collect spiders with the intent to yield HMW DNA, such data are important. Data are scarce, so any facts are significant.

Any homologs of the Pyriform spidroin (PySp) in Acanthoscurria? Piriform silk attachment points are a synapomorphy of araneomorph or "true" spiders. Liphistiomorph and mygalomorph spiders do not (cannot?) make point attachments, and the inability to make point attachments either to substrate or silk-silk point attachments probably constrains/ed the evolution of web architectures in non-araneomorph spiders. Therefore finding homologs to PySp spidroins in non-araneomorph spiders is of great interest to explain araneomorph web architecture diversity.

Likewise, tubuliform spidroin (TuSp) is probably a synapomorphy of entelegyne spiders, with derived female genitalia--a "flow-though" sperm management system. Eggsacs occur widely in non-entelegyne spiders, so it is a mystery why entelegynes have specialized spigots, glands, and spidroins for the same purpose. Indeed, the particular function of tubuliform silk is not clear. Any thoughts on this? E.g.

It is good to see attention paid to the mitochondrial genome, as many whole genome studies ignore it. In spiders, early work claimed that tRNA's appeared to be peculiar. Masta and Boore. 2004. The Complete Mitochondrial Genome Sequence of the Spider Habronattus oregonensis Reveals Rearranged and Extremely Truncated tRNAs. Molecular Biology and Evolution, Volume 21, Issue 5, May 2004, Pages 893-902. Any comments on U. diversus tRNAs from that point of view?

Finally, any comments on evidence for or against the convergent evolution of the orb web? Homology between the pseudoflagelliform and flagelliform spidroins would be pertinent. The intro does raise expectations that some of the macro / larger evolutionary questions will be addressed in the paper, but many, see above, are only cursory or not too much. Perhaps include a sentence in intro acknowledging this, but saying that this paper intends to present the genome and address sex chromosomes, but other topics? For example the sections on some of the spidroins do not extensively discuss comparisons with other spider genomes.

Reviewer 2: Hui Xiang

In this study, the authors generated huge genome sequencing data and RNA-seq data and provided a genome assembly with rather complicated merging approach, of a spider with novel phylogenetic position. The genome undoubtedly added novel and important resources for deep understanding of spider evolution. However, there are still severe issues that need to be addressed. 1. There are huge sequencing data from different samples. However, I don't think that marge of different assemblies is good for a final qualified genome. Given high heterozygosity, that illumina data and ONT data from different individuals is quite difficult to use for assembling a clean genome. As shown in Table 2, assembly by Hify approach is not obviously inferior compared with the merged one, but obviously much better in avoiding redundancy. I strongly suggest that the author adopt the genome assembly of Hify data from one individual, instead of merging two sets of assemblies. Illumina and Nanopore assembly may be helpful in fully deciphering silk proteins. 2. Proportion of repeats are somewhat affected by the quality of assembly. The high heterozygous genome assembly is complicated merged by diverse batch of data, so the real quality might be not as good as the author described. The quality of repeat is especially hard to evaluate. Hence the statements on genome size (Line 193-200) are not convictive. 3. About the assembly of RNA-seq data. The authors get huge amounts of data. However, it is not so helpful to obtain novel transcripts if the data is saturated. More importantly, assembly of short reads is even not so useful to obtain long transcripts. 4. As to whole genome duplication. The authors did not provided solid evidence supporting that WGD occurred in U. diversus genome. They only demonstrated two hox clusters therein. The synteny analysis was quite confusing which is not helpful in confirmation of WGD. They need to provide more solid genome-wide evidence, or otherwise totally downplay the statements. 5. The identification of the sex chromosome is still vague. The statements are not well organized. The statements and the results are so vague and not convictive. "While 8 of the 10 pseudochromsomes had a median read depth of 40 ± 2, pseudochromosomes 3 and 10 were outliers, with read depths of 36 and 33, respectively." The difference in sequencing depth is rather convictive. As I know the authors sequenced female and male samples. So why they didn't clearly compare the depth of the two sex chromosomes between them and make more evidence? Other: 1. The information of chromosome-level spider genome are not Incomplete. As I know, there is a black widow genome with chromosome-level. The authors need to added this one. 2. The authors need to release the sequences of the spidroins the identified and described.

Reviewer 3: Zhisheng Zhang, Ph.D

The manuscript GIGA-D-22-00169 presents a chromosome-level genome of the cribellate orb-weaving spider Uloborus diversus. The assembly reinforces evidence of an ancient arachnid genome duplication and identifies complete open reading frames for every class of spidroin gene. And the authors identified the two X chromosomes for U. diversus and identify candidate sex-determining genes.

The methods of work are well fited to the aims of the study, clearly described, and well written.

Minor comments:

1. In the Figure 1B, I noticed that it noted the estimated divergence times of the Araneae, I think there should be add the reference, or detail describe how to do.

2. There is something wrong with the table format, such as Table1, 2, 5 and Table 6.

3. Line 70: "chromosome- scale" changes to "chromosome-scale".

4. Line 147 to lines 148: Line breaks error.

5. Line 458: "[48]" in the wrong location.

6. Line 511-512: In the genome of spider Uloborus diversus, which chromosome the genes of "sex lethal (sxl)" and "doublesex (dsx)" located at?

7. Line 515-516: "The 534 shared sex-linked genes in these three species, 14 are predicted to be DNA/RNA-binding", if these sex-linked genes have difference on RNA level between male and female?

8. Line 685: "Dovetail Chicago and Dovetail Hi-C Sequencing" should be bold.

9. Line 764: "We then used the Trinity assembler43 v.2.12.0", the number of 43 may be redundancy.

10. Some softwares lack the number of RRID, such as line 223 "BRAKER2", line 245 of "NOVOplasty", line 790 of "tRNAscan-SE", line 773 of "RepeatModeler", line 774 of "RepeatMasker", line 797 of "EMBOSS", and so on.

11. Lines 780 "using the BRAKER 2 pipeline" changes to "using the BRAKER2 pipeline".

12. Lines 950: "Literature Cited" changes to "Reference".

13. Lines 952-953: wrong cite. The World Spider Catalog is a web online, the version and the data you accessed from should also added, and the author's name should change to World Spider Catalog.

This work has been peer reviewed in GigaScience (see https://doi.org/10.1093/gigascience/giac128), which carries out open, named peer-review. These reviews are published under a CC-BY 4.0 license and were as follows:

Reviewer 1: Saurabh V Laddha

Authors did a fantastic job by integrating MPM multi-omics datasets and created an integrative and interactive map for users to explore these datasets. MPM is a rare cancer type and understudied so such resources are very useful to move the field forward at a molecular level. The comprehensive data is well presented and the manuscript is well written to explain the complex genomics dataset for MPM. All the figures are well explained and very clear to understand

Minor point: - Author mentioned an evaluation of tumor purity was done using pathological review, did author used molecular data such as genomic data to find tumor purity ? and if yes, how was the consensus ? This is very important factor to interpret the genomic results as the data was sequenced at 30X - In the same line, RNAseq can also be used to identify tumor purity and it will be really helpful for users to clear picture on tumor purity. - Is it not very clear from method section that the same MPM samples were used to sequence at DNA , RNA and DNA methylation level ? A brief explanation or table will be very easy for users to understand. - Recent WHO classify MPM into three different histopathological types. Did author do any unsupervised analysis from these comprehensive data to understand MPM heterogeneity or replicate WHO classification? or did author find WHO subtypes of MPM using molecular dataset ? A brief analysis/comment on usage of histological classification Vs Molecular classification will certainly move the MPM research field forward as researcher have found vast differences between histological vs molecular classification and the field is moving towards more molecular based classification in clinic.

Reviewer 2: Jeremy Warner

In this paper, the authors describe a new public resource for the molecular characterization of malignant pleural mesothelioma (MPM), which they describe as the most comprehensive to date. They perform WGS, transcriptome, and methylation arrays for 120 patients with MPM sourced through the MESOMICS project and integrate this dataset with an additional several hundred patients from previously published datasets.

Although I cannot independently verify their claim that this is the largest and most comprehensive dataset for MPM, it is quite impressive and expansive. The pipeline utilized is well described and the results at all stages are transparently shared for prospective users of this dataset.

The description of the methods to identify and remove germline variants is interesting, although the length somewhat detracts from the main goal of the paper in describing an MPM resource. Perhaps, this part could be condensed with the technical details presented in supplement. This comment pertains to both the Point Mutations and Structural Variants sections.

Additional moderate concerns:

There are insufficient details provided on the clinical and epidemiological parameters. Indirectly, it would appear that sex, age class, and smoking status are the clinical parameters - but what are the age classes? Is smoking status binary ever/never, or more involved? There ought to be a data dictionary provided as a supplemental table which describes each clinical/epidemiological variable, along with the possible values that the variable can take on. It should additionally be explained why other important clinical parameters are not available - most importantly, the presence of accompanying pulmonary comorbidity such as chronic obstructive pulmonary disease (COPD) and the existence of conditions that might preclude the use of standard systemic therapies, such as renal disease precluding the use of platinum agents.

Context: I would like to see more here about the role of asbestos in the etiology, including what might be known about the pathophysiology of asbestos fibers at the molecular level. Also, there is nothing here about the evolution of treatment for MPM; the latest "state-of-the-art" regimens (platinum doublet + bevacizumab [MAPS; NCT00651456] and dual checkpoint inhibition [Checkmate 743; NCT02899299]) have reported median survival in the 18-month range, which is distinctly better than the median survivals quoted by the authors. Finally, I would like to see one or more direct references to the clinical trials where molecular heterogeneity has "fueled the implementation of drug trials for more tailored MPM treatments".

Data Description: All specimens in the MESOMICS study are said to be collected from surgically resected MPM; this also appears to be the case for the integrated multi-omic studies from Bueno et al. and Hmeljak et al. and this should be explicitly indicated. Somewhere, it should also be explicitly discussed that this is an important limitation in the future utility of this data - surgical specimens are convenience samples and while they do provide important information, they lack treatment exposure. Given that many if not most patients with MPM will survive to 2nd or 3rd line systemic therapy, and that 1st line is fairly standardized, a knowledge of induced mutations is going to be essential to the ultimate goal of precision medicine.

Minor concerns:

The labels in the figures (e.g., Figure 2 - "Unmapped..too.short") are human-readable but could still be presented in a more friendly fashion. All acronyms should be defined.

Reviewer 3: Mary Ann Tuli

I have been asked to review the process of accessing the controlled data cited in this study to ensure that the process is clear and smooth. The study is available from the European Genome-phenome Archive (EGA) under accession number EGAS00001004812 (https://ega-archive.org/studies/EGAS00001004812). The paper is clear about how to obtain the DAA.

The study has three datasets.

I can confirm that the author was very prompt in his response to me requesting the DAC, in providing the DAA and in responding to the queries I had when completing the DAA. The completed DAA was sent to the EGA by the author on 29-Jul, and EGA responded within 3 working days, stating access had been granted. This is an excellent response time, so I conclude that the process of obtaining the DAA and the EGA making the data available to the user is very good.

Today (1-Sep) I have attempted to gain access to the data via EGA. I was easily able to login to my EGA account and see that the datasets are available to me to download. Users need to download data using the EGA download client - pyEGA3. EGA provides a video on how to install the client, but I hit a problem and require technical support.

I emailed the EGA help desk but have not had a response yet. I was quite surprised to receive a response from the author and have learnt that EGA include the owner of the study in RT tickets so they see any communication. I commend the author for his prompt response to my ticket (though it didn't solve my problem).

I cannot hold on to this review for any longer, and I am not yet in a position to comment on the nature of the data held within this study.

I do have concerns that the process of accessing controlled data held in the EGA is not straight forward. Users need to watch a 12 minute video to learn how to install the download client and may need to install programs on their computer). There is a FAQ which is very technical. This is not an issue for the author to resolve though.

I understand the author has some minor revisions to make, so hopefully I should have a response from the EGA help desk before a final decision needs to be made (?).

This work has been peer reviewed in GigaScience (see https://doi.org/10.1093/gigascience/giac126), which carries out open, named peer-review. These reviews are published under a CC-BY 4.0 license and were as follows:

Reviewer 1: Shiping Liu

How to model the statistical distribution of the gene expression, is a basic question for the field of single cell sequencing data mining. Dharmaratne and colleagues looked details at the distribution of very gene. By using the generalized linear models (GLM), the authors present a new program scShapes, which matched a specific gene with a distribution from one of the four shapes, Poisson, Negative Binomial (NB), Zero-inflated Poisson (ZIP), and Zero-inflated Negative Binomial (ZINB). As the authors present in this manuscript, not all genes adapted to a single distribution, neither NB or Poisson, and some of the genes actually adapted to the zero-inflated models because of the property of high drop-out rate in the modern single cell sequencing, says 3' tag sequenced. It is has been popular to employ GLM in single cell data mining recently, but it also got both praise and blame. So it is a good forward step to model a specific model for an individual gene. But the bad side is the computing cost, especially for the number of cells been sequenced reach to millions in currently research, and it believed that the dataset will be reached even bigger in the future. So it make a great obstacle arise to the application of the method presented by the author here. How to speed up the calculation using the mixed model or scShapes? The authors also performed the scShapes on some datasets, including the metformin, human T cells, and PBMCs. They found some potential genes that changed the distribution shape, but didn't easy to be identified by other methods. It demonstrated that scShapes could identified the subtle change in gene expression.

Major points: (1) We didn't see any details about the metformin dataset, the segueing depth and quality, number of genes/UMIs per cell, and so on. It makes hard to evaluate the quality and reliability of the results generated by scShapes. If this dataset is another manuscript could not possible to be presented at the same time, I suggest the author could perform on alternative dataset, as there are so many single cell datasets has been published could be used in this study.

(2) Even the authors taken the cell type account in the GLM, I wonder for a specific gene, whether the distribution shape will change in different cell type. If so, it will becoming more complex, that is need to model the distribution shape for individual gene for every cell type alone.

(3) To identify the different gene expression in scShapes, the author didn't consider the influence of different cell number, or the proportion of cell number, in the different cell type. A possible way to evaluate or eliminate this bias is to down sampling from a big dataset, instead of just simulated total number 2k ~ 5k from the PBMC. To evaluate the influence both the total number cell and the proportion in cell type.

(4) The author should present the comparative results of the computational cost for different methods. Says the accuracy, time and memory consuming under different number of cells. I suggest the authors use much a larger dataset, because currently single cell research may include millions of cells, and the ability to process big data is very important to the application and becoming a widely used one.

Minor points: (1) No figure legends for Fig.2 c and d.

(2) It is unclear whether the total 30% genes undergo shape change, or just the proportion of the remaining after the pipeline. So please clarify the details.

Reviewer 2: Yuchen Yang

In this manuscript, authors presented a novel statistical framework scShapes using GLM approach for identifying differential distributions in genes across scRNA-seq data of different conditions. scShapes quantifies gene-specific cell-to-cell variability by testing for differences in the expression distribution. scShapes was shown to be able to identify biologically-relevant switch in gene distribution shapes between different conditions. However, there are still several concerns required to be addressed.

1. In this study, authors compared scShapes to scDD and edgeR. However, besides these two, there are many other methods for calling DEGs from scRNA-seq. Wang et al. (2019) systematically evaluated the performance of eight methods specifically designed for scRNA-seq data (SCDE, MAST, scDD, D3E, Monocle2, SINCERA, DEsingle, and SigEMD) and two methods for bulk RNA-seq (edgeR and DESeq2). Thus, it is also worthy to compare scShapes to other methods, such as SigEMD, DEsingle and DESeq2, which were supposed to perform better than scDD or edgeR.

2. When scShapes was compared to scDD, authors mainly focused on the distribution shifting. However, to users, it would be better to present a venn diagram showing the numbers of the genes detected by both scShapes and scDD, and the genes specifically identified by scShapes and scDD, respectively. In addition, authors showed the functional enrichment results for DEGs identified by scShapes. It is also worthy to perform enrichment analysis for the genes detected by both scShapes and scDD or specifically identified by scShapes or scDD.

3. Since scShapes detects differential gene distribution between different conditions, it would be better to show users how to interpret the significant results biologically. For example, authors mentioned that RXRA is differentially distributed between Old and Young and Old and Treated, so what does this results mean? Can this differential distribution be associated with differential expression?

4. In Discussion, authors mentioned that scRATE is another tool that can model droplet-based scRNA-seq data. It would be clearer to discuss that why authors develop their own algorithm rather than using scRATE to model the distribution.

5. In Introduction, authors talked about the zero counts in scRNA-seq data, and presented evidence in Results part. Since 2020, there are several publications also focusing on this issue, such as Svensson, 2020 and Cao 2021. These discussions should be included in this manuscript.

This work has been peer reviewed in GigaScience (see https://doi.org/10.1093/gigascience/giac125), which carries out open, named peer-review. These reviews are published under a CC-BY 4.0 license and were as follows:

Reviewer 1: Ruibang Luo

In this paper, the authors proposed xAtlas, an open-source NGS variant caller. xAtlas is a fast and lightweight caller with comparable performance with other benchmarked callers. The benchmark comparison in multiple popular short-read platforms (Illumina HiSeq X and NovaSeq) demonstrated xAtlas's capacity to identify small variants rapidly with desirable performance. Although xAtlas is limited to call multi-allelic variants, the high sensitivity (~99.75% recall for ~60x benchmarking datasets) and desirable runtime (<2 hours) enable xAtlas to rapidly filter candidates and be considered as important quality control for further utilization.

The authors presented a detailed explanation of xAtlas's workflow, design decisions and have done complete experiments in benchmarking, while there are still some points the authors need to discuss further listed as follow:

The authors reported the performance in multiple coverages of the HG001 sample and the benchmarking result of HG002-4 samples by measuring the concordance with the GIAB truth set (v3.3.2). I noticed that GIAB had updated the GIAB truth sets from v3.3.2 to v4.2.1 for the Ashkenazi trio. The updated version included more difficult regions like segmental duplications and the Major Histocompatibility Complex (MHC) to identify previously unknown clinically relevant variants. Therefore, it would be helpful if the author could give a performance evaluation using the updated truth sets to give a more comprehensive performance of the proposed caller.

In the Methods section, The authors stated the main three stages of the xAtlas variant calling process: read prepossessing, candidates identification, and candidates evaluation. The author fed hand-craft features (base quality, coverages, reference and alternative allele support, etc.) into a logistic regression model to classify true variants and reference calls in the candidate evaluation stage. But in Figure 1, the main workflow of xAtlas, only model scoring was shown, and the evaluation details were not visible. It would be useful if the authors could enrich Figure 1 to add more details to ensure consistency with Methods and facilitate reader understanding.

In Figure 2, the authors reported the xAtlas performance comparison across in HG001 dataset with other variant callers. I noticed that the x-axis was F1-score while the y-axis was true positives per second. The tendency measurement of two metrics seems irrelevant, which might confuse the readers. we suggest the authors make separate comparisons for the two metrics. (For instance, plot Precision-Recall curves for F1-score measurement and Runtime comparison of various variant callers for speed benchmarking).

Zheng, Zhenxian on behalf of the primary reviewer

Reviewer 2: Jorge Duitama

The manuscript describes a variant caller called xAtlas, which uses a logistic regression model to call SNPs after building an alignment and pileup of the reads. The manuscript is clear. The software is built with the aim of being faster than other solutions. However, I have some concerns relative to the method and the manuscript.

1. Unfortunately, the biggest issue with this work is that the gain of speed is obtained with an important sacrifice in accuracy, specially to call indels. I ran xAtlas with two different benchmark datasets and the accuracy, especially for indels and other complex regions was about 20% lower compared to other solutions. Although the difference was smaller, xAtlas is also less accurate than other software tools for SNV calling. It is well known that even a simple SNV caller can achieve high sensitivity and specificity (see results from https://doi.org/10.1101/gr.107524.110). However, several SNV errors can be generated by incorrect alignment of reads around indels and other complex regions. For that reason most of the work on variant detection is focused on mechanisms to perform indel realignment or de-novo miniassembly to increase accuracy of both SNV and indel detection. The paper of Strelka is a great example of this (https://doi.org/10.1038/s41592-018-0051-x). The manuscript does not mention if any procedure has been implemented to realign reads or to increase in some way the accuracy to call indels. This is critical if xAtlas is meant to be used in clinical settings.

2. The manuscript looks outdated in terms of evaluation datasets, metrics and available tools. Since high values of standard precision and sensitivity are easy to achieve with simple SNV callers, metrics such as the false positives per million basepair (FPPM) proposed by the developers of the synthetic diploid benchmark dataset should be used to achieve a more clear assessment of the accuracy of the different methods (https://doi.org/10.1038/s41592-018-0054-7). Regarding benchmark experiments, SynDyp should also be used for benchmarking. To actually support that xAtlas is reliable across heterogeneus datasets (as stated in the title), further datasets should be tested, as it has been done for software tools such as NGSEP (https://doi.org/10.1093/bioinformatics/btz275). In terms of tools, both DeepVariant and NGSEP should be included in the comparisons.

3. Regarding the metrics proposed by the authors, I do not think it is a good practice to merge results on accuracy and efficiency, taking into account that the accuracy in this case is lower than other solutions, and for clinical settings that is an important issue. The supplementary table should also report sensitivity and precision for indels, not only for SNVs.

4. The SNV calling method and particularly the genotyping procedure should be describe in much better detail. The manuscript describes the general pileup process, then, it mentions some general filters for read alignments and then it mentions that it applies logistic regression. However, it is not clear which data is used for such regression or in general how allele counts and quality scores are taken into account. A much deeper description of the logistic regression model should be included in the manuscript.

5. There are better methods than PCA to show clustering of the 1000g samples. A structure analysis is more suitable for population genomics data and it is more clear to show the different subpopulations.

6. Finally, about the software, genotype calls produced by the xAtlas should have a value for the genotype quality (GQ) format field to assess the genotyping accuracy. For single sample analysis the QUAL value can be used (although this is not entirely correct). However, for population VCFs, the GQ field is very important to have a measure of genotyping quality per datapoint. Regarding population VCF files it is not clear, either from the in-line help or from the github site, how population VCF files should be constructed.

This work has been peer reviewed in GigaScience (see https://doi.org/10.1093/gigascience/giac112), which carries out open, named peer-review. These reviews are published under a CC-BY 4.0 license and were as follows:

Reviewer 1: Jong Hwa Bhak

This manuscript is about assemblies of Bengal tigers. It is a great improvement over past two tiger genome assemblies. The assemblies quality is unprecedented (exceeding perhaps any feline genome in terms of contiguity).

This represented a ~50x improvement in genome contiguity (see materials and methods). PanTigT.MC.v2

What was the most important factor in this big jump of improvement in length?

the overall contiguity was better than the domestic cat reference genome

The quality comparison section is informative.

We identified the "repetitive elements" in the genome by combining both

==> repeat elements is better.

How close are the two genomes (MC & SI)?

This reviewer finds it a great contribution to existing feline genome assemblies. The authors have done all the usual QC and constructed really high quality assemblies.

Reviewer 2: Gang Li

The submitted manuscript 'Near-chromosomal de novo assembly of Bengal tiger genome reveals genetic hallmarks of apex-predation' assemble the high-quality near-chromosomal leveled reference genomes of Bengal tiger, which will be of great significance for the conservation and rejuvenation of tigers, even other endangered felids. I have some comments on this manuscript: 1. Considering this the assembled genome used the Hic technology to figure out the chromosome structure, the figure of Hic results need to be presented. While, the assemble of sex chromosome always attract attentions, especially Y chromosome of tiger. More detailed information need to be specified, such as the conserved Y chromosome genes compared to other mammals, or whether there are tiger-specific Y linked gene has been observed or not. 2. In this work, authors used four zoo-bred individuals with known pedigree to test the inbreeding index of ROH and intend to evaluate the assembly quality. But I don't find any information about these four individuals and I guess they should be Bengal tigers. If it is the case, the question is that the quantity of ROH will not be only decided by the reference quality, but also the divergence between the target resequencing date and the used reference genome. That is to say, if the resequencing data and the reference genome are all from the same tiger sub-species, Bengal tiger, the quantity of ROH supposed to be more than that of the different sub-species comparison, which may not be an appropriate method used to evaluate the assembly quality. 3. I have some advice about the evolutionary divergence calibrations. Using some other species which have closer phylogenetic relationship might be better, according to their shared similar substitution rate and generation time, for instance, other species of Panthera . 4. The format of references part need to be rechecked.