Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Referee #2

Evidence, reproducibility and clarity

The present work adds to the growing literature on sedimentation rate as a major player in the evolution of multicellularity. Via rigorous experimentation, the authors convincingly show that they can select for increase sedimentation rate and identify two mechanisms underlying this increase: incomplete cellular separation leading to multicellular groups and increases in cellular density. They also show surprising natural variation in sedimentation and argue that, along with similar evidence from other organisms, their findings cement the likely major role of sedimentation and go farther by revealing the tight genetic control that it is under.

Significance

This is a very significant study because it illuminates processes and underlying mechanisms that could have played a major role in the transition to multicellularity. Their result will likely greatly influence the conceptual and theoretical thinking and will foster additional empirical directions. My only quibble with the manuscript is that I wished for a bit more ecological context and grounding of the main findings: in that respect, both the abstract and the last paragraph of the discussion leave me wanting and occasionally puzzled. If maintaining buoyancy is such a strong selective pressure and the variation in sedimentation rate is such a challenge to it, then I think explaining a bit more exactly why sedimentation would evolve, why so much variation would exist etc etc would be really helpful to the more naive reader. Just a bit further elaboration on selective pressures (even presumed ones and even if speculative) would be helpful to put the picture together.

A more minor point:

I remember seeing a talk by Will Ratcliff a while back in which he showed that in S cerevisiae they also see the two mechanisms of increased sedimentation: increased cellular size and clumping. Yet, I didn't see a reference to that work in the context of the cell density mechanism discussion and wondered why.

Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Referee #1

Evidence, reproducibility and clarity

Hello, we wrote our review before seeing that you have special formatting requirements. We're just going to post our review in it's entirety rather than rewrite it based on these suggestions. It encompasses the above content, it's just not formatted in the suggested order. We hope that's OK!

Full review:

This manuscript makes a strong case for the evolvability of multicellular size via selection for settling rate in the icthyosporea. The use of an experimental evolution framework to assess the evolvability of multicellular phenotypes, using sedimentation rate as a selective pressure, extends the previous work of others into a new domain within the holozoan and the closest living relatives of animals. The natural, ecological significance of selection for sedimentation rate is a novel idea, and the connection between sedimentation rate and multicellular evolution in natural as opposed to contrived experimental circumstances is an interesting idea. The results are striking and well supported, with laboratory evolution rapidly adjusting both the cellular composition and the multicellular phenotypes of the organisms involved in ways that are well explained. This is an important result that brings the laboratory study of the evolution of multicellularity forward, into a different branch of the tree of life and showing its broad applicability.

Sequencing of evolved lines adds significantly to the completeness of the story. While the causal role of these mutations in the production of the observed multicellular phenotypes are not demonstrated via manipulation or breeding, this is quite understandable in the light of the unusual model organism and the observed homologies and role of the genes involved. While this is largely clear from a reading, we believe the manuscript would benefit from a brief analysis of the numerical enrichment of genes with homologs involved in cytokinesis, cell membrane composition, and cell cycle control relative to the null hypothesis of genes picked randomly from the genome. If this is beyond the scope of this research in an unusual model organism with many poorly annotated genes, then a slightly expanded verbal discussion of the potential roles of the apparent functions of these genes in the evolution of multicellular clumping would be an appropriate substitute.

We wholeheartedly recommend the publication of this manuscript with a number of minor revisions, which while not affecting the main conclusions or points of the manuscript will clarify important points, adjust small errors, and point the reader at relevant literature and concepts.

Major points:

none.

Minor points:

Line 79 - is sedimentation rate really invariably associated with multicellularization? Active swimming would seem to prevent this.

Line 164 - the precise phenotype in the evolution experiment being referred to is unclear without further context, with the ordering of paragraphs possibly needing a little work.

Line 178 - is sorting them into three classes informative? Are there different mutations associated with these, or is it just visual clumping on the numberline? Perhaps not a useful classification, but the existence of great variation is an important point to get across. A more useful classification might be those that increase sedimentation with large density changes versus exclusively by clumping.

Line 254 - excess cellular density is referred to interchangeably with density, when these are very different figures. This continues in line 269, and in the figure legends of Figure 4.

Line 341 - the rule of RCC1 homolog in other organisms could be expanded on in slightly more detail. Similarly, other mutations in this same section known to affect cytokinesis could have potential mechanisms for affecting clumping commented upon, especially given the cell membrane results in the figures.

Line 387 - awkward formatting or sentence structure, with dashes and commas.

Line 395 - this cellular process, or this evolutionary process of selection for faster settling?

Line 408 - per unit volume

Line 425 - the idea of clumpiness as ancestral is quickly put forward and dismissed within a single sentence. This could be explored in slightly more detail as an option, before concluding that what is clear is that the phenotype is easy to change.

Line 437 - sedimentation as a highly variable trait, or a highly evolvable trait?

Figure 1G, 1H: We are fairly certain that the logarithmic scale of DNA content and coenocyte volume are mislabeled. The scale that is labeled log2 in 1G in the legend goes up by factors of 2 rather than single digits. The axis is obviously logarithmic, and the log2 in the legend is superfluous and misleading. Similarly, in 1H a scale labeled as log10 goes from 1 to 30, which on a logarithmic scale would be a sphere approximately 100 kilometers wide. The numbers can remain, but the legend should remove the log10.

General:

Were there any head to head competitions performed? Not suggesting you need to, but it's a nice way to directly examine fitness consequences of multicellularity, and is commonly done in the field. If you have done this it wasn't clear to us.

Significance

see the above comments about writing the review before realizing there were specific formatting suggestions. I hope you understand us not wanting to re-write the review having already written it once.

Note: This rebuttal was posted by the corresponding author to Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Reply to the reviewers

General Statements [optional]

We thank the reviewers for their thoughtful, constructive, and highly actionable critique. The reviewers mentioned that “the experiments presented are well-designed, the methods well-implemented, and communication of the authors' findings is clear and concise”. We are happy to hear that “figure presentation and manuscript layout are top notch and... these data are easy to read and interpret”.

We appreciate reviewers’ suggestions in improving the interpretability of the morphodynamic representation and address each of the Reviewers’ comments (typeset in blue) in the document below.

Description of the planned revisions

Insert here a point-by-point reply that explains what revisions, additional experimentations and analyses are planned to address the points raised by the referees.

Reviewer # 1 (major points)

* The Trajectory Feature Vectors (TFVs) are averaged over time - this seems to lose a lot of the salient information in the trajectories themselves, resulting in the low(ish) accuracy of the GMM. Could a Hidden Markov Model trained on the trajectories in state space help to identify/classify those trajectories that change their morphology/motion over time?

Thanks for the suggestion. We did recognize that averaging will smooth the dynamics in each cell trajectory and reduce diversity of phenotypes. On the other hand, the temporal smoothing serves to reduce the noise, especially when the cells have reached steady state dynamics after being stimulated with pro- or anti-inflammatory cytokines. Our experiments were constructed to probe steady state dynamics and therefore we opted to use temporal smoothing.

It is possible to identify rare transitions even with some temporal smoothing.

In our analysis of rare transitions (Fig. 4C), we extracted long trajectories and split them into segments (10~15 frames, 1.5~2 hours). By applying Gaussian Mixture Model (GMM) to each segment, we identified a sequence of states along the full trajectory, from which state transitions were identified.

During the revision, we will employ the Hidden Markov Model (HMM) to model state transitions in the latent shape space as suggested by the reviewer to detect rare transitions. Our expectation is that HMM will be able to identify more transition events due to its higher time resolution (frame instead of segment), though it may also be affected by unexpected imaging artifacts and noise.

Reviewer # 1 (minor points)

Could the authors provide some example images showing interpolation of each PC using the generative decoder?

Thanks for the suggestion, however the discrete nature of the latent codebook of VQ-VAE makes it challenging to use interpolation as a proxy for utility of interpolation. A possible link between interpolation abilities and usefulness of representation learned by autoencoders has been explored in this paper by Berthelot et al. As Berthelot et al. note, “We perform interpolation in the VQ-VAE by interpolating continuous latents, mapping them to their nearest codebook entries, and decoding the result. Assuming a sufficiently large codebook, a semantically “smooth” interpolation may be possible. On the lines task, we found that this procedure produced poor interpolations. Ultimately, many entries of the codebook were mapped to unrealistic datapoints, and the interpolations resembled those of the baseline autoencoder.”

Reviewer # 2 (major points)

-It's unclear what the effect of speed is on the final state determination. TFVs were composed of auto-encoder-based features (PCs from latent space) and speed of the cells. Would the states be very different without speed as part of the TFVs or with TFVs consisting only of speed features? Please quantify and discuss.

Thanks for your comment. We agree that speed of the cell is a main factor that contributes to the clustering, though shape features (from VQ-VAE) do contribute (Fig. 3B, histograms) to discrimination of cell states. In the revision, we will perform the clustering analysis with only shape features and compare with current results of Fig. 4.

Reviewer # 3 (major points)

1. Temporal consistency regularization

In the authors' framework, models are regularized to minimize the l2 norm between embeddings of adjacent timepoints.

This is approach is conceptually well-motivated, but could have some unintended effects.

For instance, some cells may make a rapid state transition such that state(t-1) = A, state(t) = B, state(t+1) = A'.

In these cases, a regularized model may best minimize the joint loss by returning an embedding at time t that interpolates between state A and A', rather than returning an embedding that reflects the true distinct state B.

The work would be strengthened if the authors analyzed the impact of this regularization term on the detection of rapid state transitions that occur for only a few frames (e.g. when cells that exhibit filopodial motility "jump" in an actin/myosin contraction).

This might be accomplished through experiments scanning different regularization hyperparameters on some of the authors' real data, fitting models on temporally downsampled versions of the real data where "slow" multi-timestep transitions now occur in a few timesteps, or perhaps using simulations where rapid state transitions are known to occur.

Even if the regularization does have some negative impacts, it does not argue against the utility of the general approach, but it is important for users to understand the constraints on downstream applications.

In our revision, we will evaluate the optimal matching loss for our dataset by training the model with a series of temporal matching loss weights. With this computational experiment, we will illustrate the trade-offs introduced by the relative strengths of matching and reconstruction losses.

Our expectation is that with very high matching loss, the embeddings (latent vectors) of the frames of the same trajectory will collapse regardless of morphology. For, a relatively wide range of matching loss weights, rank relations between transition pairs ([A->B] + [B->A'] >> [A->A']) should be preserved, from which the rare transitions can be robustly identified. In our experiments, most cells reached steady state morphodynamics when imaged, i.e., the matching loss between two adjacent frames arises primarily due to variations in background/noise. Fast transitions are “rare” in our data. Numerically, fast transitions contribute less to the matching loss during training and therefore their latent representations are not minimized. In other words, if B is a morphologically different state from A/A', the model is driven more by the reconstruction loss due to morphological difference rather than temporal smoothness across three consecutive frames.

Baseline comparisons

The authors evaluate their method by assessing the correlation of embedding PCs with heuristic features (Fig. 2C,D + supp.), variation of embedding PCs across cell treatment groups (Fig. 3), and qualitative interpretation of embedding trajectories.

In the supplement, the authors compare their VQ-VAE approach to VAEs and AAEs and chose to use a VQ-VAE based on lower reconstruction error and higher PC/heuristic feature correlation.

However, the authors do not compare their method to much simpler baseline approaches to this problem.

Existing literature suggests that heuristic features of cell shape and motion (similar to those the authors use to evaluate the relevance of their embeddings) are sufficient to perform many of the same tasks a VQ-VAE is used for in this work.

For instance, in Fig. 3 it appears that a simple analysis of cell centroid speed recovers much of same information as the complex VQ-VAE embeddings.

In Fig. 2 - Supp. 6, it appears that after regressing out many heuristic features of cell geometry, the latent space largely explains cell non-autonomous information about the background environment, suggesting the heuristic features are largely sufficient.

To demonstrate the usefulness of their deep modeling approach relative to simple baselines, the authors should compare against existing heuristics and embeddings of heuristics (e.g. PCA) using some of the tasks shown for the VQ-VAE (recovery of perturbation state, state transition detection, qualitative trajectory analysis, discrimination of cell types).

Heuristics might include those already calculated here, or a more comprehensive set as cited in the Introduction.

The authors may also consider comparing against baselines that don't include time information for some of their tasks (e.g. recovery of perturbation state could arguably be achieved with CNNs either ignorant of the timestep with simple temporal conditioning, not including trajectory information).

If these features are sufficient for many of the same tasks performed in this work, the authors should provide a clear argument for readers as to why the unsupervised VQ-VAE approach may be preferable (e.g. ability to recover potentially unknown cell changes, for which no heuristic exists).

The VQ-VAE doesn't need to be superior along every axis to hold merit, but the work would be strengthened if the authors could show clear superiority along some dimension.

Thanks for your comments. We agree that through our exploration, specific heuristic features are found to be correlated with latent shape features. We did not start with heuristic features, but instead identified them after observing how cell morphology changes along the principal components of the latent shape space. Discovering the heuristic shape features that describe the variation in shape space, in our view, reinforces the value of self-supervised learning of complex cellular morphologies.

We’d argue that the dynamorph pipeline complements heuristic approaches: it enables discovery of cell states through unbiased encoding and clustering, and the correlation of learned features with heuristic features enables interpretation of the cell state/data distribution more quantitatively than using either approach in isolation. Our argument is further reinforced by the related work (e.g., Zaritsky et al. and others mentioned in the introduction) on self-supervised learning of cell shape and interpretation of its latent space.

More specifically, self-supervised learning with temporal matching generates unbiased and smooth encodings for cell morphologies, from which we identified the rank correlations between top PCs and certain geometric properties. However, this does not indicate that the set of heuristics chosen a priori will be equally descriptive of the shape distribution. For example, optical density of cells (phase) is a heuristic feature that has not been used in previous studies, which we recognized after sampling the PCs of shape space. Further identification of such correlations is by itself an interesting discovery enabled by self-supervised learning.

In the current manuscript, we compared learned latent features (PCA on VQ-VAE latent embeddings) against a simple baseline (top PCs of raw images) and showed superior performances, which already illustrate the advantage of self-supervised learning in denoising data and extracting key diversities. In the revision, we will compare PCs of multiple heuristic features (e.g., cell size) with latent features to further strengthen the above point.

Reviewer # 3 (minor points)

For Fig. 4 - supp 1 -- isn't it expected that the GMM cluster of a vector can be predicted from the vector? The GMM clusters were derived from the vectors to begin with, so this seems like a bit of a circular analysis. If I'm missing something, this figure might benefit from more exposition.

Thanks for your question. The original purpose of having this confusion matrix is to parallel Fig. 3 - supp 2, showing that GMM generated distinct cell states that describe population better than perturbation conditions. The confusion matrix itself is trivial, so we will evaluate how to make this point more precisely during the revision.

For Fig. 4 - Supp 3, the authors should consider changing the "state" and "cluster" colors on the embedding projections so that they do not match. As presented, it appears as if the states and clusters were co-assayed and linked by some experimental label, when in fact the State 1::Cluster1, State 2::Cluster 2 relationship is just inferred.

Thanks for your comment, we will change the color scheme for Fig. 4 - supp 3 to avoid confusion in the revision.

* Description of the revisions that have already been incorporated in the transferred manuscript

Please insert a point-by-point reply describing the revisions that were already carried out and included in the transferred manuscript. If no revisions have been carried out yet, please leave this section empty.

Reviewer # 1 (major points)

* The temporal matching to enforce a smooth latent space representation is interesting. The authors mention that they mask out surrounding cells with a median pixel value. Have the authors considered using a pixel weighting in the reconstruction/matching loss to differentiate foreground/background? Also, does this affect detection of any fast (or indeed rare) transitions in the trajectories?

Thanks for your comment and question. Yes, we indeed incorporated a pixel weighting strategy during training. In addition to masking out surrounding cells, we used a smoothed and enlarged version of individual cell's segmentation mask to emphasize accurate reconstruction of the center cell in each patch, and reduce the influence of the surrounding cells/artifacts/background fluctuations. Matching loss is computed from latent vectors, which will be indirectly affected by the pixel weighting as well.

More detailed description of the weighting strategy will be added to the methods section. The code for our weighting strategy can be found at: https://github.com/czbiohub/dynamorph/blob/b3321f4368002707fbe39d727bc5c23bd5e7e199/HiddenStateExtractor/vq_vae_supp.py#L287

Reviewer # 1 (minor points)

I was a little confused by the labels given to the PCs, as they seem to vary between figures. For example, In Fig2, PC1 and PC2 are Size and Peak Retardance, but in Fig3 they are referred to as Size and Cell Density (which could be interpreted as the number of cells per unit area). Could the authors clarify these in the captions?

We have clarified the text to distinguish between cell density (population) and optical density (phase).

The authors note that single-cell tracking is of vital importance. This should be elaborated upon. Also - could the VQ-VAE encodings be used to help track linking in cases of high density?

We added a clearer reference to the methods section containing details of the tracking procedure. Additionally, we clarified in the discussion that the methods used for segmentation and tracking cells can be refined for high density cultures. Since we rely on the tracks to compute the temporal matching loss and regularize the VQ-VAE encodings (shape space) during the training, the encodings are not useable for refining tracking in high density population.

Reviewer # 2 (major points)

-'Cell state' in the field of cell biology has been operationally defined in so many different ways and with so many different types of measurement data, that 'cell state' is becoming a somewhat vacuous term. This is not only a problem of this paper but a challenge for the field. In this case, clustering of cells using a Gaussian mixture model that uses the first few principal components of the latent space coefficients as well speed - both averaged across the frames of cell tracks. This is fine and descriptive, but it's unclear whether this definition of 'cell state' is easily applied to other datasets and how this definition can be operationalized for hypothesis generation and experimentation. For other datasets, e.g. other cell types and other processes, such as differentiation, where e.g. tracking and segmentation may be more difficult and images would look quite different, can one still apply the same approach towards describing cell states? One could state that this definition of cell state is very specific to the dataset and therefore not generally useful. How would the authors respond to such a statement?

This is an excellent point. We agree that the meaning of a “cell state” or a “cell type” can depend on the context. Cell state can be rigorously described in terms of measurements of the cells, and recent developments of new cell probing techniques, including imaging modalities and single-cell genomics keep adding to the growing list of the features that can be measured. Time-lapse imaging is high dimensional and therefore admits multiple definitions of cell state. Our use of the terms ‘latent shape space’ and ‘trajectory feature vectors’ clarifies how we define the cell state. Given the increasingly wider use of live cell imaging for biological studies and drug discovery, both of these descriptors of cell state are valuable. In the current manuscript, we focus on a combination of morphodynamic features, including but not limited to the cell shape, size, and speed. We use these features to cluster cells in an unbiased manner to detect morpho-dynamic “states” unique for this particular culture system. Our approach can be generalized to other cell culture systems, such as cell differentiation, where cell architecture evolves substantially.

To clarify this point, we add the following text in the manuscript:

Line 85: “The meaning of a "cell state" can vary with the physiological and methodological context. In this work, we refer to "morphodynamic states" as a combination of morphological and temporal features. From the trajectory of cells in the latent shape space, we identified transitions among morphodynamic states of single cells. The same approach enabled detection of transitions in the morphodynamic states of cells as a result of immunogeneic perturbations.

”

In the discussion:

Line 333: “ Our work formalizes an analytical approach for data-driven discovery of morphodynamic cell states based on the quantitative shape and motion descriptors. A cell state can be rigorously described in terms of measurements of the cells, and recent developments in measurement techniques, including imaging modalities and single-cell genomics keep adding to the growing list of the features that can be measured. Time-lapse imaging is high dimensional and therefore admits multiple definitions of a cell state.”

-It's unclear to the reviewer whether the training data (unperturbed microglia) are close enough to the test data (perturbed microglia) such that application of the trained model to the test data makes sense. The authors provide reconstruction loss numbers, but they are difficult to interpret. Can the authors create plots of the unperturbed microglia cells and unperturbed microglia cells in the latent space and show overlap, or in other ways, show that training data and test data are close enough for this application.

Thank you for pointing out the lack of clarity in generalizability of the model. We trained the model on control, untreated microglia acquired during one experiment, and then applied it to a separate dataset acquired during another experiment that included perturbed and control microglia. The reconstructions shown in Fig. 2 are from the test dataset that was not used during training. The quality of reconstructions supports that the shape space of the training set is representative of the shape space of the larger test set. We will add a density plot in the supplementary figures showing the overlapping latent space distribution of unperturbed (training dataset) and perturbed (test dataset) microglia.

We now include the revised sentence in the manuscript to clarify the results:

Line 132: “Comparison of reconstructed shapes from the test set and training set along with the analysis of the shape space described in the next section show that our self-supervised model trained on training dataset generalized well between independent experiments and can be used to compare cell state changes between control microglia and cells treated with multiple perturbations”.

-Only a small amount of intensity variation is explained; 17% using the first 4 PC components which are mainly used in the analyses. This seems like a very low number. There is a lot of variation in the intensity images that is not explained by the autoencoder. The autoencoder seems to be doing a bad job. At the same time, the downstream analyses using the latent space are insightful and sensible. Can the authors provide more explanation?

Thanks for your question. We would like to first clarify that the autoencoder (VQ-VAE) used in this work follows the design of the original reference, which doesn't have a very large compression. Given the latent space size (16x16x16), it is understandable that the 4 top PCs captured relatively smaller portions of the variance. The fact that cell shape cannot be described with few principal components is likely due to: a) diversity of morphology of microglia, b) diversity of modalities used to train the model.

We now include the following text in the manuscript: Line 158: “The high variance of the shape space of microglia can be due to more complex shapes of microglia, such as diversity of protrusions, sub-cellular structures and variations in cell optical density, location of nuclei in migrating cells, etc. As we mentioned above, the inclusion of several imaging channels (brightfield, phase, and retardance) increases the performance of the model, possibly by increasing the diversity of morphological information encoded in our input data.”

As you note, the downstream analyses from the learned latent space are insightful, e.g., we do detect substantial changes in top PCs upon perturbations. This supports our view that the shape space of microglia as encoded by our data is intrinsically high dimensional and the transients in the shape space are informative.

Reviewer # 2 (minor points)

-The motivation for GMMs over k-means is unclear. K-means clustering leads to spatial separation between clusters (states) since all cells/tracks that closest to their cluster mean are per definition further away from the means of other clusters. This is not the case with the more flexible GMMs; e.g. they allow one to have a smaller cluster (with small variance components) inside of a larger cluster (with large variance). The latter scenario seems undesirable for interpretation in terms of states.

Thanks for your comments. The major reason for choosing GMMs over K-means clustering is that GMM allows different prior distributions for different perturbations. In practice, K-means would be capable of generating clusters regardless of perturbation conditions, while GMM enables a finer separation of states which are very likely correlated with perturbations. We agree that GMM has certain caveats as you mentioned in the comment. In our analyses, we didn’t notice the issues such as ‘nesting of components’ that you described.

-Related to the previous point, 'self-supervised' sounds nice, but it's still optimizing towards something, in this case explaining the variation in input intensity images. A lot of the variation in the intensity images may not be of interest for the biological investigation of shape and dynamics. Did the authors uncover that indeed some of the latent dimensions are encoding other aspects of the images which may be less related to the biology and more to image properties/artifacts/biases?

We agree with your assessment. Precisely for the reasons you point out, we counter the dependence of learned representation on non-biological variations in data using temporal regularization. This point is recognized by the reviewer #3. We clarify this concept. We clarify that not all the latent features represent biology of the cells and some represent the features of the instrument and the experiment. We report this for the top few PCs of latent representation and provide the code for the interested reader to discover what other PCs report.

-The original images are 3D (5 z-planes). The analyzed images were 2D. The reviewer missed how the authors went from 3D to 2D. And since cells are 3D, can the authors describe what they gained by going to 2D and what they potentially lost?

We added additional text to the methods subsection describing the Dynamorph Pipeline (line 590):

“The input data for both semantic segmentation and VQ-VAE models are 2D-images of computed phase and retardance that measure integrated optical density and anisotropy across the depth of the cell. The raw collected data is 3-dimensional (5 z-slices acquired in multiple polarization channels). The 2D phase is computed from the full stack of brightfield images via deconvolution. The retardance is computed from an average of the intensities across the 5 z-slices. Subsequent model training is more tractable with 2D data instead of 3D, while capturing the cell architecture across the depth.”

Reviewer # 3 (major points)

Cell state transition interpretation

In line 278, the authors propose that the unbalanced nature of transitions such that p(1 -> 2) >> p(2 -> 1) must represent some difference in timescales across the transitions because "cell states should have reached equilibrium after several days in culture at the time of the imaging experiments".

This logic is unclear to me for two reasons.

* If the population obeys detailed balance (e.g. transitions have equal frequency), then observed transitions should be balanced on a reasonably long time window, even if individual transitions occur on different timescales.

* The assumption that cell states are balanced after a few days in culture is at odds with a few different aspects of the biology. Cell density and nutrient availability are continually changing in the dish, so culture conditions are non-stationary. Imaging apparatuses also commonly impact the cell biology of imaged samples due to imperfect incubation, etc. (2 or 3)

It seems likelier that these data represent an unbalanced transition due to the non-stationary nature of the culture system.

Given the authors' emphasis on the value of measuring these transitions, the work would be strengthened by a more careful interpretation of these results, additional analysis details (e.g. how large are most state transitions? are these mostly small shifts "over the border" in state space, or large jumps?), and an attempt at biological interpretation of the observed phenomenon.

The authors' RNA-seq data may be helpful in this latter regard.

This is an excellent point. We agree that the cell culture conditions, including nutrient availability, accumulating presence of metabolites and imagine-induced changes constantly introduce new variations to the system. In an attempt to mitigate these dynamic changes to the system, we maintained cells in culture for six days before starting the experiment. To avoid cell stimulation due to freshly added nutrients and growth factors from the culture media, we consistently exchanged the media and performed cytokine treatments 24 hours before each imaging experiment. Each imaging round was started after the cells were allowed to equilibrate to the environmental chamber for at least one hour before imaging. Despite these efforts, we agree with the reviewer that the conditions cannot be considered fully stationary. We removed the sentence “ Given that cell states should have reached equilibrium after several days in culture at the time of the imaging experiments, these results suggest that the transitions from state 2 to state 1 occur at a different time scale (i.e., much slower)” and changed the text to reflect this point:

Line 294:

“In our analysis, transition events are very rare among cells treated with IFN beta, while the most frequent cell transitions were observed among cells treated with GBM supernatant. One possible explanation for this imbalance is that IFN-treated cells represent a single polarization axis, while a heterogeneous cell signaling milieu derived from cancer cells provides conflicting pro- and anti-inflammatory signals, instructing cells to transition between the states. While both directions of transitions were observed within the imaging period, cells in state-1 are more likely to transition to state-2 than vice versa within the chosen time frame. This imbalance between the rates of state transitions correlates with the higher state-2/state-1 ratio in GBM and control environment and may explain the longitudinal accumulation of cells in a more activated state under these culture conditions.”

1. Single cell RNA-seq analysis

The authors performed a very interesting experiment where they profiled the same cell population using both timelapse imaging and single cell RNA-seq.

The authors argue that the global structure of the state space resolved by each modality is analogous, but this seems a bit of a stretch to me.

The behavior state space is unimodal (bifurcated into two states by GMM clustering), while the mRNA-seq space has several distinct clusters.

The argument that these states are analogous would be significantly strengthened by biological interpretation of the RNA-seq data.

Do the mRNA profiles exhibit differentially expressed genes that might explain differences in behavior in the cell behavior states?

The analyses in Fig. 4 - Supp 4 are suggestive that "State 1" contains interferon-responsive cells and not control cells, but broader conclusions don't appear well supported by current analyses.

We agree with the reviewer’s comment that the analogy between molecular cell states defined with scRNAseq analysis and morphodynamic cell states defined with dynamorph needs to be clarified. In our current work, the correlative measurement of morphodynamics and transcriptome was exploratory and relied on population statistics measured with each modality. More detailed studies linking morphodynamic states to the single cell transcriptomics, such as Patch-Seq or laser microdissection, are needed to decisively link morphodynamics and molecular programs underlying these phenotypes.

Single cell transcriptomics simultaneously measures thousands of mRNA species in individual cells. Therefore, it can provide a nuanced interpretation for the molecular states of each population, as can be seen at a more granular separation of sub-states in scRNAseq clustering. For example, Cluster 1-2 was defined by high expression of interferon response genes, and predictably, this cluster was primarily derived from the cells treated with IFNb. Interferon exposure induces morphological changes associated with increased cell perimeter, which reports ramification of microglia plasma membrane (Aw et al., PMID: 33183319). It was also shown that infections with neurotropic viruses, leading to interferon response, also leads to decreased velocity and distance traveled for cultured microglia cells (Fekete et al., PMID: 30027450). These observations are in direct agreement with our morphodynamic analysis demonstrating a higher proportion of cells in State 1, characterized by lower cell velocity. Interestingly, scRNAseq analysis also identified a population of cells with high expression of cell cycle genes (Cluster 1-3), which would also be predicted to have a slower speed and potentially larger cell body. These results point to the fact that different molecular states may be underlying very similar morphodynamic states.

We now provide a revised statement to reflect the above.

Line 290: “We further compared the detected morphodynamic states with scRNA measurements of the same cell populations. Interestingly, the separation of cells in state-1 and state-2 from control and IFN group parallels the clusters identified with cell transcriptome, suggesting that correlative analysis of gene expression and morphodynamics can reveal molecular programs underlying these phenotypes. In our preliminary analysis, scRNAseq revealed a greater degree of granularity in each of the cell populations, such as cluster 1 of the scRNAseq separating into three additional subclusters. Cluster 1-2 was defined by high expression of interferon response genes, and predictably, this cluster was primarily derived from the cells treated with IFNb. Interferon exposure induces morphological changes associated with increased cell perimeter, which reports ramification of microglia membrane (Aw et al., 2020). It was also shown that infections with neurotropic viruses, leading to interferon response, also leads to decreased velocity and distance traveled for cultured microglia cells (Fekete et al., 2018). These observations are in direct agreement with the higher proportion of cells in State 1, characterized by lower cell velocity. Interestingly, scRNAseq analysis also identified a population of cells with high expression of cell cycle genes (Cluster 1-3), which would also be predicted to have a slower speed and potentially larger cell body. These results point to the fact that different molecular states may be underlying very similar morphodynamic states. Correlative single-cell measurements of morphodynamic states and single cell transcriptomics, such as Patch-Seq or laser microdissection, are needed to decisively link morphodynamics and molecular programs underlying these phenotypes.”

Reviewer # 3 (minor points)

1. Check grammar. Some articles are missing and some subject-verb agreements are mismatched. e.g. line 624 "we regularized [the] latent space", line 713 "after both loss[es] achieved".

Thanks for pointing this out, we have thoroughly checked grammar and typos in this submission.

Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Referee #3

Evidence, reproducibility and clarity

Summary

Here, the authors present Dynamorph, an unsupervised learning framework for timelapse cell microscopy data built on VQ-VAEs.

The authors apply this method to the analysis of microglial cell behavior under a series of perturbation conditions.

Methodologically, the primary contributions of this work are the introduction of a temporal consistency regularization penalty on the latent space of a VQ-VAE model for application to timeseries data and the introduction of a "temporal feature vector"-ization procedure to summarize complex temporal trajectories in a single low-dimensional vector for analysis. Biologically, the primary contributions are the demonstration that microglial responses to different perturbogens and dynamics state transitions can be resolved by transmitted light microscopy.

Overall, the experiments presented are well-designed, the methods well-implemented, and communication of the authors' findings is clear and concise.

However, there are unaddressed potential caveats to the proposed framework and the manuscript fails to compare the proposed method to any existing baselines, such that the particular strengths and weaknesses of the method are unclear to readers.

Major Points

1. Temporal consistency regularization

In the authors' framework, models are regularized to minimize the l2 norm between embeddings of adjacent timepoints. This is approach is conceptually well-motivated, but could have some unintended effects.

For instance, some cells may make a rapid state transition such that state(t-1) = A, state(t) = B, state(t+1) = A'. In these cases, a regularized model may best minimize the joint loss by returning an embedding at time t that interpolates between state A and A', rather than returning an embedding that reflects the true distinct state B.

The work would be strengthened if the authors analyzed the impact of this regularization term on the detection of rapid state transitions that occur for only a few frames (e.g. when cells that exhibit filopodial motility "jump" in an actin/myosin contraction). This might be accomplished through experiments scanning different regularization hyperparameters on some of the authors' real data, fitting models on temporally downsampled versions of the real data where "slow" multi-timestep transitions now occur in a few timesteps, or perhaps using simulations where rapid state transitions are known to occur.

Even if the regularization does have some negative impacts, it does not argue against the utility of the general approach, but it is important for users to understand the constraints on downstream applications.

1. Baseline comparisons

The authors evaluate their method by assessing the correlation of embedding PCs with heuristic features (Fig. 2C,D + supp.), variation of embedding PCs across cell treatment groups (Fig. 3), and qualitative interpretation of embedding trajectories. In the supplement, the authors compare their VQ-VAE approach to VAEs and AAEs and chose to use a VQ-VAE based on lower reconstruction error and higher PC/heuristic feature correlation.

However, the authors do not compare their method to much simpler baseline approaches to this problem. Existing literature suggests that heuristic features of cell shape and motion (similar to those the authors use to evaluate the relevance of their embeddings) are sufficient to perform many of the same tasks a VQ-VAE is used for in this work. For instance, in Fig. 3 it appears that a simple analysis of cell centroid speed recovers much of same information as the complex VQ-VAE embeddings. In Fig. 2 - Supp. 6, it appears that after regressing out many heuristic features of cell geometry, the latent space largely explains cell non-autonomous information about the background environment, suggesting the heuristic features are largely sufficient.

To demonstrate the usefulness of their deep modeling approach relative to simple baselines, the authors should compare against existing heuristics and embeddings of heuristics (e.g. PCA) using some of the tasks shown for the VQ-VAE (recovery of perturbation state, state transition detection, qualitative trajectory analysis, discrimination of cell types). Heuristics might include those already calculated here, or a more comprehensive set as cited in the Introduction. The authors may also consider comparing against baselines that don't include time information for some of their tasks (e.g. recovery of perturbation state could arguably be achieved with CNNs either ignorant of the timestep with simple temporal conditioning, not including trajectory information).

If these features are sufficient for many of the same tasks performed in this work, the authors should provide a clear argument for readers as to why the unsupervised VQ-VAE approach may be preferable (e.g. ability to recover potentially unknown cell changes, for which no heuristic exists). The VQ-VAE doesn't need to be superior along every axis to hold merit, but the work would be strengthened if the authors could show clear superiority along some dimension.

1. Cell state transition interpretation

In line 278, the authors propose that the unbalanced nature of transitions such that p(1 -> 2) >> p(2 -> 1) must represent some difference in timescales across the transitions because "cell states should have reached equilibrium after several days in culture at the time of the imaging experiments". This logic is unclear to me for two reasons.

• If the population obeys detailed balance (e.g. transitions have equal frequency), then observed transitions should be balanced on a reasonably long time window, even if individual transitions occur on different timescales.
• The assumption that cell states are balanced after a few days in culture is at odds with a few different aspects of the biology. Cell density and nutrient availability are continually changing in the dish, so culture conditions are non-stationary. Imaging apparatuses also commonly impact the cell biology of imaged samples due to imperfect incubation, etc.

It seems likelier that these data represent an unbalanced transition due to the non-stationary nature of the culture system. Given the authors' emphasis on the value of measuring these transitions, the work would be strengthened by a more careful interpretation of these results, additional analysis details (e.g. how large are most state transitions? are these mostly small shifts "over the border" in state space, or large jumps?), and an attempt at biological interpretation of the observed phenomenon. The authors' RNA-seq data may be helpful in this latter regard.

1. Single cell RNA-seq analysis

The authors performed a very interesting experiment where they profiled the same cell population using both timelapse imaging and single cell RNA-seq. The authors argue that the global structure of the state space resolved by each modality is analogous, but this seems a bit of a stretch to me. The behavior state space is unimodal (bifurcated into two states by GMM clustering), while the mRNA-seq space has several distinct clusters.

The argument that these states are analogous would be significantly strengthened by biological interpretation of the RNA-seq data. Do the mRNA profiles exhibit differentially expressed genes that might explain differences in behavior in the cell behavior states? The analyses in Fig. 4 - Supp 4 are suggestive that "State 1" contains interferon-responsive cells and not control cells, but broader conclusions don't appear well supported by current analyses.

Minor Points

1. Check grammar. Some articles are missing and some subject-verb agreements are mismatched. e.g. line 624 "we regularized [the] latent space", line 713 "after both loss[es] achieved".
2. For Fig. 4 - supp 1 -- isn't it expected that the GMM cluster of a vector can be predicted from the vector? The GMM clusters were derived from the vectors to begin with, so this seems like a bit of a circular analysis. If I'm missing something, this figure might benefit from more exposition.
3. For Fig. 4 - Supp 3, the authors should consider changing the "state" and "cluster" colors on the embedding projections so that they do not match. As presented, it appears as if the states and clusters were co-assayed and linked by some experimental label, when in fact the State 1::Cluster1, State 2::Cluster 2 relationship is just inferred.

Positive comments

1. Figure presentation and manuscript layout are top notch. Thanks to the authors for making these data easy to read and interpret.

Significance

See above.

Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Referee #2

Evidence, reproducibility and clarity

Provide a short summary of the findings and key conclusions (including methodology and model system(s) where appropriate). Please place your comments about significance in section 2.

-The authors describe Dynamorph; a deep-learning based autoencoder to represent - in an interpretable latent space - live cell microscopy image data of motile microglia in unperturbed and perturbed situations. Using Dynamorph, the authors identify and describe 'morphodynamic' states of the microglia.

Major comments:

Are the key conclusions convincing?

-Yes, the methodology, observations and conclusions are clearly explained and convincing.

Should the authors qualify some of their claims as preliminary or speculative, or remove them altogether?

-'Cell state' in the field of cell biology has been operationally defined in so many different ways and with so many different types of measurement data, that 'cell state' is becoming a somewhat vacuous term. This is not only a problem of this paper but a challenge for the field. In this case, clustering of cells using a Gaussian mixture model that uses the first few principal components of the latent space coefficients as well speed - both averaged across the frames of cell tracks. This is fine and descriptive, but it's unclear whether this definition of 'cell state' is easily applied to other datasets and how this definition can be operationalized for hypothesis generation and experimentation. For other datasets, e.g. other cell types and other processes, such as differentiation, where e.g. tracking and segmentation may be more difficult and images would look quite different, can one still apply the same approach towards describing cell states? One could state that this definition of cell state is very specific to the dataset and therefore not generally useful. How would the authors respond to such a statement?

Would additional experiments be essential to support the claims of the paper? Request additional experiments only where necessary to evaluate the paper as it is, and do not ask authors to open new lines of experimentation.

-It's unclear to the reviewer whether the training data (unperturbed microglia) are close enough to the test data (perturbed microglia) such that application of the trained model to the test data makes sense. The authors provide reconstruction loss numbers, but they are difficult to interpret. Can the authors create plots of the unperturbed microglia cells and unperturbed microglia cells in the latent space and show overlap, or in other ways, show that training data and test data are close enough for this application.

-It's unclear what the effect of speed is on the final state determination. TFVs were composed of auto-encoder-based features (PCs from latent space) and speed of the cells. Would the states be very different without speed as part of the TFVs or with TFVs consisting only of speed features? Please quantify and discuss. -Only a small amount of intensity variation is explained; 17% using the first 4 PC components which are mainly used in the analyses. This seems like a very low number. There is a lot of variation in the intensity images that is not explained by the autoencoder. The autoencoder seems to be doing a bad job. At the same time, the downstream analyses using the latent space are insightful and sensible. Can the authors provide more explanation?

-Related to the previous point, 'self-supervised' sounds nice, but it's still optimizing towards something, in this case explaining the variation in input intensity images. A lot of the variation in the intensity images may not be of interest for the biological investigation of shape and dynamics. Did the authors uncover that indeed some of the latent dimensions are encoding other aspects of the images which may be less related to the biology and more to image properties/artifacts/biases? Are the suggested experiments realistic for the authors? It would help if you could add an estimated cost and time investment for substantial experiments. -These are computational experiments based on already existing data/results/code. It should be relatively straightforward to do these additional computational experiments. Careful analysis and interpretation require time.

Are the data and the methods presented in such a way that they can be reproduced? -The methods are described with sufficient detail.The complicated experimental and computational processes seem reproducible to a decent extent. The code is captured in Github repos. The reviewer did not attempt to reproduce computational results. The reviewer did not check whether the available data meets FAIR requirements. Are the experiments adequately replicated and statistical analysis adequate?

-Yes, and there is lots of useful supplementary material which helps with interpretation of the results. Minor comments: Specific experimental issues that are easily addressable. -The motivation for GMMs over k-means is unclear. K-means clustering leads to spatial separation between clusters (states) since all cells/tracks that closest to their cluster mean are per definition further away from the means of other clusters. This is not the case with the more flexible GMMs; e.g. they allow one to have a smaller cluster (with small variance components) inside of a larger cluster (with large variance). The latter scenario seems undesirable for interpretation in terms of states.

-The original images are 3D (5 z-planes). The analyzed images were 2D. The reviewer missed how the authors went from 3D to 2D. And since cells are 3D, can the authors describe what they gained by going to 2D and what they potentially lost? Are prior studies referenced appropriately?

-Yes, citations are amply and relevant. Are the text and figures clear and accurate?

-Yes, the figures are informative. Do you have suggestions that would help the authors improve the presentation of their data and conclusions?

-No specific suggestions

Significance

Describe the nature and significance of the advance (e.g. conceptual, technical, clinical) for the field.

-This is a technological/computational advance using a large integrative (experimental+computational) approach.

Place the work in the context of the existing literature (provide references, where appropriate).

-The authors have done an excellent job at this.

State what audience might be interested in and influenced by the reported findings.

-Cell biologists, brain researchers, computer vision computational biologists

Define your field of expertise with a few keywords to help the authors contextualize your point of view. Indicate if there are any parts of the paper that you do not have sufficient expertise to evaluate.

-Cell biology, cancer biology, systems biology, machine learning, statistics, data integration

-Brain biology aspects (biological significance of the findings on morphodynamic microglial states) are difficult to assess for the reviewer

Referee Cross-commenting

Comments by Reviewer #1 look great and useful. I think they are in line with my comments. I think this manuscript would benefit from a reviewer that could comment on the biological significance. The review reports are skewed towards questions and remarks about the computational approach.

Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Referee #1

Evidence, reproducibility and clarity

Summary:

The authors use a combination of quantitative phase microscopy and machine learning to determine the state space of microglia cells. The key conclusions are that a VQ-VAE is able to capture a compact latent representation of the cell morphology, and combined with motion features, can predict state changes in single cell trajectories, and discriminate between purturbations.

Major comments:

Overall - I very much enjoyed reading the manuscript. The work has been carefully performed and the results are interesting.

• The temporal matching to enforce a smooth latent space representation is interesting. The authors mention that they mask out surrounding cells with a median pixel value. Have the authors considered using a pixel weighting in the reconstruction/matching loss to differentiate foreground/background? Also, does this affect detection of any fast (or indeed rare) transitions in the trajectories?
• The Trajectory Feature Vectors (TFVs) are averaged over time - this seems to lose a lot of the salient information in the trajectories themselves, resulting in the low(ish) accuracy of the GMM. Could a Hidden Markov Model trained on the trajectories in state space help to identify/classify those trajectories that change their morphology/motion over time?

Minor comments:

• Could the authors provide some example images showing interpolation of each PC using the generative decoder?
• I was a little confused by the labels given to the PCs, as they seem to vary between figures. For example, In Fig2, PC1 and PC2 are Size and Peak Retardance, but in Fig3 they are referred to as Size and Cell Density (which could be interpreted as the number of cells per unit area). Could the authors clarify these in the captions?
• The authors note that single-cell tracking is of vital importance. This should be elaborated upon. Also - could the VQ-VAE encodings be used to help track linking in cases of high density?
• I was pleased to see the full source code available!

Significance

Nature and significance:

This is a significant, mostly technical piece of work, that explores a complex new area of science -- using ML and large datasets to gain insight into biological systems. There are significant challenges, not least that interpreting ML models can be challenging.

Existing literature/context:

There have been relatively few examples of using self-supervised learning to gain insight into these complex datasets. Much of the work has concentrated on learning morphological descriptors. The present work starts to introduce the time dimension more explicity.

Target Audience:

Broadly applicable to those studying cell biology, microscopy and machine learning.

My expertise:

ML applied to microscopy data. Single cell tracking.

Note: This rebuttal was posted by the corresponding author to Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Reply to the reviewers

The authors do not wish to provide a response at this time.

Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Referee #3

Evidence, reproducibility and clarity

The authors examine the important and challenging question in current biology, the role of RNA in the structural maintenance of nuclear and cytoplasmic membrane-less organelles including stress granules, processing bodies, nucleolus, Cajal bodies, and nuclear speckles. Furthermore, the authors explored super-enhancer complexes involved in the regulation of gene expression. The authors used RNase L, an interferon-induced ribonuclease which, upon activation in the cytoplasm or targeted to the nucleus, degrades all RNAs within the cell. Then they took the quantitative approach to analyze the effect of RNA degradation on disassembly or reorganization of membrane-less organelles. Interestingly, the authors observed that RNAs present within nuclear organelles are susceptible to RNase IL degradation leading to their disassembly. In contrast, super-enhancer-containing eRNAs are largely unaffected.

Major concerns

Many studied organelles are challenging to see in many of the figures. Thus this reviewer encourages the authors to present clearer insets at higher magnification to illustrate what is being quantified, and then show that quantification in the central figure next to the immunofluorescent images.

The amount of specific RNAs degradation after induction of RNase L for several assemblies should be analyzed by qRT-PCR and quantified. This will justify observations provided by microscopy on an individual cell basis. The main issue regards the connection between RNA and its role in the formation and structural integrity of nuclear organelles. There is consensus that these nuclear assemblies are built on specific nascent transcripts which act as a nucleation scaffold. If specific RNA synthesis is impaired, these assemblies collapse. The authors should discuss it. It would be relevant to mention two experimental works on this topics, DOI: 10.1038/ncb2140 and DOI: 10.1038/ncb2157 The study is limited to observed macroscopical changes in the appearances of assemblies. The authors must dig deeper and provide more conclusive results by several colocalizing components of these assemblies. It has been documented that the visualization of a selective marker for a specific assembly is not enough to prove its functionality/dysfunctionality but also the level of its disassembly. For example, in Figure 4A the authors should more convincible visualize nascent 47/45S pre-rRNA transcript to demonstrate that the nucleolus is built on ongoing pre-rRNA synthesis reflected by the tripartite nucleolar substructures. The loss of the GC component after rRNA depletion should be better presented with NPM1 colocalization.

In Figure 4C, D the authors used the term "coilin assemblies". That's confusing for a reader. The Cajal body after activation of RNase L likely undergoes the structural rearrangement which cannot be justified only by the presence of rearranged coilin foci. The authors should colocalize them with at least one or two functional markers.

Enhancer RNAs likely play the role in gene control rather than as a nucleation element to build nuclear assemblies. This should be discussed in the explanation of observed differences between MED1 foci and other assemblies.

Significance

Understanding changes in the nuclear and cellular organization that accompany and drive changes in the formation and maintenance of cellular structures is an essential and not well-understood topic. Thus, this manuscript is relevant. However, the presented data in this paper are based on a limited approach, and particularly their interpretation and presentation could be substantially improved. Consequently, the conclusions are not convincingly supported by published data. However, some open questions need to be addressed. Specific criticisms are outlined above.

Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Referee #2

Evidence, reproducibility and clarity

The manuscript by Decker et al. "RNA is required for the maintenance of multiple cytoplasmic and nuclear membrane-less organelles" investigates the structural role of RNA in membraneless organelles. The authors show that degradation of RNA in transient or constitutive membraneless organelles results in the altered formation and structure of many but not all organelles studied. The main assay is the activation of RNAseL activity by dsRNA which then destroys mRNA in the cell. The collected data leads the authors to highlight the possible roles of RNA in membraneless organelle formation and categorize the organelles: some relying more on the RNA-RNA interactions while others on protein-RNA or protein-protein interactions. The manuscript is well written and the data is sound.

Major comments:

The authors study the maintenance of organelles by RNA. For the transient ones, like stress granules (SG), it would be very interesting to see the formation/clearance kinetics with and without RNA. Also maybe using something other than dsRNA to trigger the formation. The idea being - if RNA is needed for SG maintenance, then the clearance kinetics with RNA would differ from that of the depleted RNA.

The experiments were done in cells. It is known that core components of the organelles can form granule like structures in vitro without RNA. If it is possible to show that RNA presence improves the integrity in vitro, that would support the authors claim. For example studying SG maintenance with and without RNAseL using the previously developed SG extraction protocol.

Minor comments:

In the Figure 1a, it is not clear if the smaller granules are different from SGs as mentioned in the text, maybe using additional markers can make it clearer. Figure 3 and 4 requires quantification.

Significance

This is a solid paper that advances our understanding of membraneless organelle formation and dynamics. This field is of high general interest for the broader scientific community. My expertise is in the field of membraneless organelles.

Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Referee #1

Evidence, reproducibility and clarity

In the paper entitled. "RNA is required for the maintenance of multiple cytoplasmic and nuclear membrane-less organelles" Decker et al set out to test the rolw of RNA in maintaining the integrity of a variety of biomolecular condensates. To do this, they assess how multiple different assemblies in the cytoplasm and nucleus retain their structural integrity following RNAseL activation. They identified many condensates which are solubilized and have protein components redistributed following RNAse L activation and presumably subsequent RNA digestion. These experiments largely concur with previous findings from RNAseA treatment. The implication is that RNA rather than protein is the essential organizing component for most tested condensates. The manuscript is well written, and the data are convincing. It is my judgement that this is worthy of publication following a few additional experiments/clarifications.

1. The authors identify condensates which are sensitive to or refractory to RNAaseL. It would be good if the authors more conclusively eliminate the possibility that remaining condensates contain specific residual antiviral RNAs and this is the reason why these condensates remain intact. Are any of these condensates enriched in anti-viral RNAs like IFNbeta following polyIC treatment by FISH, for example (PMID: 31494035)?
2. Is there a particular protein feature, charge, IDR-types etc. which is common to solubilized versus not solubilized groups? What about dissolved and novel formed assemblies? A simple table comparing protein features in the three groups would suffice, with particular emphasis on RNA binding domains PMID: 32243832 and intrinsic disordered regions PMID: 24773235.
3. Demonstrate that the RNAseL treatment is reversible (i.e. withdraw polyIC, particularly for a protein that ends up in a novel assembly) or remove the word maintenance from the title.
4. Control for RNA-dependence of the activity. Try to dissolve a non-RNA dependent/enriched condensate with RNAseL. SPOP mutations (PMID: 30244836) might be interesting as both SPOP and RNAseL loss of function mutations (PMID: 11799394) are associated with prostate cancer.
5. A caveat is that certain regions of condensates enriched in RNA may not be accessible to RNAseL protein. A way to address this might be to attempt to directly target the enzyme to a compartment that is deemed refractory to the activity (and inferred to not require RNA) via an inducible systsem (ie FKBP12/FK506)
6. Overall, this paper would be greatly enhanced by including a more extensive discussion on the basic biological implications for these findings. Why are some condensates RNA dependent? What function(s) are common to these condensates? How does disruption of this lead to disease?

Significance

This work addresses the neglected role of RNA in structuring condensates throughout the cell. Despite the prevalence of RNA in many condensates and the enrichment of RNA-binding proteins in condensates, there is still a highly limited understanding of the structural roles RNA plays in their assembly s most work has been protein/IDR-centric. This work seeks to systematically assess the RNA-dependence of the assemblies.

Transparent Peer Review

---

Download the complete Review Process [PDF] including:

• reviews
• authors' reply
• editorial decisions

</br>

Transparent Peer Review

---

Download the complete Review Process [PDF] including:

• reviews
• authors' reply
• editorial decisions

</br>

Note: This rebuttal was posted by the corresponding author to Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Reply to the reviewers

The authors do not wish to provide a response at this time.

Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Referee #3

Evidence, reproducibility and clarity

In mice, failures in conducting meiosis during spermatogenesis can be rescued by injecting prophase I male chromosomes into oocytes, to allow them to undergo the two meiotic divisions within the oocyte, together with the chromosomes of the oocyte. However, segregations are highly error prone and rarely lead to a live birth when the resulting embryos are reimplanted into foster mothers. In this study, the authors show that segregation errors in meiosis I oocytes harboring both male and female chromosomes are mainly affecting the male chromosome set. Most errors are due to precocious segregation of sister chromatids in unpaired male chromosomes (univalents). A delay in alignemnt of male chromosomes compared to female chromosomes was also observed. Reducing the volume of the oocyte cytoplams to half leads to a signifncant reduction in the errors occuring, and hence, a significant increase in successful birth after re-implantation. Excitingly, with this technique, live births were obtained from male mice with a spermatogenic arrest phenotype.

Main points:

1)The authors conclude that halving the oocyte cell size is helping in proper segregation of male meiosis I chromosomes in the cytoplasm of meiosis I oocytes. It is also possible that the experimental procedure involved in removing half of the cytoplasm is promoting proper segregation for some unknown reason. The authors should include a condition where half of the cytoplasm is aspirated but then put back again, so oocytes have the same volume as before but the cytoplasm underwent the same treatment as in the halved oocytes. Also, increasing the cytoplasm volume of the oocyte should not lead to a better segregation of male chromosomes but make things worse, have the authors checked for that?

2)The authors mention that male chromosomes align with a delay, compared to the female chromosomes. Does this delay depend on activation of error correction, or the spindle asembly checkpoint? Is it possible that dilution of factors required for checkpoint control and hence, assuring proper chromosome segregation, are the reason for error prone segregation in oocytes harboring twice the amount of chromosomes? If yes, have the authors stained for SAC proteins at the kinetochores? Maybe slight overepxression of the SAC protein were sufficient to rescue male meiotic divisions in the oocyte- have the authors tested this hypothesis?

3) The authors state that male chromosomes have a hard time segregating in the hugh cytoplasm of the oocytes. Maybe it is not the fact that the chromosomes came from a male pronucleus, but this is just a manner of double the chromosomes that have to be segregated in the oocyte cytoplams. How do male chromosomes behave in enucleated oocytes undergoing meiosis I? Conversely, if female chromosomes coming from another oocyte are injected into the recipient oocyte instead of ale chromosomes, are those segregating correctly, or the delay in chromosome alignment and error rate comparable to the situation when the additional chromosome set comes from the male?

4) In the rescue of mice with spermatogenic arrest the authors find aneuploidies of sex-chromosomes in the off-spring, not of autosomes. To my best of knowledge, autosome aneuploidies are not viable in the mouse, hence this result does not indicate that sex-chromosomes are the main source of aneuploidies. Nevertheless, it is attractive to speculate that aneuploidies are mainly due to sex chromosomes, because the oocyte is not prepared to segregate a male sex-chromosome bivalent. The authors should determine whether the segregation errors in meiosis I in oocytes harboring the additional male chromosome set concern mainly the male sex-chromosomes, by doing Fish analysis after meiosis I.

Significance

This study is very interesting and of high significance, and very well executed. I think the study can go much further as far as mechanistic insights are concerned, only requiring techniques and tools that the authors have at their disposition.

Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Referee #2

Evidence, reproducibility and clarity

Summary:

Previously, the team has shown that primary spermatocyte nucleus can undergo meiosis when transplanted into immature oocytes, and later obtained normal mice from the fertilized oocytes (Zygotes 1997, PMID: 9276513; PNAS 1998, PMID: 9576931). However, the efficiency was quite low (~ 1%) due to chromosome aberration, thus not feasible for basic/clinical research applications. In this study, Ogonuki et al., extrapolated from the recent study showing the reduction of the ooplasm ameliorate the error of chromosome segregation during meiosis (Dev Cell 2017, PMID: 28486131), injected the spermatocyte nucleus into the half-sized GV oocytes, and succeeded to obtain live murine pups with a high incidence (the birth rate improved from 1% with full-sized oocytes to 19% with half-sized oocytes). Further, through detailed observation with high-resolution 3D live imaging, the authors clarified that the misalignment of paternal chromosomes could be ameliorated by reducing the volume of ooplasm. Finally, the authors applied this technology and obtained live pups from azoospermic mice, suggesting the potential application in human infertility treatment.

Major comments:

This is a great study combining the expertise on both sperm and oocytes. The experiments are well designed and performed. The key conclusions are convincing.

Line 228. The authors claimed that all the pups born following the injection of wild-type or mutant spermatocytes grew into fertile adults.

Because the authors tested 3 males from wt spermatocytes (line 197), the above sentence should be rephrased.

The authors found one XXY male among the three male mice from wt spermatocytes. Was the XYY male mouse fully fertile without XY/XYY mosaicism?

How many females and males were obtained from wt spermatocytes?

Minor comments:

The authors clearly showed the technique can be applied to rescue the spermatogenic arrest. The readers would appreciate if the authors include any unsuccessful cases.

To prevent sex-chromosome aberration, are there any potential markers for selecting most developed spermatocytes?

Significance

One in six couples suffers from infertility, and 70-90% of male infertility cases are related to defects in spermatogenesis. Clinically, intracytoplasmic injection of sperm is common, but it is not applicable to men who lack haploid germ cells. Injection of primary spermatocyte nucleus can give pups but the efficiency was poor (~1%, PNAS 1998, PMID: 9576931). In the present study, by using halved oocytes as recipient, the authors improved the efficiency from 1% to 19%. With the great improvement, they further obtained healthy fertile offspring from the male mice genetically lacking haploid cells. This approach opens up the window for the infertile patients suffering from spermatogenic arrest.

The reviewer's field of expertise: knockout mice, male infertility, spermatogenesis, sperm function, fertilization.

Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Referee #1

Evidence, reproducibility and clarity

Ogonuki et al developed a new technique using primary spermatocyte-injected oocytes for offspring production. They examined chromosome segregation error in biparental meiosis using spermatocyte-injected oocytes. They showed that artificially reducing ooplasmic volume rescued highly error-prone chromosome segregation by preventing sister separation in biparental meiosis. Their live-imaging analysis demonstrated that erroneous chromosome segregation derived from univalent-like chromosomes followed by predivision of sister chromatids during prometaphase I in biparental meiosis. They showed that the birth rate was improved using halved oocytes. Furthermore, they showed that production of offspring was successful using spermatocyte from azoospermic mice.

Overall data are convincing and the manuscript addresses important questions. The data was produced in a technically high level. Presented data are sufficient to support conclusions of the authors, and further provide a significant insight into application to production of offspring for azoospermia animals. Thus, the manuscript could be open for the fields and are supposed to deserve publication, if they could address following minor concerns.

Fig1A, Line 117 This is an amazing experiment to set up biparental meiosis using spermatocyte nuclei. Since spermatocytes are in different stages during progression through meiotic prophase, some of them (late pachytene) should yield crossover but others (before mid-pachytene) are yet to complete recombination. Thus, whether donor paternal chromosomes have bivalents or univalents depends on which stage spermatocytes derived from. The authors should describe how spermatocytes were picked up for injection and whether they used a particular stage of spermatocytes.

Line 159-160 The authors stated that paternal chromosomes are susceptible to errors in ooplasm-hosted biparental meiosis. This is nice demonstration to trace the origin of separated chromatids. In Fig2C right graph, 1 to 2 paternal chromosomes showed misalignment. It is unclear whether premature separation is biased to any particular paternal chromosome, eg XY ? The authors should discuss more about it.

Line 176-177 The authors stated that most of errors were preceded by premature separation of bivalent chromosomes into univalent-like structures. This implies that premature separation of bivalent chromosomes happens prior to anaphase onset. Does this depend on spindle force? Or is cohesion intrinsically fragile in donor spermatocyte chromosomes? The authors should discuss more about it.

Fig3E, The authors depicted that in normal sized oocytes, univalent-like chromosomes undergo predivision at anaphase. This is somewhat too simplified, because Fig3B shows that a certain population exhibits nondisjunction. This model and description should be corrected to fit the data they demonstrated. If sister segregation at anaphase is predominant, I wonder what happens to sister kinetochore mono-orientation and sister centromeric protection in such univalent-like chromosomes. It would be nice to show centromeric proteins MEIKIN, SGO2 in donor spermatocyte chromosomes versus those of oocyte to examine centromeric cohesion. The authors should clarify this issue.

Line296-294 What do the authors mean by the sentence " It is known that sex chromosomes are prepared to undergo meiosis later than autosomes."?

Significance

The manuscript will provide biological significance for the reproduction fields. There are two major biological significances : They addressed the mechanism of erroneous chromosome segregation in biparental meiosis. They showed that biparental meiosis using spermatocyte-injected oocytes can be applied to production of offspring of azoospermic mice, which would have great impact on reproductive biology field. The data was produced with their high level of technique.

Referee Cross-commenting

I agree to the point described in Reviewer #3's Main points2. It would be better to see SAC proteins.

Transparent Peer Review

---

Download the complete Review Process [PDF] including:

• reviews
• authors' reply
• editorial decisions

</br>

Transparent Peer Review

---

Download the complete Review Process [PDF] including:

• reviews
• authors' reply
• editorial decisions

</br>

Transparent Peer Review

---

Download the complete Review Process [PDF] including:

• reviews
• authors' reply
• editorial decisions

</br>

Transparent Peer Review

---

Download the complete Review Process [PDF] including:

• reviews
• authors' reply
• editorial decisions

</br>

Note: This rebuttal was posted by the corresponding author to Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Reply to the reviewers

1. General Statements [optional]

The comments of the reviewers were highly insightful and enabled us to greatly improve the quality of our manuscript. We provided point-by-point responses to each of the reviewers’ comments. Revisions in the text are highlighted in yellow. We hope that the revisions in the manuscript and our accompanying responses will be sufficient to make our manuscript suitable for publication.

2. Point-by-point description of the revisions

Reviewer #1

- The authors provide no rationale for using the PTI score to measure the protein-coding potential of transcripts. The only attempt to justify this measure is given in the methods: "The definition of PTI score is motivated by our hypothetical concept that translation of pPTI is limited by alternate competing sPTIs." (lines 426-427, page 20). What the PTI score measures is the dominance of the largest predicted ORF over the predicted ORFs, in terms of length. It is not clear why there would be competition for translation of putative ORFs for genuine protein-coding transcripts. An alternative hypothesis, briefly touched upon in the discussion (lines 318-320) is that translation of non-functional ORFs could give rise to the production of toxic proteins, in addition to being costly in terms of energy. The authors should provide the reasoning behind the PTI score and should explain the biological mechanisms that may underlie differences between coding and non-coding transcripts.

Thank you for your comment. We previously identified a de novo gene, NCYM, and showed that its protein has a biochemical function (Suenaga et al 2014; Suenaga et al 2020). However, NCYM was previously registered as a non-coding RNA in the public database, and the established predictors for protein-coding potential, coding potential assessment tool (CPAT), showed a coding probability of NCYM of 0.022, labeling it as a noncoding RNA (new Supplementary Figure 1B). Therefore, we sought to identify a new indicator for coding potential, comparing NCYM with a small subset of coding and non-coding RNAs to determine whether NCYM has sequence features that would allow it to be registered as a coding transcript (data not shown). We found that predicted ORFs, other than major ORFs, seem to be short in coding RNAs. In addition, it has been reported that upstream ORFs inhibit the translation of major ORFs (Calvo et al 2009). Therefore, we hypothesized that the predicted ORFs may reduce the translation of major ORFs, thereby becoming short in the coding transcripts, including NCYM, during evolution. The term ORF refers to an RNA sequence that is translated into an actual product; however, the biological significance of non-translating, predicted ORFs has been largely ignored and remains to be characterized. Therefore, we defined a PTI as an RNA sequence from the start codon sequence to the end codon sequence and did not assume that it would result in a translated product. Thus, PTI can be defined even in genuine non-coding RNAs. The major ORFs are often the longest PTIs (hereafter, primary PTIs or pPTIs) in coding transcripts. Thus, to investigate the importance of pPTIs relative to other PTIs (hereafter, secondary PTIs, or sPTIs) for the evolution of coding genes, we defined a PTI score as the occupancy of the pPTI length to the total PTI length (Figure 1A–B) and assumed that the PTI score was high in coding transcripts. These are the rationale for using the PTI score for protein-coding potential and are now included in the revised manuscript (lines 92-115, page 5-6).

To examine the biological mechanism underlying the difference between coding and noncoding RNAs, we investigated the relationship between translation and PTI scores. We chose a dataset of non-coding RNAs that translated small proteins derived from the databases SmProt and sORF.org. From ribosome profiling and mass spectrometry data, the databases include noncoding RNAs that encode small proteins (less than 100 residues) as well as mRNAs that have extra-small ORFs in addition to major ORFs. The SmProt database divides these small ORFs into three categories: upstream (uORF), small (sORF), and downstream (dORF). The definitions are based on their locations: uORFs and dORFs are located in 5’ and 3’ UTRs, respectively, and sORFs overlap with major ORFs using different reading frames (new Figure 2B). We first calculated PTI scores of lincRNAs encoding small proteins and found that the distribution of these lincRNAs shifted to higher PTI scores compared with the distribution of all lincRNAs (new Figure 2A). Therefore, lincRNA translation is correlated with higher PTI scores. Next, we examined whether PTI scores were associated with the translation occupancy of major ORFs in coding RNAs. We calculated PTI scores in mRNAs with uORF, sORF, or dORFs and found that the distribution of mRNAs encoding such small proteins shifted to lower PTI scores (new Figure 2C). Similar data were obtained from the sORF org dataset (Supplementary Figure 5). These data support the idea that the PTI score is related to the occupancy of the major ORF during translation. These results are now included in the results of the revised manuscript (lines 241-271, pp 12-13).

Translation of small proteins from noncoding RNAs seems to inhibit noncoding functions because of ribosome binding and subsequent translation. On the other hand, translation of sPTIs in coding RNAs seems to inhibit the translation of major ORFs because of competing translations (Calvo et al 2009). At the same time, however, the translation of such proteins may have the advantage of producing new functional proteins/regulatory mechanisms during evolution. Therefore, the right and left shifts of the PTI score that we observed for noncoding and coding RNAs, respectively, seem to be slightly deleterious or beneficial. As further discussed in the responses below, the overlap of distributions of PTI scores between coding and noncoding transcripts was negatively correlated with the effective population size of the species. Therefore, as nearly neutral theory predicts, mutations causing such slightly deleterious/beneficial effects of translation in coding and noncoding transcripts seem to be fixed in species with small effective population sizes (including humans) by genetic drift (Kimura 1968, 1983; Ohta 1992). Clearly, PTI scores are related to translation of PTIs, and their distributions suggest a mechanism for producing bifunctional RNAs that are simultaneously coding and noncoding. The discussion has now been included in the revised manuscript (lines 487-503, pp 23-24).

Calvo SE, Pagliarini DJ, Mootha VK. Upstream open reading frames cause widespread reduction in protein expression and are polymorphic among humans. Proc Natl Acad Sci U S A. 2009 May 5;106(18):7507-12. doi: 10.1073/pnas.0810916106. Epub 2009 Apr 16. PMID: 19372376; PMCID: PMC2669787.

Kimura M. 1968. Evolutionary rate at the molecular level Nature. 217(5129):624-6. PMID: 5637732. https://doi.org/10.1038/217624a0

Kimura, M. (1983). Neutral Theory of Molecular Evolution Cambridge: Cambridge University Press. https://doi.org/10.1093/obo/9780199941728-0132

Ohta T. 1992. The Nearly Neutral Theory of Molecular Evolution. Annu Rev Ecol Syst. 23:263-86.

- The presence of ORFs in transcripts has long been used as a predictor of their protein-coding potential. For example, the ORF size and the ORF coverage are part of the set of predictors implemented in CPAT (Wang et al., 2013). The PTI score is necessarily related to these methods, yet no comparison is provided. If the PTI score is to be used as a measure to classify transcripts as coding or non-coding, its performance should be compared to other classifiers, including those that use the presence of ORFs as a predictor (e.g., CPAT) but not only (e.g., PhyloCSF, based on the pattern of sequence evolution).

Thank you for your comment. As you noted, our reasons for using the PTI score were not clearly described in the original manuscript and are now included in the Results section (lines 92-115, page 5-6). As mentioned in response to comment 1, CPAT was not able to predict NCYM as a coding transcript (Supplementary Figure 1B). Furthermore, we intended to use this new concept to identify the RNA sequence elements that determine protein-coding potential, but did not intend to use the score as a classifier of coding or non-coding RNAs. Many studies have identified bifunctional RNAs that are simultaneously coding and noncoding (Li and Liu 2019; Huang Y et al. 2021). Moreover, neutrally evolving peptides are encoded by small ORFs of noncoding RNAs, possibly contributing to the evolutionary origin of new functional proteins (Ruiz-Orera et al. 2014). Therefore, we argue that such dichotomous classification is often misleading, by unconsciously ignoring ncRNAs that encode functional or nonfunctional small proteins. Additionally, this approach has several technical problems. For a training set for use with such a classification, we need a dataset of genuine noncoding RNAs. However, it is quite difficult to define such noncoding RNAs without bias, for example, for cell or tissue types, including cancer or normal cells/tissues. Increasing evidence has shown peptide translation from known noncoding RNAs (Li and Liu 2019; Huang Y et al. 2021); moreover, some of these peptides are specific to the cellular context (Dohka et al 2021). Therefore, we cannot be certain that we are identifying genuine noncoding RNAs from the datasets from ribosome profiling or mass spectrometry, which neither cover all cell/tissue types nor all physiological contexts.

We agree with you in that we need to compare PTI scores with other indicators of coding potential, such as transcript length, ORF size, and ORF coverage. ORFs of less than 100 residues have been used to define noncoding RNAs; thus, such RNAs necessarily have shorter ORF sizes relative to coding RNAs. Therefore, we calculated these indicators by focusing on noncoding RNAs that encode proteins, but not coding RNAs (new Supplementary Figure 4). The PTI score distribution shifted to the right for lincRNAs encoding small proteins, indicating that the PTI score is related to translation (new Figure 2C). In contrast, the distributions of transcript length, ORF size, and ORF coverage did not shift higher for noncoding RNAs encoding small proteins (new Supplementary Figure 4), although a slight shift to higher ORF coverage was found. Therefore, we argue that the PTI score is a better indicator of translation than transcript length, ORF size or ORF coverage. These results are now included in the results of the revised manuscript (lines 241-255, page 12).

- The authors compare the observed PTI score distributions with the PTI scores from random or shuffled sequences. They conclude that the PTI scores do not depend on transcript lengths but on transcript sequences (lines 122-123). However, this is not true for non-coding RNAs, for which the observed and randomized distributions are very similar. The relationship between transcript length and PTI scores should be analyzed into more detail. Are the annotated non-coding transcripts with high PTI scores particular in terms of length?

We analyzed the length of high-PTI-score transcripts compared to all lncRNA transcripts. The average high-PTI-score with high coding potential (0.6 PTI score −29), consistent with the distribution of transcript length in lincRNAs translating small proteins (new Supplementary Figure 4C). Therefore, the high PTI scores are not simply due to the larger ORF size derived from longer transcript length, but also because of the occupancy of pPTI among all PTIs. The occupancy of pPTI can be estimated by ORF coverage or PTI score, and we can easily see that transcript length (the denominator of ORF coverage) correlates with the sum of the lengths of all PTIs (the denominator of the PTI score). Thus, we need to clarify which indicators have more biological significance in terms of gene evolution. Higher PTI scores in noncoding RNAs cause overlap of the coding and noncoding transcripts in eukaryotes, especially in multicellular eukaryotes (new Figure 4 and 5). The overlaps of PTI score distributions between coding and noncoding RNAs (Opti) were positively and negatively correlated with mutation rate and effective population size, respectively, and approximated by logarithmic or exponential relationships (new Figure 6). Because the inverse of the effective population size defines the strength of genetic drift relative to the strength of selection, the overlaps quantified by Opti seem to be derived from genetic drift. These results clearly suggest that the observed PTI score distribution of noncoding RNAs is not random. In contrast, ORF coverage (Ocov) showed a weaker relationship with mutation rates and effective population sizes (new Supplementary Figure 8 and 9). These results suggest that ORF coverage is less related to gene evolution than PTI score, with the weak relationship seemingly indirectly derived from the correlation with the PTI score. We have now included these results in the revised manuscript (lines 306-322, page 15).

- The authors discuss in depth the correlation between PTI scores and PTI-based protein-coding potential measures (e.g., section "PTI scores correlate with protein coding potential in humans and mice", starting line 125; section "Relationship between the PTI score and protein-coding potential", starting line 243). Given that the protein-coding potential is directly derived from the PTI score distributions for coding and non-coding transcripts, it is not surprising that the two should be correlated. The significance of observing a linear or a sigmoid relationship is not clearly explained.

As you noted, the protein-coding potential was directly derived from the PTI score distributions. Therefore, if the distribution for coding RNA shows a higher or lower PTI score compared to that of noncoding RNA, the protein-coding potential is expected to be positively or negatively correlated with the PTI score. If the distributions of coding and noncoding RNA significantly overlapped (Opti > 0.7), the protein-coding potential became constant and was not correlated with the PTI score (new Figure 7 and new Supplementary Figure 10). Thus, the PTI score is not always positively correlated with the protein-coding potential.

We had divided the species into three groups; the sigmoidal group, the linear group, or others based on the intercept and slope in the linear approximation, but considering the fit of the linear approximation, there is no essential difference between the sigmoidal and linear groups. Therefore, in the revised text, we classify the species into two groups: linear and constant (new Figure 7 and Supplementary Figure 10). We have now replaced the figures and added a new interpretation of the results in the revised manuscript (lines 341-353, pages 16-17).

- The authors use the entire set of annotated coding and non-coding transcripts to assess the distribution of PTI scores and to define the protein-coding potential. Traditionally, for methods that aim to classify transcripts as coding or non-coding, this is done using "bona fide" coding and non-coding transcripts, which are used　as training sets. The efficiency of the method can then be evaluated using a test set of transcripts. This aspect is lacking here and should be implemented.

As we wrote in response to your comment 2, we aimed to examine what RNA sequence elements determine genuine-coding RNA but not to identify the classifier of coding and noncoding RNA. Technically, the “bona fide” coding and noncoding RNAs cannot be rigorously defined, given the possible existence of unidentified bifunctional RNAs in the testing sets; therefore, more traditional approaches often eliminate such possibilities.

- The comparisons among species are likely biased by the quality of lncRNA annotations in non-model organisms - cf. high variations among primates, which are likely driven by the annotation quality and depth.

As written in the response to comment 3, the variation of PTI score distribution in lncRNA is not random, and overlaps with the distribution of coding RNA are negatively correlated with effective population size (new Figure 6). In addition, we found that the tissue-specific expression of lncRNA influences the PTI score distribution in multicellular eukaryotes (new Figure 8 C and D and new Supplementary Figure 11 and 12). Therefore, the variation is caused, at least in part, by the specificity of gene expression, and it thus contains biological significance. These results are now included in the revised manuscript (lines 383-402, pages 18-19).

Based on these results, we expect that the quality of the lncRNA annotations derived from two major databases, Ensenbl and RefSeq, are well curated and sufficient to compare the PTI score distribution. Realistically, there is no database that catalogs a number of curated lncRNAs from various species other than these two. However, we also expect that recent progress in whole genome sequencing and transcriptome analysis of vertebrates may improve the annotation of lncRNAs, including non-model organisms, and provide more ideal datasets for comparisons among species.

- The differences among bacteria, archaea and eukaryotes should be discussed into more depth. In bacteria, the genuine ORF is well defined by the presence of translation signals (e.g., Shine-Dalgarno sequence). Other factors are also at work in both prokaryotes and eukaryotes, including RNA secondary structures. The relationship between these factors and the PTI score should be discussed.

The Shine–Dalgarno sequence in bacteria and the Kozak sequence in eukaryotes have been identified as important regulatory elements for ribosome binding, but these sequences are not essential for all coding RNAs, and their significance is not well characterized, especially in noncoding RNAs that are translated. Recent research has sought to identify the determinants that regulate ribosome binding to lncRNAs using 99 characteristics, including the weight of each base at the −6 to +1 positions relative to the start codon (Kozak-like sequence) or RNA secondary structure (Zeng et al 2018). They found that transcript length is a stronger indicator than either of these characteristics for ribosome binding in human lncRNAs. Because the PTI score is a better indicator for translation of lincRNAs than transcript length (new Supplementary Figure 4C), we would argue that Kozak sequences and RNA secondary structures are not reliable indicators for ribosome binding of lncRNAs, and their significance should be limited to more specific transcript classes. Furthermore, Hata et al. recently showed that the Kozak sequence is a negative regulator of de novo gene birth in plants (Hata et al. 2021). Therefore, these sequence characteristics seem to evolve after the birth of coding transcripts and are not generally involved in new coding gene origination from noncoding RNAs.

Zeng C, Hamada M. 2018. Identifying sequence features that drive ribosomal association for lncRNAs BMC Genomics. 19(Suppl 10):906. PMID: 30598103; PMCID: PMC6311901. https://doi.org/10.1186/s12864-018-5275-8

Hata T, Satoh S, Takada N, Matsuo M, Obokata J. 2021. Kozak sequence acts as a negative regulator of de novo transcription initiation of newborn coding sequences in the plant genome. Mol Biol Evol. 38:2791-2803. PMID: 33705557; PMCID: PMC8233501. https://doi.org/10.1093/molbev/msab069

- From an evolutionary perspective, the effective population size (Ne) is also likely related to the "quality" of the ORFs. An analysis of Ne vs. the PTI score distributions would be an interesting addition to this manuscript.

We appreciate this comment. We now include an analysis of the relationship between Ne and PTI scores by defining an indicator of the extent of overlap in the PTI score distributions between coding and noncoding transcripts. This overlapping score was calculated based on PTI scores or ORF coverage and named Opti or Ocov, respectively. Opti showed positive and negative correlations with mutation rates (Up) and effective population size (Ne), respectively (new Figure 6A), suggesting that the overlap of PTI score distribution is related to slightly deleterious or beneficial mutations fixed in populations due to genetic drift. Furthermore, using the relationship between Ne and Opti, we calculated the minimum effective population size to be approximately 1000, which is consistent with the results from conservation biology (Frankham et al. 2014). Indeed, species at risk of extinction had significantly higher Opti than species with little risk of extinction (left panel, new Figure 6B). In addition, Opti was higher for species with a decrease compared to those with stable population sizes (right panel, new Figure 6B). These results are now included in the revised manuscript (lines 323-332, page 15-16).

Frankham R, Bradshaw CJA. 2014. Genetics in conservation management: Revised recommendations for the 50/500 rules, Red List criteria and population viability analyses, Biological Conservation, 170:56-63, https://doi.org/10.1016/j.biocon.2013.12.036

Reviewer #1 (Significance (Required)):

This manuscript is lacking in novelty and is not well positioned in the field. If the aim of this work is to provide a method to classify transcripts as coding or noncoding, the authors should provide detailed comparisons with existing methods (see above). If the aim is to understand what defines a genuine protein-coding transcript, then the biological mechanisms should be better described and the comparisons among species and among functional categories of genes should be further developed. The idea of using the "dominance" of the largest ORF compared to the other predicted ORFs is interesting, and provides a new element compared to existing methods that rely exclusively on ORF length and ORF coverage. I would recommend that the authors develop this idea further and discuss the advantages of using the ORF dominance compared to just the ORF length or coverage.

Thank you for your comment. To address this, we have revised the description of our aim to investigate what defines a genuine protein-coding transcript and found that doing so prompted us to learn that the extent of overlap of PTI score distribution between coding and noncoding transcripts is negatively correlated with effective population size. In addition, we have added characterizations of functional categories of high-PTI-score lncRNAs in mice (new Supplementary Tables 6 to 8) and C. elegans (new Supplementary Tables 9, 10, and 11). Comparison of ORF size and coverage to PTI score showed that PTI score is a better indicator for translation of lncRNAs than these indicators and has biological significance in molecular evolution because of the clear correlation between mutation rate and effective population size. These results and related descriptions are now included in the revised manuscript (lines 323-332, pages 15-16; lines 210-218, pages 10-11).

**Referee Cross-commenting**

I fully agree with Reviewer 2's remarks. In particular, adding ribosome profiling analyses is an excellent idea and could substantially improve the manuscript.

We investigated the PTI scores in lncRNAs that are translated, using ribosome profiling data, and found that PTI scores correlated with translation (lines 241-271, pages 12-13). Thank you for this excellent suggestion.

Reviewer 2

**Major comments:**

- some validation of their predictions of coding potential would be good to add. There are plenty of ribosome profiling experiments out there for some of the studied organisms (human, mouse, E. coli) that could be used to show that indeed some of the non-coding RNAs are misclassified and have ribosome density across the predicted open reading frames.

Thank you for your comment. As noted in our response to Reviewer 1 above, we calculated the PTI scores of translated lncRNAs from the two databases and found that the PTI score correlates with translation of both coding and noncoding RNAs (new Figure 2 and new Supplementary Figures 4 and 5). As noted above, such translation seems to produce slightly deleterious/beneficial effects, thereby becoming fixed in species with smaller effective population sizes by genetic drift. These results and related discussion are now included in the revised manuscript (lines 241-271, pages 12-13; lines 323-332, pages 15-16; lines 487-503, page 23-24).

- the manuscript is at times difficult to follow and the implication of the statements may not be immediately clear to the readers, particularly those without formal training in bioinformatic methods; even in the abstract. Some examples: "The relationship between the PTI score and protein-coding potential was sigmoidal in most eukaryotes; however, it was linear passing through the origin in three distinct eutherian lineages, including humans". Here it is not clear what this means (without reading the paper) - and even after reading the paper the importance of noting the sigmoidal vs linear relationship of PTI vs. protein-coding potential is unclear. I would encourage the authors to double-check that they provide a clear interpretation of their results, with readers unschooled in proper statistics in mind.

Thank you for these comments. As we noted in response to comment 4 of Reviewer 1, considering the fit of the linear approximation, there was no essential difference between the sigmoidal and linear groups. Therefore, in the revised manuscript, we classify the species into two groups: linear and constant (new Figure 7 and Supplementary Figure 10). We also propose and diagram a new gene birth model to help readers understand our interpretations more easily (Figure 9). These results and discussion are now included in the revised manuscript (lines 341-353, pages 16-17; lines 514-538, pages 24-25).

- For the definition of PTI and protein-coding potential the authors refer to the Materials and Methods. I would encourage to explain in plain terms in the results section 1.) how they decided on this particular formalization and 2.) explain clearly what this means.

Thank you for your suggestion. We have included a concise definition in the revised text in plain terms (lines 107-115, page 5-6; lines 144-146, page 7).

- The definition of protein coding potential for appears to be dependent on database classification of a transcript as either coding and non-coding. Particularly for organisms with complex transcriptomes, databases may not contain the proper information - what are the implications for their protein-coding potential score?

Organisms with complex transcriptomes, such as multicellular organisms, present difficulties in classifying coding vs. noncoding transcripts because RNAs classified as noncoding based on proteomic data from a subset of cell types may encode functional proteins in other cell types for which proteomic data are not available. To examine whether cell types affect the PTI distribution of coding and noncoding transcripts, we analyzed transcriptomic data from five mammals (human, mouse, rat, macaque, and opossum) and found that the PTI score distributions were similar in most cell or tissue types for noncoding transcripts (new Figure 8C and Supplementary Figure 11). However, PTI score distributions for noncoding RNA in mature testes showed a rightward shift for all five species (new Figure 8C and Supplementary Figure 11).

Furthermore, we found that tissue specificity of RNA expression was correlated with PTI score (new Figure 8D and new Supplementary Figure 12 and 13), with more specific expression associated with higher PTI scores in all five species, with the majority of the tissue-specific expression in mature testis. Therefore, the mature testis is a special tissue that expresses noncoding RNAs with high coding potentials. These results support the hypothesis that the testis is a special organ for new gene origination (Kaessmann 2010). We have added these results and discussion to the revised manuscript (lines 383-402, pages 18-19; lines 427-434, pages 20-21; lines 435-445, page 21).

Kaessmann H. 2010. Origins, evolution, and phenotypic impact of new genes. Genome Res, 20:1313-26. Epub 2010 Jul 22. PMID: 20651121; PMCID: PMC2945180. https://doi.org/10.1101/gr.101386.109

- The authors completely ignore plants - would it make sense to expand their analysis to this branch of the tree of life?

In Supplementary Figure 5 of our original manuscript (new Supplementary Figure 7), we have included the PTI score distributions from plants. We also present their overlapping scores (Opti) in the revised manuscript.

Reviewer 2 (Significance (Required)):

The manuscript presents an elegant way to predict protein-coding and non-coding RNAs, which may be very relevant to the study of organisms with complex transcriptomes. The audience for the manuscript at the moment may be more limited to scientists trained and working in the field of bioinformatics, but with some integration of transcriptomics and ribosome profiling data, as well as an effort to make the results accessible to scientists not trained in bioinformatics, this manuscript may be relevant and of interest to researchers working on the biology of long non-coding RNAs and translation in general. My expertise: systems biology of RNA binding proteins, transcriptomics, RNA biology.

**Referee Cross-commenting**

I fully agree with my co-reviewer regarding additional analyses to strengthen the manuscript.

Thank you for these comments. We analyzed noncoding RNAs using ribosome profiling data and transcriptomes in different tissues. We found that high PTI scores correlated with translation of noncoding RNAs, and that such high PTI-score noncoding RNAs were specifically expressed in mature testes. Because the effective population size was inversely correlated with the overlap of PTI distributions, the slightly deleterious or beneficial mutations in germ cells of matured testis seem to generate high-PTI score noncoding RNAs as candidates for new coding genes in the next generation. This idea is consistent with the hypothesis that new coding transcripts are derived from noncoding transcripts expressed in spermatocytes and spermatids in mature testes. In addition, we found that human noncoding transcripts with high PTI scores tended to be involved in transcriptional regulation, and the target gene of MYCN was significantly enriched as the original gene. A recent study showed that binding sites for transcription factors, including MYCN, are mutational hotspots in human spermatogonia (Kaiser et al. 2021). Therefore, the PTI score offers an opportunity to integrate the concept of gene birth with classical molecular evolutionary theory, thereby contributing to our understanding of evolution.

Kaiser VB et al. 2021. Mutational bias in spermatogonia impacts the anatomy of regulatory sites in the human genome. Genome Res. Epub ahead of print. PMID: 34417209. https://doi.org/10.1101/gr.275407.121

Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Referee #2

Evidence, reproducibility and clarity

In the manuscript "Potentially translated sequences determine protein-coding potential of RNAs in cellular organisms" Suenaga and colleagues analyze the available transcriptomes from 100 prokaryotes and eukaryotes, as well as >100 viruses to understand whether transcripts tend to be translated or not. They develop a potentially translated island score (PTI) that combines the number and length of open reading frames in a transcript. From there they develop a protein-coding potential score that combines PTI with database information on coding and non-coding transcripts in various organisms and that in some sense predicts whether a transcript would fall in the coding or non-coding category. The main takeaway appears to be that in prokaryotes PTIs and protein coding potential strongly differentiates coding and non-coding transcripts, while in eukaryotes these differences appear to be more fluid. The manuscript presents an interesting bioinformatic analysis of coding properties across the phylogenetic field and may represent an interesting resource. The audience for the manuscript at the moment may be more limited to scientists trained and working in the field of bioinformatics, but with some integration of transcriptomics and ribosome profiling data, as well as an effort to make the results accessible to scientists not trained in bioinformatics, this manuscript may be relevant and of interest to researchers working on the biology of long non-coding RNAs and translation in general.

Major comments:

• some validation of their predictions of coding potential would be good to add. There are plenty of ribosome profiling experiments out there for some of the studied organisms (human, mouse, E. coli) that could be used to show that indeed some of the non-coding RNAs are misclassified and have ribosome density across the predicted open reading frames.
• the manuscript is at times difficult to follow and the implication of the statements may not be immediately clear to the readers, particularly those without formal training in bioinformatic methods; even in the abstract. Some examples: "The relationship between the PTI score and protein-coding potential was sigmoidal in most eukaryotes; however,it was linear passing through the origin in three distinct eutherian lineages, including humans". Here it is not clear what this means (without reading the paper) - and even after reading the paper the importance of noting the sigmoidal vs linear relationship of PTI vs. protein-coding potential is unclear. I would encourage the authors to double-check that they provide a clear interpretation of their results, with readers unschooled in proper statistics in mind.
• For the definition of PTI and protein-coding potential the authors refer to the Materials and Methods. I would encourage to explain in plain terms in the results section 1.) how they decided on this particular formalization and 2.) explain clearly what this means.
• The definition of protein coding potential for appears to be dependent on database classification of a transcript as either coding and non-coding. Particularly for organisms with complex transcriptomes, databases may not contain the proper information - what are the implications for their protein-coding potential score?
• The authors completely ignore plants - would it make sense to expand their analysis to this branch of the tree of life?

Significance

The manuscript presents an elegant way to predict protein-coding and non-coding RNAs, which may be very relevant to the study of organisms with complex transcriptomes.

The audience for the manuscript at the moment may be more limited to scientists trained and working in the field of bioinformatics, but with some integration of transcriptomics and ribosome profiling data, as well as an effort to make the results accessible to scientists not trained in bioinformatics, this manuscript may be relevant and of interest to researchers working on the biology of long non-coding RNAs and translation in general.

My expertise: systems biology of RNA binding proteins, transcriptomics, RNA biology.

Referee Cross-commenting

I fully agree with my co-reviewer regarding additional analyses to strengthen the manuscript.

Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Referee #1

Evidence, reproducibility and clarity

Summary

The manuscript submitted by Suenaga and co-authors presents a method to evaluate the protein-coding potential of transcripts. This method is based on an index that they name the PTI (potentially translated island) score, which represents the ratio between the length of the largest predicted ORF and the sum of all the predicted ORF lengths, for each transcript. The author compare PTI score distributions between transcripts classified as protein-coding and as non-coding in public nucleotide databases, for a wide range of species, including bacteria, archaea, eukaryotes and viruses. They derive from this comparison a measure of the protein-coding potential of transcripts. To validate this approach, the authors evaluated the distributions of Ka/Ks values for transcripts annotated as coding or non-coding, in various classes of PTI-based protein-coding potential. The main finding of the manuscript stems from the comparison among species: the authors find that bacteria and archaea have narrow, non-overlapping PTI distributions for coding and non-coding transcripts, while eukaryotes have broader and more overlapping PTI distributions.

Major comments

• The authors provide no rationale for using the PTI score to measure the protein-coding potential of transcripts. The only attempt to justify this measure is given in the methods: "The definition of PTI score is motivated by our hypothetical concept that translation of pPTI is limited by alternate competing sPTIs." (lines 426-427, page 20). What the PTI score measures is the dominance of the largest predicted ORF over the predicted ORFs, in terms of length. It is not clear why there would be competition for translation of putative ORFs for genuine protein-coding transcripts. An alternative hypothesis, briefly touched upon in the discussion (lines 318-320) is that translation of non-functional ORFs could give rise to the production of toxic proteins, in addition to being costly in terms of energy. The authors should provide the reasoning behind the PTI score and should explain the biological mechanisms that may underlie differences between coding and non-coding transcripts.
• The presence of ORFs in transcripts has long been used as a predictor of their protein-coding potential. For example, the ORF size and the ORF coverage are part of the set of predictors implemented in CPAT (Wang et al., 2013). The PTI score is necessarily related to these methods, yet no comparison is provided. If the PTI score is to be used as a measure to classify transcripts as coding or non-coding, its performance should be compared to other classifiers, including those that use the presence of ORFs as a predictor (e.g., CPAT) but not only (e.g., PhyloCSF, based on the pattern of sequence evolution).
• The authors compare the observed PTI score distributions with the PTI scores from random or shuffled sequences. They conclude that the PTI scores do not depend on transcript lengths but on transcript sequences (lines 122-123). However, this is not true for non-coding RNAs, for which the observed and randomized distributions are very similar. The relationship between transcript length and PTI scores should be analyzed into more detail. Are the annotated non-coding transcripts with high PTI scores particular in terms of length?
• The authors discuss in depth the correlation between PTI scores and PTI-based protein-coding potential measures (e.g., section "PTI scores correlate with protein-coding potential in humans and mice", starting line 125; section "Relationship between the PTI score and protein-coding potential", starting line 243). Given that the protein-coding potential is directly derived from the PTI score distributions for coding and non-coding transcripts, it is not surprising that the two should be correlated. The significance of observing a linear or a sigmoid relationship is not clearly explained.
• The authors use the entire set of annotated coding and non-coding transcripts to assess the distribution of PTI scores and to define the protein-coding potential. Traditionally, for methods that aim to classify transcripts as coding or non-coding, this is done using "bona fide" coding and non-coding transcripts, which are used as training sets. The efficiency of the method can then be evaluated using a test set of transcripts. This aspect is lacking here and should be implemented.
• The comparisons among species are likely biased by the quality of lncRNA annotations in non-model organisms - cf. high variations among primates, which are likely driven by the annotation quality and depth.
• The differences among bacteria, archaea and eukaryotes should be discussed into more depth. In bacteria, the genuine ORF is well defined by the presence of translation signals (e.g., Shine-Dalgarno sequence). Other factors are also at work in both prokaryotes and eukaryotes, including RNA secondary structures. The relationship between these factors and the PTI score should be discussed.
• From an evolutionary perspective, the effective population size (Ne) is also likely related to the "quality" of the ORFs. An analysis of Ne vs. the PTI score distributions would be an interesting addition to this manuscript.

Significance

This manuscript is lacking in novelty and is not well positioned in the field. If the aim of this work is to provide a method to classify transcripts as coding or non-coding, the authors should provide detailed comparisons with existing methods (see above). If the aim is to understand what defines a genuine protein-coding transcript, then the biological mechanisms should be better described and the comparisons among species and among functional categories of genes should be further developed. The idea of using the "dominance" of the largest ORF compared to the other predicted ORFs is interesting, and provides a new element compared to existing methods that rely exclusively on ORF length and ORF coverage. I would recommend that the authors develop this idea further and discuss the advantages of using the ORF dominance compared to just the ORF length or coverage.

Referee Cross-commenting

I fully agree with Reviewer 2's remarks. In particular, adding ribosome profiling analyses is an excellent idea and could substantially improve the manuscript.

Transparent Peer Review

---

Download the complete Review Process [PDF] including:

• reviews
• authors' reply
• editorial decisions

</br>

Transparent Peer Review

---

Download the complete Review Process [PDF] including:

• reviews
• authors' reply
• editorial decisions

</br>

Transparent Peer Review

---

Download the complete Review Process [PDF] including:

• reviews
• authors' reply
• editorial decisions

</br>

Transparent Peer Review

---

Download the complete Review Process [PDF] including:

• reviews
• authors' reply
• editorial decisions

</br>

Note: This rebuttal was posted by the corresponding author to Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Reply to the reviewers

Manuscript number: RC-2021-01041 Corresponding author(s): Gregory P. Way, PhD

1. General Statements

On behalf of the authors, I’d like to thank the Review Commons team for sending our manuscript out for review. I’d also like to thank the three anonymous reviewers for providing valuable feedback that will improve the clarity, focus, and analysis interpretation presented in our manuscript.

To prompt the editorial team, our paper provides two well-controlled innovations:

We are the first to train variational autoencoders (VAEs) on classical image features extracted from Cell Painting images. VAEs are commonplace in, and have contributed major discoveries to, other biomedical data types (e.g. transcriptomics), but they have been underexplored in morphology data. In our paper, we trained and optimized three different VAE variants using Cell Painting readouts and compared these variants against shuffled data, against PCA (a nonlinear dimensionality reduction algorithm commonly used as a VAE control), and against L1000 (mRNA) readouts from the same perturbations. We found that cell morphology VAEs train with different settings than gene expression data, and that they generate interpretable latent spaces that depend on the chosen VAE variant.

We tested special VAE properties to predict polypharmacology cell states in a novel way. Polypharmacology is a major reason why drugs fail to reach the bedside. Off-target effects cause unintended toxicity, and lead to adverse clinical events. In our paper, we used VAE latent space arithmetic (LSA) to predict polypharmacology cell states; in other words, what cells might look like if we perturbed them with a compound that had two mechanisms of action (MOA). We compared our results to shuffled data, PCA, and to LSA performed with VAEs trained using L1000 readouts. We found that cell morphology and gene expression provide complementary information, and that we could predict some polypharmacology cell states robustly, while others were more difficult to predict.

We found value in all of the reviewer comments. We intend to conduct all but four of the proposed analyses to supplement our aforementioned innovations.

In the following revision plan, we include all reviewer comments exactly as they were written. The reviewers often had overlapping suggestions. In these cases, we grouped together similar reviewer comments and responded to them once.

We include three sections: 1) A description of the revisions we plan to conduct in the near future; 2) A description of changes we have already made; and 3) A description and rationale of changes we will not pursue.

Lastly, we would like to highlight that all reviewers provided positive feedback in their reviews. They discussed our paper as “conceptually and technically unique” and were positive about our methods section, stating that we did a “good job making everything available and reproducible”. Our methods section is complete, and we provide a fully reproducible and versioned github repository. We will release a second version of our github repository when we complete our revision plan to maintain clarity for our submitted version and the peer-reviewed version.

1. Description of the planned revisions

2.1. Address UMAP interpretability to provide a deeper description of MOA performance

Reviewer 1: Instead of using UMAP embedding, it would be better to compare reconstruction error or show a reconstructed image with the original image to claim that models reliably approximate the underlying morphology data.

Reviewer 1: Rather than just stating that the VAE's did not span the original data distribution and saying beta-VAE performed best by eye, some simple metrics can be drawn to analyze the overlap in data for a more direct and quantified comparison. Researchers should also explain what part of the data is not being captured here. Some analysis of what the original uncaptured UMAP represents is important in understanding the limitations of the VAEs' capacity.

Reviewer 2: The authors compare generation performance based on UMAP. In the UMAP space, data tend to cluster together even though they might be far from each other in the feature space. I would like to see more quantitive metrics on how well these methods capture morphology distributions. You can compute metrics like MMD distance, kullback leibler (KL), earthmoving distance, or a simple classifier trained on actual MoA classes tested on generated data.

We agree with the reviewers that evaluating reconstruction loss in addition to providing the UMAP coordinates would improve understanding of VAE limitations and enable a better comparison of VAE performance. We will analyze reconstruction loss across models and include these data as a new supplementary figure, which will enable direct comparisons across models and across different MOAs.

We also agree that UMAP interpretation can be misleading. While currently state-of-the-art, UMAP has mathematical limitations that prevent interpretation of global data structures. However, there are emerging tools, including a new dimensionality reduction algorithm, called PaCMAP, which aims to preserve both local and global structure (Wang et al, 2021). We will explore this tool to determine, both mathematically and empirically, which is most appropriate for our dataset by cross-referencing the visualization with our added supplementary figure describing per-MOA reconstruction loss.

We would also like to emphasize that we trained our VAEs using CellProfiler readouts from Cell Painting images and not the raw Cell Painting images themselves. As this was one of our primary innovations, this detail is extremely important. Therefore, we have improved clarity and added emphasis to this point in the manuscript introduction and discussion (see section 3).

2.2. More specific comparisons of MOA predictions to shuffled data and improved description of MOA label accuracy

Reviewer 1: It is difficult to know the clear threshold for successful performance is on figures like Figure 7 and SFigure 9, but by and large, it appears that the majority of predicted combination MOAs were not successful. Without the ability to either A) adequately predict most all combinations from individual profiles that were used in training or B) an explanation prior to analysis of which combination will be able to predict, it is difficult to see this method being used since the combinatorial predictions are more likely not informative.

Reviewer 1: The researchers justify the poor performance compared to shuffled data, by saying that A) MOA annotations are noisy and unreliable and B) they MOAs may only manifest in other modalities like what was seen in the L1000 vs morphology predictability. While these might be true, knowing this the researchers should make an effort to clean and de-noise their data and select MOAs that are well-known and reliable, as well as, selecting MOAs for which we have a known morphological or genetic reaction.

Reviewer 3: Figure 6 is missing error bars (standard deviation of the L2 distance) and, as such, is hard to draw conclusions from.

We thank the reviewers for raising this concern. We agree that it is critical, and we appreciate the opportunity to address it.

All three of these comments relate to being unable to draw conclusions from our results when most A∩B predictions appear to have no difference from shuffled controls. Therefore, to address this comment, we will update our LSA evaluation to compare each MOA to a matched set of randomly shuffled data. Specifically, in our existing comparison, we realized a methodological fallacy in how we're displaying these data shuffles. We should be comparing specific MOA combinations to their corresponding shuffled results instead of comparing all to all, which will artificially decrease performance when there are polypharmacology predictions that fail to recapitulate the ground truth cell states.

We have connected with Paul Clemons, the senior director Director of Computational Chemical Biology Research at the Broad Institute of MIT and Harvard, who has informed us that the Drug Repurposing Hub annotations are among the most well documented. Therefore, while we know that biological annotations are often incomplete, our original text overemphasized the amount of noise contributed by inaccurate labels. We therefore added the following sentence to the discussion to clarify this important point:

“However, the Drug Repurposing Hub MOA annotations are among the most well-documented resources, so other factors like different dose concentration and non-additive effects contribute to weak LSA performance for some compound combinations (Corsello et al, 2017).”

We will also update our supplementary figure to account for specific MOA shuffling and include additional text comparing Cell Painting and L1000 showing which MOAs perform best in which modality.

2.3. More detailed evaluation of MOA performance across drug variance and drug classes

Reviewer 1: With the small number of combinations that are successfully predicted, to build confidence in the performance, it would be necessary to explain the reason for the differences in performance. Further experimentations should be done looking into any relationship between the type of MOAs (and their features) and the resulting A|B predictability. Looking at Figure 7, the top-performing combinations are comprised entirely of inhibitor MOAs. If the noisiness of the data is a factor, there should be some measurable correlation between feature noisiness and variation and the resulting A|B predictability from LSA.

We agree with the reviewer that further experimentation would be helpful to gain confidence in our LSA performance. We plan to perform two different analyses to address this question. First, we will compare profile reproducibility (median pairwise correlations among MOAs) to MOA predictability. This will provide insight to determine the relationship between MOA measurement variance and performance. Second, we will split MOAs by category (e.g. inhibitor, activator) and test if there are significant performance differences between categories across VAE models in both L1000 and Cell Painting data. This will tell us if there are certain trends in the type of MOAs we’re able to predict. If there is, this would be useful knowledge since it could suggest that certain types of MOAs are associated with a more consistent cell state.

2.4. Higher confidence in LSA overfitting assessment

Reviewer 1: To show that the methodology works well on unseen data, researchers withheld the top 5 performing A|B MOAs (SFig 9) and showed they were still well predicted. This is not the most compelling demonstration since the data to be held out was selected with bias as the top-performing samples. It would be much more interesting to withhold an MOA that was near or only somewhat above the margin of acceptability and see how many holdouts affected the predictability of those more susceptible data points. From my best interpretation, the hold-out experiment also only held out the combination MOA groups from training. It would be better if single MOAs (for example A) which were a part of a combination of MOA (A|B) were also held out to see if predictability suffered as a result and if generalizability did extend to cells with unseen MOAs (not just cells which had already highly performing combinations of seen MOAs).

We believe our original analysis was extremely compelling. Even if we removed the top MOAs from training, we were still able to capture their combination polypharmacology cell states through LSA. We find this similar to removing all pictures of sunglasses in an image corpus of human faces, but still being able to reliably infer pictures of people wearing sunglasses. Specifically, this tells us that our model is learning some fundamental data generating function that our top performing MOAs tap into regardless of if they are present or not in training.

However, we agree with the reviewer that withholding intermediate-performing MOAs would also be informative, but for a separate reason. Unlike the best predicted MOAs, the intermediate MOAs are likely more susceptible to changes in the training data, so it would be interesting to determine if intermediate MOAs’ performance is a result of overfitting instead of truly learning aspects of the data generating function. We plan to perform this new analysis and add the results to Supplementary Figure 8 as a subpanel and add a full description of the approach to the appropriate methods subsection.

2.5. Additional metrics to evaluate LSA predictions to provide more confident interpretation

Reviewer 2: The predictions are evaluated using L2 distances, which I find not that informative. I would like to see other metrics (correlation or L1 or distribution distances in previous comments)

We agree with the reviewer that using more than one metric would be helpful because oftentimes a single metric does not tell a complete story. We will add a panel to the LSA supplementary figure (Supplementary Figure 7), using Pearson correlation instead. While L2 distances will tell us how close predictions are to ground truth, Pearson correlations will tell us how consistent, on average, we are able to predict feature direction.

2.6. Adding a performance-driven feature level analysis to categorize per-feature modeling ability

Reviewer 2: I would like to see feature-level analysis, which features are well predicted and which ones are more challenging to predict?

We agree with the reviewer that feature level analysis would be interesting to study. We believe that understanding which features are easy and hard to model could give insight into why certain MOAs (which could be associated with more signal in certain Cell Painting features) are predicted better than others.

However, we are concerned that it is difficult to have an objective measurement of which features are easier to model because features that have less variation might be easier to model. So, we will analyze the correlation between individual feature reconstruction loss vs. feature variance across profiles. We will color-code the points to represent feature groups or channels. This analysis will not only demonstrate the relationship between feature variance and modeling ability, but also provide insight into the difficulty of modeling individual CellProfiler features.

1. Description of the revisions that have already been incorporated in the transferred manuscript

3.1. Documenting positive feedback as provided by the three reviewers

Reviewer 1: With access to the dataset, the posted GitHub, and documentation in the paper, I believe that the experiments are reproducible.

Reviewer 1: The experiments are adequately replicated statistically for conventions of deep learning.

Reviewer 1: This paper proposes a conceptually and technically unique proposal in terms of application, taking existing technologies of VAEs and LSA and, and as far as I know, uses them in a novel area of application (predicting and simulating combination MOAs for compound treatments). If this work is shown to work more broadly and effectively, is seen through to it completion, and is eventually successfully implemented, it will help to evaluate the effects of drugs used in combination on gene expression and cell morphology. An audience in the realm of biological deep learning applications as well as an audience working in the compound and drug testing would be interested in the results of this paper. Authors successfully place their work within the context of existing literature, referencing the numerous VAE applications that they build off of and fit into the field of (Lafarge et al, 2018; Ternes et al, 2021, etc...), citing the applications of LSA in the computer vision community (Radford et al, 2015, Goldsborough et al, 2017), and discussing the biological context that they are working in (Chandrasekaran et al, 2021).

Reviewer 2: The main novelty of the work is applying VAEs on cell painting data to predict drug perturbations. The final use case could be guiding experimental design by predicting unseen data. However, the authors do not show such an example and use case which is understandable due to the need for doing further experiments to validate computational results and maybe not the main focus of this paper. The authors did a good job of citing existing methods and relevant. The potential audience could be the computational biology and applied machine learning community.

Reviewer 3: The manuscript is beautifully written in a crystal clear manner. The authors have made a visible effort towards making their work understandable. The methods section is clear and comprehensive. All experiments are rigorously conducted and the validation procedures are sound. The conclusions of the paper are convincing and most of them are well supported by the data. Both the data and the code required to reproduce this work are freely available. Overall, the article is of high quality and relevance to several scientific communities.

We thank the reviewers for their encouraging remarks and overall positive sentiment. As early-career researchers, we feel empowered by these words.

3.2. Moved Figure 2 to supplement and removed Figure 5

Reviewer 1: Fig 2 is not informative so it can go to supplementary.

Reviewer 2: I liked the paper's GitHub repo, the authors did a good job making everything available and reproducible. As a suggestion, you can move the learning curves in two the sup figures cause they might not be the most exciting piece of info for the non-technical reader.

Reviewer 3: I would suggest removing Figure 5 (or moving it to the supplementary) as it revisits the content of Figure 1 and does not bring much extra information.

We agree that Figure 2 might not be informative to a non-technical reader, so we have accepted this suggestion by both reviewers 1 and 2, and we have moved Figure 2 to supplementary.

We agree with the reviewer and have removed Figure 5.

3.3. Clarified our data source as CellProfiler readouts, not raw Cell Painting images

Reviewer 1: In Fig 4, it would be useful to show a few sample representative images with respect to CellProfiler feature groups.

Reviewer 1: Figure 6, what does it means original input space? Does it mean raw pixel image? As researchers extracted CellProfiler feature groups already, it would be interesting to compare mean L2 distance based on CellProfiler features so that whether VAE improves performance or not (compared to handcrafted features) as a baseline.

Reviewer 3: While what "morphological readouts" concretely mean becomes clearer later on in the paper, it would be useful to give a couple of examples early on when introducing the considered datasets.

We thank the reviewer for these suggestions, which bring to light a common source of confusion, which we must alleviate. We are working with CellProfiler readouts (features extracted using classical algorithms) of the Cell Painting images and not the images themselves. We have made several edits throughout the manuscript to improve clarity and remove this confusion, including the introduction, in which we clearly state our model input data:

“Because of the success of VAEs on these various datasets, we sought to determine if VAEs could also be trained using cell morphology readouts (rather than directly on images), and further, to carry out arithmetic to predict novel treatment outcomes. We derive the cell morphology readouts using CellProfiler (McQuin et al, 2018), which measures the size, structure, texture, and intensity of cells, and use these readouts to train all models.”

This decision comes with tradeoffs: The benefit of using CellProfiler readouts instead of images is that they are more manageable but we might lose some information. We more thoroughly discuss this important tradeoff in the discussion section:

“We determined that VAEs can be trained on cell morphology readouts rather than directly using the cell images from which they were derived. This decision comes with various trade-offs. Compared to cell images, cell morphology readouts as extracted by image analysis tools (e.g. CellProfiler) are a more manageable data type; the data are smaller, easier to distribute, substantially less expensive to analyze and store, and faster to train (McQuin et al, 2018). However, it is likely some biological information is lost, because these tools might fail to measure all morphology signals. The so-called image-based profiling pipeline also loses information, by nature of aggregating inherently single-cell data to bulk consensus signatures (Caicedo et al, 2017).”

3.4. Clarified future directions to infer cell health readouts from simulated polypharmacology cell states

Reviewer 1: Authors also make the claim that they can infer toxicity and simulate the mechanism of how two compounds might react. This is a claim that would not be supported even if the method were able to successfully predict morphology or gene profiles. Drug interaction and toxicity are quite complex and goes beyond just morphology and expression. VAEs predicting a small set of features would not be able to capture information beyond the readouts, especially when dealing with potentially unseen compounds for which toxicity is not yet known. For example, two compounds might produce a morphology that appears similar to other safe compounds but has other factors that contribute to toxicity. Further, here they show no evidence of toxicity or interaction analysis.

The reviewer is correct that such a claim is unsupported by our research. Our message was actually that inferring toxicity could be a potential future application of our work. Specifically, for example, we can apply orthogonal models of cell toxicity that we previously derived using other data (Way et al, 2021a) to our inferred polypharmacology cell states. We thank this reviewer for noticing our lack of clarity, and we have made changes in the discussion to make it clear that inferring toxicity is something we may do in the future and is not something that is discussed in the manuscript:

“In the future, by predicting cell states of inferred polypharmacology, we can also infer toxicity using orthogonal models (e.g. Way et al. 2021) and simulate the mechanisms of how two compounds might interact.”

3.5. Clarified our method of splitting data, and noting how a future analysis will answer overfitting extent

Reviewer 2: Could authors outline detailed data splits? Which MoA are in train and which are held out from training? As I understood, there were samples from MoAs that were supposed to be predicted in the calculation of LSA? Generally, the predicted MoA should not be seen during training and not in LSA calculation.

We now more explicitly detail how we split our data in the methods:

“As input into our machine learning models, we split the data into an 80% training, 10% validation, and 10% test set, stratified by plate for Cell Painting and stratified by cell line for L1000. In effect, this procedure evenly distributes compounds and MOAs across data splits.”

We also thank the reviewer for this comment, because they express an important concern about making sure that we are not overfitting to the data. We have explained in the manuscript that because of lack of data, MOAs were repeated in training and LSA. However, we believe overfitting is not playing a large role in model performance. Through our hold 5 out experiment, we are able to show that our models are able to predict the same MOAs irrespective of whether they were in the training data, indicating that we did not overfit to the distribution of certain MOAs.

Reviewer 1 also suggested that we do the hold 5 out experiment on A∩Bs that were barely predicted. After we do that, we will explicitly demonstrate the extent of overfitting.

3.6. Introduced acronyms when they first appear in the manuscript

Reviewer 3: The Kullback-Leibler divergence is properly introduced in the methods part, but not at all in the introduction (it directly appears as "the KL divergence"). To enhance readability, it would be better to fully spell it before using the acronym, and maybe give a one-sentence intuition of what it is about before pointing out to the methods part for more details.

We thank the reviewer for bringing this to our attention. We have carefully reviewed the entire manuscript and have corrected such instances of clear introductions to acronyms.

3.7. Fixed minor text changes

Reviewer 3: In Figure 1, I would recommend changing "compression algorithms" to "dimension reduction algorithm" or "embedding algorithm". In a compression setting, I would expect the focus to be on the number of bits of information each method requires (or the dimension of the resulting embedding) to encode the data while guaranteeing a certain quality threshold. This is obviously not the case here as the dimension of the embedding is fixed and the focus is on exploring how the embedding is constructed (eg how much it decorrelates the different features, etc) - which may be misleading.

Reviewer 3: I recommend using "A n B" or "A & B" or "(A, B)" to denote the combination of two independent modes of action A and B. The current notation "A | B" overloads the statistical "A given B" which appears in the VAE loss and is therefore misleading.

We agree with the reviewer, and aim to minimize all sources of potential confusion. We have made the change in the figure.

We also agree that our current notation can be confusing. We have updated all instances of “A|B” with “A ∩ B”.

3.8. Added hypothesis of MMD-VAE oscillations to supplementary figure legend

Reviewer 3: Do the authors have a hypothesis of what may be causing MMD-VAE to oscillate during validation when data are shuffled? This seems to be the case on two of the three considered datasets (Figure 2 and SuppFigure 1) and is not observed for the other models. Including a few sentences on that in the text would be interesting.

We believe a big reason for this is because of the fact that the optimal MMD-VAE had a much higher regularization term, which puts a greater emphasis on forming normal latent distributions, than the optimal Beta or Vanilla VAE. Forcing the VAE to encode a shuffled distribution into a normally distributed latent distribution would be difficult to do consistently across different randomly shuffled data subsets, and therefore might cause oscillations in the training curve across epochs when the penalty for that term is high. As these observations may be interesting to a certain population of readers, we have incorporated this explanation into the supplementary figure legend (which is where this figure is shown):

“Forcing the VAE to consistently encode a shuffled distribution into a normally distributed latent distribution would be difficult, and therefore might cause oscillations in the training curve across epochs.”

3.9. Explained our selection of VAE variants

Reviewer 3: The different types of considered VAE and their differences are very clearly introduced. It may however be good to motivate a bit more the focus on beta-VAE and MMD-VAE among all the possible VAE models. This is partly done through examples in the second paragraph of page 2, but could be elaborated further.

We thank the author for their encouraging remarks. We have made edits to the manuscript’s introduction, explaining why we chose these two variants out of all the possible choices:

“We trained vanilla-VAEs, β-VAEs, and MMD-VAEs only, and not other VAE variants and other generative model architectures, such as generative adversarial networks (GANs), because these VAE variants are known to facilitate latent space interpretability.”

1. Description of analyses that authors prefer not to carry out

4.1. We will not explore additional latent space dimensions in more detail, as this is out of scope

Reviewer 1: As both reconstructed and simulated data did not span the full original data distribution, it might be better to look at reconstruction error and increase the dimension of latent space.

We thank the reviewer for bringing up this important point. Our VAE loss function consists of the sum of reconstruction error and some form of KL divergence. Specifically, this reviewer is suggesting that if we only minimize reconstruction error (or focus more on reconstruction over KLD by lowering beta), a higher latent dimension would result in better overall reconstruction. This is true, but doing so would have negative consequences. While we would perhaps get the UMAPs to show the full data distribution, the UMAPs are not our focus; predicting polypharmacology through LSA is. We found that when we have a higher focus on the reconstruction term, we have more feature entanglement, as indicated by lower performance when simulating data and overlapping feature contribution per latent feature. The fact that simulating data would logically require less disentanglement than performing LSA shows that we require higher regularization (and hence lower focus on reconstruction) than the one we got from simulating data.

Essentially, while the reviewer's comments would improve reconstruction and allow us to improve the UMAPs, doing so would likely worsen LSA performance, which is the main focus of the project. Also, increasing the latent dimension without changing beta would likely have caused little to no change because since beta is encouraging disentanglement, it would cause the newly added dimension to have little variation and encode little new information that wasn’t already encoded before.

We have also previously explored the concept of toggling the latent dimensions in a separate project (Way et al, 2020). We are very interested in this area of research in general, and any additional analyses (beyond hyperparameter optimization) deserves a much deeper dive than what we can provide in this paper.

Lastly, we intend to include a deeper description and analysis of reconstruction loss across models, datasets, and MOAs as was suggested by a previous reviewer comment (see section 2.1 above)

4.2. We will not review Gaussian distribution assumptions of the VAE as we feel it is not informative

Reviewer 1: By looking at SFigure 6, I am wondering whether latent distribution actually met gaussian distribution (assumption of VAE). It may show skew distribution as some of latent features shows low contribution.

This reviewer’s comment is interesting, but we do not believe it would change the findings of our study. Suppose we find that the latent dimensions aren’t normally distributed. This wouldn’t change much; a gaussian distribution isn’t the most critical to perform LSA. We need the latent code to be disentangled, but having normally distributed latent features doesn't necessarily mean that we have good disentanglement (see https://towardsdatascience.com/what-a-disentangled-net-we-weave-representation-learning-in-vaes-pt-1-9e5dbc205bd1)

4.3. In this paper, we will not train or compare conditional VAEs nor cycle GANs

Reviewer 2: While authors provided a comparison between vanilla VAE and MMD-VAE, B-VEA, there are other methods capable of doing similar tasks (data simulation, counterfactual predictions ), I would like to see a comparison with those methods such as conditional VAE( https://papers.nips.cc/paper/2015/hash/8d55a249e6baa5c06772297520da2051-Abstract.html, CVAE + MMD : https://academic.oup.com/bioinformatics/article/36/Supplement_2/i610/6055927?login=true) or cycle GANs(https://arxiv.org/abs/1703.10593 ).

While such comparisons would be interesting, they are not the main focus of the manuscript, which is to benchmark the use of VAEs in cell morphology readouts and to predict polypharmacology.

We think that CVAE would not be appropriate for our study. In a CVAE, the encoder and decoder are both conditioned to some variable. In our situation where we are predicting the cell states of different MOAs, it would make most sense to condition on the MOA. However, because we’re using the MOA labels in our LSA experiment, conditioning on them is likely to bias our results and not be effective for MOAs outside the conditioning.

For cycle GANs, we have found that training using these data, in a separate study in our lab, is extremely difficult. Our lab has not published this yet, but once we are able to better understand cycleGAN behavior in these data, it will require a separate paper in which we compare performance and dissect model properties in much greater detail.

Nevertheless, we have added citations to multi-modal approaches like cycle GANs (see section 4.4) as they will point a reader to useful resources for future directions.

4.4. We will not be comparing with multi-modal integration, but we clarified our focus on Cell Painting VAE novelty and added multi-modal citations

Reviewer 1: Researchers found that the optimal VAE architectures were very different between morphology and gene expression, suggesting that the lessons learned training gene expression VAEs might not necessarily translate to morphology. It would be interesting to compare the result with multimodal integration as baseline (i.e., Seurat).

Our focus in this paper was to train and benchmark different variational autoencoder (VAE) architectures using Cell Painting data and to demonstrate an important, unsolved application in predicting polypharmacology that we show is now possible for a subset of compounds. It was a natural and useful extension to compare Cell Painting VAE performance with L1000 VAE performance especially since our data set contained equivalent drug perturbations. We feel that any extension including multi-modal data integration will distract focus away from the Cell Painting VAE novelty, and requires a much deeper dive beyond scope of our current manuscript.

Additionally, there have been other, more in-depth and very recent multi-modal data integration efforts using the same or similar datasets (Caicedo et al, 2021; Haghighi et al, 2021). In a separate paper that we just recently submitted, we also dive much deeper to answer the question of how the two modalities complement one another in various ways and for various tasks (Way et al, 2021b). These two papers already provide a deeper and more informative exploration of Cell Painting and L1000 data integration.

Therefore, because multi-modal data integration, while certainly interesting, will distract from the Cell Painting VAE novelty and is redundant with other recent publications, we feel it is beyond scope of this current paper.

Nevertheless, multi-modal data integration is important to mention, so we add it to the discussion. Specifically, we discuss how multi-modal data integration might help with predicting polypharmacology in the future and include pertinent citations so that we, or another reader, might be able to follow-up in the future. The new section reads:

“Because we had access to the same perturbations with L1000 readouts, we were able to compare cell morphology and gene expression results. We found that both models capture complementary information when predicting polypharmacology, which is a similar observation to recent work comparing the different technologies’ information content (Way et al, 2021). We did not explore multi-modal data integration in this project; this has been explored in more detail in other recent publications (Caicedo et al, 2021; Haghighi et al, 2021). However, using multi-modal data integration with models like CycleGAN or other style transfer algorithms might provide more confidence in our ability to predict polypharmacology in the future (Zhu et al, 2017).”

1. References

Caicedo JC, Cooper S, Heigwer F, Warchal S, Qiu P, Molnar C, Vasilevich AS, Barry JD, Bansal HS, Kraus O, et al (2017) Data-analysis strategies for image-based cell profiling. Nat Methods 14: 849–863

Caicedo JC, Moshkov N, Becker T, Yang K, Horvath P, Dancik V, Wagner BK, Clemons PA, Singh S & Carpenter AE (2021) Predicting compound activity from phenotypic profiles and chemical structures. bioRxiv: 2020.12.15.422887

Corsello SM, Bittker JA, Liu Z, Gould J, McCarren P, Hirschman JE, Johnston SE, Vrcic A, Wong B, Khan M, et al (2017) The Drug Repurposing Hub: a next-generation drug library and information resource. Nat Med 23: 405–408

Haghighi M, Singh S, Caicedo J & Carpenter A (2021) High-Dimensional Gene Expression and Morphology Profiles of Cells across 28,000 Genetic and Chemical Perturbations. bioRxiv: 2021.09.08.459417

McQuin C, Goodman A, Chernyshev V, Kamentsky L, Cimini BA, Karhohs KW, Doan M, Ding L, Rafelski SM, Thirstrup D, et al (2018) CellProfiler 3.0: Next-generation image processing for biology. PLoS Biol 16: e2005970

Wang Y, Huang H, Rudin C & Shaposhnik Y (2021) Understanding how dimension reduction tools work: an empirical approach to deciphering t-SNE, UMAP, TriMAP, and PaCMAP for data visualization. J Mach Learn Res 22: 1–73

Way GP, Kost-Alimova M, Shibue T, Harrington WF, Gill S, Piccioni F, Becker T, Shafqat-Abbasi H, Hahn WC, Carpenter AE, et al (2021a) Predicting cell health phenotypes using image-based morphology profiling. Mol Biol Cell 32: 995–1005

Way GP, Natoli T, Adeboye A, Litichevskiy L, Yang A, Lu X, Caicedo JC, Cimini BA, Karhohs K, Logan DJ, et al (2021b) Morphology and gene expression profiling provide complementary information for mapping cell state. bioRxiv: 2021.10.21.465335

Way GP, Zietz M, Rubinetti V, Himmelstein DS & Greene CS (2020) Compressing gene expression data using multiple latent space dimensionalities learns complementary biological representations. Genome Biol 21: 109

Zhu J-Y, Park T, Isola P & Efros AA (2017) Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks. arXiv [csCV]

Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Referee #3

Evidence, reproducibility and clarity

Summary:

In this paper, the authors explore the use of VAE to learn low-dimensional representations of morphological features of cells. They demonstrate that the representations learned by the different VAE models considered accurately model the distribution of features in real data and can be complemented by other biological readouts such as gene expression. Additionally, the structure of the learned feature space appears to be sufficient to generate accurate predictions relying on latent space arithmetic - for instance allowing to predict the morphology of samples subjected to two perturbations knowing the morphology of samples affected by either of these perturbations in isolation.

Comments:

The manuscript is beautifully written in a crystal clear manner. The authors have made a visible effort towards making their work understandable. The methods section is clear and comprehensive. All experiments are rigorously conducted and the validation procedures are sound. The conclusions of the paper are convincing and most of them are well supported by the data. Both the data and the code required to reproduce this work are freely available.

Overall, the article is of high quality and relevance to several scientific communities. I only have a couple of minor comments that I think could help improve it further:

• The Kullback-Leibler divergence is properly introduced in the methods part, but not at all in the introduction (it directly appears as "the KL divergence"). To enhance readability, it would be better to fully spell it before using the acronym, and maybe give a one-sentence intuition of what it is about before pointing out to the methods part for more details.
• While what "morphological readouts" concretely mean becomes clearer later on in the paper, it would be useful to give a couple of examples early on when introducing the considered datasets.
• The different types of considered VAE and their differences are very clearly introduced. It may however be good to motivate a bit more the focus on beta-VAE and MMD-VAE among all the possible VAE models. This is partly done through examples in the second paragraph of page 2, but could be elaborated further.
• In Figure 1, I would recommend changing "compression algorithms" to "dimension reduction algorithm" or "embedding algorithm". In a compression setting, I would expect the focus to be on the number of bits of information each method requires (or the dimension of the resulting embedding) to encode the data while guaranteeing a certain quality threshold. This is obviously not the case here as the dimension of the embedding is fixed and the focus is on exploring how the embedding is constructed (eg how much it decorrelates the different features, etc) - which may be misleading.
• Do the authors have a hypothesis of what may be causing MMD-VAE to oscillate during validation when data are shuffled? This seems to be the case on two of the three considered datasets (Figure 2 and SuppFigure 1) and is not observed for the other models. Including a few sentences on that in the text would be interesting.
• I recommend using "A n B" or "A & B" or "(A, B)" to denote the combination of two independent modes of action A and B. The current notation "A | B" overloads the statistical "A given B" which appears in the VAE loss and is therefore misleading.
• I would suggest removing Figure 5 (or moving it to the supplementary) as it revisits the content of Figure 1 and does not bring much extra information.
• Figure 6 is missing error bars (standard deviation of the L2 distance) and, as such, is hard to draw conclusions from.

Significance

Nature and significance:

This work does not hold new conceptual or technical contributions per se as it focuses on showcasing the use of existing techniques established in other fields (eg in the context of natural image processing for latent space arithmetics) to biological data analysis. That said, popularizing successful methodologies beyond the scientific community where they have been developed, as done in this work, is immensely valuable. As such, the approach presented in the paper is likely to inspire and enable many other studies and is therefore a significant contribution (especially so thanks to the code availability!)

Comparison to existing published knowledge:

While a bunch of published works use VAEs on biological data, I am not aware of existing ones that study the relative merit of the representations obtained with different VAE models as done here and explore their use in a generative setting with latent space arithmetics. As such, this work is novel and distinguishes itself from existing published knowledge.

Audience:

This work is likely to be of interest to life scientists with an enthusiasm for state-of-the-art data analysis techniques. Because the paper is clearly written and makes very few assumptions of prior expert knowledge, it is also likely to be a good entry point to the wider VAE/generative models literature for non-experts. I also believe that this manuscript can be of interest to computer scientists and machine learning researchers as it presents a concrete example of the use of published methods in the context of biological data analysis.

My expertise:

Computer vision and machine learning. I do not feel qualified to assess the clinical relevance of this work.

Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Referee #2

Evidence, reproducibility and clarity

Ler et al. propose a series of VAE based methods to predict compound polypharmacology For cell painting data. They first learn a latent space and try to answer the following counterfactual:

how would cell morphology or gene expression of a cell perturbed with Drug A change if was perturbed with Drug A and B (A+B) given we have the measurement for drug A and drug B. They address the problem by doing latent space arithmetics (LSA) and decoding the predicted morphology measurements. They first train different VAE models to compare the training stability and simulation performance by sampling from the latent space. Further analysis is the learned latent space to deconvolve latent space to feature space. I like the application of LSA+VAE on cell painting datasets which is the main novelty of the paper. However, I have some major comments and concerns:

Major comments:

While authors provided a comparison between vanilla VAE and MMD-VAE, B-VEA, there are other methods capable of doing similar tasks (data simulation, counterfactual predictions ), I would like to see a comparison with those methods such as conditional VAE( https://papers.nips.cc/paper/2015/hash/8d55a249e6baa5c06772297520da2051-Abstract.html, CVAE + MMD : https://academic.oup.com/bioinformatics/article/36/Supplement_2/i610/6055927?login=true) or cycle GANs(https://arxiv.org/abs/1703.10593 ). The authors compare generation performance based on UMAP. In the UMAP space, data tend to cluster together even though they might be far from each other in the feature space. I would like to see more quantitive metrics on how well these methods capture morphology distributions. You can compute metrics like MMD distance, kullback leibler (KL), earthmoving distance, or a simple classifier trained on actual MoA classes tested on generated data.

The predictions are evaluated using L2 distances, which I find not that informative. I would like to see other metrics (correlation or L1 or distribution distances in previous comments) I would like to see feature-level analysis, which features are well predicted and which ones are more challenging to predict?

• Could authors outline detailed data splits? Which MoA are in train and which are held out from training? As I understood, there were samples from MoAs that were supposed to be predicted in the calculation of LSA? Generally, the predicted MoA should not be seen during training and not in LSA calculation.

Minor comments:

I liked the paper's GitHub repo, the authors did a good job making everything available and reproducible. As a suggestion, you can move the learning curves in two the sup figures cause they might not be the most exciting piece of info for the non-technical reader.

Significance

The main novelty of the work is applying VAEs on cell painting data to predict drug perturbations. The final use case could be guiding experimental design by predicting unseen data. However, the authors do not show such an example and use case which is understandable due to the need for doing further experiments to validate computational results and maybe not the main focus of this paper.

• The authors did a good job of citing existing methods and relevant

• The potential audience could be the computational biology and applied machine learning community.

• My expertise is in computational biology and machine learning.

Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Referee #1

Evidence, reproducibility and clarity

Summary:

Researchers used two primary data modalities (L1000 sequencing data, and Cell painting morphology features) for cell data perturbed by a series of compounds, each with labeled (individual and combination) mechanisms of action. Using several VAEs and ML methods, they evaluated their ability to encode interpretable latent spaces (evaluated by subtracting +/-3stds and checking the contribution off features to the latent space) and adequately reconstruct the input data. Using the constructed latent spaces and labeled MOAs, researchers performed latent space arithmetic, to remove base DMSO features and add features of individual MOAs to produce the features of combination MOAs (evaluated by the significance of difference to shuffled data). Researchers found that MDD-VAE encoded the most information and that VAEs successfully simulated morphology and gene expression features. They found that the optimal VAE architectures were very different between morphology and gene expression. Researchers found that VAEs were able to use individual MOA profiles to simulate some combination MOA profiles with varied success.

Comments:

• Researchers found that the optimal VAE architectures were very different between morphology and gene expression, suggesting that the lessons learned training gene expression VAEs might not necessarily translate to morphology. It would be interesting to compare the result with multimodal integration as baseline (i.e., Seurat).

-Instead of using UMAP embedding, it would be better to compare reconstruction error or show a reconstructed image with the original image to claim that models reliably approximate the underlying morphology data. As both reconstructed and simulated data did not span the full original data distribution, it might be better to look at reconstruction error and increase the dimension of latent space.

-Fig 2 is not informative so it can go to supplementary. -In Fig 4, it would be useful to show a few sample representative images with respect to CellProfiler feature groups.

-By looking at SFigure 6, I am wondering whether latent distribution actually met gaussian distribution (assumption of VAE). It may show skew distribution as some of latent features shows low contribution.

-Figure 6, what does it means original input space? Does it mean raw pixel image? As researchers extracted CellProfiler feature groups already, it would be interesting to compare mean L2 distance based on CellProfiler features so that whether VAE improves performance or not (compared to handcrafted features) as a baseline.

-It is difficult to know the clear threshold for successful performance is on figures like Figure 7 and SFigure 9, but by and large, it appears that the majority of predicted combination MOAs were not successful. Without the ability to either A) adequately predict most all combinations from individual profiles that were used in training or B) an explanation prior to analysis of which combination will be able to predict, it is difficult to see this method being used since the combinatorial predictions are more likely not informative.

-Authors also make the claim that they can infer toxicity and simulate the mechanism of how two compounds might react. This is a claim that would not be supported even if the method were able to successfully predict morphology or gene profiles. Drug interaction and toxicity are quite complex and goes beyond just morphology and expression. VAEs predicting a small set of features would not be able to capture information beyond the readouts, especially when dealing with potentially unseen compounds for which toxicity is not yet known. For example, two compounds might produce a morphology that appears similar to other safe compounds but has other factors that contribute to toxicity. Further, here they show no evidence of toxicity or interaction analysis.

-The researchers justify the poor performance compared to shuffled data, by saying that A) MOA annotations are noisy and unreliable and B) they MOAs may only manifest in other modalities like what was seen in the L1000 vs morphology predictability. While these might be true, knowing this the researchers should make an effort to clean and de-noise their data and select MOAs that are well-known and reliable, as well as, selecting MOAs for which we have a known morphological or genetic reaction.

-With the small number of combinations that are successfully predicted, to build confidence in the performance, it would be necessary to explain the reason for the differences in performance. Further experimentations should be done looking into any relationship between the type of MOAs (and their features) and the resulting A|B predictability. Looking at Figure 7, the top-performing combinations are comprised entirely of inhibitor MOAs. If the noisiness of the data is a factor, there should be some measurable correlation between feature noisiness and variation and the resulting A|B predictability from LSA.

-To show that the methodology works well on unseen data, researchers withheld the top 5 performing A|B MOAs (SFig 9) and showed they were still well predicted. This is not the most compelling demonstration since the data to be held out was selected with bias as the top-performing samples. It would be much more interesting to withhold an MOA that was near or only somewhat above the margin of acceptability and see how many holdouts affected the predictability of those more susceptible data points. From my best interpretation, the hold-out experiment also only held out the combination MOA groups from training. It would be better if single MOAs (for example A) which were a part of a combination of MOA (A|B) were also held out to see if predictability suffered as a result and if generalizability did extend to cells with unseen MOAs (not just cells which had already highly performing combinations of seen MOAs).

-Rather than just stating that the VAE's did not span the original data distribution and saying beta-VAE performed best by eye, some simple metrics can be drawn to analyze the overlap in data for a more direct and quantified comparison. Researchers should also explain what part of the data is not being captured here. Some analysis of what the original uncaptured UMAP represents is important in understanding the limitations of the VAEs' capacity.

-My suggestions are realistic and feasible. The cost for the recommended tests and validations would cost no additional money (outside of researcher labor and re-training on the existing GPUs) as my recommendations are simply further analysis and training on the same data. Time would be dependent on the time required to train the VAE models, but seeing as 2-layer VAEs are relatively small for the deep learning community, time to train and analyze through existing pipelines should be minimal. This is confirmed by looking at their GitHub code, where jupyter notebooks show that models can be trained in a few minutes.

-With access to the dataset, the posted GitHub, and documentation in the paper, I believe that the experiments are reproducible.

-The experiments are adequately replicated statistically for conventions of deep learning.

Significance

My background of expertise is developing and applying deep learning and VAEs applied to single cell imaging and expression data. There is no part of this paper that I do not have sufficient expertise to evaluate.

This paper proposes a conceptually and technically unique proposal in terms of application, taking existing technologies of VAEs and LSA and, and as far as I know, uses them in a novel area of application (predicting and simulating combination MOAs for compound treatments). If this work is shown to work more broadly and effectively, is seen through to it completion, and is eventually successfully implemented, it will help to evaluate the effects of drugs used in combination on gene expression and cell morphology. An audience in the realm of biological deep learning applications as well as an audience working in the compound and drug testing would be interested in the results of this paper. Authors successfully place their work within the context of existing literature, referencing the numerous VAE applications that they build off of and fit into the field of (Lafarge et al, 2018; Ternes et al, 2021, etc...), citing the applications of LSA in the computer vision community (Radford et al, 2015, Goldsborough et al, 2017), and discussing the biological context that they are working in (Chandrasekaran et al, 2021).

Transparent Peer Review

---

Download the complete Review Process [PDF] including:

• reviews
• authors' reply
• editorial decisions

</br>

Transparent Peer Review

---

Download the complete Review Process [PDF] including:

• reviews
• authors' reply
• editorial decisions

</br>

Transparent Peer Review

---

Download the complete Review Process [PDF] including:

• reviews
• authors' reply
• editorial decisions

</br>

Transparent Peer Review

---

Download the complete Review Process [PDF] including:

• reviews
• authors' reply
• editorial decisions

</br>

Transparent Peer Review

---

Download the complete Review Process [PDF] including:

• reviews
• authors' reply
• editorial decisions

</br>

Transparent Peer Review

---

Download the complete Review Process [PDF] including:

• reviews
• authors' reply
• editorial decisions

</br>

Transparent Peer Review

---

Download the complete Review Process [PDF] including:

• reviews
• authors' reply
• editorial decisions

</br>

Transparent Peer Review

---

Download the complete Review Process [PDF] including:

• reviews
• authors' reply
• editorial decisions

</br>

Note: This rebuttal was posted by the corresponding author to Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Reply to the reviewers

Reviewer #1 (Evidence, reproducibility and clarity (Required)): **Summary:** The manuscript submitted by Djekidel et al entitled: "CovidExpress: an interactive portal for intuitive investigation on SARS-CoV-2 related transcriptomes" reports on a new web portal to search and analyze RNAseq data related to SARS-CoV-2 infections. The authors downloaded and reprocessed data of more than 40 different studies, which is available on the web portal along with all available meta data. The web portal allows to perform numerous differential expression and gene set enrichment analyses on the data and provides publication ready figures. Because of batch effects that could not be removed, the authors do not recommend to analyze data across studies at this point. The authors conclude that the web portal is unique and will allow scientists to rapidly analyze gene expression signatures related to SARS-CoV-2 infections with the potential to make new discoveries. **Major comments:** Based on the scientific literature, the web portal seems to be an unprecedented resource to search and analyze SARS-CoV-2-related RNAseq data and as such would certainly be a useful resource for the SARS-CoV-2 scientific community. The authors argue that new discoveries are possible by using their web portal in providing use cases. However, the section detailing the analyses the authors did to generate new hypotheses about genes potentially relevant in SARS-CoV-2 infections are very difficult to follow and without more guidance very difficult to reproduce with the web portal. It would require substantial expert knowledge in RNAseq data analysis without more information being provided. It also seems that key candidate genes identified by their analyses have all been studied or identified to be related to SARS-CoV-2 infections, so it is somewhat unclear whether new hypotheses can be generated by the reanalysis of RNAseq datasets, especially because combining the data from different studies is currently not recommended by the authors. The manuscript would benefit from providing fewer use cases but for each of them providing more information on how the portal and which studies were used to generate them and which findings were not described in the publication of the used studies. Some observations in the manuscript are not substantiated with significance calculations (see below). At times, the English writing (grammar) should be improved.

We thank the reviewer for the positive comments. We suppose the reviewer conclude it need substantial expert knowledge in RNAseq data analysis were due to lacking Video Tutorial. We have now put up several Video Tutorials and more tutorials would be added along later along with users’ feedbacks. We believed this would help ease reviewers’ concern.

In response to whether new hypothesis can be generated. Sorry if it’s not clear, for all the case studies and our “CovidExpress Reveals Insights and Potential Discoveries”, our portal has provided information not reported by their original publications, as listed below:

1. Case study #1: The original publication employed a multiomics approach to find the predictor genes between ICU and non-ICU patient. But it’s not obviously to know which genes were mainly due to expression level, which might be due to other data they included (e.g. mass spectrometry data). Our portal allow user to quickly check their expression level and find SESN2 does not have strong expression differences.
2. Case study #2: We replace this case study with bacterial-susceptibility genes to show such questions could be quickly asked and answered using our portal. Such investigation has not been reported before.
3. FURIN’s function have been well related to SARS-CoV-2. However, for all reports we could find, they focused on Furin cleavage sites of SARS-CoV-2 or whether FURIN were expressed in the SARS-CoV-2 sensitive tissues. SARS-CoV-2 infection could up-regulate FURIN expression have never been reported before. The study published the data didn’t mentioned FURIN at all. We have made this discovery simply by using CovidExpress portal to find the differential expressed genes and overlap with the literature-based gene list (Supplementary Table S2), we believe more discoveries could be made by users by selecting different data.
4. If we search OASL AND " SARS-CoV-2" on pubmed, only 5 results shown up indicated it’s under-studied. And none of them indicated OASL could be up-regulated both by SARS-CoV-2 infected lung and Rhinovirus-infected nasal in human. It is not clear to us if we might misunderstand reviewers’ suggestion as “fewer use cases”. Thus, we haven’t removed any use cases, instead we provided more details to help users understand what and how did we made those discoveries not reported by their original studies using CovidExpress.

At last, we have gone through substantial scientific editing to improve the grammar. **Minor comments:** Page 6 last sentence: The statement of this sentence is very much what one would expect. It remains unclear whether the authors mean this as a result to validate the processing of the RNAseq data or as a new discovery. Please, clarify.

We apologize for the confusion. We intended this statement to be a result confirming what we had expected. We have now amended the text to make this point clearer.

Figure 3A: The violin plots are so tiny that it is impossible to see any trends. It is also difficult to understand which categories one should compare with each other. If there is anything significant to observe, please, add a statistical test and better guide the reader.

We agree with the reviewer; therefore, we have removed this figure from the paper. The goal of this figure was to demonstrate how to use violin plots for exploratory analysis; however, in this case, the violin plot did not show a clear trend. By using more filtering and other plots (e.g., Figure 3B-C), we believe we now provide better insight.

Figure 3C: A legend for the color scale is missing. The signal (I guess expression amounts) for SESN2 seems very weak and the same between ICU and non-ICU samples. What is the significance for assigning this gene to the group of genes being upregulated in ICU samples? Also contrary to what the authors state on page 8, SESN2 does not seem to be highly expressed in ICU samples, however, without knowing what the colors represent (fold changes or absolute expression values?) this is somewhat speculative.

We thank the reviewer for bringing this to our attention. We have now added a legend for the color scale in the revised figure. In Figures 3A-C, we are showcasing how an exploratory analysis can be performed using CovidExpress. As an example, we investigated the expression of the top 20 genes identified by the random forest classifier of Overmyer et al., 2021, as predictors of ICU and non-ICU cases. In the original Overmyer et al. paper, only the general performance metrics of the models are presented (Fig. 6c-g), but the authors do not show the expression patterns of the top predictors. Hence, we demonstrate how CovidExpress can be used to further investigate some questions not explored in the original paper. SESN2 was listed as a top predictor; however, its expression did not vary between ICU and non-ICU samples, as was also observed by the reviewer. We suspect SESN2 was a top predictor due to other data the Overmyer et al. paper included, such as mass spectrometry data. Our statement about SESN2 was not accurately reflected in the figure; therefore, we have rewritten this section to make it clearer.

Page 9 first sentence: Please, specify what you mean by "starting list". Furthermore, in this paragraph, how do your results compare to the results from the study that you re-analyze here?

We thank the reviewer for the question. By “starting list,” we meant the top genes from the Overmyer et al., 2021, article as predictors of ICU and non-ICU cases. We have now rewritten this section to make it clearer. We did not expect our results to differ from their data. Our goal was to ask which of their top predictors (by multi-omics data) show a difference in gene expression. When we downloaded their TPM values from their GEO records, the values were very similar overall (see below).

Figure 3F: Please add labels to your axes and is there a particular reason why in a correlation plot like this one, the y and x axis are not shown with the same range and why does the y axis not start at 0?

We thank the reviewer for this helpful comment. Our reasoning for presenting the figure in this way is that different genes can have very different expression levels but still be correlated. For example, if gene A expressed 1, 5, and 10 in samples 1,2, and 3, while gene B expressed 100, 500, and 1000 for samples 1, 2, and 3, then their range would be very different but still perfectly correlated (see panel A below). If we draw the x- and y-axes using the same range, this correlation will not be visually obvious (see panel B below).

This comparison is different from the correlation plots that compare the expression of one gene in different samples. We apologize for the confusion and to avoid misleading readers, we have enlarged the gene names in the Figure labels to ensure that readers notice their differences. We have also added an option to the correlation plot on our portal so that users can choose the optimal format (see below).

Page 9 second last sentence: It remains unclear which kind of analysis the authors intend to do here and what the starting question is. Please, try to rewrite with less technical terms (i.e. what do you mean by "precalculated contrasts"). In line with this, it remains unclear what Figure 3I is supposed to show. Please, provide some more information to readers who are not RNAseq analysis experts.

We thank the reviewer for this suggestion. To avoid any misleading claims, we followed Reviewer #2’s suggestion and replaced the coagulation gene list with a filtered gene list from the “Coronavirus disease - COVID-19” KEGG pathway (hsa05171) to showcase how to identify experiments in which this gene signature is enriched or depleted. We also replaced the related figures and text with new results and rewrote this section to avoid using technical terms.

Figure 3J is somewhat confusing. Why is the mean expression range indicated from 0 to 1 and why are all genes apparently having a mean expression of 1?

We thank the reviewer for this question. Because the levels of expression of different genes can vary greatly, in Figure 3J (new Figure 3A and 3I), we normalized the mean expression levels of the genes to their maximum values across groups to improve the visualization. We have now made this clearer in the figure, legend, and text.

Page 10 line 5-6. Are you referring to coagulation markers here or general expression patterns? In case of the latter, how does this statement fit to the paragraph about analyzing expression patterns of coagulation markers? Please, specify. And in line with this, are the highlighted genes in Figure 3K coagulation markers? If not, what is the relevance of these to make the point that one can use the portal to investigate the role of coagulation markers in SARS-CoV-2 infections?

As mentioned above, to avoid any misleading claims, we followed Reviewer #2’s suggestion and replaced the coagulation gene list with a filtered gene list from the “Coronavirus disease - COVID-19” KEGG pathway (hsa05171). This revision enables us to show how to identify experiments in which this gene signature is enriched or depleted. We have now replaced these figures and text with new results.

The appearance of describing batch effects and attempts to remove them from the studies was somewhat surprising on page 10 as I would expect this kind of results rather earlier in the results section before describing use cases of the data. You may consider changing the order of your results for a better flow.

We apologize for the confusion. However, we want to make it clear that the analysis before page 10 did not involve “batch effect”; all analyses were performed within each study. Thus, it is not necessary to change the order in which the results are presented. Also, based on Reviewer #2’s comments, we did not accurately use the term “batch effect,” because “batch effects are purely due to technical differences.” We have now revised the corresponding text to make this point clearer.

Page 11, second paragraph. Please, explain briefly what the silhouette score is supposed to reflect and thus how Figure S4G should be interpreted. The difference of both bars in Figure S4G is very marginal and thus, does not seem to support the statement of the authors that the ssGSEA scores-based projection is better unless you perform a significance test or I misunderstood. Please, clarify.

We thank the reviewer for this suggestion. We have now added an explanation of the silhouette score in the manuscript. Briefly, a silhouette score is a metric of the degree of separability of gene clusters from the nearest cluster. For a given sample, lets be the mean intra-cluster distance, and be the mean distance to the nearest cluster. The silhouette score (sil) will be calculated as follows

The silhouette score ranges between -1 and 1. A value near 1 means that the clusters are well separated, and a value near -1 means that the clusters are intermingled. Using a Wilcoxon rank test, we showed that using ssGSEA scores significantly improves the separability of global GTEx tissues (in Figure S4G; p=8.75e-26).

Page 11, third paragraph: Figure 4B, to the best of my understanding, does not support the claim that samples clustered less according to study cohorts using the ssGSEA approach. Please, quantify the effect and test for significance or better explain.

We apologize for the confusion. We quantified the separability between cohorts (GSE ids) by using the silhouette score. In Figure S4H (panel A below), we show that the TPM-based PCA leads to more separation by studies than does the Covid contrast ssGSEA scores in which the separation between studies is less prominent (p-value=0.0045, paired Wilcoxon test).

For the analyses described starting on page 12 it remains largely unclear whether they were conducted across studies or within studies and which studies were used. This section until the end of the results would especially benefit from providing more information on how the analyses were performed, either in the results or in the methods section.

We apologize for the confusion. The goal of the analysis on page 12 and the corresponding Figure 4G was to identify genes whose expression increased in both the SARS-CoV-2 infection lung and rhinovirus-infected nasal tissue. Hence, we did a log2(fold-change) vs log2(fold-change) comparison. The log2(fold-change) values were independently calculated for each study. Because we compared values by using the same ranking metric, the cross-samples comparison was possible, as shown in Figure 4G. We have now added more details to the Methods section to clarify this point.

Figures 4J and 4K miss axis labels and since we look at correlations, the figures could be redrawn using the same ranges on x and y axis.

We thank the reviewer for this suggestion. We have now added axes labels to the new figures. However, we have not used the same range on the x and y axes because they depict expression levels of different genes. For example, if gene A is expressed 1, 5, and 10 in samples 1, 2, and 3, while gene B is expressed 100, 500 and 1000 for samples 1, 2, and 3, their range would be very different but still perfectly correlated (panel A below). If we draw x and y axes using the same range, this correlation will not be visually obvious (panel B below).

This comparison is different from the correlation plots that compare the expression of one gene in different samples. We apologize for the confusion and to avoid misleading readers, we have enlarged the gene names in Figure labels to ensure that readers notice they are different genes. We have also added an option to the correlation plot on our portal so that users can choose the optimal format (see below).

Page 14 line 5: Is this the right figure reference here to Figure 4G? If yes, then it is unclear how Figure 4G supports the statement in this sentence. Please, clarify.

We apologize for the confusion. In Figure 4G, we labeled several important genes and used different colors to indicate whether the gene was regulated by SARS-CoV-2 only (purple), Rhinovirus only (black), or both(red). FURIN was the gene that is only significantly upregulated by SARS-CoV-2. The data in Figure 4G were from GSE160435(“SARS-CoV-2 infection of primary human lung epithelium for COVID-19 modeling and drug discovery”); that study used lung organoid alveolar type 2 (AT2) cells as the model. We think this confusion was caused by our failure to provide the details about the GSE160435 study. We have now amended the manuscript to include these details in the Methods section to avoid confusion. We also enlarged the gene labels in the figure to make them more visible. In the manuscript, we have changed from “our results found FURIN gene was also upregulated in SARS-CoV-2–infected lung organoid alveolar type 2 cells (Figure 4G, Supplementary Table S3).” to “We found that FURIN was upregulated in SARS-CoV-2-infected lung organoid alveolar type 2 cells (Figure 4G, Supplementary Table S4) (Mulay, Konda et al., 2021), it has reported that TGF-β signaling could also regulates FURIN (Blanchette, Rivard et al., 2001). Our gene enrichment analysis also found TGF-β signaling enriched only for up-regulated genes in SARS-CoV-2-infected lung cells (FDR correct p=7.58E-05, Supplementary Table S4), these observations implicated a positive feedback mechanism only for SARS-CoV-2-infected lung but not RV-infected nasal cells.”

Figure 2 is of too low resolution. Many details cannot be read. Please, provide a higher resolution figure.

We apologize for the inconvenience. However, we did not expect the reader to read the details on Figure 2, as it is just an overview of the CovidExpress portal. The aim is give the reader an impression about what functions CovidExpress could offer.

Reviewer #1 (Significance (Required)):

Providing a single platform for the analysis of SARS-CoV-2-related RNAseq data is certainly of high value to the scientific community. However, as the portal and manuscript are currently presented, for scientists that are not RNAseq analysis specialists, more guidance would be required to understand and use correctly the functionalities of the portal. Unfortunately, because batch effects could not be removed from the studies, the authors, correctly, do not recommend to combine data from different studies for analyses, however, this likely will also limit the potential of the resource to make new discoveries beyond what the original studies have already published. As indicated above, the authors could support their claim by comparing their findings with findings published from the studies they reanalyzed. The portal is only of use to scientists studying SARS-CoV-2. I am not an expert in RNAseq data analysis and thus cannot comment on the technicalities, especially the processing of the RNAseq datasets. We thank the reviewer for the positive comments. We apologize for the confusion and acknowledge that we should not describe our effort using the term “batch effect.” As described by Reviewer #2 (and we agree), batch effect should be used only to indicate a purely technical difference in the same biological system; for example, differences in experiments performed on different days or by different lab personnel. Thus, we cannot correct for “batch effect” by using CovidExpress. We hope that the reviewer realizes that what we did was correct for the effect caused by differences in software and parameters across the studies. For example, in our approach, the DEGs from GSE155518 and GSE160435 (both primary lung alveolar AT2 cells (both from Mulay et al., Cell Report, 2021) were significantly correlated (panel A below; p = 1.36e-24, F-test). However, when we downloaded the TPM values from their GEO records, GSE155518 appeared to have a genome-wide decrease in the expression of SARS-CoV-2–infected samples (panel B below). We suspect that this is because in their data processing, the expression of virus themselves were also considered. Thus, using the proceed data directly without careful reviewing the method might lead to false hypothesis.

At last, researchers can make new discoveries, such as our OASL and FURIN findings, by using many other features that CovidExpress provides.

Reviewer #2 (Evidence, reproducibility and clarity (Required)): Djekidel and colleagues describe a web portal to explore several SARS-CoV-2 related datasets. The authors applied a uniform reprocessing pipeline to the diverse RNA-seq datasets and integrated them into a cellxgene-based interface. The major strengths of the manuscript are the scale of the compiled data, with over one thousand samples included, and the data portal itself, which has useful visualization and analysis functions, including GSEA and DEG analysis. My primary concerns with the study are centered on the analysis examples that are presented and their interpretation, as well as the user interface for the data portal. **Major Comments:**

1. The literature analysis feels out of place and is not informative (Fig 1E), as the conclusions that can be drawn from literature mining are minimal. In evidence of this, the authors highlight that CRP is a top-studied "gene" and later voice their interest in how CRP is not a differentially expressed gene (pg6). This illustrates the problems with the literature-based analysis, since in the context of COVID-19, CRP is a common blood laboratory measurement that is used as a general marker of inflammation. Transcription of CRP is essentially exclusively in hepatocytes as an acute phase reactant (see GTEx portal for helpful reference), and would therefore not be expected to be found in the various datasets collected by the authors. The one exception might be liver RNA-seq samples from COVID-19 patients, but I do not think these are available in the current collection. I would therefore suggest to remove the literature analysis parts from the manuscript.

We thank the reviewer for sharing knowledge about CRP. As discussed in our manuscript, we agree that not all top genes from literature-based analysis were expected to be included in RNA-seq analysis. We apologize for the confusion, and we have amended our description to make this point clearer. However, we still believe that literature-based analyses are very useful in the following aspects:

1. This type of analysis bridges the gap between data-driven research and hypothesis-driven research. For example, we found many genes in our meta-analysis, but it is not feasible to describe the functions of all of them. Thus, in Figure 1F, we color-coded genes in red if they also appeared as top genes in the literature-based analysis and read related manuscripts to build confidence that the meta-analysis is useful. Then we expanded our review to more top genes and found more interesting evidence (Supplementary Table S2, “TopGenesbyDifferentialAnalysis” tab).
2. Literature-based analyses also reduce the time researchers spend prioritizing their investigations. For example, in our comparison of SARS-CoV-2–infected lung and Rhinovirus-infected nasal tissue, we found >2000 genes upregulated only in SARS-CoV-2–infected lung but not in Rhinovirus-infected nasal cells. It is not easy to derive a hypothesis from so many genes. When we overlapped the gene list with literature-based analysis, FURIN popped up as the most well-studied gene, and we did not find any report that mentioned that SARS-CoV-2 can regulate FURIN This raised our interest and led to a suggested mechanism in which SARS-CoV-2 could evolve to induce FURIN expression and gain superior infectivity. FURIN’s upregulation is significant but not among the top genes, in terms of fold change (>2-fold change, FDR p th by fold change). Thus, without the literature-based analysis, this observation could have easily been neglected.
3. Such analyses help researchers to prime their hypotheses for novel findings. For example, in our comparison between SARS-CoV-2–infected lung and Rhinovirus-infected nasal tissues (Figure 4G, Supplementary Figure 5D and E), we found many upregulated genes, but OASL was not in our literature-based analysis, which indicated that it is under-studied and worth highlighting. We hope the reviewer will agree that we should retain the literature-based analysis in our paper. These analyses were not meant to be conclusive but rather a way to prioritize investigations. Finally, we removed CRP from Fig 1E and the main text to avoid confusion.

1. The data portal, implemented through cellxgene, is accessible for non-programmers to use. However, it is very easy to end up with an "Unexpected HTTP response 400, BAD REQUEST" error, with essentially no description of the cause of the error or how to rectify it. When this occurs (and in my experience it occurs very frequently), this also forces the user to refresh the page entirely, losing any progress they may have made. I see that the authors describe this error in their FAQ page, but their answer is not very intuitive and I was unsure of what they meant: "This happens because the samples you selected doesn't contain all "Group by" you want compare for each "Split by" group. You could confirm using the "Diff. groups" buttons.".

We apologize for the confusion. This excellent point made by the reviewer required an improvement in the software engineering, which we have now completed. We have figured out how to avoid this error and have run thorough tests to ensure that it does not appear anymore. We also added a gitter chat channel to our landing page, so that users can report if they encounter this or other errors.

I would therefore ask that the authors provide more detailed tutorials (ideally step-by-step) on common analyses that users will want to perform, hopefully minimizing the amount of frustration that users will encounter.

We thank the reviewer for this suggestion. We have uploaded several video tutorials to our landing page and will gradually add more. We also added a gitter chat channel, so users can ask questions, report bugs, or suggest new studies to include in the portal.

1. Selection of samples is not very quick or intuitive. If I wanted to select only the samples from one specific GEO accession, I had to resort to individually checking the boxes of the sample IDs that I wanted. If I instead selected the GEO accession under the samples source ID, then used the "Subset to currently selected samples" button, I invariable got the HTTP error 400 message. Of course, this may simply reflect my lack of familiarity with cellxgene; I would nevertheless encourage the authors to improve the FAQ to include a step-by-step example for how to do common analyses/procedures.

We apologize for the confusion. To select an individual GEO accession, users can simply tick the box beside “Samples Source ID.”

Then all boxes would be clear for “Samples Source ID” that allow you to select only the one you want. We also have uploaded video tutorials to help users learn how to navigate the portal.

We apologize for the “HTTP error 400” messages. We figured out that users would encounter that message frequently after they encounter it once due to a back-end cache mechanism. We have now improved the portal from the software-engineering side. In our recent tests of the latest version, this error does not appear anymore. We also added a gitter chat channel on our landing page so that users can report encountering this or other errors.

1. The second case study, centered on coagulation genes, is misguided. Alteration of coagulation lab values in severe COVID-19 patients is reflecting the general inflammatory state of these patients, and would not be expected to manifest on the transcriptional level in infected cells/tissues. Coagulation labs are measuring the functional status of the coagulation cascade, which is far-removed from the direct transcription of the corresponding genes - proteolytic processing of clotting factors, etc. As with CRP (see above comment), most clotting factors are transcribed almost exclusively in the liver (check GTEx portal); I would not expect upregulation of coagulation factors in lung cell lines/organoids/cultures etc after infection with SARS-CoV-2. I would recommend the authors to pick a different gene ontology set for a case study, as the current one focusing on coagulation is confusing in a pathophysiologic sense.

We thank the reviewer for this suggestion. To avoid any misleading claims, we have replaced the coagulation gene list with a filtered gene list from the “Coronavirus disease - COVID-19” KEGG pathway (hsa05171) to showcase how to identify experiments in which this gene signature is enriched or depleted. We also replaced Figures 3G-J with new results.

1. The two large clusters of blood-derived samples vs other tissues is not surprising and the authors' interpretation is confusing. The authors write that "the COVID-19 signature was not able to overcome the tissue specificity and that immune cells might respond to SARS-CoV-2 differently." This should be immediately obvious given the pathophysiology of COVID-19 infection; the cell types that are directly infected by SARS-CoV-2 will of course have a distinct response compared to the circulating blood cells of COVID-19 patients, which are responding by mounting an immune response. There is no reason to expect a priori that the DEGs in the directly infected lung cells would be similar to that of immune cells that are mounting a response against the virus.

We thank the reviewer for these comments. We agree that it should be obvious that directly infected lung cells would differ from immune cells. However, this has never been shown in a large dataset. Also, it is not obviously whether all other different tissues would respond to SARS-CoV-2 differently. Thus, we believe it is important to present this overview. We have amended the description to deliver clearer message as “This confirmed immune cells respond to SARS-CoV-2 differently from other tissues also suggested the response of most other tissues might sharing similar features.”.

1. The authors devote considerable space in the manuscript to exploring "batch effects" and trying to minimize them (pg10-11 Fig 4A-D, Fig S4). However, given that the compiled datasets are from entirely different experimental and biological systems (e.g. in vitro infection vs patient infection, different cell lines, timepoints after virus exposure, diverse tissues, varying disease severity), it is inappropriate to simply refer to all of these differences as "batch effects" alone. Usually, the term "batch effect" would refer to the same biological experiment/system (i.e. A549 cells infected with CoV vs control), but performed on different days or by different lab personnel - in other words, batch effects are purely due to technical differences. This term clearly does not apply when comparing samples from entirely different cell lines, or tissues, etc, and the authors should not keep describing these differences as batch effects that should be "corrected" out.

We thank the reviewer for the insight. We apologize for the confusion caused by using the phrase “batch effect correction” to describe our approach. We agree that the difference between studies should not be referred to as a “batch effect correction” and have now amended the descriptions to avoid confusion.

Indeed, the authors themselves state that the main point of their "batch effect correction" efforts is only for PCA visualization. I therefore feel this section contributes very little to the overall manuscript, especially given the authors' own recommendation that all analyses should be performed on individual datasets (which I certainly agree with). I assume that the authors were required to provide some sort of dimensional reduction projection for the cellxgene browser, but this is more a quirk in their choice of platform for the web portal. Thus, this section of the manuscript should be deemphasized.

We thank the reviewer for these comments and again apologize for the confusion caused by our use of the term “batch effect correction” to describe our approach. However, we believe these parts of the paper should be retained for the following reasons:

• In practice, sample mislabeling can happen. PCA or simple clustering approaches are very useful for helping raise researchers’ attention, so they could further check the possibility of sample mislabeling.
• Even within a study, one sample can be an outlier due to low or unequal sample quality. Removing outliers would help boost the significance of real findings. Without our approach, it would be harder for users to notice and remove outliers from their investigations.
• Finally, these efforts are useful for generating hypotheses. For example, although we collected a lot of data, it is not feasible for us to read all the details in all the manuscripts published. We observed a similarity between SARS-CoV-2–infected lung samples and Rhinovirus–infected nasal samples by exploring our portal’s capabilities (Figure 3E-F). Then we read the manuscripts in which those data were published and found that our discovery was consistent with the original studies’ results. We believe these efforts are essential to help researchers generate or refine their hypotheses. As we update the database with more samples, this approach will become increasingly powerful.

  1. Given the limitations of any combined multi-dataset analyses, one very useful feature would be to conduct "meta-analyses" across multiple datasets. For instance, it would be informative to find which genes are commonly DEGs in user-selected comparisons, calculated separately for each dataset and then cross-referenced across the relevant/user-selected datasets.

We thank the reviewer for this comment. Indeed, we agree that “meta-analyses” are useful and have now compiled Supplementary Table S2 and Figure 1F to demonstrate the commonly regulated genes. To enable user-selected comparisons across studies on our portal, we need to design a thoughtful user interface. Otherwise, the results from our portal could easily cause fatal misinterpretation. For example, GSE154613 includes samples like DMSO, Drug, SARS-CoV-2, and DMSO+SARS-CoV-2. If a user simply selected to compare SARS-CoV-2 versus Control, the results would be SARS-CoV-2 and DMSO+SARS-CoV-2 versus DMSO and Drug. Such functions need time to design and implement; therefore, we will consider this suggestion for further development of our portal.

**Minor comments:**

1. Fig S1G, color legend should be added (I understand that these colors are the same from S1H).

We thank the reviewer for the comment. We have now added information about the colors in the figure legend.

1. Mouseover text for trackPlot on the data portal is incorrect (it says the heatmap text instead).

We thank the reviewer for this comment. We have now corrected this bug.

1. Abstract should be revised to describe only the 1093 final remaining RNA-seq samples after filtering/QC steps.

We thank the reviewer for this comment. We have now amended the Abstract to include this information.

1. Text in many figures is too small to be legible. I would suggest pt 6 font minimum for all figure text, including the various statistics in the figure panels.

We thank the reviewer for this comment. We have now amended the font sizes and will provide high-resolution figures in revision.

1. Are the DE analyses in Fig 1F specifically limited to control vs SARS-CoV-2/COVID-19 comparisons? Many of the samples included in this study are from other respiratory infections (labeled "other" in Fig 1B).

We thank the reviewer for the question. Figure 1F was not originally limited to control vs SARS-CoV-2/COVID-19 comparisons, because we thought control vs virus, drug vs mock, or difference between time points would also be interesting. If we narrow the analysis to contrasts only between control vs SARS-CoV-2/COVID-19, Figure 1F would be still look similar (as below) because the genes in that comparison comprise the largest share of genes included in the original graphic.

In the end, we replaced Figure 1F to avoid confusion and added more details in the Methods.

1. The word cloud format is not conducive for understanding or interpretation. It would be much more informative to simply have a barplot or similar to clearly indicate the relative "abnudance" of a given gene among all 315 DE analyses.

We thank the reviewer for this comment but respectfully disagree with this point. Visualization of the relative “abundance” of genes with word clouds is a relatively novel concept in computational biology. However, we believe, that in this case, it has certain advantages over visualization using traditional bar plots for example. The word cloud format allows us to highlight genes relative to their importance, with the word “importance” being used here in the sense of combined metrics from DEGs, as shown in Figure 1F, or the frequency with which genes are mentioned/discussed in various literature sources, as shown in Figure 1E. For this purpose, the exact values will most likely not be important for most users/readers. Be presenting a word cloud visualization, readers can easily discern the top genes and use them in the exploration of their own data or the CovidExpress portal. However, if users want to analyze raw values, we provide in Supplementary Table S3 a full list of all genes and gene sets that can be download from our landing page (section “CovidExpress Expression Data Download”) in GMT format. Also, when we visualized the ranks of genes by using bar plots as the reviewer suggested, the results were much harder to read (as shown in the bar graph below) than simply looking at the raw data in supplementary tables.

1. Claims of increased/decreased dataset separability should have statistical analysis on the silhouette score boxplots (Fig S4G-I).

We thank the reviewer for the reminder. We have added statistical tests to referred silhouette score boxplots (Wilcoxon rank test)

1. Regarding Fig 4E-F - what are the key genes that contribute to PC1, and how do they relate to the DEGs in Fig 4G?

We thank the reviewer for this question and apologize for the confusion. In Figure 4E-F, the PCA were based on ssGSEA score, as each gene set would have a score for a sample, not individual genes. Thus, the top contributed to PC1 were gene sets upregulated or down-regulated in certain contrasts. We provided on the portal’s landing page detailed results for top gene sets (for the ssGSEA approach) and genes (for the TPM approach) that contributed to various PCs (“Clustering Results for Reviewing and Download” section). This allows users to download and further explore these data.

1. Statistics describing the relation between OASL And TNF/PPARGC1A should be included to justify the author's statements. This could be correlation, mutual information, regression, etc.

We thank the reviewer for this suggestion, and we have updated Figures 4J-K to show the correlation values and corresponding F-statistics. The Pearson correlation between OASL and TNF was significant (Pearson Correlation=0.75 and p-value = 6.85e-72), but the correlation between OASL and PPARGC1A had a negative slope and showed a moderately significant p-value (Pearson Correlation=-0.08 and p-value=0.12), confirming to a certain degree our statement. We have now updated the corresponding text in the manuscript.

1. There are several studies now that have performed scRNA-seq on the lung resident and peripheral immune cells of COVID-19 patients. To more definitively tie in their analyses in Fig 4J-K/Fig S5D-E (to affirm "its important role in the innate immune response in lungs"), the authors should assess whether OASL is upregulated in the lung macrophages of COVID-19 patients vs controls.

We thank the reviewer for this suggestion. Indeed, Liao, et al. recently reported “BALFs of patients with severe/critical COVID-19 infection contained higher proportions of macrophages and neutrophils and lower proportions of mDCs, pDCs, and T cells than those with moderate infection.” (Nature Medicine, 2020, https://doi.org/10.1038/s41591-020-0901-9). They further refined macrophage data into subclusters and reported top enriched GO terms as “response to virus” (group 1), “type I interferon signaling pathway” (group 2), “neutrophile degranulation” (group 3), and “cytoplasmic translational initiation” (group 4). When we investigated their data, we found that group1 and group2 both identified OASL as a marker gene, indicated OASL might response to virus and help type I interferon signaling. Furthermore, another data set (from Ren et al., Cell, 2021, https://dx.doi.org/10.1016%2Fj.cell.2021.01.053) showed several clusters in patients with severe COVID-19 (left panel below) that were enriched for OASL expression(right panel below).

We have now added these observations to strengthen our hypothesis about the role of OASL.

1. The visualization and analysis functions in the data portal appear to work reasonably well out of the box. However, the download buttons for plots did not work in my hands. I realized that a workaround is to right click -> "Save image as" (which then downloads a .svg file), but this is not ideal and should be fixed to improve usability. I had tested the data portal on both Firefox and Edge browsers, using a Windows 10 PC.

We agree with the reviewer. Due to some technical issues with the figure javascript plugin, the download feature does not work unless the figure is saved as a file on the server side. To avoid any security issues, we tried to minimize new file generations, hence, for the moment we have disabled this feature. Users can still download high-resolution .svg figures by using the right-click -> “save image as.” This information is now included in the FAQ section on the portal’s landing page.

Reviewer #2 (Significance (Required)): The data portal appears to have useful analysis and visualization features, and the data collection appears to be quite comprehensive. I would strongly encourage the authors to continue collecting datasets as they become available and further improving the usability of the portal. As noted in the above comments, I think there is potential for their cellxgene-based browser to be useful to non-computational biologists, but at present, the data portal is not as simple to use as it should be. With further efforts to developing step-by-step tutorials for common analysis/visualization tasks, more informative case studies, and the other revisions suggested above, this study could be a valuable resource for the community. Of note, this review is written from the perspective of a primary wet-lab biologist with extensive bioinformatics experience but limited web development expertise.

We thank the reviewer for the positive comments. We understand the importance of data updating. Our plan is to complete quarterly updates once this manuscript has been accepted or when 10 new studies have been either collected by us or suggested by users. This information is also now included in the FAQs of the portal’s landing page. We have also uploaded several tutorials videos to the landing page and will gradually add more. We also added a gitter chat channel, so users can ask questions, report bugs, or suggest new studies to add to the database.

**Referee Cross-commenting** I agree with the comments of the other reviewers. Reviewer #3 (Evidence, reproducibility and clarity (Required)): **Summary:** The ongoing COVID-19 pandemic is a big threat to human health. The researchers have conducted studies to explore the gene expression regulations of human cells responding to COVID-19 infection. A website that integrating those datasets and providing user-friendly tools for gene expression analysis is a valuable resource for the COVID-19 study community. The authors collected published RNASeq datasets and developed a database and an interactive portal for users to investigate the gene expression of SARS-CoV-2 related samples. This website would be of great value for the SARS-CoV-2 research community if the batch normalization problems are solved. **Major comments:** 1) The major concern of CovidExpress is the batch effects from different studies. As the authors have shown and mentioned in their discussion that "For the current release, we strongly suggest investigators to perform gene expression comparison within individual study." This limits the usage of CovidExpress as integrating analysis from multiple datasets of different studies is the key value and purpose of CovidExpress.

We thank the reviewer for the comment. Reviewer #2 reminded us, and we agree, that differences between studies should not be considered “batch effects.” We apologize for the confusion. The GSEA function provided in the portal does not suffer from batch effect, because all the pre-ranked lists of genes are based on contrasts from the same studies. Although we cannot correct for the differences between studies, we did correct for effect caused by differences in software and parameters used. For example, in our approach, the DEGs from GSE155518 and GSE160435 (both studies of primary lung alveolar AT2 cells from Mulay et al., Cell Report, 2021) were significantly correlated (below panel A, p-value = 1.36e-24, F-test). However, if we simply download the TPM values from their GEO records, GSE155518 appears to show a genome-wide decrease in expression in SARS-CoV-2–infected samples (below panel B). These errors might lead to false hypotheses.

2) The authors should include experimental protocols as one key parameter in the description and further integrating analysis of different datasets. As the authors showed that QuantSeq is a 3' sequencing protocol of RNA sequencing. However, it is not convincing to me that simply excluding QuantSeq samples is the ideal solution for downstream integrating analysis as QuantSeq has been shown that it has pretty good correlations with normal RNASeq methods in gene quantifications. It is interesting that there are 21.2% of samples were biased toward intronic reads. What protocol differences or experimental variations would explain the biases?

We thank the reviewer for the comment and apologized for not being clearer. One of our main goals re-processing all samples is to correct for pipeline processing–related batch effects. We tried to reduce those effects introduced by using different software or parameters. QuantSeq or similar protocols are heavily bias to 3’ UTR; thus, the software and parameters used for RNA-seq data will not be suitable. In contrast, we agree that the downstream results from QuantSeq have good correlation to RNA-seq (we observed a correlation of ~0.75, when compared to the log2 fold-change from Quant-Seq to RNA-seq). However, we could not reconcile QuantSeq always correlated well with RNA-seq, in terms of individual quantification. For example, Jarvis et al. recently reported only ~0.35 correlation between QuantSeq and RNA-seq (https://doi.org/10.3389/fgene.2020.562445). Theoretically, the correlation would be weaker for genes with a small 3’ UTR. Thus, we will not include QuantSeq data in this portal. However, if we collect enough studies in the future, we will consider uploading a separate portal just for QuantSeq using a pipeline optimized for protocol bias to 3’ UTR.

For the 21.2% samples that were biased towards intronic reads, we believe they reflect differences in the kits used. For example, of the 162 samples “BASE_INTRON (%)” >30% (Supplementary Table S1) that passed QC, 76 samples were total RNA obtained using the SMARTer kit and 36 were total RNA obtained using the Trio kit. Given that we have 105 samples of total RNA derived using the SMARTer kit and 38 samples of total RNA derived using the Trio kit, we conclude that the Trio kit was more biased toward introns, and the SMARTer kit was also strongly biased. This finding is consistent with those of others who have reported the bias of the SMARTer kit (Song et al., https://doi.org/10.1186/s12864-018-5066-2). Users can find these results in our Supplementary Table S1. We have also uploaded the protocol information to our portal.

3) How do the authors plan to update and maintain CovidExpress?

We thank the reviewer for this question. We understand the importance of data updating. Our plan is to update the database quarterly once this manuscript has been accepted or when 10 new studies have been collected by us or suggested by users. We have added this information to the FAQs on the portal’s landing page. We also understand the importance of maintaining the service for a feasible amount of time for research. Therefore, we will keep the server activated for at least 2 years after the WHO announces that COVID-19 is no longer a global pandemic. We will also ensure that, even after we take down the server , scientists with programming skills will be able to create local servers based on the data provided on CovidExpress.

**Minor comments:** 1) Some texts in figures are not readable. For example, Fig2B, 2C, 2D, 2E.

We thank the reviewer for this comment. We have now increased the font sizes and provided high-resolution figures in revision.

2) The authors could use Videos to demonstrate how to use CovidExpress on the website as they have shown in Fig3.

We thank the reviewer for this suggestion. We have uploaded several video tutorials to the landing page and will gradually add more. We also added a gitter chat channel so that users can ask questions, report bugs, or suggest new studies to include in the database.

Reviewer #3 (Significance (Required)): The ongoing COVID-19 pandemic is a big threat to human health. Many molecular and cellular questions related to COVID-19 pathophysiology remain unclear and many researchers have conducted studies to explore the gene expression regulations of human cells responding to COVID-19 infection. However, there is no database/website that integrating all RNASeq data to provide user-friendly tools for gene expression analysis for COVID-19 researchers. The authors collected the published RNASeq datasets and developed a database and an interactive portal, named CovidExpress, to allow users to investigate the gene expressions response to COVID-19 infection. CovidExpress is a valuable resource for the COVID-19 study community once the batch normalization problems are solved. The users who came up with ideas about the regulation of COVID-19 response could use the system to test their hypothesis, without experience in bioinformatics and RNASeq data analysis. This will be more important when more RNASeq data from samples with different tissues, cell lines, and conditions are integrated into the database.

We thank the reviewer for the positive comments. We apologize for the confusion and acknowledge that we should not describe our effort using the term “batch effect.” As described by Reviewer #2 (and we agree), batch effect should be used only to indicate a purely technical difference in the same biological system; for example, differences in experiments performed on different days or by different lab personnel. Thus, we cannot correct for “batch effect” by using CovidExpress. We hope that the reviewer realizes that what we did was correct for the effect caused by differences in software and parameters across the studies. For example, in our approach, the DEGs from GSE155518 and GSE160435 (both primary lung alveolar AT2 cells (both from Mulay et al., Cell Report, 2021) were significantly correlated (panel A below; p = 1.36e-24, F-test). However, when we downloaded the TPM values from their GEO records, GSE155518 appeared to have a genome-wide decrease in the expression of SARS-CoV-2–infected samples (panel B below).

Thus, using the proceed data directly without careful reviewing the method might lead to false hypothesis. At last, researchers can make new discoveries, such as our OASL and FURIN findings, by using many other features that CovidExpress provides.

Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Referee #3

Evidence, reproducibility and clarity

Summary:

The ongoing COVID-19 pandemic is a big threat to human health. The researchers have conducted studies to explore the gene expression regulations of human cells responding to COVID-19 infection. A website that integrating those datasets and providing user-friendly tools for gene expression analysis is a valuable resource for the COVID-19 study community. The authors collected published RNASeq datasets and developed a database and an interactive portal for users to investigate the gene expression of SARS-CoV-2 related samples. This website would be of great value for the SARS-CoV-2 research community if the batch normalization problems are solved.

Major comments:

1) The major concern of CovidExpress is the batch effects from different studies. As the authors have shown and mentioned in their discussion that "For the current release, we strongly suggest investigators to perform gene expression comparison within individual study." This limits the usage of CovidExpress as integrating analysis from multiple datasets of different studies is the key value and purpose of CovidExpress.

2) The authors should include experimental protocols as one key parameter in the description and further integrating analysis of different datasets. As the authors showed that QuantSeq is a 3' sequencing protocol of RNA sequencing. However, it is not convincing to me that simply excluding QuantSeq samples is the ideal solution for downstream integrating analysis as QuantSeq has been shown that it has pretty good correlations with normal RNASeq methods in gene quantifications. It is interesting that there are 21.2% of samples were biased toward intronic reads. What protocol differences or experimental variations would explain the biases?

3) How do the authors plan to update and maintain CovidExpress?

Minor comments:

1) Some texts in figures are not readable. For example, Fig2B, 2C, 2D, 2E.

2) The authors could use Videos to demonstrate how to use CovidExpress on the website as they have shown in Fig3.

Significance

The ongoing COVID-19 pandemic is a big threat to human health. Many molecular and cellular questions related to COVID-19 pathophysiology remain unclear and many researchers have conducted studies to explore the gene expression regulations of human cells responding to COVID-19 infection. However, there is no database/website that integrating all RNASeq data to provide user-friendly tools for gene expression analysis for COVID-19 researchers. The authors collected the published RNASeq datasets and developed a database and an interactive portal, named CovidExpress, to allow users to investigate the gene expressions response to COVID-19 infection. CovidExpress is a valuable resource for the COVID-19 study community once the batch normalization problems are solved. The users who came up with ideas about the regulation of COVID-19 response could use the system to test their hypothesis, without experience in bioinformatics and RNASeq data analysis. This will be more important when more RNASeq data from samples with different tissues, cell lines, and conditions are integrated into the database.

Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Referee #2

Evidence, reproducibility and clarity

Djekidel and colleagues describe a web portal to explore several SARS-CoV-2 related datasets. The authors applied a uniform reprocessing pipeline to the diverse RNA-seq datasets and integrated them into a cellxgene-based interface. The major strengths of the manuscript are the scale of the compiled data, with over one thousand samples included, and the data portal itself, which has useful visualization and analysis functions, including GSEA and DEG analysis. My primary concerns with the study are centered on the analysis examples that are presented and their interpretation, as well as the user interface for the data portal.

Major Comments:

1. The literature analysis feels out of place and is not informative (Fig 1E), as the conclusions that can be drawn from literature mining are minimal. In evidence of this, the authors highlight that CRP is a top-studied "gene" and later voice their interest in how CRP is not a differentially expressed gene (pg6). This illustrates the problems with the literature-based analysis, since in the context of COVID-19, CRP is a common blood laboratory measurement that is used as a general marker of inflammation. Transcription of CRP is essentially exclusively in hepatocytes as an acute phase reactant (see GTEx portal for helpful reference), and would therefore not be expected to be found in the various datasets collected by the authors. The one exception might be liver RNA-seq samples from COVID-19 patients, but I do not think these are available in the current collection. I would therefore suggest to remove the literature analysis parts from the manuscript.
2. The data portal, implemented through cellxgene, is accessible for non-programmers to use. However, it is very easy to end up with an "Unexpected HTTP response 400, BAD REQUEST" error, with essentially no description of the cause of the error or how to rectify it. When this occurs (and in my experience it occurs very frequently), this also forces the user to refresh the page entirely, losing any progress they may have made. I see that the authors describe this error in their FAQ page, but their answer is not very intuitive and I was unsure of what they meant: "This happens because the samples you selected doesn't contain all "Group by" you want compare for each "Split by" group. You could confirm using the "Diff. groups" buttons.".

I would therefore ask that the authors provide more detailed tutorials (ideally step-by-step) on common analyses that users will want to perform, hopefully minimizing the amount of frustration that users will encounter.

1. Selection of samples is not very quick or intuitive. If I wanted to select only the samples from one specific GEO accession, I had to resort to individually checking the boxes of the sample IDs that I wanted. If I instead selected the GEO accession under the samples source ID, then used the "Subset to currently selected samples" button, I invariable got the HTTP error 400 message. Of course, this may simply reflect my lack of familiarity with cellxgene; I would nevertheless encourage the authors to improve the FAQ to include a step-by-step example for how to do common analyses/procedures.
2. The second case study, centered on coagulation genes, is misguided. Alteration of coagulation lab values in severe COVID-19 patients is reflecting the general inflammatory state of these patients, and would not be expected to manifest on the transcriptional level in infected cells/tissues. Coagulation labs are measuring the functional status of the coagulation cascade, which is far-removed from the direct transcription of the corresponding genes - proteolytic processing of clotting factors, etc. As with CRP (see above comment), most clotting factors are transcribed almost exclusively in the liver (check GTEx portal); I would not expect upregulation of coagulation factors in lung cell lines/organoids/cultures etc after infection with SARS-CoV-2. I would recommend the authors to pick a different gene ontology set for a case study, as the current one focusing on coagulation is confusing in a pathophysiologic sense.
3. The two large clusters of blood-derived samples vs other tissues is not surprising and the authors' interpretation is confusing. The authors write that "the COVID-19 signature was not able to overcome the tissue specificity and that immune cells might respond to SARS-CoV-2 differently." This should be immediately obvious given the pathophysiology of COVID-19 infection; the cell types that are directly infected by SARS-CoV-2 will of course have a distinct response compared to the circulating blood cells of COVID-19 patients, which are responding by mounting an immune response. There is no reason to expect a priori that the DEGs in the directly infected lung cells would be similar to that of immune cells that are mounting a response against the virus.
4. The authors devote considerable space in the manuscript to exploring "batch effects" and trying to minimize them (pg10-11 Fig 4A-D, Fig S4). However, given that the compiled datasets are from entirely different experimental and biological systems (e.g. in vitro infection vs patient infection, different cell lines, timepoints after virus exposure, diverse tissues, varying disease severity), it is inappropriate to simply refer to all of these differences as "batch effects" alone. Usually, the term "batch effect" would refer to the same biological experiment/system (i.e. A549 cells infected with CoV vs control), but performed on different days or by different lab personnel - in other words, batch effects are purely due to technical differences. This term clearly does not apply when comparing samples from entirely different cell lines, or tissues, etc, and the authors should not keep describing these differences as batch effects that should be "corrected" out.

Indeed, the authors themselves state that the main point of their "batch effect correction" efforts is only for PCA visualization. I therefore feel this section contributes very little to the overall manuscript, especially given the authors' own recommendation that all analyses should be performed on individual datasets (which I certainly agree with). I assume that the authors were required to provide some sort of dimensional reduction projection for the cellxgene browser, but this is more a quirk in their choice of platform for the web portal. Thus, this section of the manuscript should be deemphasized.

1. Given the limitations of any combined multi-dataset analyses, one very useful feature would be to conduct "meta-analyses" across multiple datasets. For instance, it would be informative to find which genes are commonly DEGs in user-selected comparisons, calculated separately for each dataset and then cross-referenced across the relevant/user-selected datasets.

Minor comments:

1. Fig S1G, color legend should be added (I understand that these colors are the same from S1H).
2. Mouseover text for trackPlot on the data portal is incorrect (it says the heatmap text instead).
3. Abstract should be revised to describe only the 1093 final remaining RNA-seq samples after filtering/QC steps.
4. Text in many figures is too small to be legible. I would suggest pt 6 font minimum for all figure text, including the various statistics in the figure panels.
5. Are the DE analyses in Fig 1F specifically limited to control vs SARS-CoV-2/COVID-19 comparisons? Many of the samples included in this study are from other respiratory infections (labeled "other" in Fig 1B).
6. The word cloud format is not conducive for understanding or interpretation. It would be much more informative to simply have a barplot or similar to clearly indicate the relative "abnudance" of a given gene among all 315 DE analyses.
7. Claims of increased/decreased dataset separability should have statistical analysis on the silhouette score boxplots (Fig S4G-I).
8. Regarding Fig 4E-F - what are the key genes that contribute to PC1, and how do they relate to the DEGs in Fig 4G?
9. Statistics describing the relation between OASL And TNF/PPARGC1A should be included to justify the author's statements. This could be correlation, mutual information, regression, etc.
10. There are several studies now that have performed scRNA-seq on the lung resident and peripheral immune cells of COVID-19 patients. To more definitively tie in their analyses in Fig 4J-K/Fig S5D-E (to affirm "its important role in the innate immune response in lungs"), the authors should assess whether OASL is upregulated in the lung macrophages of COVID-19 patients vs controls.
11. The visualization and analysis functions in the data portal appear to work reasonably well out of the box. However, the download buttons for plots did not work in my hands. I realized that a workaround is to right click -> "Save image as" (which then downloads a .svg file), but this is not ideal and should be fixed to improve usability. I had tested the data portal on both Firefox and Edge browsers, using a Windows 10 PC.

Significance

The data portal appears to have useful analysis and visualization features, and the data collection appears to be quite comprehensive. I would strongly encourage the authors to continue collecting datasets as they become available and further improving the usability of the portal. As noted in the above comments, I think there is potential for their cellxgene-based browser to be useful to non-computational biologists, but at present, the data portal is not as simple to use as it should be. With further efforts to developing step-by-step tutorials for common analysis/visualization tasks, more informative case studies, and the other revisions suggested above, this study could be a valuable resource for the community. Of note, this review is written from the perspective of a primary wet-lab biologist with extensive bioinformatics experience but limited web development expertise.

Referee Cross-commenting

I agree with the comments of the other reviewers.

Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Referee #1

Evidence, reproducibility and clarity

Summary:

The manuscript submitted by Djekidel et al entitled: "CovidExpress: an interactive portal for intuitive investigation on SARS-CoV-2 related transcriptomes" reports on a new web portal to search and analyze RNAseq data related to SARS-CoV-2 infections. The authors downloaded and reprocessed data of more than 40 different studies, which is available on the web portal along with all available meta data. The web portal allows to perform numerous differential expression and gene set enrichment analyses on the data and provides publication ready figures. Because of batch effects that could not be removed, the authors do not recommend to analyze data across studies at this point. The authors conclude that the web portal is unique and will allow scientists to rapidly analyze gene expression signatures related to SARS-CoV-2 infections with the potential to make new discoveries.

Major comments:

Based on the scientific literature, the web portal seems to be an unprecedented resource to search and analyze SARS-CoV-2-related RNAseq data and as such would certainly be a useful resource for the SARS-CoV-2 scientific community. The authors argue that new discoveries are possible by using their web portal in providing use cases. However, the section detailing the analyses the authors did to generate new hypotheses about genes potentially relevant in SARS-CoV-2 infections are very difficult to follow and without more guidance very difficult to reproduce with the web portal. It would require substantial expert knowledge in RNAseq data analysis without more information being provided. It also seems that key candidate genes identified by their analyses have all been studied or identified to be related to SARS-CoV-2 infections, so it is somewhat unclear whether new hypotheses can be generated by the reanalysis of RNAseq datasets, especially because combining the data from different studies is currently not recommended by the authors. The manuscript would benefit from providing fewer use cases but for each of them providing more information on how the portal and which studies were used to generate them and which findings were not described in the publication of the used studies. Some observations in the manuscript are not substantiated with significance calculations (see below). At times, the English writing (grammar) should be improved.

Minor comments:

Page 6 last sentence: The statement of this sentence is very much what one would expect. It remains unclear whether the authors mean this as a result to validate the processing of the RNAseq data or as a new discovery. Please, clarify.

Figure 3A: The violin plots are so tiny that it is impossible to see any trends. It is also difficult to understand which categories one should compare with each other. If there is anything significant to observe, please, add a statistical test and better guide the reader.

Figure 3C: A legend for the color scale is missing. The signal (I guess expression amounts) for SESN2 seems very weak and the same between ICU and non-ICU samples. What is the significance for assigning this gene to the group of genes being upregulated in ICU samples? Also contrary to what the authors state on page 8, SESN2 does not seem to be highly expressed in ICU samples, however, without knowing what the colors represent (fold changes or absolute expression values?) this is somewhat speculative.

Page 9 first sentence: Please, specify what you mean by "starting list". Furthermore, in this paragraph, how do your results compare to the results from the study that you re-analyze here?

Figure 3F: Please add labels to your axes and is there a particular reason why in a correlation plot like this one, the y and x axis are not shown with the same range and why does the y axis not start at 0?

Page 9 second last sentence: It remains unclear which kind of analysis the authors intend to do here and what the starting question is. Please, try to rewrite with less technical terms (i.e. what do you mean by "precalculated contrasts"). In line with this, it remains unclear what Figure 3I is supposed to show. Please, provide some more information to readers who are not RNAseq analysis experts.

Figure 3J is somewhat confusing. Why is the mean expression range indicated from 0 to 1 and why are all genes apparently having a mean expression of 1? Page 10 line 5-6. Are you referring to coagulation markers here or general expression patterns? In case of the latter, how does this statement fit to the paragraph about analyzing expression patterns of coagulation markers? Please, specify. And in line with this, are the highlighted genes in Figure 3K coagulation markers? If not, what is the relevance of these to make the point that one can use the portal to investigate the role of coagulation markers in SARS-CoV-2 infections?

The appearance of describing batch effects and attempts to remove them from the studies was somewhat surprising on page 10 as I would expect this kind of results rather earlier in the results section before describing use cases of the data. You may consider changing the order of your results for a better flow. Page 11, second paragraph. Please, explain briefly what the silhouette score is supposed to reflect and thus how Figure S4G should be interpreted. The difference of both bars in Figure S4G is very marginal and thus, does not seem to support the statement of the authors that the ssGSEA scores-based projection is better unless you perform a significance test or I misunderstood. Please, clarify.

Page 11, third paragraph: Figure 4B, to the best of my understanding, does not support the claim that samples clustered less according to study cohorts using the ssGSEA approach. Please, quantify the effect and test for significance or better explain.

For the analyses described starting on page 12 it remains largely unclear whether they were conducted across studies or within studies and which studies were used. This section until the end of the results would especially benefit from providing more information on how the analyses were performed, either in the results or in the methods section.

Figures 4J and 4K miss axis labels and since we look at correlations, the figures could be redrawn using the same ranges on x and y axis.

Page 14 line 5: Is this the right figure reference here to Figure 4G? If yes, then it is unclear how Figure 4G supports the statement in this sentence. Please, clarify. Figure 2 is of too low resolution. Many details cannot be read. Please, provide a higher resolution figure.

Significance

Providing a single platform for the analysis of SARS-CoV-2-related RNAseq data is certainly of high value to the scientific community. However, as the portal and manuscript are currently presented, for scientists that are not RNAseq analysis specialists, more guidance would be required to understand and use correctly the functionalities of the portal. Unfortunately, because batch effects could not be removed from the studies, the authors, correctly, do not recommend to combine data from different studies for analyses, however, this likely will also limit the potential of the resource to make new discoveries beyond what the original studies have already published. As indicated above, the authors could support their claim by comparing their findings with findings published from the studies they reanalyzed. The portal is only of use to scientists studying SARS-CoV-2. I am not an expert in RNAseq data analysis and thus cannot comment on the technicalities, especially the processing of the RNAseq datasets.

Transparent Peer Review

---

Download the complete Review Process [PDF] including:

• reviews
• authors' reply
• editorial decisions

</br>

Note: This rebuttal was posted by the corresponding author to Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Reply to the reviewers

Manuscript number: RC-2021-01024

Corresponding author(s): Martin Spiess

1. Description of the planned revisions — point-by-point response

---

Reviewer #1 (Evidence, reproducibility and clarity (Required)):

Apart from the default constitutive pathway for protein secretion some specialized cells (e.g., neuroendocrine cells, exocrine cells, peptidergic neurons and mast cells) exhibit additional regulated secretory pathway, where peptide hormones are stored as highly concentrated ordered manner inside electron opaque "dense core" of secretory granule for long duration until secretagogue mediated burst release. Although the general sorting receptor for packaging hormones in secretory granules is not yet identified, self-aggregation in the trans-Golgi network is a common shared property of peptide hormones and is a well-accepted potential sorting mechanism. Here the authors have hypothesized that cysteine containing small disulphide loop (CC loop), which is abundant in several hormone precursors, acts as aggregation mediator in TGN for sorting into secretory granule. They have tested the aggregation propensity of a misfolded reporter protein, NPΔ, in ER by attaching the CC loop segment of different hormones which promoted the pathological aggregation in endoplasmic reticulum (ER) of mutant provasopressin in the case of diabetes insipidus. Immunofluorescence and immunogold electron microscopy revealed accumulation of aggregates in the ER when CC loop of different hormonal origin fused NPΔ was transiently transfected in COS-1 fibroblasts and Neuro-2a neuroblastoma cells. The authors have also shown small disulphide loop mediated functional aggregation in TGN can sort a constitutively secreted protein, α1-protease inhibitor, into the secretory granule. The rerouting capacity of CC loop was tested in stably expressed AtT-20 cell line by confirming their localization with CgA-positive secretory granule as well as by studying BaCl2 mediated stimulated secretion and by testing secretory granule specific lubrol insolubility.

**Major comments:**

The study is highly impressive, and the results fully support the CC loop mediated hormone sorting hypothesis. However, it would be nice if the authors characterize the nature of the CC-loop mediated aggregates as hormones are reported to be stored inside secretory granules as functional amyloid (Maji et al., 2009). The mechanistic reason behind the small disulfide loop mediated aggregation was not explained in the paper. Authors may propose the probable molecular reasons behind CC loop mediated aggregation to completely justify their hypothesis.

Although the hypothesis and the experimental results are highly impressive, the authors may consider adding the following experiments.

The authors replaced CC-loop by the proline/glycine repeat sequence (Pro1) as a negative control which was previously reported to abolish aggregation as well. However, the authors may completely delete the small loop forming segment, CCv, and may check the status of His-tagged fused neurophysin II (NPΔ) segment as an additional negative control. We plan to use a NP∆ construct completely lacking any N-terminal extension as a further negative control, as proposed by the reviewer.

To find the ultrastructure authors have done immunogold assay with anti-His antibody which indicated different CC loop mediated ER aggregation. Since the amyloid-like fibril nature of pro-vasopressin mutant mediated ER aggregates was previously reported (Beuret et al., 2017), authors must check the nature of the CC loop mediated ER aggregates with amyloid specific antibody.

We will test staining ER aggregates of our CC loop–NP∆ constructs with anti-amyloid antibodies. A caveat is that CC loops cannot form a classical cross-b structure (strict b-sheets) because of the ring closure – which is why we suggest their aggregation to be "amyloid-like". These structures may not be recognized by anti-amyloid antibodies.

Since hormones are known to form reversible functional amyloid during their storage inside secretory granule, authors may consider characterizing the nature of the aggregates formed by CC loop fused constitutive protein in AtT-20 cell line by immunostaining, immunoprecipitation and dot blot assay using amyloid specific antibody. Endogenous AtT20 granules are expected to be positive for amyloid stains or antibodies anyway (if the size and mass of the granules is sufficient for detection; Maji et al. used pituitary tissue and purified granules).

**Minor comments:**

In the quantification study (Figure 2C) CCc and CCr showed almost similar ER aggregates (around 40%). But authors have commented that all constructs except CCc produce statistically significant increases in cells compared to background. Authors must clarify the statement.

CCc also increased, but in a statistically not significant manner (p = 0.08). We will change the sentence to: "It confirmed the ability of all constructs to produce an increase of cells with aggregates above background in COS-1 cells (Figure 2C), although not statistically significant for CCc (p = 0.08)."

In lubrol insolubility assay, the otherwise constitutively secreted protein A1Pimyc (negative control) showed 23% insolubility. The authors explained the observation by commenting about trapping of the protein inside granule aggregate. But CCv and CCa fused proteins showed a very slight increase (around 30%). Only CCc construct showed more than 40% insolubility. If the trapping of constitutive protein may result in 23% insolubility, all the insolubility data except CCc is not satisfactory to claim as secretory granular content of aggregated protein. The authors must explain that.

Lubrol insolubility is an empirical assay with high specificity for Golgi/post-Golgi forms, but with a relatively high background that we suggest to be due to trapping. Interpretation is based on statistical analysis of several independent experiments. It supports the conclusion of the other assays from an independent angle.

We present the data of the paired t-test

The authors have satisfactorily referenced prior studies in the field. However, authors may consider adding the following papers as they are directly connected with the hypothesis. The sorting of POMC hormone into secretory granules by disulphide loop was previously studied. (Cool et al.,1995). The N-terminal loop segment was also previously used to reroute a constitutive protein chloramphenicol acetyltransferase (Tam and Peng, 1993). S K. Maji and his coworker had previously shown that disulphide bond maintains native reversible functional amyloid structure relevant to hormone storage inside secretory granule whereas disulphide bond disruption led to rapid irreversible amyloid aggregation using cyclic somatostatin as model peptide. (Anoop et al., 2014). We will be happy to add these references (Anoop et al., 2014, is already discussed in the text).. Authors must check grammar and may reconstruct a few sentences where sentence construction seems complicated. We will go through the text to improve readability.

Reviewer #1 (Significance (Required)):

This manuscript has a significant contribution to enrich academia with fundamental research knowledge of hormone sorting mechanisms. Although constitutive and regulated secretory pathways are known for long times, the exact sorting mechanism is not yet elucidated. There is no common receptor identified yet for recruiting regulated secretary proteins inside the secretory granules.

Aggregation in the TGN is a well-accepted mechanism for sorting. However, the triggering factor for aggregation is not yet known. This study has shed light on a novel hypothesis, which has considered intramolecular disulfide bond mediated small CC loop in hormone may act as aggregation mediator. Since many regulated secretory proteins contain the short disulphide loop, the hypothesis proposed in the manuscript is interesting.

It has been confirmed that TGN is the last compartment which is common to both regulated and constitutive pathways (Kelly, 1985). There is no sorting mechanism required for the constitutive one as this is the default mechanism, whereas a regulated secretory pathway requires a specific sorting mechanism to be efficiently packaged in the secretory granules. There are two popular hypotheses about protein sorting in regulated secretory pathways. They are "sorting for entry" and "sorting for retention" (Blázquez and Kathleen, 2000). In "sorting for entry" hormones destined to go to the regulated secretory pathway start to form aggregates in the TGN specific environment excluding other proteins destined to go to the constitutive pathway. Arvan and Castle proposed the second mechanism as some hormones, like proinsulin, are initially packaged with lysosomal enzymes in immature secretory granules (ISG) (Arvan and Castle, 1998). But with time they start to aggregate and lysosomal enzymes are removed from ISG by small constitutive-like vesicles. Although, in both the mechanisms aggregation is an essential sorting criterion the molecular events that lead to aggregation is not yet elucidated. TGN specific environmental conditions including pH (around 6.5), divalent metal ions (Zn2+, Cu2+), Glycosaminoglycans (GAGs) have potential to trigger aggregation (Dannies, Priscilla S, 2012). Though each hormone has aggregation prone regions in the amino acid sequence, there is no common amino acid sequence responsible for aggregation. The authors in this manuscript, have pointed out an interesting observation that many hormones contain small disulfide loops which are exposed due to their presence in N or C terminal or close to the processing site. Based on their observation, they hypothesized CC loop may act as aggregation driver for hormone sorting. In-cell study with CC construct from different hormones successfully rerouted a constitutively secretory protein into the regulated pathway which supported their novel hypothesis.

However, the hypothesis raises some questions to be answered regarding the molecular mechanism of CC loop mediated aggregation. Why does CC-loop promote aggregation? Does the amino acid sequence, size of the loop play a role in aggregation? The granular structure shown in the manuscript from different CC loops has different size and shape (Figure 2 and 3). What is the reason for the structural heterogeneity of the CC loop mediated dense core? Since authors have shown CC loop mediated aggregation both in functional as well as in diseased aggregation, a very important aspect to address would be the structure-function relationship of the aggregates. Since authors have rightly pointed out that not all hormones or prohormones contain CC loop, another curious question would be about the sorting mechanism of those without CC loop. The best part of the study is that it has tried to explain the well-established aggregation mediated sorting mechanism from a new perspective, which raises room for many questions to be addressed by further research. These are very valid questions, but beyond the scope of this study in which we address the contribution of CC loops in a cellular context. This is a novel extension to published in vitro studies, where a few CC loop proteins (vasopressin, oxytocin, somatostatin-14) have already been shown to enable amyloid(-like) aggregation in vitro.

From this study, the audience will get to know about the role of small disulphide loop in functional and diseased associated protein/peptide aggregation. The audience will also get an idea about the sorting mechanism in the regulated secretory pathway from the study. According to my expertise and knowledge where I do protein aggregation related to human diseases and hormone storage, I see this manuscript is a fantastic addition to understand the secretory granules biogenesis of hormones with storage and subsequent release.

Reference: Maji, Samir K., et al. "Functional amyloids as natural storage of peptide hormones in pituitary secretory granules." Science 325.5938 (2009): 328-332. Beuret, Nicole, et al. "Amyloid-like aggregation of provasopressin in diabetes insipidus and secretory granule sorting." BMC biology 15.1 (2017): 1-14. Cool, David R., et al. "Identification of the sorting signal motif within pro-opiomelanocortin for the regulated secretory pathway." Journal of Biological Chemistry 270.15 (1995): 8723-8729. Tam, W. W., K. I. Andreasson, and Y. Peng Loh. "The amino-terminal sequence of pro-opiomelanocortin directs intracellular targeting to the regulated secretory pathway." European journal of cell biology 62.2 (1993): 294-306.

Anoop, Arunagiri, et al. "Elucidating the Role of Disulfide Bond on Amyloid Formation and Fibril Reversibility of Somatostatin-14: RELEVANCE TO ITS STORAGE AND SECRETION." Journal of Biological Chemistry 289.24 (2014): 16884-16903. Kelly, Regis B. "Pathways of protein secretion in eukaryotes." Science 230.4721 (1985): 25-32. Blázquez, Mercedes, and Kathleen I. Shennan. "Basic mechanisms of secretion: sorting into the regulated secretory pathway." Biochemistry and Cell Biology 78.3 (2000): 181-191. Arvan, Peter, and David Castle. "Sorting and storage during secretory granule biogenesis: looking backward and looking forward." Biochemical Journal 332.3 (1998): 593-610. Dannies, Priscilla S. "Prolactin and growth hormone aggregates in secretory granules: the need to understand the structure of the aggregate." Endocrine reviews 33.2 (2012): 254-270.

---

Reviewer #2 (Evidence, reproducibility and clarity (Required)):

**Summary:**

This manuscript by Reck and colleagues aim at determining the importance of short disulfide loops for the correct sorting to, and release from, secretory granules. They utilize hybrid secretory proteins where sequences encoding disulfide loop from different hormones are cloned in frame with the same secretory peptide, and assess how the presence of the disulfide loop affect the ability of the protein to aggregate in the ER and to get sorted for secretion. By immunofluorescence analysis they show that the presence of a disulfide loop increases the ability of the peptide hormone to form aggregates in the ER, and these observations are confirmed by immunogold-EM. Importantly, aggregate formation is seen both in professional secretory (N-2a) and non-secretory (COS-1) cells. Using immunofluorescence and quantitative immuoblotting, they also show that the ability to aggregate the secretory proteins coincide with increased localization to secretory granules and in increased release from cells in response to stimuli.

The results from this study are interesting and suggest that small disulfide loops may be an important part of the cargo sorting mechanism in secretory cells, and perhaps also a cause of sorting defects in certain diseases. The study is overall well conducted and worthy of publication after revision.

**Major comments:**

1) It is unclear to me what the relationship between the CC-loop and amyloid is. They are not involved in the formation of fibrils and amyloid, yet the authors conclude that they support the amyloid hypothesis of granule biogenesis. This must be clarified.

Maji et al. (2009) concluded in their Science paper that secretory granules of the pituitary are made of functional amyloids formed by the protein hormones themselves. Evidence for this is that many purified protein hormones formed fibrillar aggregates in vitro with amyloid characteristics. Among the hormones analyzed were 4 CC loop-containing ones: vasopressin, oxytocin, somatostatin-14 (these are just the CC loop segments of the respective precursors), and full-length prolactin (199 aa, containing an N- and a C-terminal CC loop). Amyloid formation of somatostatin-14 was further analyzed in vitro with and without the disulfide bond by Anoop et al. (2014). On the tissue level, it was only shown that granules are stained by amyloid dyes (Maji et al., 2009). Our own lab found that folding-deficient mutant forms of provasopressin formed fibrillar aggregates in vitro (Birk et al., 2009) and in the ER of expressing cells (Birk et al., 2009; Beuret et al., 2011). These ER aggregates likely represent mislocalized amyloid formation that normally happens at the TGN for granule sorting.

In the present study, we therefore tested the role of different CC loops in cells with respect to (1) inducing ER aggregation of a folding-incompetent reporter and (2) inducing granule sorting of a folded constitutive cargo protein. Unfortunately, the ER aggregates were all very compact and did not reveal fibrillarity. However, secretory granules, which contain functional amyloids, similarly do not have a fibrillar appearance.

In this study, we do not directly provide evidence for the amyloid (or rather amyloid-like) character of aggregation. The concept of granules consisting of functional amyloids of peptide hormones was the starting point for our analysis. Our results are in line with the functional amyloid hypothesis and thus provide first functional support for it.

2) What is the actual function of the CC-loops? The authors show that the loops promote aggregation of cargo proteins, yet the mechanism behind this is unclear. For example, would the proteins used in this study be able to aggregate in vitro (i.e. the CC-loop enable aggregation) or do they require some co-factor/chaperone? It would also be good if the authors could clarify or explain why some CC-loops cause aggregation and others not.

Maji et al. (2009) showed for 3 different CC loops (vasopressin, oxytocin and somatostatin-14) that they aggregate in an amyloid-like form in vitro in purified form in the absence of chaperones or other protein cofactors. Anoop et al. (2014) analyzed in vitro amyloid formation of somatostatin-14 with and without disulfide bond in more detail. The proposed function is aggregation of the hormone into secretory granules as functional amyloids, which is supported by the finding that secretory granules are positive for amyloids.

In the present study, we tested a variety of CC loops for aggregation in cells rather than in vitro. Many proteins and peptides have been shown to be able to form amyloids in vitro. The hallmark of pathological or functional amyloids is that they are still able to do it in living cells despite the presence of chaperones, whose function is to generally prevent aggregation.

We found all CC loops to have the ability to mediate ER aggregation and granule sorting, although to different extents. The differences are likely due to their intrinsic potency and/or the way they are presented by the reporter proteins, since we used the same rather short linkers.

We plan to go through the manuscript text to make our points clearer.

3) The MS data in table 2 is very confusing, since half of the data points are missing. It is also not clear what the numbers in the table represent and if they are from a single experiment or multiple. As it is presented now, and as I interpret it, these results do not give support to the conclusion that CC loops form disulfide bonds. Since this is an important conclusion from the paper, these experiments need to be clarified, repeated or a different experimental approach used.

Thanks to this comment, we realize that Table II may have presented the result in a confusing way, making the impression that a lot of data are missing, while in fact the data was measured to be 0. To improve it, we will write 0 instead of – to indicate that no signal could be detected for a particular peptide. In addition, we will move the missing results for CCpN-NP∆ into the figure legend to avoid confusion. In the legend, we will also note that the intensities detected by mass spectrometry differ strongly for different peptides. One experiment is shown, because the numbers for peak areas inherently differ between experiments. We will revise the text to make the experiment clearer.

Proposed new Table II:

Table II. Cysteines of CC loops are oxidized in secreted reporter fusion proteins.

__nonreduced

• IAA__

__reduced

• IAA__

Diagnostic peptide*

CCv disulf

1637

10

CYFQNCPR↓

CCv 2xmod

0

696

CCa disulf

4

0

↓CNTATCATQTGEDPQGDAAQK↓

CCa 2xmod

0

23

CCc disulf

6

0

↓CGNLSTCMLGTTGEDPQGDAAQK↓

CCc 2xmod

0

32

CCr disulf

570

152

↓CSRLYTACVYHK↓

CCr 2xmod

0

246

CC loop fusion proteins with A1Pimyc were immunoprecipitated from the media of producing AtT20 cell lines, reduced with TCEP or not, before treatment with iodoacetic acid (+IAA). Samples analyzed by mass spectrometry for the expected peptide masses and the peak areas, normalized to the intensity of the peptide LQHLENELTHDIITK within A1Pi in arbitrary units are shown. It should be noted that intensities detected by mass spectrometry differ strongly by peptide. *CC loop sequences are shown in green with red cysteines, the N-terminal sequence of A1Pi in blue, linker sequence in black. CCv-, CCa-, and CCc-NP∆ containing samples were digested with trypsin, CCr- and CCpN-NP∆ containing samples with Lys-C. The peptides for CCpN-NP∆ (↓LPICPGGAARCQVTTGEDPQGDAAQK↓, disulfide bonded or carbamidomethylated) could not be detected.

4) As the authors state, it is well-known that the concentration of proteins in the ER will influence the ability to aggregate. In figure 1 and 2, the authors use transient overexpression to assess the ability of different CC-loops to induce aggregation in the ER. How were these results normalized to expression levels of the proteins? In later experiments the authors instead use stable cell lines expressing similar amounts of the different proteins. However, in these cells there is no obvious aggregation in the ER (see figure 4). It therefore becomes unclear what the role of ER aggregation for sorting to granules is.

The ER aggregation experiments were not normalized for expression levels. Plasmids were identical except for the short CC loop segments and produced similar transfection efficiencies. Stable cell lines with useful expression levels of CC-NP∆ could not be obtained, most likely because expression of mutant proteins inhibits growth.

To analyze granule sorting, we expressed CC fusion proteins with rapidly folding A1Pi as a reporter that does not accumulate in the ER. Stable cell lines were important to select clones with moderate and very similar expression levels.

5) What is the basal secretion of the different proteins, i.e. how much goes through the constitutive secretory pathway and how much goes through the regulated secretory pathway? The authors should show the resting secretion (before BaCl2 addition) for all conditions tested instead of just the change in relation to control (i.e. the way data is presented now it is not possible to tell whether BaCl2 stimulation actually cause an increased release of the peptides).

The experiment is done by comparing resting secretion (– lanes) with BaCl2 stimulated secretion (+ lanes) in Fig. 5A and C. Stimulated secretion is calculated as a ratio of resting secretion / stimulated secretion (after normalization for cell number and supernatant loading).

6) Lastly, the importance of CC-loops for the sorting of native peptides is unclear. The authors should test the importance of these loops for aggregation, sorting and secretion of a non-hybrid hormone with naturally occurring CC-loops (and a mutated version lacking the loop). This is important, since it is so far only shown that loops can affect the secretion of non-biologically relevant hybrid hormones.

In our previous study Beuret et al. (2017), we analyzed the segments contributing to ER aggregation of folding-incompetent mutant provasopressins and to granule sorting for folding-competent mutants of provasopressins by self-aggregtion at the TGN. We found separate protein segments – vasopressin (=CCv) and the glycopeptide – to contribute to aggregation in both localizations. Our study is a follow up on the finding for vasopressin, expanding to other CC loops found in peptide hormones. Our results show that CC loops in general have the ability to aggregate and contribute to granule sorting.

As exemplified by provasopressin, the CC loop may not be the only contributor. Preliminary experiments suggest the same for growth hormone. The detailed analysis of the aggregating sequences in one or more prohormone is clearly beyond the scope of our study.

**Minor comments:**

1) Stated that the 2x CC-loop constructs showed a positive effect in the cases of CCv and CCr, but this is not evaluated statistically.

We will add the statistics to the respective figures.

2) Explain the abbreviation POMC

We will add the full name to the text.

3) Figure 6D. Paired Student's t-test is not appropriate for determining significance when data is not paired (unpaired t-tests used throughout the rest of the paper).

Only in the lubrol insolubility experiment did we find considerable shifts between experiments (particularly obvious for the yellow experiment). Instead of normalizing to the control construct, we used the paired t-test. However, using the unparied t-test does not produce fundamentally different significance. If required, we will change the figure as suggested.

Figure 6D using unpaired t-test: [Figure]

Reviewer #2 (Significance (Required)):

The work in this paper builds on previous work from the same group and reinforces the notion that peptide aggregation is an important part of the sorting process that controls efficient delivery of certain proteins to nascent secretory granules, and suggest that short loops formed by disulfide bridges between closely apposed cysteine residues may be part of this sorting mechanism. The paper is of general cell biological interest, but perhaps of special interest to researches working on professional secretory cells and mechanisms of secretory protein sorting and secretion. My own research focuses on stimulus-secretion coupling pathways in secretory cells and we primarily use live cell imaging approaches to visualize different steps of secretory granule biogenesis and release.

---

Reviewer #3 (Evidence, reproducibility and clarity (Required)):

**Summary:**

Since the small disulfide loop of the nonapeptide vasopressin has been previously demonstrated to play a role the self-aggregation and secretory granule targeting of vasopressin precursor (Beuret et al., 2017), and as several other peptide hormones contain small disulfide loops, Reck and colleagues investigate in this study the requirement of small disulfide loops coming from four additional peptide hormones for the self-aggregation and secretory granule targeting of their precursors. Then, they studied the aggregation role of small disulfide loops in the ER and the TGN of two cell lines, COS1 and Neuro-2a. Using confocal and TEM, an aggregation has indeed been observed, although to different extents depending on the cell line. When fused to a constitutively secreted reporter protein, these disulfide loops induced their sorting into secretory granules, increased the stimulated secretion and Lubrol insolubility in endocrine AtT20 cells. All these results led the authors to hypothesize that small disulfide loops may act as a general device for peptide hormone aggregation and sorting, and therefore for secretory granule biogenesis.

**Major comments:**

The authors demonstrated the ability of small disulfide loops of peptide hormones to induce peptide precursor aggregation in ER using confocal microscopy, in COS1 and Neuro-2a cell lines, with a higher extent in COS1 cells. The authors have to moderate this conclusion and to include in their interpretation that distinct results may be due to the distinct secretory phenotype of these two cell lines: COS1 are epithelial cells, i.e. with a unique constitutive secretory pathway, while Neuro-2a as well as AtT20 cells also possess a regulated secretory pathway. Thus, the differences could be explained by the distinct molecular mechanisms involved in the formation of constitutive vesicles or secretory granules, and therefore aggregation and/or sorting processes could be distinct in the two cell types. We can also suggest to remove COS1-related results, to avoid hasty conclusions. As suggested, we will amend the text to point out that the two cell lines differ with respect to regulated secretion and to explain why they were used. COS-1 and Neuro-2a cells were previously used by Birk et al. (2009) to study ER aggregation of disease mutants of provasopressin. COS-1 cells were used because they are large with an extensive ER suitable for immunofluorescence microscopy. Neuro-2a cells are of neuroendocrine origin and thus more comparable to the cell types where ER aggregation of disease mutants of provasopressin or growth hormone was observed. However, the presence or absence of a regulated pathway has no relevance for ER aggregation experiments, since the different pathways diverge only at the TGN.

The data and the methods can be reproduced and the experiments are adequately replicated, using timely statistical analysis.

**Minor comments:**

• Figure 3: to complete TEM study, the concomitant use of an ER specific antibody would definitely demonstrate that small disulfide loop-containing aggregates are linked to ER compartment.

In our previous study Birk et al. (2009), we performed double-immunogold staining for provasopressin mutants and calreticulin to confirm aggregation in the ER. This anti-calreticulin antibody is unfortunately not commercially available anymore and other antibodies we tested were not suitable for immuno-EM. Instead, we colocalized PDI with CC-NP∆ constructs for immunofluorescence microscopy. Colocalization is so extensive that we believe EM confirmation to be unnecessary.

• Along abstract, introduction and discussion sections, the authors should avoid to conclude on the role of small disulfide loops on secretory granule biogenesis, but rather limit their conclusion on prohormone aggregation and targeting. Indeed, the present study did not highlight any direct molecular / physical link between disulfide loops and TGN membrane to drive secretory granule formation. Granule biogenesis involves a number of processes including interaction of cargo components with the membrane and of the actomyosin complex with the forming buds, but also selfaggregation of cargo as functional amyloids. However, we will reword our statements in the Abstract avoiding the term "**granule biogenesis".

Reviewer #3 (Significance (Required)):

• This study highlights small disulfide loops as novel signals for self-aggregating and secretory granule sorting of prohormone precursors in cells with a regulated secretory pathway. These results help to understand the molecular mechanism driving peptide hormone secretion, a physiological process which is crucial for interorgan communication and functional synchronization. Moreover, their previous study revealed that vasopressin small disulfide loop is involved in toxic unfolded mutant aggregation in the ER (Beuret et al., 2017), which highlights the clinical potential of the work.
• Audience that might be interested in and influenced by the reported findings: cell biologists interested in cell trafficking, peptide hormone secretion
• My field of expertise: secretory granule biogenesis, hormone sorting, secretory cells, neurosecretion.

2. Description of the revisions that have already been incorporated in the transferred manuscript

The manuscript has not yet been revised.

3. Description of analyses that authors prefer not to carry out

As indicated in the point-by-point response above, we consider additional analyses of in vitro aggregation with purified proteins to be beyond the scope of our study.

Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Referee #3

Evidence, reproducibility and clarity

Summary:

Since the small disulfide loop of the nonapeptide vasopressin has been previously demonstrated to play a role the self-aggregation and secretory granule targeting of vasopressin precursor (Beuret et al., 2017), and as several other peptide hormones contain small disulfide loops, Reck and colleagues investigate in this study the requirement of small disulfide loops coming from four additional peptide hormones for the self-aggregation and secretory granule targeting of their precursors. Then, they studied the aggregation role of small disulfide loops in the ER and the TGN of two cell lines, COS1 and Neuro-2a. Using confocal and TEM, an aggregation has indeed been observed, although to different extents depending on the cell line. When fused to a constitutively secreted reporter protein, these disulfide loops induced their sorting into secretory granules, increased the stimulated secretion and Lubrol insolubility in endocrine AtT20 cells. All these results led the authors to hypothesize that small disulfide loops may act as a general device for peptide hormone aggregation and sorting, and therefore for secretory granule biogenesis.

Major comments:

The authors demonstrated the ability of small disulfide loops of peptide hormones to induce peptide precursor aggregation in ER using confocal microscopy, in COS1 and Neuro-2a cell lines, with a higher extent in COS1 cells. The authors have to moderate this conclusion and to include in their interpretation that distinct results may be due to the distinct secretory phenotype of these two cell lines: COS1 are epithelial cells, i.e. with a unique constitutive secretory pathway, while Neuro-2a as well as AtT20 cells also possess a regulated secretory pathway. Thus, the differences could be explained by the distinct molecular mechanisms involved in the formation of constitutive vesicles or secretory granules, and therefore aggregation and/or sorting processes could be distinct in the two cell types. We can also suggest to remove COS1-related results, to avoid hasty conclusions.

The data and the methods can be reproduced and the experiments are adequately replicated, using timely statistical analysis.

Minor comments:

• Figure 3: to complete TEM study, the concomitant use of an ER specific antibody would definitely demonstrate that small disulfide loop-containing aggregates are linked to ER compartment.
• Along abstract, introduction and discussion sections, the authors should avoid to conclude on the role of small disulfide loops on secretory granule biogenesis, but rather limit their conclusion on prohormone aggregation and targeting. Indeed, the present study did not highlight any direct molecular / physical link between disulfide loops and TGN membrane to drive secretory granule formation.

Significance

• This study highlights small disulfide loops as novel signals for self-aggregating and secretory granule sorting of prohormone precursors in cells with a regulated secretory pathway. These results help to understand the molecular mechanism driving peptide hormone secretion, a physiological process which is crucial for interorgan communication and functional synchronization. Moreover, their previous study revealed that vasopressin small disulfide loop is involved in toxic unfolded mutant aggregation in the ER (Beuret et al., 2017), which highlights the clinical potential of the work.
  • Audience that might be interested in and influenced by the reported findings: cell biologists interested in cell trafficking, peptide hormone secretion
  • My field of expertise: secretory granule biogenesis, hormone sorting, secretory cells, neurosecretion.

Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Referee #2

Evidence, reproducibility and clarity

Summary:

This manuscript by Reck and colleagues aim at determining the importance of short disulfide loops for the correct sorting to, and release from, secretory granules. They utilize hybrid secretory proteins where sequences encoding disulfide loop from different hormones are cloned in frame with the same secretory peptide, and assess how the presence of the disulfide loop affect the ability of the protein to aggregate in the ER and to get sorted for secretion. By immunofluorescence analysis they show that the presence of a disulfide loop increases the ability of the peptide hormone to form aggregates in the ER, and these observations are confirmed by immunogold-EM. Importantly, aggregate formation is seen both in professional secretory (N-2a) and non-secretory (COS-1) cells. Using immunofluorescence and quantitative immuoblotting, they also show that the ability to aggregate the secretory proteins coincide with increased localization to secretory granules and in increased release from cells in response to stimuli.

The results from this study are interesting and suggest that small disulfide loops may be an important part of the cargo sorting mechanism in secretory cells, and perhaps also a cause of sorting defects in certain diseases. The study is overall well conducted and worthy of publication after revision.

Major comments:

1) It is unclear to me what the relationship between the CC-loop and amyloid is. They are not involved in the formation of fibrils and amyloid, yet the authors conclude that they support the amyloid hypothesis of granule biogenesis. This must be clarified.

2) What is the actual function of the CC-loops? The authors show that the loops promote aggregation of cargo proteins, yet the mechanism behind this is unclear. For example, would the proteins used in this study be able to aggregate in vitro (i.e. the CC-loop enable aggregation) or do they require some co-factor/chaperone? It would also be good if the authors could clarify or explain why some CC-loops cause aggregation and others not.

3) The MS data in table 2 is very confusing, since half of the data points are missing. It is also not clear what the numbers in the table represent and if they are from a single experiment or multiple. As it is presented now, and as I interpret it, these results do not give support to the conclusion that CC loops form disulfide bonds. Since this is an important conclusion from the paper, these experiments need to be clarified, repeated or a different experimental approach used.

4) As the authors state, it is well-known that the concentration of proteins in the ER will influence the ability to aggregate. In figure 1 and 2, the authors use transient overexpression to assess the ability of different CC-loops to induce aggregation in the ER. How were these results normalized to expression levels of the proteins? In later experiments the authors instead use stable cell lines expressing similar amounts of the different proteins. However, in these cells there is no obvious aggregation in the ER (see figure 4). It therefore becomes unclear what the role of ER aggregation for sorting to granules is.

5) What is the basal secretion of the different proteins, i.e. how much goes through the constitutive secretory pathway and how much goes through the regulated secretory pathway? The authors should show the resting secretion (before BaCl2 addition) for all conditions tested instead of just the change in relation to control (i.e. the way data is presented now it is not possible to tell whether BaCl2 stimulation actually cause an increased release of the peptides).

6) Lastly, the importance of CC-loops for the sorting of native peptides is unclear. The authors should test the importance of these loops for aggregation, sorting and secretion of a non-hybrid hormone with naturally occurring CC-loops (and a mutated version lacking the loop). This is important, since it is so far only shown that loops can affect the secretion of non-biologically relevant hybrid hormones.

Minor comments:

1) Stated that the 2x CC-loop constructs showed a positive effect in the cases of CCv and CCr, but this is not evaluated statistically.

2) Explain the abbreviation POMC

3) Figure 6D. Paired Student's t-test is not appropriate for determining significance when data is not paired (unpaired t-tests used throughout the rest of the paper).

Significance

The work in this paper builds on previous work from the same group and reinforces the notion that peptide aggregation is an important part of the sorting process that controls efficient delivery of certain proteins to nascent secretory granules, and suggest that short loops formed by disulfide bridges between closely apposed cysteine residues may be part of this sorting mechanism. The paper is of general cell biological interest, but perhaps of special interest to researches working on professional secretory cells and mechanisms of secretory protein sorting and secretion. My own research focuses on stimulus-secretion coupling pathways in secretory cells and we primarily use live cell imaging approaches to visualize different steps of secretory granule biogenesis and release.

Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Referee #1

Evidence, reproducibility and clarity

Apart from the default constitutive pathway for protein secretion some specialized cells (e.g., neuroendocrine cells, exocrine cells, peptidergic neurons and mast cells) exhibit additional regulated secretory pathway, where peptide hormones are stored as highly concentrated ordered manner inside electron opaque "dense core" of secretory granule for long duration until secretagogue mediated burst release. Although the general sorting receptor for packaging hormones in secretory granules is not yet identified, self-aggregation in the trans-Golgi network is a common shared property of peptide hormones and is a well-accepted potential sorting mechanism. Here the authors have hypothesized that cysteine containing small disulphide loop (CC loop), which is abundant in several hormone precursors, acts as aggregation mediator in TGN for sorting into secretory granule. They have tested the aggregation propensity of a misfolded reporter protein, NPΔ, in ER by attaching the CC loop segment of different hormones which promoted the pathological aggregation in endoplasmic reticulum (ER) of mutant provasopressin in the case of diabetes insipidus. Immunofluorescence and immunogold electron microscopy revealed accumulation of aggregates in the ER when CC loop of different hormonal origin fused NPΔ was transiently transfected in COS-1 fibroblasts and Neuro-2a neuroblastoma cells. The authors have also shown small disulphide loop mediated functional aggregation in TGN can sort a constitutively secreted protein, α1-protease inhibitor, into the secretory granule. The rerouting capacity of CC loop was tested in stably expressed AtT-20 cell line by confirming their localization with CgA-positive secretory granule as well as by studying BaCl2 mediated stimulated secretion and by testing secretory granule specific lubrol insolubility.

Major comments:

The study is highly impressive, and the results fully support the CC loop mediated hormone sorting hypothesis. However, it would be nice if the authors characterize the nature of the CC-loop mediated aggregates as hormones are reported to be stored inside secretory granules as functional amyloid (Maji et al., 2009). The mechanistic reason behind the small disulfide loop mediated aggregation was not explained in the paper. Authors may propose the probable molecular reasons behind CC loop mediated aggregation to completely justify their hypothesis.

Although the hypothesis and the experimental results are highly impressive, the authors may consider adding the following experiments.

The authors replaced CC-loop by the proline/glycine repeat sequence (Pro1) as a negative control which was previously reported to abolish aggregation as well. However, the authors may completely delete the small loop forming segment, CCv, and may check the status of His-tagged fused neurophysin II (NPΔ) segment as an additional negative control.

To find the ultrastructure authors have done immunogold assay with anti-His antibody which indicated different CC loop mediated ER aggregation. Since the amyloid-like fibril nature of pro-vasopressin mutant mediated ER aggregates was previously reported (Beuret et al., 2017), authors must check the nature of the CC loop mediated ER aggregates with amyloid specific antibody. Since hormones are known to form reversible functional amyloid during their storage inside secretory granule, authors may consider characterizing the nature of the aggregates formed by CC loop fused constitutive protein in AtT-20 cell line by immunostaining, immunoprecipitation and dot blot assay using amyloid specific antibody.

Minor comments:

In the quantification study (Figure 2C) CCc and CCr showed almost similar ER aggregates (around 40%). But authors have commented that all constructs except CCc produce statistically significant increases in cells compared to background. Authors must clarify the statement.

In lubrol insolubility assay, the otherwise constitutively secreted protein A1Pimyc (negative control) showed 23% insolubility. The authors explained the observation by commenting about trapping of the protein inside granule aggregate. But CCv and CCa fused proteins showed a very slight increase (around 30%). Only CCc construct showed more than 40% insolubility. If the trapping of constitutive protein may result in 23% insolubility, all the insolubility data except CCc is not satisfactory to claim as secretory granular content of aggregated protein. The authors must explain that. The authors have satisfactorily referenced prior studies in the field. However, authors may consider adding the following papers as they are directly connected with the hypothesis. The sorting of POMC hormone into secretory granules by disulphide loop was previously studied. (Cool et al.,1995). The N-terminal loop segment was also previously used to reroute a constitutive protein chloramphenicol acetyltransferase (Tam and Peng, 1993). S K. Maji and his coworker had previously shown that disulphide bond maintains native reversible functional amyloid structure relevant to hormone storage inside secretory granule whereas disulphide bond disruption led to rapid irreversible amyloid aggregation using cyclic somatostatin as model peptide. (Anoop et al., 2014).

Authors must check grammar and may reconstruct a few sentences where sentence construction seems complicated.

Significance

This manuscript has a significant contribution to enrich academia with fundamental research knowledge of hormone sorting mechanisms. Although constitutive and regulated secretory pathways are known for long times, the exact sorting mechanism is not yet elucidated. There is no common receptor identified yet for recruiting regulated secretary proteins inside the secretory granules.

Aggregation in the TGN is a well-accepted mechanism for sorting. However, the triggering factor for aggregation is not yet known. This study has shed light on a novel hypothesis, which has considered intramolecular disulfide bond mediated small CC loop in hormone may act as aggregation mediator. Since many regulated secretory proteins contain the short disulphide loop, the hypothesis proposed in the manuscript is interesting.

It has been confirmed that TGN is the last compartment which is common to both regulated and constitutive pathways (Kelly, 1985). There is no sorting mechanism required for the constitutive one as this is the default mechanism, whereas a regulated secretory pathway requires a specific sorting mechanism to be efficiently packaged in the secretory granules. There are two popular hypotheses about protein sorting in regulated secretory pathways. They are "sorting for entry" and "sorting for retention" (Blázquez and Kathleen, 2000). In "sorting for entry" hormones destined to go to the regulated secretory pathway start to form aggregates in the TGN specific environment excluding other proteins destined to go to the constitutive pathway. Arvan and Castle proposed the second mechanism as some hormones, like proinsulin, are initially packaged with lysosomal enzymes in immature secretory granules (ISG) (Arvan and Castle, 1998). But with time they start to aggregate and lysosomal enzymes are removed from ISG by small constitutive-like vesicles. Although, in both the mechanisms aggregation is an essential sorting criterion the molecular events that lead to aggregation is not yet elucidated. TGN specific environmental conditions including pH (around 6.5), divalent metal ions (Zn2+, Cu2+), Glycosaminoglycans (GAGs) have potential to trigger aggregation (Dannies, Priscilla S, 2012). Though each hormone has aggregation prone regions in the amino acid sequence, there is no common amino acid sequence responsible for aggregation. The authors in this manuscript, have pointed out an interesting observation that many hormones contain small disulfide loops which are exposed due to their presence in N or C terminal or close to the processing site. Based on their observation, they hypothesized CC loop may act as aggregation driver for hormone sorting. In-cell study with CC construct from different hormones successfully rerouted a constitutively secretory protein into the regulated pathway which supported their novel hypothesis.

However, the hypothesis raises some questions to be answered regarding the molecular mechanism of CC loop mediated aggregation. Why does CC-loop promote aggregation? Does the amino acid sequence, size of the loop play a role in aggregation? The granular structure shown in the manuscript from different CC loops has different size and shape (Figure 2 and 3). What is the reason for the structural heterogeneity of the CC loop mediated dense core? Since authors have shown CC loop mediated aggregation both in functional as well as in diseased aggregation, a very important aspect to address would be the structure-function relationship of the aggregates. Since authors have rightly pointed out that not all hormones or prohormones contain CC loop, another curious question would be about the sorting mechanism of those without CC loop. The best part of the study is that it has tried to explain the well-established aggregation mediated sorting mechanism from a new perspective, which raises room for many questions to be addressed by further research.

From this study, the audience will get to know about the role of small disulphide loop in functional and diseased associated protein/peptide aggregation. The audience will also get an idea about the sorting mechanism in the regulated secretory pathway from the study. According to my expertise and knowledge where I do protein aggregation related to human diseases and hormone storage, I see this manuscript is a fantastic addition to understand the secretory granules biogenesis of hormones with storage and subsequent release.

Reference: Maji, Samir K., et al. "Functional amyloids as natural storage of peptide hormones in pituitary secretory granules." Science 325.5938 (2009): 328-332. Beuret, Nicole, et al. "Amyloid-like aggregation of provasopressin in diabetes insipidus and secretory granule sorting." BMC biology 15.1 (2017): 1-14. Cool, David R., et al. "Identification of the sorting signal motif within pro-opiomelanocortin for the regulated secretory pathway." Journal of Biological Chemistry 270.15 (1995): 8723-8729. Tam, W. W., K. I. Andreasson, and Y. Peng Loh. "The amino-terminal sequence of pro-opiomelanocortin directs intracellular targeting to the regulated secretory pathway." European journal of cell biology 62.2 (1993): 294-306.

Anoop, Arunagiri, et al. "Elucidating the Role of Disulfide Bond on Amyloid Formation and Fibril Reversibility of Somatostatin-14: RELEVANCE TO ITS STORAGE AND SECRETION." Journal of Biological Chemistry 289.24 (2014): 16884-16903. Kelly, Regis B. "Pathways of protein secretion in eukaryotes." Science 230.4721 (1985): 25-32. Blázquez, Mercedes, and Kathleen I. Shennan. "Basic mechanisms of secretion: sorting into the regulated secretory pathway." Biochemistry and Cell Biology 78.3 (2000): 181-191. Arvan, Peter, and David Castle. "Sorting and storage during secretory granule biogenesis: looking backward and looking forward." Biochemical Journal 332.3 (1998): 593-610. Dannies, Priscilla S. "Prolactin and growth hormone aggregates in secretory granules: the need to understand the structure of the aggregate." Endocrine reviews 33.2 (2012): 254-270.

Note: This rebuttal was posted by the corresponding author to Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Reply to the reviewers

FULL REVISION

Manuscript number: RC-2021-00934

Corresponding author(s): Seiya, Mizuno

General Statements

We would like to thank all the reviewers for their comments on improving the manuscript. We are encouraged by the overall positive responses from the reviewers. According to the reviewers’ comments, we have further refined our manuscript. We are confident that we have addressed all the reviewers’ comments and suggestions by incorporating them into the revised manuscript. We highlighted the changed text in the manuscript in red. The point-by-point responses to all comments follow.

Point-by-point description of the revisions

Reviewer 1:

The study by Akihiro and colleagues describe the generation of multiplex genotyping method for detecting CRISPR gene editing alleles using nanopore sequencing and a machine learning program. The method is based on long-range PCR amplification of intended targeted loci from gene edited animals followed by nanopore sequencing. A PCR-index is introduced to the sample pooling system before sequencing, thus allow sequencing up to 100 sample in one flowcell. The study developed a machine learning program for allele binning, analysis, and presentation. To demonstrate the applicability of the method, the study has validated their methods for detection of point mutations, deletion, and flox insertion. The study has in principal provided sufficient investigation and data to demonstrate the validity of the method. All the figures are very nicely and clearly presented. However, there is a few concerns that it should be taken in to consideration.

We appreciate the constructive and important comments from the reviewer.

Reviewer 1_Comment #1:

Many previous reported unintended structure variations caused by CRISPR off-targets are typically much larger deletion/insertion/insertion/translocation occurred outside the target sites. The current study is more for targeted allele genotyping. The use of structure variable (SV) in the whole study should be considered to revise thoroughly.

SV is typically referred to genomic variation of approximately 1kb and above. What the study describe in this study is still within indel types instead. Thus, comparing the DAJIN with NanoSV and Sniffles on reads with 50, 100 and 200 bases deletions is not appropriate.

The detection of SV alleles in the whole study is most likely a cause of minor indel alleles and sequencing errors. Figure 2b, BC32, WT mice also contains a proportion of SV allele, which is apparently caused by sequencing error. Such SV which is not related to CRISPR gene editing is also seen in other genotyping results e.g. Figure 3a. Figure 4b, Figure 5c, Figure 6b.

Another co-factor that contributes to the SVs is the PCR-error from the method.

Thank you very much for your comments. We agree that structural variation traditionally referred to genomic alterations that are larger than 1 kb in length. Although the application of sequencing technology has expanded the spectrum of structural variation to include smaller events >50 bp in length (PMID: 21358748, PMID: 26432246), there are no common understanding on the definition of the name of genomic rearrangements >50 bp in length through genome editing. We changed the name of the unexpected mutation reads more than approximately 50 bp in length “Large rearrangements (LAR)”. We changed description on the name of reads that DAJIN annotates in the Methods (Page 6, Line 205) and Results section (Page8, Line 249) as well as all other parts throughout the manuscript.

Although we believe most of the LAR alleles are the real alleles generated through genomic rearrangements (Fig. 3b&3c, S12, and S16), we recognize that minor fractions of the LAR alleles, including those observed in WT mice, are composed of reads with high sequencing error rate. Visualized BAM files and consensus sequences can be indicators of the annotation results, providing information to the users of DAJIN that minor alleles that are similar in proportion to the one in the WT sample can be artificial alleles. We also cannot exclude the possibility that LAR alleles include those generated through PCR error. ‘Pseudo-LoxP’ alleles could be generated if the PCR products, which included one-side LoxP but not another-side LoxP, worked as a PCR primer to anneal WT allele in the next PCR step (Page 12, Line 425-427). Recently developed methods may address these limitations. We added description in the Discussion section (Page 17-18, Line 608-620).

Reviewer 1_Comment #2:

The reason that current method detect more than two alleles from one animal is probably due to the chimerism of the animal. However, when looking at the BAM file and figures presented in Figure 1b, 2c, 3b, 3d, 4c, as well as those in the Supplementary figures, there seems to be more than one allele (indels reads with different size) presented in one category.

For example, Figure 2C, mice BC12, it is not fully aligned between the all alleles and the allele1 and allele 2 presented. For allele 1, which is called SV, there are reads with different size of indels. For allele 2, which is called intended PM, some reads are a hybrid of deletion and intended substitution.

Thank you for checking the data in detail. As the reviewer pointed out, some of the reads in each allele showed indels with different sizes. We think these indel mutations are due to nanopore sequencing errors. Although the error rate of nanopore sequencing has improved, it has been reported that an error rate of 5% occurs in 1D sequencing of R9.4 flow cells that is the same flow cells used in our study (DOI: 10.1002/wfs2.1323). In this study, DAJIN mitigated the nanopore sequencing errors by calculating the MIDS score (Fig. S7), but the visualization using the BAM file showed the raw reads including the sequence errors. For this reason, the one allele seems to include different indel alleles.

To evaluate the point, we performed Sanger sequencing and found that there were no hybrid sequences containing indel mutations, but only intended point mutation in BC12 allele 2 (Fig. 2d). The results of Sanger sequencing suggested that the indel mutations visualized by the BAM file were due to nanopore sequencing errors. To clarify the points, we updated the description in the Discussion section (Page 15-16, Line 528-548).

Reviewer 1_Comment #3:

What is the advantage of the current method as compared to the one reported by Bi et al., 2020, genome biology, previously?

Thank you for pointing it out. We believe that one of the advantages of IDM-seq developed by Bi et al. is performing quantitative analysis by correcting PCR bias via Unique Molecular Identifiers (UMIs). However, when multiple samples are processed simultaneously, it is impractical in terms of cost and workability to prepare primers for the UMIs. While IDM-seq has the advantage to quantify the precise amount of each allele in a single sample, DAJIN is more suitable for primary and comprehensive analysis of multiple genome-edited samples. We have described these points in the Discussion section (Page 15, Line 509-513).

Reviewer 1_Comment #4:

The report machine learning method is developed for calling the different alleles. Has the authors compare DAJIN with e.g. NanoCaller, which is developed for SNPs and small indels calling based on DNN.

We are thankful to the referee for bringing the comparison with NanoCaller to our attention. We conducted NanoCaller and found it performed better to detect the point mutation than Medaka and Clair. However, because NanoCaller could not detect the LAR (formerly labelled as “SV”) alleles, it incorrectly reported the genotype of BC25 as 'point mutation', not 'LAR with point mutation'. We added the results of NanoCaller in Table S9 and described these points in the Results section (Page 10, Line338-339).

Reviewer 1_Comment #5:

Apart from genotyping, many CRISPR studies performed in cells are focusing on profiling the indel profiles in a pool of edited cells. It would broaden the applicability of the method for detecting different indels types in such samples and conditions. Current methods, such as TIDE/ICE, NGS-based amplicon sequencing, IDAA can only detect smaller indels. DAJIN will add the advantage of detecting longer indels for such application.

Thank you very much for your comments. We added description on application of DAJIN in the Discussion section (Page 17, Line 592-596).

Reviewer #1 Significance :

Although similar methods are reported for genotyping of the CRISPR editing outcome, the current study introduce the PCR barcoding and particularly the bioinformatic tool box for allele binning and calculation contribute with useful tool to the filed. The study has demonstrated with multiple applications demonstrating the broad applicability of it.

Reviewer 2:

CRISPR nucleases typically generate DNA double strand breaks (DSBs) at target site, which typically generate small insertion and deletion (indel) enabling precise gene knockout or knock-in. However, accompanied DNA DSBs often induce unwanted large deletions or chromosomal translocation. Thus, to assess such large variations as well as small indels is crucial in the genome editing field. In this manuscript, the authors developed a long-range assessment tool, named Determine Allele mutations and Judge Intended genotype by Nanopore sequencer (DAJIN), using a long-read sequencer, Nanopore sequencing. Overall, the topic will be interesting for broad readers and this tool looks technologically sound. I would suggest a few comments that may strengthen this manuscript, as follows.

We are grateful for the referee’s valuable suggestions to improve our manuscript.

Reviewer 2_Comment #1:

Another key study is missed in this manuscript. Recently, a tool with similar concept to DAJIN was published in Nat Methods, which uses also long-read sequencers, Nanopore or PacBio [PMID: 33432244]. It is necessary to describe the benefits of DAJIN against the previous study.

Thank you for pointing this out. Our method has an advantage over those utilizing unique molecular identifiers (UMIs) in its automatic identification and classification of genomic rearrangements including unexpected mutations in multiple samples obtained under different editing conditions (different target loci). As per our response to the Reviewer #1_Comment #3, one of the disadvantages of UMIs is the cost. More accessible methods of routine assessment of on-target genome editing outcomes are required, as well as unbiased assessment of editing products (PMID: 32643177). We showed in the manuscript that the machine-learning-based model could bypass molecular tagging to provide a feasible approach for routine assessment of genome editing outcomes. DAJIN will make a very significant contribution to speeding up and improving the accuracy of this experimental process.

We agree that the approach reported by Karst et al. has certainly contributed to generation of highly accurate single-molecule consensus sequences. Analysis of small portion of samples using UMI-based methods may compensate for the limitations of DAJIN such as PCR bias and/or PCR-mediated recombination as you described in your comment #6. We added description in the Discussion section (Page 15, Line 509-513; Page 17, Line 615-618).

Reviewer 2_Comment #2:

In Figure 1a, the authors used Barcoding but details information is not present in the main text. The length and context information are necessary to be described in the main text.

We thank the reviewer for these comments. According to the comments, we illustrated the process of PCR-based barcoding in Fig. 1a. Besides, we described the length of barcodes at "Library preparation and nanopore sequencing" in the Methods section (Page 4, Line 137 & 140).

Reviewer 2_Comment #3:

The term "SV (structural variation)" over "Single-nucleotide variant (SNV)" seems ambiguous. Does "SV" include large deletion and chromosomal translocation? In this manuscript, I guess that SNV indicates small indels, whereas SV indicates large indels. The detailed definition is needed for better understanding.

Thank you very much for your comments. We intended to classify and label large genomic rearrangements including large deletion and chromosomal translocation as “SV (structural variation)”. We agree that structural variation traditionally referred to genomic alterations that are larger than 1 kb in length. Although the application of sequencing technology has expanded the spectrum of structural variation to include smaller events >50 bp in length (PMID: 21358748, PMID: 26432246), there are no common understanding on the definition of the name of genomic rearrangements >50 bp in length through genome editing. We changed the name of the unexpected mutation reads more than approximately 50 bp in length “Large rearrangements (LAR)”. We changed description on the name of reads that DAJIN annotates in the Methods (Page 6, Line 205) and Results section (Page8, Line 249) as well as all other parts throughout the manuscript.

Reviewer 2_Comment #4:

In Figure 2, IGV exhibits several SNVs (i.e., random errors) in each query sequence, which might be due to the low accuracy of Nanopore sequencing. I understand that DAJIN makes consensus sequence based on those long-read sequences. But I wonder how DAJIN pinpoint the point mutation (PM) so exactly?

Thank you for pointing it out. As you mentioned, the low accuracy of Nanopore long-read sequencing made PM detection difficult. We tackled the issue and partly solved it by (i) calculation of MIDS score (Fig. S7), (ii) reducing data's dimension by principal component analysis (PCA), and (iii) setting proper parameters of HDMSCAN.

DAJIN converts ACGT nucleotide information to MIDS (Match, Insertion, Deletion, and Substitution) (Fig. S6). Subsequently, DAJIN subtracts the relative frequency of MIDS between a control and a sample. We called the subtracted relative frequency 'MIDS score' (Fig. S7). The subtraction mitigates the sequencing errors because the error patterns are similar between a sample and a control. We next perform clustering using the MIDS score. DAJIN compresses the score by PCA and extracts five dimensions. The dimension reduction may be effective to mitigate sequencing errors because the sequencing errors have lower scores than true mutations. Subsequently, DAJIN performs HDBSCAN, a density-based clustering method. The HDBSCAN have a parameter of 'min_cluster_size' that indicates a minimum number of samples in a cluster. DAJIN finds the parameter returning the most frequent cluster numbers by searching the value in the range of 10% to 40% of reads. It means DAJIN ignores minor clusters that contain less than 10% of reads. We set the criteria because sequencing errors often made such minor clusters.

In summary, we consider the MIDS score, PCA and the parameter setting of HDBSCAN support DAJIN's accurate PM detection. To clarify the point, we updated the description in the Methods section (Page 7, Line 217-225).

Reviewer 2_Comment #5:

In page 9, the authors also used next-generation sequencing (NGS). I guess this NGS indicates illumine-based short-read sequencing. Clearer definition is necessary here.

We thank the referee for bringing this unclarity to our attention. According to the reviewer's comment, we updated the words 'NGS' to the 'illumina-based short-read next-generation sequencing' or 'short-read NGS' in the whole text.

Reviewer 2_Comment #5-1:

Whereas DAJIN could reported SVs, PM, and WT, the NGS could not capture SVs. Could you write the reason here? I guess that the short-read sequences including SVs might be discarded during the alignment process, which means that it is because of software limitation, rather than the NGS itself.

Thank you for pointing this out. In this study, we performed the short-read NGS analysis by paired-end sequencing (2 x 151 bases) for PCR amplicons of about 200 bp length. We consider the main reason that NGS could not capture LAR (formerly labelled as “SV”) is due to the PCR process. The allele 2 in BC20, BC25, and BC26 of Tyr point mutation had a large deletion including primer annealing sites, which makes it impossible to obtain the PCR amplicon of this allele. Besides, allele 1 in BC25 had about 60-70 bp insertions. The insertion might make it difficult to amplify the whole length of this allele because of the limited number of cycles in short-read NGS.

To examine whether the short-read sequencing reads were discarded during the alignment process, we calculated the mapping percentages of BC20, BC25, and BC26 and found that 97-99% of reads were successfully aligned to the mm10 reference genome. We think this result can support our hypothesis. We added the results in Table S10 and described the points in the Results section (Page 10, Line 329-332).

Reviewer 2_Comment #6:

Basically, DAJIN amplify the target region using PCR, thus PCR bias (e.g. unequal amplification according to different lengths) should be considered. The authors should address it. Moreover, it is better to describe the limitation of current DAJIN in the discussion section.

Thank you very much for your comments. PCR amplification of genomic DNA is essential in our method described in the manuscript. As we have described in a paragraph in the Discussion section (Page 17, Line 597-601), we recognize there is an unavoidable limitation with PCR bias. We also cannot exclude the possibility that large rearrangements (‘LAR’, formerly labeled as ‘SV’) include alleles generated through PCR and/or sequencing error. ‘Pseudo-LoxP’ alleles could be generated if the PCR products, which included one-side LoxP but not another-side LoxP, worked as a PCR primer to anneal WT allele in the next PCR step (Page 17, Line 608-613). We recognize that minor fractions of the ‘LAR’ alleles, including those observed in WT mice, are composed of reads with high sequencing error rate. Recently developed methods including the one you kindly mentioned in the comment #1 may address these limitations. We added description in the Discussion section (Page 17-18, Line 615-618).

Reviewer #2 Significance:

Overall, the topic will be interesting for broad readers

Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Referee #2

Evidence, reproducibility and clarity

General comments

CRISPR nucleases typically generate DNA double strand breaks (DSBs) at target site, which typically generate small insertion and deletion (indel) enabling precise gene knockout or knock-in. However, accompanied DNA DSBs often induce unwanted large deletions or chromosomal translocation. Thus, to assess such large variations as well as small indels is crucial in the genome editing field. In this manuscript, the authors developed a long-range assessment tool, named Determine Allele mutations and Judge Intended genotype by Nanopore sequencer (DAJIN), using a long-read sequencer, Nanopore sequencing. Overall, the topic will be interesting for broad readers and this tool looks technologically sound. I would suggest a few comments that may strengthen this manuscript, as follows.

Specific Comments:

1. Another key study is missed in this manuscript. Recently, a tool with similar concept to DAJIN was published in Nat Methods, which uses also long-read sequencers, Nanopore or PacBio [PMID: 33432244]. It is necessary to describe the benefits of DAJIN against the previous study.
2. In Figure 1a, the authors used Barcoding but details information is not present in the main text. The length and context information are necessary to be described in the main text.
3. The term "SV (structural variation)" over "Single-nucleotide variant (SNV)" seems ambiguous. Does "SV" include large deletion and chromosomal translocation? In this manuscript, I guess that SNV indicates small indels, whereas SV indicates large indels. The detailed definition is needed for better understanding.
4. In Figure 2, IGV exhibits several SNVs (i.e., random errors) in each query sequence, which might be due to the low accuracy of Nanopore sequencing. I understand that DAJIN makes consensus sequence based on those long-read sequences. But I wonder how DAJIN pinpoint the point mutation (PM) so exactly?
5. In page 9, the authors also used next-generation sequencing (NGS). I guess this NGS indicates illumine-based short-read sequencing. Clearer definition is necessary here.
   • 5-1. Whereas DAJIN could reported SVs, PM, and WT, the NGS could not capture SVs. Could you write the reason here? I guess that the short-read sequences including SVs might be discarded during the alignment process, which means that it is because of software limitation, rather than the NGS itself.
6. Basically, DAJIN amplify the target region using PCR, thus PCR bias (e.g. unequal amplification according to different lengths) should be considered. The authors should address it. Moreover, it is better to describe the limitation of current DAJIN in the discussion section.

Significance

Overall, the topic will be interesting for broad readers

Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Referee #1

Evidence, reproducibility and clarity

The study by Akihiro and colleagues describe the generation of multiplex genotyping method for detecting CRISPR gene editing alleles using nanopore sequencing and a machine learning program. The method is based on long-range PCR amplification of intended targeted loci from gene edited animals followed by nanopore sequencing. A PCR-index is introduced to the sample pooling system before sequencing, thus allow sequencing up to 100 sample in one flowcell. The study developed a machine learning program for allele binning, analysis, and presentation. To demonstrate the applicability of the method, the study has validated their methods for detection of point mutations, deletion, and flox insertion. The study has in principal provided sufficient investigation and data to demonstrate the validity of the method. All the figures are very nicely and clearly presented. However, there is a few concerns that it should be taken in to consideration.

1. Many previous reported unintended structure variations caused by CRISPR off-targets are typically much larger deletion/insertion/insertion/translocation occurred outside the target sites. The current study is more for targeted allele genotyping. The use of structure variable (SV) in the whole study should be considered to revise thoroughly.

SV is typically referred to genomic variation of approximately 1kb and above. What the study describe in this study is still within indel types instead. Thus, comparing the DAJIN with NanoSV and Sniffles on reads with 50, 100 and 200 bases deletions is not appropriate.

The detection of SV alleles in the whole study is most likely a cause of minor indel alleles and sequencing errors. Figure 2b, BC32, WT mice also contains a proportion of SV allele, which is apparently caused by sequencing error. Such SV which is not related to CRISPR gene editing is also seen in other genotyping results e.g. Figure 3a. Figure 4b, Figure 5c, Figure 6b.

Another co-factor that contributes to the SVs is the PCR-error from the method.

1. The reason that current method detect more than two alleles from one animal is probably due to the chimerism of the animal. However, when looking at the BAM file and figures presented in Figure 1b, 2c, 3b, 3d, 4c, as well as those in the Supplementary figures, there seems to be more than one allele (indels reads with different size) presented in one category.

For example, Figure 2C, mice BC12, it is not fully aligned between the all alleles and the allele1 and allele 2 presented. For allele 1, which is called SV, there are reads with different size of indels. For allele 2, which is called intended PM, some reads are a hybrid of deletion and intended substitution.

1. What is the advantage of the current method as compared to the one reported by Bi et al., 2020, genome biology, previously?

2. The report machine learning method is developed for calling the different alleles. Has the authors compare DAJIN with e.g. NanoCaller, which is developed for SNPs and small indels calling based on DNN.

3. Apart from genotyping, many CRISPR studies performed in cells are focusing on profiling the indel profiles in a pool of edited cells. It would broaden the applicability of the method for detecting different indels types in such samples and conditions. Current methods, such as TIDE/ICE, NGS-based amplicon sequencing, IDAA can only detect smaller indels. DAJIN will add the advantage of detecting longer indels for such application.

Significance

Although similar methods are reported for genotyping of the CRISPR editing outcome, the current study introduce the PCR barcoding and particularly the bioinformatic tool box for allele binning and calculation contribute with useful tool to the filed. The study has demonstrated with multiple applications demonstrating the broad applicability of it.

Transparent Peer Review

---

Download the complete Review Process [PDF] including:

• reviews
• authors' reply
• editorial decisions

</br>

Transparent Peer Review

---

Download the complete Review Process [PDF] including:

• reviews
• authors' reply
• editorial decisions

</br>

Transparent Peer Review

---

Download the complete Review Process [PDF] including:

• reviews
• authors' reply
• editorial decisions

</br>

Note: This rebuttal was posted by the corresponding author to Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Reply to the reviewers

Manuscript number: #RC-2021-00992

Corresponding author(s): Parisa Kakanj and Maria Leptin

Reviewer #1 (Evidence, reproducibility and clarity (Required)):

In this study, the authors use the fruit fly as a model to understand the role and regulation of autophagy in epidermal integrity during development and wound healing. They discover that hyper activation of autophagy via overexpression of Atg1 leads to disruption of epithelial organization, junctional protein localization, and syncytium formation. In addition, these epidermal defects were found to be dependent on TORC1 as knockdown or inhibition of TORC1 antagonists resulted in similar epidermal defects which could be rescued by knockdown of Atg1 or Atg5. Wound healing in fruit fly epidermis is known to induce cell fusion and here the authors show that syncytium formation is dependent on autophagy. GFP-Atg8a autophagosomes were found to accumulate in cells adjacent to the wound site, but Atg1-induced syncytium formation was dispensable for wound repair. However, the authors found that hyper activation of autophagy prior to injury slowed wound closure. This may be due to defects in actomyosin organization or another developmental defect the authors observed in the epidermis. Overall, the key conclusions of this study are convincing, but the experiments would be strengthened by validation of all the RNAi strains used as well as demonstration that epidermal barrier remains intact as described.

**Major Comments**

1. This study uses a number of UAS-RNAi strains as well as dominant negative and overexpression transgenes. There is no validation that these genetic perturbations work as expected.

   Almost all of the lines we use have been extensively used and validated by others as documented in the literature. We append a table (below, page 14) with these references. It would be close to impossible for us to show their tissue specific efficacy in the larval epidermis because it is extremely difficult to obtain clean dissections of epidermis without contamination from other tissues (muscles, nerves, etc.), and we believe we can rely on the known validation of most of the lines. It is true that some of the lines are less well characterised, and we comment on those below, and will eliminate our speculation on their effects in the manuscript.

In fact, the authors state on pg 5 that RNAi to Atg6, Atg7, and Atg12 may be less effective, but do not verify the knockdown efficiency to the gene of interest (i.e. Atg5 RNAi knock downs Atg5 transcript or protein).

Atg12 and Atg7 have been shown (PMID: 25882046) by quantitative RT-PCR to effectively reduce RNA levels in the midgut during larval to pupal transition. We will therefore have to eliminate our speculation that the weak effect in the epidermis may be due to ineffective knock-down. Rather, it seems that these components are accessory but not necessarily essential for the completion of autophagy, as also observed by others (PMID: 25882046, PMID: 1805642, PMID: 23599123, PMID: 15296714, PMID: 23873149, PMID: 23406899)

This is particularly important as authors use a single UAS-rictor RNAi strain to conclude that autophagy is dependent on TORC1 and not TORC2. If rictor RNAi is also weak or ineffective than this conclusion would be erroneous.

The function of rictor has been validated by classic genetics: Animals homozygous for deletions of rictor show no defects throughout their normal life cycle (Hietakangas and Cohen, 2007). We have also shown that epidermis of homozygous rictor∆1 larvae (marked with Src-GFP, DsNuc-Red2) shows no abnormalities in cell shapes or cell membranes. We include an image here.

Figure A __| Effect of rictor deletion on the epidermis. a,b, Fluorescence micrographs of larval epidermis expressing the indicated markers in a larva homozygous for a rictor deletion (rictorEY08986 , also named rictor∆1). a, Lower magnification showing the entire width of larval segments A3 or A4. n=16-18 larvae each genotype. Scale bars: a 50 μm; b,__ 20 µm.

A major conclusion of this study is that autophagy remodels the lateral cell membranes and not the basal or apical, so the membrane integrity remains intact. This is described and shown in Fig S3a, but it is hard to see that the apical membrane is intact. It would be helpful if authors could show a true membrane marker, such as UAS-CD8mGFP to see if there is a continuous membrane.

We will include new experiments with this marker.

Alternatively, is there a barrier assay that could help demonstrate that syncytium formation does not disrupt epithelial integrity?

This follows from the fluorescence recovery we performed (Supplementary Video 13), where we observe rapid diffusion between areas in the epidermis, but never any leakage of fluorescence in the y-axis into the body cavity. We will emphasize this more clearly in the text.

This could be performed in the fly gut, using the smurf assay (Rera M et al. 2011), since the authors also describe (pg 9) a similar role for autophagy in disruption of epithelial lateral membranes.

We had done a smurf assay, and observed no leakage from the gut, but didn’t document this at the time because of difficulties during the period of Covid restrictions of accessing a dissecting scope/camera set up in a lab outside our own. We will try to repeat this now in the hope that with current reduced restrictions we can record the result.

Is autophagy dependent syncytium formation cell autonomous?

Our clonal analysis in wound healing addresses this point (Figure 2; text page 5 and 6). Clones of GFP-expressing cells neighbouring a wound share their cytoplasmic contents with their neighbours during wound closure. However, a clonal cell that is Atg5-deficient in a wild-type background does not share its content with the neighbouring cells. This shows that for a cell to participate in syncytium formation, that every cell itself has to be competent to perform autophagy. We will expand the explanation of this point in the text.

The A58-Gal is not cell-type specific as authors describe (pg 9) similar effects in trachea, salivary glands, and intestine and it is unclear if effects are due to disruption of autophagy in epidermal cells or general disruption in fly's physiology. The authors should determine, using a more restrictive Gal driver, whether syncytium formation is due to activation of autophagy in the epidermal cells or another cell type (trachea, salivary glands, or intestine).

We apologize if our phrasing of ‘ectodermal’ led to the impression that A58-Gal4 is cell-type specific. A58 also drives expression in the tracheal system, as all other available epidermal drivers do. A58 expression in the salivary gland is presumably due to the origin of the Gla4 construct, which like many other Gal4 drivers (e.g. NP1-Gal4) includes salivary gland specific enhancers (PMID: 8223268 and PMID: 12324947). A58 is not active in the gut, and for the experiments in the gut we used the NP1 driver. We will rephrase the text in the paper to avoid confusion. There is no driver that is absolutely restricted to the epidermis.

Alternatively, if no other Gal4 is available for the larval epidermis then authors could at least show using enterocytes driver (NP1-Gal4) that overexpression of Atg1 is sufficient to induce syncytium formation and its effect on gut barrier integrity.

We did do this experiment but didn’t include the images because of the large number of figures we already had. We now show them here. As mentioned above, barrier integrity is not compromised. We can also provide images of the phenotype in tracheal cells.

Figure B __| Effect__ of uncontrolled autophagy on enterocytes and salivary glands. Larval gut or salivary glands expressing the indicated markers and overexpression (Tsc1,2 or Atg1S) or RNAi (raptori) constructs using the NP1-Gal4 driver. Images are from live imaging of gut or salivary gland of 6 to 11 larvae for each genotype. Scale bars, 20 µm.

In Fig 8, authors nicely show that Atg1 RNAi can rescue Tor RNAi and raptor RNAi, but, what about the reverse. Is overexpression of Tor sufficient to inhibit the overexpression Atg1 and reduce autophagy-induced syncytium formation?

Overexpression of Tor would affect both TORC1 and TORC2. We have done this experiment using UAS-Torwt construct but found that it leads to excessive autophagy rather than suppression, consistent with similar results reported by others (PMID: 12324961 and PMID: 15186745). This approach can therefore not be used to do the proposed experiment. Instead, one could use downregulation of the Tor inhibitor TSC1, which acts on TORC1, and we have shown to reduce autophagosome formation in wound healing (Fig. 1d). Another option is to overexpress the TORC1-specific activator Rheb (PMID: 12893813, PMID: 17208179 and PMID: 31422886). We will set up the experiments with these constructs in the hope that they will yield interpretable results.

**Minor comments:**

1. Check spelling of abbreviations, Sqh is often misspelled Shq in figures

   We will correct them. Thanks for alerting us.

The order of images in Figure 3 should match the description in the text (pg. 6).

We would prefer to retain the current order because it is then consistent with all the other figures. Re-writing the text to reflect this order would make it less clear.

AtgW is described in text, but not shown in Fig 3a-c. Also, upstream activators of TORC1 are described first, but shown last in this Figure making it difficult to follow.

We will now only mention Atg1W later in the text where we also show it in a figure.

Fig7a should show junctional effect of Atg1W alone and in combination with Atg5i which is used in 7b.

We had left this out to save space, but we will now include these data.

It is unclear why authors switched to this weak overexpression for this photobleaching assay when Atg1S was predominantly used in the rest of the study.

The reason we used Atg1W in this particular experiment is that we had it on a chromosome where it was recombined with GFP which made it genetically much easier to use for FLIP experiments. However, perhaps these constructs merit some discussion. Atg1W and Atg1S were originally called “weak” and “strong” based on studies in other tissues and other stages (PMID: 33253201). However, we found that in the epidermis their effects are practically indistinguishable, as judged by TEM (Fig.3d,e) (Fig 5e,f) (Suppl. Fig. 5a,b and Suppl. Fig. 6b,c), and all markers we used in confocal analyses (which we will include them). Thus, to avoid confusion, we will change the nomenclature we use on our paper to the neutral Atg1GS and Atg16B.

Reviewer #1 (Significance (Required)):

This study elucidates the role and regulation of TORC1 and autophagy in epithelial membrane remodeling. This is important work that is significant to both developmental and wound healing research. Many cell types become multinucleate during differentiation, aging, and wound healing and here the authors find a novel role for authophagy in remodeling lateral cellular junctions to facilitate syncytium formation.

Reviewer #2 (Evidence, reproducibility and clarity (Required)):

In their present manuscript Kakanj and colleagues show that during epithelial wound healing autophagy pathway controls plasma membrane integrity and homeostasis. Furthermore, elevated autophagic activity is sufficient to induce syncytium formation, which is essential for wound closure and healing. Authors used the epidermis of fruit fly larvae as model to study wound healing and video microscopy to examine this process. The methodology is well established, since authors already published a related study in 2016 using similar tools.

The findings presented here are interesting and promising, the quality of most experiments are satisfactory, the confocal images/videos are excellent and I truly appreciate that authors used electron microscopy to support some of their claims. Findings are well presented and the text is well written and easy to read.

Overall, my opinion is very positive about this manuscript.

I believe most of the findings are very well supported, but I have some suggestions, which may can help strengthen the authors' points.

1) Authors used GFP-Atg8a reporter to follow autophagy during wound healing. While I also believe that, the appearing GFP-Atg8a dots represent autophagic vesicles after wounding but GFP-Atg8a has some certain limitations. First: Atg8a (or LC3 in mammals) is removed from the outer surface of autophagosomes by Atg4 and the Atg8a trapped inside the autophagosomes will be degraded in the autolysosomal lumen. Thus Atg8a not always localizes to autolysosomes, actually Atg8a immunostaining mostly labels autophagosomes (and phagophores) but not autolysosomes in insect cells. Accordingly, GFP-Atg8a reporter is also subject of autolysosomal degradation and furthermore most of the GFP signal is quenched in the acidic lumen of autolysosomes, since at lower pH GFP loses fluorescence. Nevertheless, if lysosomal degradation proceeds normally, GFP-Atg8 will be degraded completely. Thus, some of the autolysosomes cannot be detected using this reporter, for this mCherry-Atg8a reporters can be used, since mCherry is more resistant than GFP and thus accumulate inside lysosomes, and retains its fluorescence in acidic environments.

This is a good suggestion and we had done these experiments. However, the red fluorophores have a serious problem in that they all tend to form small aggregates or puncta – not in all tissues and at all stages, but this is a very wide-spread phenomenon, and is even observed in in vitro experiments (own observations). This makes quantification of vesicles or other small structures such as autophagosomes complete impossible. Nevertheless, here are a few figures from our analyses. While some of the spots clearly appear to be autophagosomes, as judged by their positions, they cannot be objectively distinguished from the other spots.

Figure C __| Autophagy during epidermal wound healing. Time-lapse series of single-cell wound healing in larva expressing mCherry-Atg8a (black) to mark autophagosomes and autolysosomes (A58>mCherry-Atg8a). a, z-projections of a time-lapse series. b, Higher magnification of the areas marked by magenta boxes in (a). n=11 larvae. Each frame is a merge of 57 planes spaced 0.28 μm apart. Scale bars: a 20 μm; b,__ 10 µm.

However, I still believe that for video microscopy GFP-Atg8a was a perfect choice, I just suggest to confirm the appearance of autophagosomes after wounding by other means: for instance, immunostaining of the epidermis after wounding (120 min) against Atg8a should confirm the presence of autophagosomes. There are a few specific available antibodies working in flies which are listed in the reviews of Nagy (PMID: 25481477) or more recently in Lorincz (PMID: 28704946)

This is technically a huge challenge. We would have to induce a single cell wound, then filet and fix the epidermis, during which it rolls up and often destroys the area of interest. If it doesn’t, then the prep can be flattened out, but it still can be very difficult to find the wound in the prep.

2) One of the major claims of the authors is that elevated autophagy leads to the breakdown or removal of lateral plasma membranes to promote syncytium formation. It is clearly seen on the confocal or EM images that lateral membranes disappear after wounding. However, it is also suggested that the lateral plasma membrane material is incorporated into autophagosomes or plasma membrane is a potential membrane source of autophagosome formation. I believe this is the least supported claim of the manuscript since no direct evidence for this is presented. This is based on BodyPy staining only, that BodyPy positive vesicles accumulate inside the cells. If this is indeed the case plasma membrane components should be detected in autophagic vesicles. Thus, I recommend co-staining membrane components with autophagic markers.

This is indeed the clear next step, and we did a number of experiments along those lines, but they were once again compromised by the problem with the mCherry aggregates. This made the interpretation in the unwounded epidermis with artificially upregulated autophagy impossible. However, experiments with naturally upregulated autophagy in wound healing yielded results that are consistent with plasma membrane components being associated with autophagosomes (with the caveat that not every red dot we see represents an autophagosome). We have just repeated some of these using the septate junction marker FasIII and have obtained some beautiful movies that show FasIII labelled membrane (green) being surrounded by mCherry spots, and as the membrane begins to dissociate, the mCherry spots turn from red to yellow. We have included stills from results of these analyses here and will include them in a new figure in the revised manuscript.

Figure D __| Colocalization of Atg8a and the septate junction component FasIII during epidermal wound healing. a, Time-lapse series of single-cell wound healing in a larva expressing mCherry-Atg8a (red) (A58>mCherry-Atg8a) and endogenously tagged FasIII (GFP gene trap; green), a transmembrane component of septate junctions. b, Higher magnification of the time-lapse marked by magenta boxes in (a). n=11 larvae. a,b, Each frame is a merge of 68 planes spaced 0.28 μm apart. Scale bars: a,b __20 μm.

However if authors observe no colocalization of plasma membrane components with autophagy markers I still believe this study worth to be published. I would like to recommend the review of Ungermann and Reggiori (PMID: 29966469) in which the trafficking of Atg9 is discussed,

Yes, indeed. And there is in fact now a further paper that goes in a similar direction (PMID: 34257406). We had left this out because we did not have direct data on Atg9, but will be happy to include it in the discussion in which we cite the paper that shows that Drosophila Atg9 is localized on the lateral plasma membrane in nurse cells, and loss of it leads to syncytium formation.

since the source of autophagosomal Atg9 is in part the plasma membrane in mammalian cells. Therefore, these findings may strengthen the authors' claims.

**Minor points:**

Figure 2A: I believe authors wanted to use the word 'maintaining' not mating in their scheme.

Indeed. Thanks for alerting us.

Discussion: Authors suggest that: another function of autophagy in the cells surrounding the wound may be to clear up debris as in planarian and other cell types autophagy is activated in healthy cells, which simultaneously phagocytose cell debris. Honestly, I do not believe that this is the case here. Some of the Atg proteins are indeed required for phagocytosis during LC3-assiciated phagocytosis (LAP) (see: PMID: 30787029), but LAP is independent form Atg1

Good point, we will include this in the discussion.

and if LAP happened in the cells, surrounding the wound then GFP-Atg8a positive phagosomes would appear in those cells. However, it is clearly not the case here.

Reviewer #2 (Significance (Required)):

I highly recommend this manuscript to be uploaded to a relevant journal and I believe the findings presented here will be interesting for biologists specialized in regeneration and readers from the autophagy fields alike.

Reviewer #3 (Evidence, reproducibility and clarity (Required)):

**Summary:**

The larval epidermis of Drosophila is a prime model for studying wound healing by combining live imaging with cellular, genetic and molecular analysis of the processes involved. Autophagy is known to be activated and necessary for efficient wound healing in animal models through secretion of cytokines and clearance of bacteria. This manuscript implicates autophagy in cellular syncytium formation during wound healing. Live imaging demonstrates autophagy activation in cells surrounding the wound. Inhibition of autophagy by RNAi against atg1 or atg5, required for autophagy initiation and autophagosome formation had no effect on the rate of constriction and closing of the wound site. However, elegant live imaging demonstrates that autophagy is required cell autonomously for cell fusion, a normal process during wound healing in flies. Autophagy can also be instructive for cell fusion. Strong induction of autophagy by TORC1 inhibition, TSC1/2 overexpression or Atg1 overexpression induce cell fusion that is genetically dependent on atg5, a gene acting downstream of atg1 in autophagosome formation. As Chloroquine treatment, a chemical inhibiting autophagosome fusion to the lysosome and lysosomal breakdown showed no effect, the authors suggest that later steps of autophagy are not involved. Live imaging with a selection of cellular fluorescently tagged markers of apical, lateral and basolateral membrane domains, combined with electron microscopy show clearly that lateral membrane are disrupted and removed within the epithelium. During this process, membranous large vesicles "drift" away from the plasma membrane. If these vesicles relate to autophagy is not addressed. In addition to the effect on cell fusion, strong autophagy induction also leads to autophagy within the nucleus, chromatin condensation and distortion of the nuclear membrane. The manuscript is well written and easy to follow. Figure panels and data are clearly presented. All experiments are well described throughout and skillfully executed with appropriate controls and statistical analysis. It remains unknown what induces autophagy in response to wounding. It also remains unclear whether autophagy deconstructs or engulfs parts of the plasma membrane, or if parts of the autophagy machinery has additional roles in plasma membrane fusion.

**Major comments:**

• Are the key conclusions convincing? -Conclusions are generally balanced and convincing.

-I have seldom seen a paper so well written, presented and balanced by first pass. Hence my experimental suggestions are few.

• Should the authors qualify some of their claims as preliminary or speculative, or remove them altogether? -Claims are well founded.

• Would additional experiments be essential to support the claims of the paper? Request additional experiments only where necessary to evaluate the paper as it is, and do not ask authors to open new lines of experimentation.

  -The inhibition of autophagy is performed using knockdown of two genes acting in autophagy initiation (atg1, a part of the ULK1 kinase complex) and atg5, required for autophagosome formation. Later acting genes in the autophagy process such as autophagosome closure, fusion with the lysosome or degradation were not analyzed. In the abstract, the authors state "Proper functioning of TORC1 is needed to prevent autophagy from destroying the larval epidermis which depends on membrane isolation and phagophore expansion, but not fusion of autophagosomes to lysosomes". As far as I can see, the last statement on fusion derives from experiments with Chloroquine. Although frequently used for qualitative experiments, CQ is not suited for conclusive experiments. Without genetic experiments targeting genes for autophagosome-lysosome fusion such as snap29,stx17,vamp7 this statement is in my mind not well supported.

We agree this would strengthen our findings, and we had indeed ordered these strains from the Bloomington stock collection. However, they were dead on arrival and both our labs in Heidelberg and Cologne currently have major problems with shipments from Bloomington and German customs. Other colleagues whom we asked did not have them available either. We will continue to search for appropriate constructs, but even if we find them and they arrive alive, and are processed by customs within a reasonable time, it will take many weeks to establish and then expand them and subsequently do the multi-generation crosses to obtain the stocks with all the relevant drivers and markers to set up the experiment. Three months is the absolute lower limit provided everything works according to plan, and first time round 6 months is a more realistic assumption. We hope that the referees and the editors agree that while this is a desirable experiment, it is not essential for the publication of the other results we present.

• Are the suggested experiments realistic for the authors? It would help if you could add an estimated cost and time investment for substantial experiments. -Given the expertise of the authors, these experiments should be easy to perform within 3 months.

  • Are the data and the methods presented in such a way that they can be reproduced?

  • The manuscript is well written and an excellent example of how how methods and experiments should be presented. Methods, tools and experiments are all well described.

  • Are the experiments adequately replicated and statistical analysis adequate? -Replicates and statistics are adequate and custom for the type of analysis performed.

  **Minor comments:**

• Specific experimental issues that are easily addressable. Figure 3 h. The live imaging documents the striking disappearance of lateral cell membranes using SRC-GFP. In 3h, large vesicle formation and movement towards the cell interior is shown. How frequent is this?

This can only be seen clearly in experiments with time-controlled (Gal80ts) induction of authophagy where we can observe the process unfolding. We see these structures very frequently, but great variability in morphology and the structures are not always captured clearly in the plane of imaging. We here provide further examples.

Figure E __| Autophagy in unwounded epidermis. a-c, Three additional examples showing apparent extrusions from lateral membranes after induction of autophagy (same experiment asn Figure 3h).__ Time-lapse series of epidermal cells expressing Src-GFP and Atg1S. Transgene expression is induced at the end of the second larval instar, live imaging started 6 h later (t=0) and continued for an additional 6 hours. a-c, Src-GFP containing material appears to be taken out of and eventually detached from lateral cell membranes (arrows).

Is this believed to be the mechanism of lateral membrane removal?

We would of course like to believe that, but we have no proof, and would therefore only be able to speculate.

If so, is it dependent on the autophagy machinery. Are these vesicle positive for autophagy markers?

Some autophagy markers have indeed been reported to be associated with the plasma membrane (e.g. Atg9, Atg16), but a conclusive study, while highly desirable, in our view goes beyond the scope of this study.

Resolving this issue may lift the conclusions of the paper. Using 3xCherry-Atg8 together with SRC-GFP, this should be possible.

We are intrigued by this suggestion and will be setting up the necessary crosses to do the experiments. However, it will take a long time to generate the necessary stocks (see genetics described below), and we will then again encounter the problem with the mCherry aggregates (see response to referees # 2). We are curious about the outcome, but we do not think it will be reasonable to promise as part of this revision that we will be able to provide conclusive results in the foreseeable future. Along with the many other things to do, this may just have to become part of a future paper, especially if there turn out to be other problems to be solved along the way. Like, for example, having to make an infrared (like mIFP or mKate, with which we have had much better experience in other contexts) Atg8 construct.

Using CQ, the authors should be able to detect plasma membrane and junctional components in autophagosomes or autolysosomes (by confocal and electron microscopy) as degradation is inhibited. This should help to distinguish whether lateral membranes are engulfed and digested or if cells simply fuse, by using a part of the autophagy machiney.

We have many interesting EM images on which we have had extensive discussions with the Paolo Ronchi and Yannick Schwab at the EMBL (whom we embarrassingly forgot to acknowledge in our manuscript, which will now be corrected), and one of the authors of this paper (BM) is an expert in EM imaging of the larval epidermis. It was agreed that some structures could indeed be interpreted as autophagosomes with content resembling junctional material. However, in the absence of absolute proof, we did not include them in the paper. We now show them here.

Figure F __| Autophagosomes with junctional material in unwounded epidermis.__ Transmission electron micrographs of sections through the epidermis of a larva with elevated autophagy (A58>Atg1S) at two different magnifications. Arrows mark the autophagosomal membrane with content resembling junctional material.

The authors, state that strong autophagy activation also leads to syncytium formation of tracheal cells, salivary glands and gut EC cells. Representative images in a supplementary figure would be useful for future reference.

See response to other comments above (response to referees # 1). We have added some images in this document (Figure B) and will be happy to add additional ones in the revised manuscript.

• Are prior studies referenced appropriately? -Yes. Key literature and findings are cited and discussed.

• Are the text and figures clear and accurate? -Yes

  • Do you have suggestions that would help the authors improve the presentation of their data and conclusions?

-See suggested experiments above.

Reviewer #3 (Significance (Required)):

• Describe the nature and significance of the advance (e.g. conceptual, technical, clinical) for the field. -The findings clearly documents a role of autophagy in syncytium formation in the physiological process of wounding. This has parallels to muscle syncytium formation, but has to my knowledge not been demonstrated in any other cell type to be performed by autophagy. Moreover, the authors show that strong autophagy induction can lead to fusion of epithelial cells. This may have relevance for processes and diseases where polyploidy are observed.

• Place the work in the context of the existing literature (provide references, where appropriate).

• State what audience might be interested in and influenced by the reported findings. -The data are very strong and the demonstration that autophagy controls syncytium formation outside of muscle development is surprising and significant. It is of interest to the field of cell biology and development in general and the autophagy field in particular. It will also be of interest for the medical field that deals with multinuclear phenotypes, such as cancer.

• Define your field of expertise with a few keywords to help the authors contextualize your point of view. Indicate if there are any parts of the paper that you do not have sufficient expertise to evaluate. -Development, cell signaling, autophagy, vesicle trafficking.

Table 2 | Fly stocks used in experiments

Transgenes

Stock ID

Source

Publications using this construct

Reference

UAS-GFP-Kuk

(UAS-GFP-KukEY07696(w+))

Jörg Großhans

PMID: 16421189

https://flybase.org/reports/FBal0161312

29

UAS-Atg1i

(UAS-Atg1RNAi)

V # 16133

(GD7149)

PMID: 19363474

PMID: 31995752

PMID: 32032548

PMID: 32915229

https://flybase.org/reports/FBtp0034071.html

UAS-Atg5i

(UAS-Atg5RNAi)

V # 104461

(KK108904)

PMID: 31995752

PMID: 32032548

https://flybase.org/reports/FBtp0046851.html

UAS-Atg6i

(UAS-Atg6RNAi)

V # 110197

(KK102460)

PMID: 28581519

PMID: 23599123

PMID: 27542914

PMID: 25644700

Dissertation of Philipp Trachte, Abb. 23. https://refubium.fu-berlin.de/handle/fub188/27709

Dissertation of Sirena Soriano Rodríguez. https://roderic.uv.es/bitstream/handle/10550/50749/Tesis%20SSoriano.pdf?sequence=1

UAS-Atg7i

(UAS-Atg7RNAi)

V # 45558

(GD11671)

PMID: 25882046

PMID: 31995752

PMID: 32032548

PMID: 23599123

https://flybase.org/reports/FBtp0025106.html

UAS-Atg12i

(UAS-Atg12RNAi)

V # 29791

(GD15230)

PMID: 25882046

PMID: 17568747

PMID: 31995752

https://flybase.org/reports/FBtp0027770.html

UAS-TSC1,2

(UAS-TSC1, AUS-TSC2)

Iswar K. Hariharan

PMID: 15296714

PMID: 11348592

64

UAS-TSC1i

(UAS-TSC1RNAi)

V # 22252

(GD11836)

PMID: 23144631

PMID: 29144896

PMID: 29456138

https://flybase.org/reports/FBtp0025266.html

UAS-Tori

(UAS-TorRNAi)

BL # 33951

Nobert Perrimon

PMID: 25882046

PMID: 26395483

https://flybase.org/reports/FBtp0065159.html

65

UAS-TORDN

(UAS-TORTED)

BL # 7013

Thomas P. Neufeld

PMID: 15296714

PMID: 29144896

https://flybase.org/reports/FBtp0016360.html

66

UAS-raptori

(UAS-raptorRNAi)

BL # 34814

Nobert Perrimon

PMID: 25882046

PMID: 31048465

https://flybase.org/reports/FBtp0068814.html

65

UAS-raptori-2

(UAS-raptorRNAi)

BL # 41912

Nobert Perrimon

PMID: 32097403

https://flybase.org/reports/FBtp0081336.html

65

UAS-rictori

(UAS-rictorRNAi)

BL # 36699

Nobert Perrimon

PMID: 25882046

https://flybase.org/reports/FBtp0070835.html

65

UAS-Atg1S

(UAS-Atg16B)

Thomas P. Neufeld

PMID: 33253201

https://flybase.org/reports/FBtp0041043.html

67

UAS-Atg1W, UAS-GFP

(UAS-Atg1GS10797)

Thomas P. Neufeld

PMID: 33253201

https://flybase.org/reports/FBal0216676.html

67

UAS-S6Ki

(UAS-S6KRNAi)

BL # 41895

Nobert Perrimon

PMID: 25284370

https://flybase.org/reports/FBtp0080798.html

65

UAS-SqaKA

(UAS-SqaT279A/CyO)

Guang-Chao Chen

PMID: 21169990

https://flybase.org/reports/FBtp0071419

30

UAS-RhoAi

(UAS-RhoARNAi)

V # 12734

(GD4726)

PMID: 23853710

PMID: 33789114

https://flybase.org/reports/FBtp0031970.html

UAS-Roki

(UAS-RokRNAi)

V # 104675

(KK107802)

PMID: 24995985

PMID: 33789114

https://flybase.org/reports/FBtp0046110.html

UAS-RhebAV4

BL # 9690

Fuyuhiko Tamanoi

PMID: 31909714

PMID: 28829944

https://flybase.org/reports/FBal0141561.html

69

Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Referee #3

Evidence, reproducibility and clarity

Summary:

The larval epidermis of Drosophila is a prime model for studying wound healing by combining live imaging with cellular, genetic and molecular analysis of the processes involved. Autophagy is known to be activated and necessary for efficient wound healing in animal models through secretion of cytokines and clearance of bacteria. This manuscript implicates autophagy in cellular syncytium formation during wound healing. Live imaging demonstrates autophagy activation in cells surrounding the wound. Inhibition of autophagy by RNAi against atg1 or atg5, required for autophagy initiation and autophagosome formation had no effect on the rate of constriction and closing of the wound site. However, elegant live imaging demonstrates that autophagy is required cell autonomously for cell fusion, a normal process during wound healing in flies. Autophagy can also be instructive for cell fusion. Strong induction of autophagy by TORC1 inhibition, TSC1/2 overexpression or Atg1 overexpression induce cell fusion that is genetically dependent on atg5, a gene acting downstream of atg1 in autophagosome formation. As Chloroquine treatment, a chemical inhibiting autophagosome fusion to the lysosome and lysosomal breakdown showed no effect, the authors suggest that later steps of autophagy are not involved. Live imaging with a selection of cellular fluorescently tagged markers of apical, lateral and basolateral membrane domains, combined with electron microscopy show clearly that lateral membrane are disrupted and removed within the epithelium. During this process, membranous large vesicles "drift" away from the plasma membrane. If these vesicles relate to autophagy is not addressed. In addition to the effect on cell fusion, strong autophagy induction also leads to autophagy within the nucleus, chromatin condensation and distortion of the nuclear membrane. The manuscript is well written and easy to follow. Figure panels and data are clearly presented. All experiments are well described throughout and skillfully executed with appropriate controls and statistical analysis. It remains unknown what induces autophagy in response to wounding. It also remains unclear whether autophagy deconstructs or engulfs parts of the plasma membrane, or if parts of the autophagy machinery has additional roles in plasma membrane fusion.

Major comments:

• Are the key conclusions convincing? -Conclusions are generally balanced and convincing. -I have seldom seen a paper so well written, presented and balanced by first pass. Hence my experimental suggestions are few.

• Should the authors qualify some of their claims as preliminary or speculative, or remove them altogether? -Claims are well founded,

• Would additional experiments be essential to support the claims of the paper? Request additional experiments only where necessary to evaluate the paper as it is, and do not ask authors to open new lines of experimentation.

-The inhibition of autophagy is performed using knockdown of two genes acting in autophagy initiation (atg1, a part of the ULK1 kinase complex) and atg5, required for autophagosome formation. Later acting genes in the autophagy process such as autophagosome closure, fusion with the lysosome or degradation were not analyzed. In the abstract, the authors state "Proper functioning of TORC1 is needed to prevent autophagy from destroying the larval epidermis which depends on membrane isolation and phagophore expansion, but not fusion of autophagosomes to lysosomes". As far as I can see, the last statement on fusion derives from experiments with Chloroquine. Although frequently used for qualitative experiments, CQ is not suited for conclusive experiments. Without genetic experiments targeting genes for autophagosome-lysosome fusion such as snap29,stx17,vamp7 this statement is in my mind not well supported.

• Are the suggested experiments realistic for the authors? It would help if you could add an estimated cost and time investment for substantial experiments. -Given the expertise of the authors, these experiments should be easy to perform within 3 months.

• Are the data and the methods presented in such a way that they can be reproduced?

• The manuscript is well written and an excellent example of how how methods and experiments should be presented. Methods, tools and experiments are all well described.

• Are the experiments adequately replicated and statistical analysis adequate? -Replicates and statistics are adequate and custom for the type of analysis performed.

Minor comments:

• Specific experimental issues that are easily addressable. Figure 3 h. The live imaging documents the striking disappearance of lateral cell membranes using SRC-GFP. In 3h, large vesicle formation and movement towards the cell interior is shown. How frequent is this? Is this believed to be the mechanism of lateral membrane removal? If so, is it dependent on the autophagy machinery. Are these vesicle positive for autophagy markers? Resolving this issue may lift the conclusions of the paper. Using 3xCherry-Atg8 together with SRC-GFP, this should be possible.

Using CQ, the authors should be able to detect plasma membrane and junctional components in autophagosomes or autolysosomes (by confocal and electron microscopy) as degradation is inhibited. This should help to distinguish whether lateral membranes are engulfed and digested or if cells simply fuse, by using a part of the autophagy machiney.

The authors, state that strong autophagy activation also leads to syncytium formation of tracheal cells, salivary glands and gut EC cells. Representative images in a supplementary figure would be useful for future reference.

• Are prior studies referenced appropriately?

-Yes. Key literature and findings are cited and discussed.

• Are the text and figures clear and accurate?

-Yes

• Do you have suggestions that would help the authors improve the presentation of their data and conclusions?

-See suggested experiments above.

Significance

• Describe the nature and significance of the advance (e.g. conceptual, technical, clinical) for the field.

-The findings clearly documents a role of autophagy in syncytium formation in the physiological process of wounding. This has parallels to muscle syncytium formation, but has to my knowledge not been demonstrated in any other cell type to be performed by autophagy. Moreover, the authors show that strong autophagy induction can lead to fusion of epithelial cells. This may have relevance for processes and diseases where polyploidy are observed.

• Place the work in the context of the existing literature (provide references, where appropriate).

• State what audience might be interested in and influenced by the reported findings. -The data are very strong and the demonstration that autophagy controls syncytium formation outside of muscle development is surprising and significant. It is of interest to the field of cell biology and development in general and the autophagy field in particular. It will also be of interest for the medical field that deals with multinuclear phenotypes, such as cancer.

• Define your field of expertise with a few keywords to help the authors contextualize your point of view. Indicate if there are any parts of the paper that you do not have sufficient expertise to evaluate.

-Development, cell signaling, autophagy, vesicle trafficking.

Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Referee #2

Evidence, reproducibility and clarity

In their present manuscript Kakanj and colleagues show that during epithelial wound healing autophagy pathway controls plasma membrane integrity and homeostasis. Furthermore, elevated autophagic activity is sufficient to induce syncytium formation, which is essential for wound closure and healing. Authors used the epidermis of fruit fly larvae as model to study wound healing and video microscopy to examine this process. The methodology is well established, since authors already published a related study in 2016 using similar tools.

The findings presented here are interesting and promising, the quality of most experiments are satisfactory, the confocal images/videos are excellent and I truly appreciate that authors used electron microscopy to support some of their claims. Findings are well presented and the text is well written and easy to read.

Overall, my opinion is very positive about this manuscript.

I believe most of the findings are very well supported, but I have some suggestions, which may can help strengthen the authors' points.

1) Authors used GFP-Atg8a reporter to follow autophagy during wound healing. While I also believe that, the appearing GFP-Atg8a dots represent autophagic vesicles after wounding but GFP-Atg8a has some certain limitations. First: Atg8a (or LC3 in mammals) is removed from the outer surface of autophagosomes by Atg4 and the Atg8a trapped inside the autophagosomes will be degraded in the autolysosomal lumen. Thus Atg8a not always localizes to autolysosomes, actually Atg8a immunostaining mostly labels autophagosomes (and phagophores) but not autolysosomes in insect cells. Accordingly, GFP-Atg8a reporter is also subject of autolysosomal degradation and furthermore most of the GFP signal is quenched in the acidic lumen of autolysosomes, since at lower pH GFP loses fluorescence. Nevertheless, if lysosomal degradation proceeds normally, GFP-Atg8 will be degraded completely. Thus, some of the autolysosomes cannot be detected using this reporter, for this mCherry-Atg8a reporters can be used, since mCherry is more resistant than GFP and thus accumulate inside lysosomes, and retains its fluorescence in acidic environments. However, I still believe that for video microscopy GFP-Atg8a was a perfect choice, I just suggest to confirm the appearance of autophagosomes after wounding by other means: for instance, immunostaining of the epidermis after wounding (120 min) against Atg8a should confirm the presence of autophagosomes. There are a few specific available antibodies working in flies which are listed in the reviews of Nagy (PMID: 25481477) or more recently in Lorincz (PMID: 28704946)

2) One of the major claims of the authors is that elevated autophagy leads to the breakdown or removal of lateral plasma membranes to promote syncytium formation. It is clearly seen on the confocal or EM images that lateral membranes disappear after wounding. However, it is also suggested that the lateral plasma membrane material is incorporated into autophagosomes or plasma membrane is a potential membrane source of autophagosome formation. I believe this is the least supported claim of the manuscript since no direct evidence for this is presented. This is based on BodyPy staining only, that BodyPy positive vesicles accumulate inside the cells. If this is indeed the case plasma membrane components should be detected in autophagic vesicles. Thus, I recommend co-staining membrane components with autophagic markers. However if authors observe no colocalization of plasma membrane components with autophagy markers I still believe this study worth to be published. I would like to recommend the review of Ungermann and Reggiori (PMID: 29966469) in which the trafficking of Atg9 is discussed, since the source of autophagosomal Atg9 is in part the plasma membrane in mammalian cells. Therefore, these findings may strengthen the authors' claims.

Minor points:

Figure 2A: I believe authors wanted to use the word 'maintaining' not mating in their scheme. Discussion: Authors suggest that: another function of autophagy in the cells surrounding the wound may be to clear up debris as in planarian and other cell types autophagy is activated in healthy cells, which simultaneously phagocytose cell debris. Honestly, I do not believe that this is the case here. Some of the Atg proteins are indeed required for phagocytosis during LC3-assiciated phagocytosis (LAP) (see: PMID: 30787029), but LAP is independent form Atg1 and if LAP happened in the cells, surrounding the wound then GFP-Atg8a positive phagosomes would appear in those cells. However, it is clearly not the case here.

Significance

I highly recommend this manuscript to be uploaded to a relevant journal and I believe the findings presented here will be interesting for biologists specialized in regeneration and readers from the autophagy fields alike.

Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Referee #1

Evidence, reproducibility and clarity

In this study, the authors use the fruit fly as a model to understand the role and regulation of autophagy in epidermal integrity during development and wound healing. They discover that hyper activation of autophagy via overexpression of Atg1 leads to disruption of epithelial organization, junctional protein localization, and syncytium formation. In addition, these epidermal defects were found to be dependent on TORC1 as knockdown or inhibition of TORC1 antagonists resulted in similar epidermal defects which could be rescued by knockdown of Atg1 or Atg5. Wound healing in fruit fly epidermis is known to induce cell fusion and here the authors show that syncytium formation is dependent on autophagy. GFP-Atg8a autophagosomes were found to accumulate in cells adjacent to the wound site, but Atg1-induced syncytium formation was dispensable for wound repair. However, the authors found that hyper activation of autophagy prior to injury slowed wound closure. This may be due to defects in actomyosin organization or another developmental defect the authors observed in the epidermis. Overall, the key conclusions of this study are convincing, but the experiments would be strengthened by validation of all the RNAi strains used as well as demonstration that epidermal barrier remains intact as described.

Major Comments

1. This study uses a number of UAS-RNAi strains as well as dominant negative and overexpression transgenes. There is no validation that these genetic perturbations work as expected. In fact, the authors state on pg 5 that RNAi to Atg6, Atg7, and Atg12 may be less effective, but do not verify the knockdown efficiency to the gene of interest (i.e. Atg5 RNAi knock downs Atg5 transcript or protein). This is particularly important as authors use a single UAS-rictor RNAi strain to conclude that autophagy is dependent on TORC1 and not TORC2. If rictor RNAi is also weak or ineffective than this conclusion would be erroneous.
2. A major conclusion of this study is that autophagy remodels the lateral cell membranes and not the basal or apical, so the membrane integrity remains intact. This is described and shown in Fig S3a, but it is hard to see that the apical membrane is intact. It would be helpful if authors could show a true membrane marker, such as UAS-CD8mGFP to see if there is a continuous membrane. Alternatively, is there a barrier assay that could help demonstrate that syncytium formation does not disrupt epithelial integrity? This could be performed in the fly gut, using the smurf assay (Rera M et al. 2011), since the authors also describe (pg 9) a similar role for autophagy in disruption of epithelial lateral membranes.
3. Is autophagy dependent syncytium formation cell autonomous? The A58-Gal is not cell-type specific as authors describe (pg 9) similar effects in trachea, salivary glands, and intestine and it is unclear if effects are due to disruption of autophagy in epidermal cells or general disruption in fly's physiology. The authors should determine, using a more restrictive Gal driver, whether syncytium formation is due to activation of autophagy in the epidermal cells or another cell type (trachea, salivary glands, or intestine). Alternatively, if no other Gal4 is available for the larval epidermis then authors could at least show using enterocytes driver (NP1-Gal4) that overexpression of Atg1 is sufficient to induce syncytium formation and its effect on gut barrier integrity.
4. In Fig 8, authors nicely show that Atg1 RNAi can rescue Tor RNAi and raptor RNAi, but, what about the reverse. Is overexpression of Tor sufficient to inhibit the overexpression Atg1 and reduce autophagy-induced syncytium formation?

Minor comments:

1. Check spelling of abbreviations, Sqh is often misspelled Shq in figures
2. The order of images in Figure 3 should match the description in the text (pg. 6).<br> AtgW is described in text, but not shown in Fig 3a-c. Also, upstream activators of TORC1 are described first, but shown last in this Figure making it difficult to follow.
3. Fig7a should show junctional effect of Atg1W alone and in combination with Atg5i which is used in 7b. It is unclear why authors switched to this weak overexpression for this photobleaching assay when Atg1S was predominantly used in the rest of the study.

Significance

This study elucidates the role and regulation of TORC1 and autophagy in epithelial membrane remodeling. This is important work that is significant to both developmental and wound healing research. Many cell types become multinucleate during differentiation, aging, and wound healing and here the authors find a novel role for authophagy in remodeling lateral cellular junctions to facilitate syncytium formation.

Note: This rebuttal was posted by the corresponding author to Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Reply to the reviewers

Reviewer #1 (Evidence, reproducibility and clarity):

The manuscript is interesting and well presented. The authors propose the use of an antifibrotic drug to attenuate resistance to RTK inhibitors.

\*Specific comments***

1. • It is not entirely clear how Nintedanib decreases tumour growth. It may be due to its effect on resistant melanoma cells as proposed, but it could also be due to the effect on CAFs. This should be at least discussed. *

The reviewer asks about a potential effect of Nintedanib on CAFs in our mouse model. While we show that Nintedanib has a direct action on melanoma cells in vitro, the in vivo situation can indeed be more complex. We agree that we cannot rule out the possibility that its therapeutic efficacy could be attributed in part to inhibition of CAFs, knowing that BRAF inhibitors has been shown to activate CAFs in melanoma, generating a host-tumor niche that can mediate therapeutic escape. However, addressing the contribution of CAF in vivo is challenging and would represent an entire new study. As requested by the reviewer, we have discussed this important issue and added 3 new references (see discussion section lines 377-381).

• A potential caveat is that drug used is non-specific as it also blocks PDGFR signalling. Hyperactivation of RTKs is a mechanism of BRAFi resistance and for example in Figure 1J, they see that BIF1120/Nintedanib has a significant effect on BRAFi-resistant cells, which may indicate that the growth inhibition seen in allografts could be a combination of an "anti-fibrotic" role and its own activity inhibiting the survival of resistant cells. This needs to be considered.*

We thank the reviewer for this interesting issue. Nintedanib was chosen due to its inhibitory action on extracellular matrix deposition and as an example of a rapidly available drug to be exploited therapeutically to increase the effect of targeted therapy and delay the emergence of therapy-resistant cells. We recognize that a possible disadvantage of Nintedanib could be due to its multi-targeted nature (e.g. PDGFR (α and β), FGFR-1, -2, -3, -4 and VEGFR-1, -2, -3 as well as Src, Lck or Lyn) but it is one of the only approved molecules for the treatment of fibroproliferative diseases. Upregulation of PDGFRβ/AKT signaling was previously shown to contribute to acquired resistance in M238R (Shi et al. Cancer Res. 2011;71:5067-74 ; Nazarian et al. Nature. 2010;468:973-7). Our in vitro results indicate that Nintedanib inhibits survival of these resistant cells along with a decrease in their myofibroblast-like dedifferentiated phenotype (Fig. 1 I-J).

To meet the reviewer’s comment, we have now addressed the contribution of PDGFRβ inhibition in Nintedanib’s effects on resistant cells. We have performed experiments on M238R using the selective PDGFR inhibitor CP673451 in comparison with Nintedanib (please see results section lines 120-127 and new Supplementary Fig. S1F-H). The data show that selective inhibition of the PDGFR pathway attenuates the myofibroblast-like signature typical of resistant cells to a similar degree as Nintedanib and affects melanoma cell viability (new Supplementary Fig. S1G-H). However, administration of CP673451 showed less efficiency than Nintedanib in inducing a phenotype switch toward a more differentiated phenotype (new Supplementary Fig. S1G). To further confirm the implication of RTK pathway in the phenotype observed, we analyzed the tyrosine phosphorylation status of EGFR, PDGFR and FGFR (another RTK inhibited by Nintedanib) and activation of AKT in M238R melanoma cells upon treatment with Nintedanib or CP673451 (new Supplementary Fig. S1F and additional results for the reviewers). Nintedanib had no effect on FGFR tyrosine phosphorylation and slightly decreased pEGFR levels. However, we found that the two inhibitors showed similar efficiency in decreasing phospho-PDGFRβ and phospho-AKT levels (Supplementary Fig. S1F). The results section has been modified according to these new results (lines 126-127).

Altogether these data suggest that inhibition of PDGFR signaling likely plays a prominent role in the efficacy of Nintedanib in vitro on M238R survival. Thus, as proposed by the reviewer, we can predict that the growth inhibition induced by Nintedanib seen in vivo could be a combination of its "anti-fibrotic" action and PDGFR inhibitory activity inhibiting the survival of resistant cells. It is important to note that, compared to Nintedanib, inhibition of PDGFR/AKT signaling by the CP673451 compound is not sufficient to direct melanoma cells to a more differentiated state. This is now discussed in the manuscript (Discussion section lines 404-405).

• Does the viability decrease in BRAFi-sensitive cells? For instance, in the parental cells?*

This information was already addressed in the manuscript. As shown in Supplemental Fig. S1D, Nintedanib had no effect on BRAFi-sensitive M238P viability. We have also confirmed this result using a crystal violet viability assay on M238P and UACC62 cells treated with different doses of BIBF1120.

• Figure 1 b-e, in vivo and in vivo experiments. *How many animals were used? Collagen decrease is not quantified (statistics missing).

We apologize for this omission and have now added the number of animals in the legend of Fig.1 (n = 6). We have also performed statistics for collagen quantification and included this analysis in Fig.1F (see lines 720/723). We also provide to the referee the detailed statistical analysis of mature collagen fibers between the different treatment groups.

• The title is not accurate. "prevent" resistance in melanoma is an overestimation because the cells do become resistant, albeit later.*

We agree with the reviewer and we have modified the title accordingly. The new title is now: “Blockade of pro-fibrotic response mediated by the miR-143/-145 cluster prevents targeted therapy-induced phenotypic plasticity and delays resistance in melanoma”.

Reviewer #1 (Significance):

As the authors discussed, they and others have previously studied the contribution of ECM and stromal remodelling to resistance to targeted therapies in melanoma. Previous data from E. Sahai´s lab show that BRAFi activate CAFs and increase the production and remodelling of the extracellular matrix, but in this work, they look at a cell-autonomous mechanism mediated by miRs that promotes fibrosis and propose the use of an antifibrotic drug to attenuate resistance to RTK inhibitors.

Reviewer #2 (Evidence, reproducibility and clarity (Required)): In this very interesting study, Diazzi and colleagues show that during adaptation to MAPK-targeted therapy (MAPKi), melanoma cells upregulate a miRNA profibrotic cluster (miR-143, -145), which drives a phenotypic switch towards a drug resistant undifferentiated mesenchymal-like state. From the miRNA targets, authors identify FSCN1 as a gene that needs to be downregulated during adaptation to MAPKi by the miRNAs, since FSCN1 ablation promotes the drug resistant phenotype. Importantly, authors show in a preclinical mouse melanoma model that the anti-fibrotic drug nintedanib (BIBF) improves response to MAPKi and delays onset of resistance.

The study conclusions are convincing and the data are adequately replicated and presented, authors should be commended for having the manuscript in such good shape. However, there are a few issues that authors should clarify/expand.

We sincerely thank the reviewer for his/her careful review and constructive comments.

1. The study starts with the in vivo YUMM1.7 model and combination BRAFi+MEKi, and then authors use this combination in many in vitro experiments. However, when studying resistant lines, only BRAFi-resistant and -sensitive pairs were used. I would suggest including more validation of the upregulation of the miRNA and the fibrotic genes on BRAFi+MEKi-resistant lines, and this could be easily gathered from published transcriptomes of several BRAFi+MEKi-resistant melanoma lines from Roger Lo's lab (Song et al 2017 Cancer Discov, including M238, M229, M249 used by the authors). To complement this approach, miRNA expression could be evaluated in large collections of melanoma cell lines classified as more or less undifferentiated (correlating with more or less resistance) as in Tsoi 2018 Cancer Cell and Verfaille 2015 Nat Commun.

We thank the reviewer for these interesting suggestions. We have performed several analyses, summarized below:

• First, we have analyzed the expression of the miRNA-143/-145 cluster and pro-fibrotic signature by qPCR in A375 parental and BRAFi/MEKi double resistant melanoma cell lines described in Shen et al. Nat Commun. 2019;10:5713. We observed the upregulation of both mature miRNAs along with a pro-fibrotic signature in several A375 DR clones compared to parental cells. This new result is described in the results section (lines 147-150) and shown in new Supplementary Fig. S2B. In addition, we have included in the results section the important information that the undifferentiated/mesenchymal-like BRAFi-resistant M229R and M238R cells used in our work also displayed cross-resistance to MEKi (results section, line 112 and 1 new reference).

• Second, as recommended, we have also fully (re)analyzed the mentioned studies and associated datasets. We provide a summary of the different studies including samples number, design of the study, platform used and accession.

A general observation is that unfortunately, none of these published studies provided an available small RNA-seq dataset, which thus does not allow quantifying the expression levels of mature miRNAs. However, some interesting observations have been uncovered from these datasets, confirming at least in part some of our data:

i) The dataset from Song et al. 2017 compared 18 isogenic parental versus resistant cell lines. Two subsets of resistant cells were identified, with MAPK addiction (Ra) or Resistance with MAPK redundancy (Rr). The expression of the pri-miR-143/145 precursor, named MIR143HG, was detected in these cells and was found significantly upregulated in Rr cell lines compared to parental cells. Of note, MIR143HG was also part of the Rr specific signature associated with a mesenchymal phenotype. This interesting observation is now discussed in the manuscript (Discussion section, lines 392-394).

ii) The dataset from Tsoi et al. 2018 focused on transcriptome analysis of 53 human melanoma cell lines including paired acquired resistance sublines established from patient biopsies. Unfortunately, MIR143HG expression is not detected in this dataset, probably due to a limited sequencing depth. Interestingly, we found that FSCN1 expression was decreased in most mesenchymal-like resistant cell lines compared to their parental counterpart. These data cannot be added in the manuscript since we cannot correlate the expression of the miRNAs with their target.

iii) The dataset from Verfaillie et al. 2015 revealed transcriptomic analyses on 11 short-term cultures derived from patient biopsies before therapy and gave access to RNA-seq data of tumors with a proliferative or an invasive phenotype. MIR143HG is not detected and FSCN1 expression does not appear to be associated with a specific phenotype. We have performed qPCR-based expression of miR-143-3p and miR-145-5p in some of these short-term cultures, confirming that miR-143/-145 expression is not associated with a specific phenotype in therapy naïve melanoma cells (results for referees, see below). Expression of miR-143-3p and miR-145-5p in each short-term culture was compared to the average expression of the analyzed miRNA in the proliferative short-term cultures. These results are consistent with the findings of our study describing that expression of the miR-143/145 cluster is triggered by the inhibition of the BRAF oncogenic pathway.

Related to this, the clinical relevance would increase if findings were validated using patient samples, for example, from published transcriptomes (Hugo 2015 Cell, Song 2017 Cancer Discov, Wagle 2014 Cancer Discov...) or even from TCGA, which could be used to identify if patients with high miRNA have worse prognosis.

We agree with the reviewer about the importance of providing clinical data supporting our observations. We have carefully analyzed all these profiling studies and provide below a summary.

Overall, these studies have several limitations: i) as underlined above, expression of the miRNA cluster is specifically induced in response to therapy and is not present (or barely) in tumors at diagnosis; ii) no small RNA-seq datasets are available yet; iii) melanoma tumors are highly heterogeneous and invaded with stroma, especially CAFs and vessels that also express these miRNAs. We have looked at the expression of the MIR143HG precursor in these datasets and it was not present, probably due to low to medium sequencing depths in these clinical studies.

We have also carefully explored TCGA datasets to look at possible association between prognosis and mature / precursor miRNA as well as miRNA target (FSCN1) expression in skin cutaneous melanoma (SKCM) using the tools developed by Anaya et al. 2016, PeerJ Computer Science 2:e67. Cox regression models and Kaplan-Meier analysis (using different percentiles) did not show any association of our candidates with survival on a cohort of 459 SKCM patients (median survival of 2.4 years).

Finally, during the revision process, we could have access to 9 relapsed melanoma for research purposes from the Dermatology Department of Nice University Hospital (CHU) following treatment with targeted therapies, immunotherapies or a combination of them. We have analyzed in these biopsies the expression of fibrotic/mesenchymal genes, FSCN1 and the miR-143/145 cluster compared to the mean expression of the same genes/miRNAs in therapy naïve patient-derived xenografts (MEL003, MEL006, MEL015, MEL047). Our first results indicate that relapsed tumors acquire a strong fibrotic signature which is associated to increased expression of the miR-143/-145 cluster and decreased expression of FSCN1 (8 out of 9 patients).

These results are encouraging and represent a good indicator for further clinical validation but are not solid enough to be incorporated in the manuscript. Overall, validation of our hypotheses in patient samples would require an entire new and highly complex clinical study comparing tumors at diagnosis with relapsed tumors after targeted therapies and ideally processed using single-cell RNA-seq and/or RNA FISH to take into account the stromal compartment.

• While blocking the miRNA improves BRAFi response (Fig.3H), it is not clear that this combination would overcome resistance (using resistant lines), although authors show that BIBF does overcome resistance (Fig.1J). *This also applies to line 277 "… mirroring the effect of miR143/145 ASOs, forced expression of FSCN1 in M238R cells decreased viability in the presence of BRAFi (Fig.5H)." However, the miRNA ASOs were used in parental cells (Fig.3H).

To meet the reviewer’s comment, we have conducted new experiments in resistant melanoma cells using different approaches to silence simultaneously the 2 mature miRNAs: i) an ASO-directed RNAse H degradation of the miR-143/145 precursor, as described by Plaisance et al., JACC Basic Transl Sci. 2016, 1:472-493 to knock-down the pri-miRNA in cardiomyocytes, and ii) a combination of the 2 anti-miRs ASOs. Unfortunately, the first approach failed to efficiently inhibit the expression of mature miR-143-3p and miR-145-5, suggesting that the miR-143/145 cluster has a different precursor gene in melanoma than the one described in cardiomyocytes.

Concerning the second approach, as expected, the 2 anti-miRs ASOs as well as the combination of the 2 ASOs efficiently targeted the mature miRNAs (new Supplementary Fig.S6C). Inhibition of miR-145-5p alone and combined inhibition of the two miRNAs significantly affected the viability of BRAFi resistant melanoma cells (M238R) in the absence of BRAFi (new Supplementary Fig.S6D) in a similar way as Nintedanib/BIBF (Fig. 1J).

• Analysis of cytoskeletal changes. Text (lines 284-287) is missing references, regarding "…morphological changes with cells assuming flattened spindle-like shape" and "..function of FSCN1 in F-actin microfilaments reorganization...".*

We apologize for these omissions and have added the relevant references in the text (lines 305/306).

Besides, authors say that transient overexpression of miRNAs reproduced these morphological changes as shown by F-actin staining. These would have benefited from including also side-by-side comparison of BRAFi treatment on these cell lines. To my knowledge, these melanoma lines (M238, M229, etc) have not been characterized in that regard (F-actin, focal adhesions). In Nazarian et al 2010, only brightfield pictures are shown in a supplementary figure.

The same applies to YAP and especially MRTF activation upon miRNA overexpression, and whether this mirrors what BRAFi does to YAP and MRTF. In Misek et al 2020 and Kim et al 2015 YAP and MRTF were shown to be more enriched in the nucleus in resistant than in parental cells. Kim et al also show in time course experiments that there is significantly higher nuclear YAP after 7-14 days of BRAFi treatment. In the present manuscript, authors seemed to have assessed nuclear YAP/MRTF after 72h miRNA overexpression. Does it mirror MAPKi?

As suggested by the reviewer, we have compared side-by-side the effect of oncogenic MAPK pathway inhibition to the effect of miR-143 or miR-145 overexpression on cytoskeleton and focal adhesion dynamics as well as YAP and MRTFA nuclear translocation in M238P, M229P and UACC62P melanoma cells. These analyses clearly show that transient overexpression of miR-143-3p or miR-145-5p mirrors the effects of BRAF or BRAF/MEK inhibition after 3 days on mechanopathways and acto-myosin remodeling. We thank the referee for this comment, which is helpful for the interpretation of the data. The new additional panels have been included in new Fig. 6B-D, new Fig. 7B-D, new Supplementary Fig. S10B-D and new Supplementary Fig. S11C-D.

Regarding the decreased proliferation/survival after miRNA overexpression, is it truly slow cycling and not combined with some cell death? Table S1 has a "cell death of tumor cell lines" theme after miRNA overexpression.

Following the reviewer suggestion, Annexin V/DAPI staining has been performed in M238P cells upon transient overexpression of miR-143 or miR-145. No significant cell death was observed (new Supplementary Fig. S4D). Detailed statistical analysis and quantification of the experiment is provided. Staurosporine (Stauro) treatment was used as a positive control of cell death induction.

Related to this, in Supp. Fig.4C the effect on the cell cycle effect is very small, is this significant? It is unclear when the cell cycle was assessed after miRNA overexpression (72h?), it could be a matter of timing. According to Fig.3E, there is a reduction in growth from 60-72h onwards.

We performed, as suggested by the reviewer, cell cycle analysis at longer timing after transfection (96 hours) (new Supplementary Fig. S4C). We observed a significant accumulation of melanoma cells in G0/G1 phase upon miR-143 or miR-145 overexpression and a significant decrease of the percentage of cells in S phase. Detailed statistical analysis of the described experiment is provided.

Statistics. While multiple comparison tests were used, most graphs have asterisks on top of some bars, and it is unclear what is being compared with what. For example, Fig.2B have asterisks on top of BRAFi+MEKi group, does it mean it is significant vs vehicle group? In this and other similar cases (1J, 2C, S1B and others), a comparison against the combination group (BRAFiMEKi+BIBF) is also relevant. This should be revised throughout manuscript.

As recommended by the reviewer, statistical analysis have been modified in the mentioned figures: Fig. 1J (lines 732/733), Fig. 2B (lines 745/746), Fig. 2C (lines 749/750) and Fig. S1B (see new figures and lines 251/252 of Supplementary materials).

\*Minor:** -For all the studies using stable cell lines, authors should state how long after transduction and selection experiments were performed. *

As recommended, we have now added this information (see lines 8-12 of Supplementary materials). - Authors only show single miRNA overexpression or inhibition. However, both miRNA are upregulated upon MAPKi. Did authors try the double overexpression or blockade?

As suggested by the reviewer, we experimented the double blockade in M238P and 1205Lu cells treated with MAPK inhibitors. Results are presented in new Fig. 3B, 3D, 3H and Supplementary Fig. S6A-B. Overall, combined inhibition of the two miRNAs had an effect comparable or more significant than the single miRNA inhibition depending on the cellular parameter analyzed.

Concerning the double overexpression, we already experimented lentivirus-mediated stable overexpression of the two miRNAs in two melanoma cell lines. Results are presented in Supplementary Fig. S5A-F and confirmed the functional effects observed by the single miRNA overexpression.

- For the 1205Lu xenograft experiment, authors should also show the tumour growth curves, and explain how long treatment was and when miRNA expression was analysed (endpoint?). In addition, why in 5A there are only 3 dots (mice?) per group, while in 5B there are more (6-7 in control, 4-5 in BRAFi)?

We apologize for this omission. We have added line 270 of the manuscript the reference to the previous study in which the experiment is described. miRNA expression was analyzed in tumors at the endpoint of the experiment i.e. 2 weeks after Vemurafenib treatment start. Moreover, we performed again the analysis of FSCN1 and miR-143/145 expression with the same number of mice (n = 6), please see new Fig. 5A.

- In a few graphs, the axis legend should give more information. For example, Fig.2 says Fold change, and it should be Fold change expression, or similar; Fig.4G fold change FSCN mRNA expression; Fig. S2 log2 expression (resistant/par), S5A...

We have corrected this and modified y-axis legends in the corresponding figures.

- Fig.1E-G and S1B. **Is this at endpoint for each group?

Yes, it is as stated in the materials and methods section.

- Fig.3H and S6B. how long were these experiments?

Experiments shown in Fig. 3H and Fig. S6B were carried out during 72 h. This information has been included in the legend of the corresponding figures.

- Fig.7B and D. Why the MRTFA signal in miR-neg and siCTRL is so different? Same for UACC in S11A vs s11D.

We apologize for this inaccuracy. We have revised the figures to show more representative pictures (new Figs. 7B, 7D and S11A, S11D and new Fig. 6C).

• Fig.5C and 5E. FSCN1 knockdown in 5C is very efficient, while not so much in 5E. However, effects on MITF, AXL etc in 5C are quite impressive. are these knockdowns representative?

We again apologize for this inaccuracy. We performed a new experiment and we are now showing a more representative FSCN1 knockdown in new Fig. 5E.

- Fig.6-7 legend. When mentioning scale bar, it reads uM, should it be um?

We have corrected this mistake.

• Fig.7A. In the graph, the "YAP nuclear enrichment", do the numbers represent the nuclear/cytoplasm ratio?

Yes, numbers represent the nuclear/cytoplasm ratio. This information was added in the legend of the corresponding figures.

- When showing migration and a picture (Fig.3F, 5D, S4D, S5E...), the blue over dark background is difficult to see, using greyscale or a brighter pseudocolour would help

We thank the reviewer for this useful suggestion. We have done this and used the gray scale to improve the quality of the pictures.

Reviewer #2 (Significance):

These findings have important preclinical implications, since the study proposes a biomarker of resistance (profibrotic signature) and importantly, a potential new therapy to delay MAPKi resistance in melanoma (BIBF). It could also apply to other BRAFmutant cancers and diseases cursing with fibrosis.

Field of expertise: melanoma, drug resistance, cytoskeleton

Reviewer #3:

Major comments:

The manuscript is well written, data are convincing, well presented and supportive of the conclusions.

We thank the reviewer for his/her interest about our study and supportive comments.

\*Minor points that may be improved:***

- The expression of miR-143/145 increases in melanoma cell lines treated with BRAFi and/or MEKi for 72h (Fig. 2B, Supp. Fig. 2B-F), and also after the development of resistance to MAPK-targeted therapies (Fig. 2A, Supp. Fig. 2A). The transient overexpression of miRs in therapy-naive cells leads to cells de-differentiation toward a mesenchymal/MAPK resistant state. On the other hand, these cells become more sensitive to BRAFi treatment when combined with LNA-mediated inhibition of miRs activity. It would be important to determine if the same occurs also in resistant cells, or whether MAPKi-resistance is established, cells are no longer sensitive to miRs blockade.

The answer to this point is common to the point 2 raised by the reviewer #2.

According to reviewers suggestion, we have conducted new experiments in resistant melanoma cells using different approaches to silence simultaneously the 2 mature miRNAs: i) an ASO-directed RNAse H degradation of the miR-143/145 precursor, as described by Plaisance et al., JACC Basic Transl Sci. 2016, 1:472-493 to knock-down the pri-miRNA in cardiomyocytes, and ii) a combination of the 2 anti-miRs ASOs. Unfortunately, the first approach failed to efficiently inhibit the expression of mature miR-143-3p and miR-145-5, suggesting that the miR-143/145 cluster has a different precursor gene in melanoma than the one described in cardiomyocytes.

Concerning the second approach, as expected, the 2 anti-miRs ASOs as well as the combination of the 2 ASOs efficiently targeted the mature miRNAs (Supplementary Fig.S6C). Inhibition of miR-145-5p alone and combined inhibition of the two miRNAs significantly affected the viability of BRAFi resistant melanoma cells (M238R) in the absence of BRAFi (new Supplementary Fig.S6D) in a similar way as BIBF (Fig. 1J).

- In 2 out of 4 melanoma PDX samples naïve/resistant to combo BRAFi/MEKi therapy, the expression level of miR-143/145 cluster correlates with the de-differentiated transcriptomic profile of resistant tumor. How is Fascin1 expression in these samples?

The reviewer legitimately asks about the expression level of the miR-143/-145 target FSCN1 in the PDX samples used in the study. Expression of FSCN1 in PDX resistant vs naïve samples has been assessed by RT-qPCR. Results are provided. We observed decreased expression of FSCN1 in only 1 out of the 2 samples showing increased miR-143/145 expression. This can be due to the heterogeneity of the subpopulations composing the tumor sample. It would have been interesting and probably more informative to test FSCN1 expression also at protein level since often miRNA molecular targets are inhibited at translation level but unfortunately we did not have the access to protein extracts corresponding to these samples.

- The clinical relevance of the data could be strongly improved by assessing the expression of the miRs cluster and of its target Fascin1 in resistant subsets of patients, comparing their expression to patients before treatment, making use of available datasets.

We agree with the reviewer about the importance of providing clinical data supporting our observations. We have carefully analyzed all available profiling studies and datasets and provide below a summary.

Overall, these studies have several limitations: i) as demonstrated in our study, expression of the miRNA cluster is specifically induced in response to therapy and is not present (or barely) in tumors at diagnosis; ii) no small RNA-seq datasets are available yet; iii) melanoma tumors are highly heterogeneous and invaded with stroma, especially CAFs and vessels that also express these miRNAs. We have looked at the expression of the MIR143HG precursor in these datasets and it was not present, probably due to low to medium sequencing depths in these clinical studies.

We have also carefully explored TCGA datasets to look at possible association between prognosis and mature / precursor miRNA as well as miRNA target (FSCN1) expression in skin cutaneous melanoma (SKCM) using the tools developed by Anaya et al. 2016 PeerJ Computer Science 2:e67. Cox regression models and Kaplan-Meier analysis (using different percentiles) did not show any association of our candidates with survival on a cohort of 459 SKCM patients (median survival of 2.4 years, see Kaplan plots below).

Finally, during the revision process, we could have access to 9 relapsed melanoma for research purposes from the Dermatology Department of Nice University Hospital (CHU) following treatment with targeted therapies, immunotherapies or a combination of them. We analyzed in these samples the expression of fibrotic/mesenchymal genes, FSCN1 and the miR-143/145 cluster compared to the mean expression of the same genes/miRNAs in therapy naïve patient-derived xenografts (MEL003, MEL006, MEL015, MEL047). Our results indicate that relapsed tumors acquire a strong fibrotic signature which is associated to increased expression of the miR-143/145 cluster and decreased expression of FSCN1 (8 out of 9 patients).

This represents a good indicator for further clinical validation but is not solid enough to be incorporated in the manuscript. Overall, validation of our hypotheses in patient samples would require an entire new and highly complex clinical study comparing tumors at diagnosis with relapsed tumors after targeted therapies and ideally processed using single-cell RNA-seq and/or RNA FISH to take into account the stromal compartment.

Minor comments:

- Fig. 4C, lower legend: M238P not M238S.

We apologize for this mistake and corrected it.

Reviewer #3 (Significance):

**Nature and significance of the advances:**

The findings not only suggest the combination therapy with the anti-fibrotic drug Nintedanib to be effective in enhancing MAPKi treatment in melanoma, reducing the development of resistance, but identify the molecular mechanism via the induction o the miR-143/145 cluster and the effects on the target Fascin1.

**Compare to existing knowledge**

These two miRNAs have been shown to have both oncogenic and oncosuppressor activities and have already been involved in EMT induction. The findings add yet one more piece to the puzzle.

**Audience** This manuscript is not only of interest for oncology researchers but also of general interest or the understanding of fundamental biological processes and their effects on cancer therapy.

**Your expertise**

Molecular biologist and cancer research, transcriptional control of tumor transfromatin and progression including EMT, microRNAs -143/145

Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Referee #3

Evidence, reproducibility and clarity

Summary:

In the present work Diazzi and co-authors describe the mechanism through which the anti-fibrotic drug Nintedanib potentiates MAPK-targeted therapy efficacy in melanoma cells. Nintedanib prevents the MAPK-induced pro-fibrotic response and is associated with loss of miR-143/-145 cluster expression. These miRs promote melanoma cells de-differentiation towards a pro-fibrotic mesenchymal-like state that correlates with resistance to MAPK inhibitors. Looking for miR-143/-145 targets responsible for this phenotype switch, the authors identified Fascin1 as a crucial regulator of cytoskeleton dynamics and mechanopathways.

Major comments:

The manuscript is well written, data are convincing, well presented and supportive of the conclusions.

Minor points that may be improved:

• The expression of miR-143/145 increases in melanoma cell lines treated with BRAFi and/or MEKi for 72h (Fig. 2B, Supp. Fig. 2B-F), and also after the development of resistance to MAPK-targeted therapies (Fig. 2A, Supp. Fig. 2A). The transient overexpression of miRs in therapy-naive cells leads to cells de-differentiation toward a mesenchymal/MAPK resistant state. On the other hand, these cells become more sensitive to BRAFi treatment when combined with LNA-mediated inhibition of miRs activity. It would be important to determine if the same occurs also in resistant cells, or whether MAPKi-resistance is established, cells are no longer sensitive to miRs blockade.
• In 2 out of 4 melanoma PDX samples naïve/resistant to combo BRAFi/MEKi therapy, the expression level of miR-143/145 cluster correlates with the de-differentiated transcriptomic profile of resistant tumor. How is Fascin1 expression in these samples?
• The clinical relevance of the data could be strongly improved by assessing the expression of the miRs cluster and of its target Fascin1 in resistant subsets of patients, comparing their expression to patients before treatment, making use of available datasets.

Minor comments:

• Fig. 4C, lower legend: M238P not M238S

Significance

Nature and significance of the advances:

The findings not only suggest the combination therapy with the anti-fibrotic drug Nintedanib to be effective in enhancing MAPKi treatment in melanoma, reducing the development of resistance, but identify the molecular mechanism via the induction o the miR-143/145 cluster and the effects on the target Fascin1.

Compare to existing knowledge

These two miRNAs have been shown to have both oncogenic and oncosuppressor activities and have already been involved in EMT induction. The findings add yet one more piece to the puzzle.

Audience

This manuscript is not only of interest for oncology researchers but also of general interest or the understanding of fundamental biological processes and their effects on cancer therapy.

Your expertise

Molecular biologist and cancer research, transcriptional control of tumor transfromatin and progression including EMT, microRNAs -143/145

Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Referee #2

Evidence, reproducibility and clarity

In this very interesting study, Diazzi and colleagues show that during adaptation to MAPK-targeted therapy (MAPKi), melanoma cells upregulate a miRNA profibrotic cluster (miR-143, -145), which drives a phenotypic switch towards a drug resistant undifferentiated mesenchymal-like state. From the miRNA targets, authors identify FSCN1 as a gene that needs to be downregulated during adaptation to MAPKi by the miRNAs, since FSCN1 ablation promotes the drug resistant phenotype. Importantly, authors show in a preclinical mouse melanoma model that the anti-fibrotic drug nintedanib (BIBF) improves response to MAPKi and delays onset of resistance.

The study conclusions are convincing and the data are adequately replicated and presented, authors should be commended for having the manuscript in such good shape. However, there are a few issues that authors should clarify/expand.

1. The study starts with the in vivo YUMM1.7 model and combination BRAFi+MEKi, and then authors use this combination in many in vitro experiments. However, when studying resistant lines, only BRAFi-resistant and -sensitive pairs were used. I would suggest including more validation of the upregulation of the miRNA and the fibrotic genes on BRAFi+MEKi-resistant lines, and this could be easily gathered from published transcriptomes of several BRAFi+MEKi-resistant melanoma lines from Roger Lo's lab (Song et al 2017 Cancer Discov, including M238, M229, M249 used by the authors). To complement this approach, miRNA expression could be evaluated in large collections of melanoma cell lines classified as more or less undifferentiated (correlating with more or less resistance) as in Tsoi 2018 Cancer Cell and Verfaille 2015 Nat Commun.

Related to this, the clinical relevance would increase if findings were validated using patient samples, for example, from published transcriptomes (Hugo 2015 Cell, Song 2017 Cancer Discov, Wagle 2014 Cancer Discov...) or even from TCGA, which could be used to identify if patients with high miRNA have worse prognosis.

1. While blocking the miRNA improves BRAFi response (Fig.3H), it is not clear that this combination would overcome resistance (using resistant lines), although authors show that BIBF does overcome resistance (Fig.1J). This also applies to line 277 ".. mirroring the effect of miR143/145 ASOs, forced expression of FSCN1 in M238R cells decreased viability in the presence of BRAFi (Fig.5H)." However, the miRNA ASOs were used in parental cells (Fig.3H).
2. Analysis of cytoskeletal changes. Text (lines 284-287) is missing references, regarding "..morphological changes with cells assuming flattened spindle-like shape" and "..function of FSCN1 in F-actin microfilaments reorganization..". Besides, authors say that transient overexpression of miRNAs reproduced these morphological changes as shown by F-actin staining. These would have benefited from including also side-by-side comparison of BRAFi treatment on these cell lines. To my knowledge, these melanoma lines (M238, M229, etc) have not been characterized in that regard (F-actin, focal adhesions). In Nazarian et al 2010, only brightfield pictures are shown in a supplementary figure. The same applies to YAP and especially MRTF activation upon miRNA overexpression, and whether this mirrors what BRAFi does to YAP and MRTF. In Misek et al 2020 and Kim et al 2015 YAP and MRTF were shown to be more enriched in the nucleus in resistant than in parental cells. Kim et al also show in time course experiments that there is significantly higher nuclear YAP after 7-14 days of BRAFi treatment. In the present manuscript, authors seemed to have assessed nuclear YAP/MRTF after 72h miRNA overexpression. Does it mirror MAPKi?
3. Regarding the decreased proliferation/survival after miRNA overexpression, is it truly slow cycling and not combined with some cell death? Table S1 has a "cell death of tumor cell lines" theme after miRNA overexpression.

Related to this, in Supp. Fig.4C the effect on the cell cycle effect is very small, is this significant? It is unclear when the cell cycle was assessed after miRNA overexpression (72h?), it could be a matter of timing. According to Fig.3E, there is a reduction in growth from 60-72h onwards.

1. Statistics. While multiple comparison tests were used, most graphs have asterisks on top of some bars, and it is unclear what is being compared with what. For example, Fig.2B have asterisks on top of BRAFi+MEKi group, does it mean it is significant vs vehicle group? In this and other similar cases (1J, 2C, S1B and others), a comparison against the combination group (BRAFiMEKi+BIBF) is also relevant. This should be revised throughout manuscript.

Minor:

-For all the studies using stable cell lines, authors should state how long after transduction and selection experiments were performed.

-Authors only show single miRNA overexpression or inhibition. However, both miRNA are upregulated upon MAPKi. Did authors try the double overexpression or blockade?

-For the 1205Lu xenograft experiment, authors should also show the tumour growth curves, and explain how long treatment was and when miRNA expression was analysed (endpoint?). In addition, why in 5A there are only 3 dots (mice?) per group, while in 5B there are more (6-7 in control, 4-5 in BRAFi)?

-In a few graphs, the axis legend should give more information. For example, Fig.2 says Fold change, and it should be Fold change expression, or similar; Fig.4G fold change FSCN mRNA expression; Fig. S2 log2 expression (resistant/par), S5A...

-Fig.1E-G and S1B. Is this at endpoint for each group?

-Fig.3H and S6B. how long were these experiments? Fig.7B and D. Why the MRTFA signal in miR-neg and siCTRL is so different? Same for UACC in S11A vs s11D.

-Fig.5C and 5E. FSCN1 knockdown in 5C is very efficient, while not so much in 5E. However, effects on MITF, AXL etc in 5C are quite impressive. are these knockdowns representative?

-Fig.6-7 legend. When mentioning scale bar, it reads uM, should it be um?

-Fig.7A. In the graph, the "YAP nuclear enrichment", do the numbers represent the nuclear/cytoplasm ratio?

-When showing migration and a picture (Fig.3F, 5D, S4D, S5E...), the blue over dark background is difficult to see, using greyscale or a brighter pseudocolour would help.

Significance

These findings have important preclinical implications, since the study proposes a biomarker of resistance (profibrotic signature) and importantly, a potential new therapy to delay MAPKi resistance in melanoma (BIBF). It could also apply to other BRAFmutant cancers and diseases cursing with fibrosis.

Field of expertise: melanoma, drug resistance, cytoskeleton

Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Referee #1

Evidence, reproducibility and clarity

The manuscript is interesting and well presented. The authors propose the use of an antifibrotic drug to attenuate resistance to RTK inhibitors.

Specific comments

1. It is not entirely clear how Nintedanib decreases tumour growth. It may be due to its effect on resistant melanoma cells as proposed, but it could also be due to the effect on CAFs. This should be at least discussed
2. A potential caveat is that drug used is non-specific as it also blocks PDGFR signalling. Hyperactivation of RTKs is a mechanism of BRAFi resistance and for example in Figure 1J, they see that BIF1120/Nintedanib has a significant effect on BRAFi-resistant cells, which may indicate that the growth inhibition seen in allografts could be a combination of an "anti-fibrotic" role and its own activity inhibiting the survival of resistant cells. This needs to be considered.
3. Does the viability decrease in BRAFi-sensitive cells? For instance, in the parental cells.
4. Figure 1 b-e, in vivo and in vivo experiments. How many animals we used? Collagen decrease is not quantified (statistics missing).
5. The title is not accurate. "prevent" resistance in melanoma is an overestimation because the cells do become resistant, albeit later.

Significance

As the authors discussed, they and others have previously studied the contribution of ECM and stromal remodelling to resistance to targeted therapies in melanoma. Previous data from E. Sahai´s lab show that BRAFi activate CAFs and increase the production and remodelling of the extracellular matrix, but in this work, they look at a cell-autonomous mechanism mediated by miRs that promotes fibrosis and propose the use of an antifibrotic drug to attenuate resistance to RTK inhibitors.

Transparent Peer Review

---

Download the complete Review Process [PDF] including:

• reviews
• authors' reply
• editorial decisions

</br>

Transparent Peer Review

---

Download the complete Review Process [PDF] including:

• reviews
• authors' reply
• editorial decisions

</br>

Transparent Peer Review

---

Download the complete Review Process [PDF] including:

• reviews
• authors' reply
• editorial decisions

</br>

Note: This rebuttal was posted by the corresponding author to Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Reply to the reviewers

1. General Statements [optional]

We thank the reviewers for their critical comments and suggestions. We are glad that the reviewers appreciated the quality of the data and the novel findings connecting the secretory trafficking machinery with extracellular matrix-related signaling.

2. Description of the planned revisions

Reviewer #1 (Evidence, reproducibility and clarity (Required)):

The manuscript by Jung et al reports on an interesting finding that focal adhesion signaling regulates the expression of Sec23A and thereby regulates COPII-dependent trafficking. The data presented a mostly solid and the finding itself is highly novel, as it tackles an area of secretory trafficking that remains poorly understood, namely the connection between the ECM and secretion.

I will list below all comments that I have mixing both technical and conceptual topics:

\*Technical issues:***

1-The authors should provide a better description of how the designed this siRNA library. What were the inclusion criteria for these 378 genes? I might have missed it, but I could not find this information easily.

Reply: The library has been designed in-house based on gene annotations and literature to include cytoskeleton structural proteins, motor proteins, and other associated and regulatory proteins. We will add this information in the Materials and Methods section.

2-Figure 2: I know this is challenging for EM images, but is there a way the authors could quantify these data? How many images were looked at? What was the average width of ER cisterne?

Reply: We will provide image quantifications and statistics

3-Figure 4: I think that the characterization of the FA phenotype is a bit underdeveloped. There is no quantification of these data. Is the size of FA changing? Is the number of FA per cell changing? Is the length of FAs changing? I think that more work is needed to increase the confidence in these data.

I could also not easily see what type of cells these are. A better description of this experiment is also required. Also, how many cells were analyzed. I think it is important that this experiment is done with a sufficient number of cells to increase the confidence in the data.

Reply: We agree with the reviewer that our observations regarding the focal adhesion (FA) phenotype will benefit from image quantification and we intend to include this in the revised manuscript. All FA experiments were performed on HeLa cells. We will update the materials and methods sections to better describe this experiment.

\*Conceptual issues:***

1-The finding that focal adhesion signaling negatively affects ER-export is surprising, because cancer cells that grow on stiff substrates have more focal adhesions and are more invasive and migratory. Both migration and invasion are expected to depend on ER-export. Although the authors did not formally test Sec23A expression under different stiffnesses, I would expect that stiff substrates would lower Sec23A expression and thereby negatively affect ER-export. It would certainly increase the breadth of this work to include data like this and to also discuss this highly surprising finding. However, it is of course the decision of the authors and the editors to decide whether such an experiment would benefit the entire story.

Reply: In this work, we have shown that cells plated on ECM or matrigel have decreased SEC23A expression compared to control cells. We have also shown that inhibition of FA kinase leads to an increase in SEC23A expression (Figure 5). Whether this translates into a change in ER transport, is a fair point that we will address in the revision. Regarding stiffness, we have done a preliminary experiment that shows that cells plated on a soft synthetic substrate have less SEC23A than cells plated on plastic.This goes in line with our ECM experiments because Matrigel and fibroblast-derived ECM are softer than plastic.

2-The authors postulate that this novel mechanism could be part of a feedback loop. If this were the case one would expect the acute effect of FA to increase ER-export (or secretion) and the negative feedback will then reduce secretion. However, the acute effect of FA is not addressed in this manuscript. In order to postulate a feedback loop, the authors would need to test the individual nodes of this loop.

Reply: The question appears to be whether an acute effect on FA would affect the expression of SEC23A and therefore ER transport. If by the acute effect the reviewer means a pharmacological manipulation, we have shown that upon treatment with the FAK inhibitor the expression of SEC23A increases (Fig 5A). Whether this increase in SEC23A expression translates into a corresponding increase in ER transport remains to be seen. This will be tested in our revised manuscript as mentioned above in reply to point # 1.

Our data encouraged us to propose a hypothetical feedback loop that would connect the deposition of ECM through the expression of SEC23A. We will have more data to support (or reject) this idea once we do the transport experiments as mentioned above. However, we think that a full characterization of this hypothetical loop by testing individual nodes is beyond the scope of this manuscript

Reviewer #1 (Significance (Required)):

I think that the basic finding of this manuscript is highly novel, by showing the impact of the ECM and focal adhesions on COPII-dependent trafficking. I think that this will not only appeal to people from the trafficking community, but also to people working on cell migration and on mechanobiology. The work in its current form does not require much extra efforts (max. 3 month). However, if the authors would decide to increase the breadth of data, they would require 3-6 months.

Reply: We thank reviewer #1 for the comments. We also believe that this story will appeal to a broader audience and would help to bridge the gap between membrane trafficking and mechanobiology communities.

\*Referees cross-commenting***

I went through the comments of the two other reviewers and agree with their verdict. Some extra work on the characterization of the early secretory pathway would be good. Both reviewers provided a nice catalogue of possible experiments to choose from.

Reply: We have characterized the early secretory pathway in terms of ER exit sites, Beta-COP, and Golgi morphology (FIG. 2B-H and S1A-B). Together, these data strongly characterize the nature of ER-block. Moreover, the finding that our interactors affect the expression of SEC23A allows us to explain mechanistically why an ER transport block occurs. This is further strengthened by the rescue experiments (FIG. 3F). We believe that further characterization of the secretory pathway will not contribute substantially to the main message of this manuscript.

Reviewer #2 (Evidence, reproducibility and clarity (Required)):

The manuscript by Jung et al which based on a targeted siRNA screen, demonstrates regulation of SEC23A (component of the SEC23 complex of the COP coat) levels at transcriptional level downstream of focal adhesion signaling. By regulating siRNA mediated downregulation, the authors were able to identify proteins which either increased or decreased traffic of VSVG through the secretory pathway when combined with downregulation in the levels of with either SEC23A or SEC23B. Authors have focused on a group of SEC23B functional interactors, downregulation of which shows them increased size of focal adhesions which also downregulate SEC23A levels, thus providing an explanation for reduced secretory traffic. Authors further show that plating cells on fibronectin or Matrigel, which activate Focal adhesion kinase signaling also results in downregulation of SEC23A transcript levels. The screen is conducted in a well-controlled manner for most parts with a clear explanation of the analysis routines and the data presentation if of very good quality. Most important results have been validated by more than one experimental strategy which lends substantial confidence to the findings. The results also open further avenues for understanding the transcriptional regulation in different physiological and disease contexts.

There are certain issues, which the authors should address with regards to controls and some conflicting observations with published results with respect the phenotypes associated with downregulating proteins on focal adhesions size. Additionally, authors don't tie the ends by monitoring secretory traffic in cells grown on different matrices but include it in the model. Addressing/explaining these issues could improve this manuscript and the model may have to be tweaked a bit.

\*Major comments:***

1)I wonder why the authors only used siRNA control in their screen when the effects are scored in context of double knockdown fashion in combination with mild knockdown of SEC23A and SEC23B to get functional interactors. Control siRNA in combination with SEC23A and SEC23B should have been two ideal negative controls in the screen. Nevertheless, in data presented Figure 1E and whole of Figure 2, using control siRNA in combination with SEC23B siRNA would have been ideal control to show that the combination does not induce any trafficking defects which could impact the findings of the study. Hence, a few of the data presented from some of these figures should have sicontrol+SEC23B siRNA combination as a control.

Reply: There seems to be a misunderstanding. In the screen, the negative controls are only used as a reference as the scoring is based on a 5X5 matrix centered on the siRNA of interest. This is done to overcome possible plate effects and to normalize data across different biological replicas. As seen in figure 1B, the negative controls (Control siRNA or Control siRNA + SEC23A siRNA or Control siRNA + SEC23B siRNA are very close to 0 (but not exactly 0) as they were not used in the normalization process. It is important to mention that all single knockdowns also contain our control siRNA to keep the same final siRNA concentration in single and double knockdowns. In Fig 1E we will include the images from Control + SEC23A siRNAs and Control + SEC23B siRNA as a reference. For Figure 2 all except 2A and 2H have the single knockdowns as controls.

2)What is the identity of post-ER structures which authors refer to in Figure 2A? Could the images represent VSVG concentrated at ER exit sites? Authors should stain with markers for ERES to see if the VSVG puncta colocalize with it.

Reply: We have done the experiment, and indeed these structures colocalize with an ER exit site marker (SEC31A). We intend to include this data into the revised manuscript. Our observations are in agreement with what is known in the literature about VSVG transport.

3)Based on RNA sequencing results, authors chose to follow up on SEC23A levels in background of siRNA knockdown of components (like MACF1, ROCK1, FERMT2 etc.) which regulate Focal adhesions in cells and show that there is a reduction in both transcript and protein levels of SEC23A. In images shown in Figure 2B and Figure 2C, levels or SEC31A and β-Cop1 are reduced. Authors should test using qPCR and western blots whether there is a downregulation SEC31A, β-Cop1 and SEC23B in siRNA knockdowns of MACF1, ROCK1, FERMT2 etc. It would provide new insights if there were a co-regulation of secretory machinery to modulate the secretory traffic in response to Focal Adhesion based signaling.

Reply: Our transcriptomics data (FIG 3C and Table 5) shows that SEC31A and COPB1 mRNAs are not altered upon any of the knockdowns. For SEC23B, we observed only a slight decrease in ROCK1 knockdown. This data suggests that a co-regulation of the secretory machinery might not be present. Instead, the curation of secretory pathway genes in our transcriptome data shows that SEC23A is the only commonly differentially expressed gene.

4)Most major concern in this manuscript surrounds around results presented in Figure 4C. Authors show that in response to all the knockdowns, they see more focal adhesions as monitored by Vinculin staining and this along with the experiments with cells plated on Matrigel and Fibronectin arrive at the conclusion that increased Focal adhesion signaling downregulates SEC23A levels which presumably modulates secretory traffic. I am not an expert on Focal adhesions but based on my understanding of the literature on that topic, downregulation of ROCK1, FEMRT2 disrupts focal adhesions. (See: Theodosiou et. al., Elife, 2016 or Lock et. al., Plos One, 2012 for example). How do authors explain their results in siRNA knockdown of ROCK1 and FEMRT2 which leads to an increased size of focal adhesions which seems contradictory to the published results? To clarify these results authors should test phosphorylation of FAK in their siRNA backgrounds which is another read out of focal adhesion signaling.

The experiments from cells grown on Fibronectin and Matrigel favor the argument which authors put forth, but authors may have to tweak the model a bit based on FAK phosphorylation and FAK signaling in context of above-mentioned knockdowns.

Reply: Based on the images for vinculin staining, in our current manuscript we propose that changes in FAs occur upon knocking down our interactors. In our revised manuscript we will provide a more robust quantitative assessment of those changes (change in number, size, or intensity) as mentioned in our reply to Reviewer #1.

As for the discrepancies in the relation of FA phenotype upon depletion of ROCK1 and FERMT2, we want to point out that this effect depends on the cell type used. For instance, the papers listed by the reviewer here use fibroblasts and keratinocytes respectively while we have used Hela Kyoto cells which are epithelial in nature. Another example is that while in fibroblasts depletion of FERMT2 leads to a rounded morphology and almost an absence of FAs (Theodosiou et. al., Elife, 2016), in podocytes (Qu et al JCS, 2011), it leads to fewer FAs but an increase in their size. Nonetheless, this is a very keen observation from the reviewer and we will address this point in our revised manuscript discussion.

5)What happens to VSVG traffic or RUSH-Cadherin traffic when cells are plated on Matrigel and Fibronectin? Reduction in secretory traffic of these is an important experiment which is missing to close the loop and validate the model presented. Authors must test these experiments either with cells grown on matrix alone or in combination with siRNA to SEC23B. Authors should also monitor ERES and transport carriers in this background.

Reply: We agree with the reviewer and intend to perform these experiments.

6)This is not such a major issue, but it would be good to see a comparison in SEC23A levels in siRNA knockdown condition in comparison to those when cells are grown on different substrates and in ROCK1, FEMRT2 knockdowns (blots of which authors already have in this manuscript).

Reply: We will assess the level of SEC23A at the protein level for cells plated on matrigel or Fibroblast-derived ECM.

\*Minor comments:***

1)Scale bars are missing in EM images in Figure 2H.

Reply: We will add the scales in our EM images

2)Show molecular weight markers in Western blots in main figure 3E and supplementary figure S1E.

Reply: We will add molecular weight markers in our Western-Blots

Reviewer #2 (Significance (Required)):

I have looked at the manuscript from through the lens of a cell biologist as that is predominantly my area of expertise. In that respect I find the screen conducted by authors particularly interesting as they aim to connect how extracellular cues regulate the secretory pathway. A screen seems justified as there is no comprehensive understanding linking the two above-mentioned processes. Authors have done a functional interaction screen and analyzed a lot of images to identify candidates which either increase or decrease secretory traffic in combination with SEC23A and SEC23B. Such a functional screen has helped authors identify candidates which were otherwise missed in single siRNA knockdowns in their previous work from 2012. This definitely opens up interesting avenues to test the candidates identified in the screen in different physiological contexts and in disease as also the transcriptional program connecting Focal adhesion signaling with the regulation of components governing secretion. Such functional interaction screens could also be employed to identify crosstalk of different cellular processes with the regulation secretory pathway at ER as well as at the Golgi apparatus.

Reply: We thank reviewer #2 for the comments. As we mentioned in our reply to reviewer #1, we strongly believe that these results will encourage further research at the crossroads of membrane trafficking and mechanobiology.

\*Referees cross-commenting***

I agree with the comments from both the referees that the manuscript is very interesting, most experiments are well controlled, but the quantification of focal adhesion phenotype in knockdowns need to be done in an extensive manner and secretion phenotypes need to measured upon plating cells on different matrix to validate the model presented.

Reply: These two experiments will be included in our revision

Reviewer #3 (Evidence, reproducibility and clarity (Required)):

\*Summary***

The authors use a synchronized cargo release assay following codepletion of either Sec23 paralog with cytoskeletal and associated proteins to identify potential functional interactions between COPII trafficking and the cytoskeleton. This screen yields a number of Sec23b functionally interacting molecules that stall cargo trafficking to various degrees within the secretory pathway upon codepletion, and in the case of MACF1 reduce ERES number despite not physically interacting. Depletion of the majority of the identified Sec23b functional interactors alone surprisingly caused the downregulation of Sec23a at the mRNA and protein levels, and cargo trafficking could be partially or fully rescued by Sec23a overexpression depending on the codepleted cytoskeletal factor. RNA-seq enrichment analysis and imaging of a focal adhesion marker suggest that genes involved in cell adhesion were differentially regulated following depletion of the cytoskeletal functional interactors. Finally, the authors show that Sec23a expression levels are reduced when cells are cultured on dishes with high amounts of ECM to induce focal adhesions, and that inhibition of focal adhesion kinase can rescue Sec23a expression levels.

\*Major comments***

#1 The authors successfully implicate a group of cytoskeletal proteins and their actions at focal adhesions in negatively regulating Sec23a expression levels and COPII trafficking. This description of a shared, novel mode of COPII transcriptional regulation by cytoskeletal factors is convincingly shown to be at least a contributor to the delayed trafficking in the presence of focal adhesions. In general, the data are reproducible and use appropriate statistical analysis. However, a more robust description of the architecture of early secretory pathway would be beneficial, especially in the case of MACF1 codepletion which cannot be fully rescued by Sec23a-YFP overexpression. In contrast, trafficking during codepletion of FERMT2 is fully rescued by Sec23a-YFP despite both MACF1 and FERMT2 showing similar loss of Sec23a mRNA levels upon codepletion. This data suggests that while the trafficking delay in FERMT2 codepletion might be exclusively due to reduced Sec23a expression levels, there are likely additional causes for the trafficking delay observed in MACF1 codepletion.

Reply: We thank the reviewer for the appreciation of our results and the importance they might bear for the field. The reviewer has very neatly highlighted that each of our interactor hits might have roles in the secretory pathway beyond the ER or independent of the expression levels of SEC23A. This phenomenon could also explain the differential rescue of the arrival of VSVG at the plasma membrane upon SEC23A overexpression in FERMT2 and MACF1 knockdowns (FIG 3F). For instance, MACF1 has been involved in Golgi to Plasma Membrane transport as well (Kakinuma et al. Exp. Cell Res. 2004, Burgo et al. Dev. Cell 2012). So a possibility is that SEC23A overexpression rescues only ER to Golgi transport but the lack of rescue in the compartment between Golgi and plasma membrane independent of SEC23A expression levels would result in reduced rescue In the case of MACF1 compared to FERMT2. To support this, in our revised manuscript, we will provide example images from the experiment.

Nonetheless, we agree that these are very important observations from Reviewer #3 and warrant a detailed discussion in the light of other interactors as well, which we intend to highlight in our revised manuscript.

#2 While there is indeed a reduction in the number of ERESs following MACF1 codepletion, the authors report an even more dramatic reduction in 'transport intermediates / cell' as marked by COPI. However, as recent cyro-EM analysis of ERESs has definitively show, COPI exists stably at ERGIC membranes (1). Thus, an alternative possibility for the more dramatic reduction of COPI sites compared to Sec31a sites in Figures 2B-E is that ERGIC membranes are destabilized following MACF1 codepletion in a manner independent of Sec23a expression, and this destabilization compounds with reduced ERES number to ultimately delay trafficking. To more directly determine whether ERGIC membranes stability is regulated by MACF1, the authors should compare COPI and ERGIC-53 staining among MACF1 codepleted and FERMT2 codepleted cells with and without Sec23a-YFP overexpressed to levels that rescue cargo trafficking. If Sec23a-YFP restores the number of ERGIC puntae marked by these stains in FERMT2 but not MACF1 codepleted cells, it would suggest a role for MACF1 in forming or stabilizing ERGIC membranes which are known to associate with microtubules and WHAMM, an actin nucleator. Additionally, it would be useful to costain COPII with COPI or ERGIC-53 in control, MACF1 depleted, MACF1 codepleted, and MACF1 codepleted and Sec23a-YFP rescued cells to determine their colocalization. COPII and ERGIC membranes should be almost entirely coupled and juxtaposed in control cells and may be decoupled upon loss of MACF if plays a role in ERGIC membrane localization and stability. These proposed experiments are relevant because ERGIC membranes are sites of COPII cargo delivery and changes in ERGIC stability or localization would suggest an additional mechanism for cytoskeletal regulation of COPII trafficking. These immunofluorescence studies should be straightforward and completed in a few weeks.

Reply: Although a possible additional role of MACF1 in the organisation of early secretory pathway, stability of ERES, etc., independent of the expression of SEC23A is interesting on its own, we believe that an extensive characterization of these possible roles/ pathways as proposed by the reviewer is beyond the scope this manuscript.

#3 The choice to use VSVG and E-Cadherin for the synchronized release assays unfortunately convolutes interpreting the 'transport ratios' used by the authors to compare the effects of the various codepletions. Each protein progresses beyond the Golgi during secretion, and the authors choose to calculate the ratio of cargo intensity at the plasma membrane normalized to the total cellular cargo. This means that the synchronized release assays and calculated 'transport ratios' assay not only ER to Golgi trafficking, but also trafficking from the Golgi to the plasma membrane. In instances where Sec23a-YFP overexpression does not fully rescue the codepletion, it is possible that additional trafficking delays occur during Golgi to plasma membrane trafficking that cause the 'transport score' to decrease. Thus, the 'transport score' as the authors calculate it is needlessly nonspecific to COPII trafficking and should not be used to compare the codepletions for COPII functional interactors.

Reply: We agree that the “transport score” used here and in our previous genome-wide screen (Simpson et. al Nat. Cell Biol. 2012) does not allow us to distinguish between the individual transport substeps in the transport of VSVG from the ER to the plasma membrane. However, as we see in Fig 1E, the proteins that we have decided to follow in more detail in this study do have a clear ER transport block phenotype (except for CRKL). So for 6 out of 7 of these proteins, the images clearly show that the decrease in the “transport score” is due to a decreased ER to Golgi transport.

#4 To mitigate unwanted contributions of post-COPII trafficking events from altering 'transport scores,' the authors should use a cargo for synchronized release assays that does not progress past the Golgi such as α-Mannosidase II and quantify a ratio of the perinuclear cargo signal to whole cell signal. Ideally, the screen would be repeated with a more appropriate cargo generating new 'transport scores' for the full list of cytoskeletal proteins. However, this may not be feasible, and as such 'transport scores' based on a Golgi resident protein should at least be produced for the 7 Sec23b functional interactors featured in this manuscript. These Golgi 'transport scores' would add much needed quantification of ER to Golgi transport delays that currently can only be inferred from the representative images in Figure 1E, which unfortunately show significant heterogeneity among cells from the same image. The authors should also explicitly state that any 'transport score' from a synchronous release assay using a cargo destined for the plasma membrane will take into account trafficking rate changes due not only to COPII, but also COPI from the ERGIC to the Golgi, and transport carriers departing from the TGN. These synchronized release assays would likely take between a few weeks to a few months depending on their ability to automate image analysis.

Reply: We consider that having a “Golgi transport score” won't add any new information as the proteins that we have chosen to follow are the ones that show a strong ER-block phenotype. However, we agree that such a “Golgi score” would indeed be useful if one would like to study other interactors, for instance, the ones that induce transport acceleration.

Also, we don't expect all cells to behave similarly as the level of knockdown might be slightly different or because of the cell to cell variability. Even in control conditions (no knockdown), this heterogeneity is evident. As suggested by the reviewer, in our revised manuscript we will explicitly state that a change in the transport scores could mean a change in any sub-step of the transport from the ER to the PM in our assay.

\*Minor comments***

It would be useful for the authors to quantify the number of focal adhesions present from Vinculin stains from Figure 4C and 5C instead of just showing representative images. It would be interesting to determine if there is a meaningful relationship between focal adhesion number induced by the codepletions or tissue culture coating and Sec23a expression levels like in Figure 3D. Generally, the figures, text, and references were appropriate.

Reply: As also pointed out by the other reviewers we will quantify the FA changes

Reviewer #3 (Significance (Required)):

In recent years, significant effort has been devoted to elucidating mechanisms by which COPII trafficking is modulated in response to cellular cues. These studies have revealed that changes in nutrient availability, growth factors, ER stress, autophagy, and T-cell activation all cause changes in COPII trafficking via unique gene expression, splicing, or post-translational control (2-7). This work elucidates a novel mechanism of transcriptional control driven by focal adhesions. Additionally, it provides a number of potentially useful Sec23a and Sec23b functional interactors among cytoskeletal factors for further study. These unexplored factors may have unique mechanism of COPII regulation that could contribute to our understanding ER export modulation. Altogether, this and similar works are building an increasingly complex set of regulatory pathways that when integrated ultimately dictate COPII trafficking kinetics.

The reported findings are not only relevant to those who study COPII trafficking, but also other fields where secretion is studied in the context of the ECM. This work would suggest that secretion of factors involved in crosstalk between cells, including in tumors, is likely to be controlled by the interactions of cells with ECM.

Reply: We thank reviewer #3 for the comments and insightful discussion about the limitations of our assay that we will highlight in the revised manuscript and in general for the insight into the early secretory pathway regulation. Furthermore their explicit summary of how our study could bridge COPII trafficking, ECM signaling and the relevance to various pathophysiologies is highly appreciated.

Expertise keywords: cell biology, light microscopy, membrane trafficking

References

1.Weigel A V., Chang CL, Shtengel G, Xu CS, Hoffman DP, Freeman M, et al. ER-to-Golgi protein delivery through an interwoven, tubular network extending from ER. Cell. 2021 Apr;184(9):2412-2429.e16.

2.Farhan, H., Wendeler, M. W., Mitrovic, S., Fava, E., Silberberg, Y., Sharan, R., Zerial, M., & Hauri, H. P. (2010). **MAPK signaling to the early secretory pathway revealed by kinase/phosphatase functional screening. Journal of Cell Biology, 189(6), 997-1011.

3.Zacharogianni, M., Kondylis, V., Tang, Y., Farhan, H., Xanthakis, D., Fuchs, F., Boutros, M., & Rabouille, C. (2011). ERK7 is a negative regulator of protein secretion in response to amino-acid starvation by modulating Sec16 membrane association. **EMBO Journal, 30(18), 3684-3700.

4.Lillmann, K.D., V. Reiterer, F. Baschieri, J. Hoffmann, V. Millarte, M.A. Hauser, A. Mazza, N. Atias, D.F. Legler, R. Sharan, et al 2015. **Regulation of Sec16 levels and dynamics links proliferation and secretion. J. Cell Sci. 128:670-682.

5.Liu, L., Cai, J., Wang, H., Liang, X., Zhou, Q., Ding, C., Zhu, Y., Fu, T., Guo, Q., Xu, Z., Xiao, L., Liu, J., Yin, Y., Fang, L., Xue, B., Wang, Y., Meng, Z. X., He, A., Li, J. L., ... Gan, Z. (2019). Coupling of COPII vesicle trafficking to nutrient availability by the IRE1α-XBP1s axis. Proceedings of the National Academy of Sciences of the United States of America, 116(24), 11776-11785.

6.Jeong, Y.-T., Simoneschi, D., Keegan, S., Melville, D., Adler, N. S., Saraf, A., Florens, L., Washburn, M. P., Cavasotto, C. N., Fenyö, D., Cuervo, A. M., Rossi, M., & Pagano, M. (2018). The ULK1-FBXW5-SEC23B nexus controls autophagy. ELife, 1-25.

7.Wilhelmi, I., Kanski, R., Neumann, A., Herdt, O., Hoff, F., Jacob, R., Preußner, M., & Heyd, F. (2016). Sec16 alternative splicing dynamically controls COPII transport efficiency. Nature Communications, 7, 12347. https://doi.org/10.1038/ncomms12347

3. Description of the revisions that have already been incorporated in the transferred manuscript

4. Description of analyses that authors prefer not to carry out

Reviewer #3 suggested to robustly characterise the early secretory pathway, in response to the depletion of our interactors, for instance, the role of MACF1 in the organization and the stability of ERES. This view is also supported by reviewer #1. However, in our revised manuscript we would like to focus more on the novel aspect of our study (as highlighted by all the reviewers), namely how ECM signaling and changes in FAs affect SEC23A and possibly ER transport. For this, we would like to present a more quantitative outlook of the FA phenotype and concentrate on the transport experiments. The reason for not dwelling into a more extensive characterization of the early secretory pathway is that these experiments are very interesting on their own, and merit a separate study that would deconvolve in detail the individual trafficking steps, and their relation to SEC23A expression, ERES stability, and ECM signaling.

Reviewer #2 suggested that to better characterize the FA phenotype and solve the apparent discrepancies between our data and the literature, we could test FAK phosphorylation. As we mentioned in our reply to this point, we think that most of the discrepancies arise from the different cell types used. Nevertheless, we agree that a quantitative approach is needed for a better characterisation of FA phenotype, therefore we intend to perform quantification of the vinculin stainings.

Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Referee #3

Evidence, reproducibility and clarity

Summary

The authors use a synchronized cargo release assay following codepletion of either Sec23 paralog with cytoskeletal and associated proteins to identify potential functional interactions between COPII trafficking and the cytoskeleton. This screen yields a number of Sec23b functionally interacting molecules that stall cargo trafficking to various degrees within the secretory pathway upon codepletion, and in the case of MACF1 reduce ERES number despite not physically interacting. Depletion of the majority of the identified Sec23b functional interactors alone surprisingly caused the downregulation of Sec23a at the mRNA and protein levels, and cargo trafficking could be partially or fully rescued by Sec23a overexpression depending on the codepleted cytoskeletal factor. RNA-seq enrichment analysis and imaging of a focal adhesion marker suggest that genes involved in cell adhesion were differentially regulated following depletion of the cytoskeletal functional interactors. Finally, the authors show that Sec23a expression levels are reduced when cells are cultured on dishes with high amounts of ECM to induce focal adhesions, and that inhibition of focal adhesion kinase can rescue Sec23a expression levels.

Major comments

The authors successfully implicate a group of cytoskeletal proteins and their actions at focal adhesions in negatively regulating Sec23a expression levels and COPII trafficking. This description of a shared, novel mode of COPII transcriptional regulation by cytoskeletal factors is convincingly shown to be at least a contributor to the delayed trafficking in the presence of focal adhesions. In general, the data are reproducible and use appropriate statistical analysis. However, a more robust description of the architecture of early secretory pathway would be beneficial, especially in the case of MACF1 codepletion which cannot be fully rescued by Sec23a-YFP overexpression. In contrast, trafficking during codepletion of FERMT2 is fully rescued by Sec23a-YFP despite both MACF1 and FERMT2 showing similar loss of Sec23a mRNA levels upon codepletion. This data suggests that while the trafficking delay in FERMT2 codepletion might be exclusively due to reduced Sec23a expression levels, there are likely additional causes for the trafficking delay observed in MACF1 codepletion.

While there is indeed a reduction in the number of ERESs following MACF1 codepletion, the authors report an even more dramatic reduction in 'transport intermediates / cell' as marked by COPI. However, as recent cyro-EM analysis of ERESs has definitively show, COPI exists stably at ERGIC membranes (1). Thus, an alternative possibility for the more dramatic reduction of COPI sites compared to Sec31a sites in Figures 2B-E is that ERGIC membranes are destabilized following MACF1 codepletion in a manner independent of Sec23a expression, and this destabilization compounds with reduced ERES number to ultimately delay trafficking. To more directly determine whether ERGIC membranes stability is regulated by MACF1, the authors should compare COPI and ERGIC-53 staining among MACF1 codepleted and FERMT2 codepleted cells with and without Sec23a-YFP overexpressed to levels that rescue cargo trafficking. If Sec23a-YFP restores the number of ERGIC puntae marked by these stains in FERMT2 but not MACF1 codepleted cells, it would suggest a role for MACF1 in forming or stabilizing ERGIC membranes which are known to associate with microtubules and WHAMM, an actin nucleator. Additionally, it would be useful to costain COPII with COPI or ERGIC-53 in control, MACF1 depleted, MACF1 codepleted, and MACF1 codepleted and Sec23a-YFP rescued cells to determine their colocalization. COPII and ERGIC membranes should be almost entirely coupled and juxtaposed in control cells and may be decoupled upon loss of MACF if plays a role in ERGIC membrane localization and stability. These proposed experiments are relevant because ERGIC membranes are sites of COPII cargo delivery and changes in ERGIC stability or localization would suggest an additional mechanism for cytoskeletal regulation of COPII trafficking. These immunofluorescence studies should be straightforward and completed in a few weeks.

The choice to use VSVG and E-Cadherin for the synchronized release assays unfortunately convolutes interpreting the 'transport ratios' used by the authors to compare the effects of the various codepletions. Each protein progresses beyond the Golgi during secretion, and the authors choose to calculate the ratio of cargo intensity at the plasma membrane normalized to the total cellular cargo. This means that the synchronized release assays and calculated 'transport ratios' assay not only ER to Golgi trafficking, but also trafficking from the Golgi to the plasma membrane. In instances where Sec23a-YFP overexpression does not fully rescue the codepletion, it is possible that additional trafficking delays occur during Golgi to plasma membrane trafficking that cause the 'transport score' to decrease. Thus, the 'transport score' as the authors calculate it is needlessly nonspecific to COPII trafficking and should not be used to compare the codepletions for COPII functional interactors.

To mitigate unwanted contributions of post-COPII trafficking events from altering 'transport scores,' the authors should use a cargo for synchronized release assays that does not progress past the Golgi such as α-Mannosidase II and quantify a ratio of the perinuclear cargo signal to whole cell signal. Ideally, the screen would be repeated with a more appropriate cargo generating new 'transport scores' for the full list of cytoskeletal proteins. However, this may not be feasible, and as such 'transport scores' based on a Golgi resident protein should at least be produced for the 7 Sec23b functional interactors featured in this manuscript. These Golgi 'transport scores' would add much needed quantification of ER to Golgi transport delays that currently can only be inferred from the representative images in Figure 1E, which unfortunately show significant heterogeneity among cells from the same image. The authors should also explicitly state that any 'transport score' from a synchronous release assay using a cargo destined for the plasma membrane will take into account trafficking rate changes due not only to COPII, but also COPI from the ERGIC to the Golgi, and transport carriers departing from the TGN. These synchronized release assays would likely take between a few weeks to a few months depending on their ability to automate image analysis.

Minor comments

It would be useful for the authors to quantify the number of focal adhesions present from Vinculin stains from Figure 4C and 5C instead of just showing representative images. It would be interesting to determine if there is a meaningful relationship between focal adhesion number induced by the codepletions or tissue culture coating and Sec23a expression levels like in Figure 3D. Generally, the figures, text, and references were appropriate.

Significance

In recent years, significant effort has been devoted to elucidating mechanisms by which COPII trafficking is modulated in response to cellular cues. These studies have revealed that changes in nutrient availability, growth factors, ER stress, autophagy, and T-cell activation all cause changes in COPII trafficking via unique gene expression, splicing, or post-translational control (2-7). This work elucidates a novel mechanism of transcriptional control driven by focal adhesions. Additionally, it provides a number of potentially useful Sec23a and Sec23b functional interactors among cytoskeletal factors for further study. These unexplored factors may have unique mechanism of COPII regulation that could contribute to our understanding ER export modulation. Altogether, this and similar works are building an increasingly complex set of regulatory pathways that when integrated ultimately dictate COPII trafficking kinetics.

The reported findings are not only relevant to those who study COPII trafficking, but also other fields where secretion is studied in the context of the ECM. This work would suggest that secretion of factors involved in crosstalk between cells, including in tumors, is likely to be controlled by the interactions of cells with ECM.

Expertise keywords: cell biology, light microscopy, membrane trafficking

References

1.Weigel A V., Chang CL, Shtengel G, Xu CS, Hoffman DP, Freeman M, et al. ER-to-Golgi protein delivery through an interwoven, tubular network extending from ER. Cell. 2021 Apr;184(9):2412-2429.e16.

2.Farhan, H., Wendeler, M. W., Mitrovic, S., Fava, E., Silberberg, Y., Sharan, R., Zerial, M., & Hauri, H. P. (2010). MAPK signaling to the early secretory pathway revealed by kinase/phosphatase functional screening. Journal of Cell Biology, 189(6), 997-1011.

3.Zacharogianni, M., Kondylis, V., Tang, Y., Farhan, H., Xanthakis, D., Fuchs, F., Boutros, M., & Rabouille, C. (2011). ERK7 is a negative regulator of protein secretion in response to amino-acid starvation by modulating Sec16 membrane association. EMBO Journal, 30(18), 3684-3700.

4.Lillmann, K.D., V. Reiterer, F. Baschieri, J. Hoffmann, V. Millarte, M.A. Hauser, A. Mazza, N. Atias, D.F. Legler, R. Sharan, et al 2015. Regulation of Sec16 levels and dynamics links proliferation and secretion. J. Cell Sci. 128:670-682.

5.Liu, L., Cai, J., Wang, H., Liang, X., Zhou, Q., Ding, C., Zhu, Y., Fu, T., Guo, Q., Xu, Z., Xiao, L., Liu, J., Yin, Y., Fang, L., Xue, B., Wang, Y., Meng, Z. X., He, A., Li, J. L., ... Gan, Z. (2019). Coupling of COPII vesicle trafficking to nutrient availability by the IRE1α-XBP1s axis. Proceedings of the National Academy of Sciences of the United States of America, 116(24), 11776-11785.

6.Jeong, Y.-T., Simoneschi, D., Keegan, S., Melville, D., Adler, N. S., Saraf, A., Florens, L., Washburn, M. P., Cavasotto, C. N., Fenyö, D., Cuervo, A. M., Rossi, M., & Pagano, M. (2018). The ULK1-FBXW5-SEC23B nexus controls autophagy. ELife, 1-25.

7.Wilhelmi, I., Kanski, R., Neumann, A., Herdt, O., Hoff, F., Jacob, R., Preußner, M., & Heyd, F. (2016). Sec16 alternative splicing dynamically controls COPII transport efficiency. Nature Communications, 7, 12347. https://doi.org/10.1038/ncomms12347

Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Referee #2

Evidence, reproducibility and clarity

The manuscript by Jung et al which based on a targeted siRNA screen, demonstrates regulation of SEC23A (component of the SEC23 complex of the COP coat) levels at transcriptional level downstream of focal adhesion signaling. By regulating siRNA mediated downregulation, the authors were able to identify proteins which either increased or decreased traffic of VSVG through the secretory pathway when combined with downregulation in the levels of with either SEC23A or SEC23B. Authors have focused on a group of SEC23B functional interactors, downregulation of which shows them increased size of focal adhesions which also downregulate SEC23A levels, thus providing an explanation for reduced secretory traffic. Authors further show that plating cells on fibronectin or Matrigel, which activate Focal adhesion kinase signaling also results in downregulation of SEC23A transcript levels. The screen is conducted in a well-controlled manner for most parts with a clear explanation of the analysis routines and the data presentation if of very good quality. Most important results have been validated by more than one experimental strategy which lends substantial confidence to the findings. The results also open further avenues for understanding the transcriptional regulation in different physiological and disease contexts.

There are certain issues, which the authors should address with regards to controls and some conflicting observations with published results with respect the phenotypes associated with downregulating proteins on focal adhesions size. Additionally, authors don't tie the ends by monitoring secretory traffic in cells grown on different matrices but include it in the model. Addressing/explaining these issues could improve this manuscript and the model may have to be tweaked a bit.

Major comments:

1)I wonder why the authors only used siRNA control in their screen when the effects are scored in context of double knockdown fashion in combination with mild knockdown of SEC23A and SEC23B to get functional interactors. Control siRNA in combination with SEC23A and SEC23B should have been two ideal negative controls in the screen. Nevertheless, in data presented Figure 1E and whole of Figure 2, using control siRNA in combination with SEC23B siRNA would have been ideal control to show that the combination does not induce any trafficking defects which could impact the findings of the study. Hence, a few of the data presented from some of these figures should have sicontrol+SEC23B siRNA combination as a control.

2)What is the identity of post-ER structures which authors refer to in Figure 2A? Could the images represent VSVG concentrated at ER exit sites? Authors should stain with markers for ERES to see if the VSVG puncta colocalize with it.

3)Based on RNA sequencing results, authors chose to follow up on SEC23A levels in background of siRNA knockdown of components (like MACF1, ROCK1, FERMT2 etc.) which regulate Focal adhesions in cells and show that there is a reduction in both transcript and protein levels of SEC23A. In images shown in Figure 2B and Figure 2C, levels or SEC31A and β-Cop1 are reduced. Authors should test using qPCR and western blots whether there is a downregulation SEC31A, β-Cop1 and SEC23B in siRNA knockdowns of MACF1, ROCK1, FERMT2 etc. It would provide new insights if there were a co-regulation of secretory machinery to modulate the secretory traffic in response to Focal Adhesion based signaling.

4)Most major concern in this manuscript surrounds around results presented in Figure 4C. Authors show that in response to all the knockdowns, they see more focal adhesions as monitored by Vinculin staining and this along with the experiments with cells plated on Matrigel and Fibronectin arrive at the conclusion that increased Focal adhesion signaling downregulates SEC23A levels which presumably modulates secretory traffic. I am not an expert on Focal adhesions but based on my understanding of the literature on that topic, downregulation of ROCK1, FEMRT2 disrupts focal adhesions. (See: Theodosiou et. al., Elife, 2016 or Lock et. al., Plos One, 2012 for example). How do authors explain their results in siRNA knockdown of ROCK1 and FEMRT2 which leads to an increased size of focal adhesions which seems contradictory to the published results? To clarify these results authors should test phosphorylation of FAK in their siRNA backgrounds which is another read out of focal adhesion signaling. The experiments from cells grown on Fibronectin and Matrigel favor the argument which authors put forth, but authors may have to tweak the model a bit based on FAK phosphorylation and FAK signaling in context of above-mentioned knockdowns.

5)What happens to VSVG traffic or RUSH-Cadherin traffic when cells are plated on Matrigel and Fibronectin? Reduction in secretory traffic of these is an important experiment which is missing to close the loop and validate the model presented. Authors must test these experiments either with cells grown on matrix alone or in combination with siRNA to SEC23B. Authors should also monitor ERES and transport carriers in this background.

6)This is not such a major issue, but it would be good to see a comparison in SEC23A levels in siRNA knockdown condition in comparison to those when cells are grown on different substrates and in ROCK1, FEMRT2 knockdowns (blots of which authors already have in this manuscript).

Minor comments:

1)Scale bars are missing in EM images in Figure 2H.

2)Show molecular weight markers in Western blots in main figure 3E and supplementary figure S1E.

Significance

I have looked at the manuscript from through the lens of a cell biologist as that is predominantly my area of expertise. In that respect I find the screen conducted by authors particularly interesting as they aim to connect how extracellular cues regulate the secretory pathway. A screen seems justified as there is no comprehensive understanding linking the two above-mentioned processes. Authors have done a functional interaction screen and analyzed a lot of images to identify candidates which either increase or decrease secretory traffic in combination with SEC23A and SEC23B. Such a functional screen has helped authors identify candidates which were otherwise missed in single siRNA knockdowns in their previous work from 2012. This definitely opens up interesting avenues to test the candidates identified in the screen in different physiological contexts and in disease as also the transcriptional program connecting Focal adhesion signaling with the regulation of components governing secretion. Such functional interaction screens could also be employed to identify crosstalk of different cellular processes with the regulation secretory pathway at ER as well as at the Golgi apparatus.

Referees cross-commenting

I agree with the comments from both the referees that the manuscript is very interesting, most experiments are well controlled, but the quantification of focal adhesion phenotype in knockdowns need to be done in an extensive manner and secretion phenotypes need to measured upon plating cells on different matrix to validate the model presented.

Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Referee #1

Evidence, reproducibility and clarity

The manuscript by Jung et al reports on an interesting finding that focal adhesion signaling regulates the expression of Sec23A and thereby regulates COPII-dependent trafficking. The data presented a mostly solid and the finding itself is highly novel, as it tackles an area of secretory trafficking that remains poorly understood, namely the connection between the ECM and secretion.

I will list below all comments that I have mixing both technical and conceptual topics:

Technical issues:

1-The authors should provide a better description of how the designed this siRNA library. What were the inclusion criteria for these 378 genes? I might have missed it, but I could not find this information easily.

2-Figure 2: I know this is challenging for EM images, but is there a way the authors could quantify these data? How many images were looked at? What was the average width of ER cisterne?

3-Figure 4: I think that the characterization of the FA phenotype is a bit underdeveloped. There is no quantification of these data. Is the size of FA changing? Is the number of FA per cell changing? Is the length of FAs changing? I think that more work is needed to increase the confidence in these data. I could also not easily see what type of cells these are. A better description of this experiment is also required. Also, how many cells were analyzed. I think it is important that this experiment is done with a sufficient number of cells to increase the confidence in the data.

Conceptual issues:

1-The finding that focal adhesion signaling negatively affects ER-export is surprising, because cancer cells that grow on stiff substrates have more focal adhesions and are more invasive and migratory. Both migration and invasion are expected to depend on ER-export. Although the authors did not formally test Sec23A expression under different stiffnesses, I would expect that stiff substrates would lower Sec23A expression and thereby negatively affect ER-export. It would certainly increase the breadth of this work to include data like this and to also discuss this highly surprising finding. However, it is of course the decision of the authors and the editors to decide whether such an experiment would benefit the entire story.

2-The authors postulate that this novel mechanism could be part of a feedback loop. If this were the case one would expect the acute effect of FA to increase ER-export (or secretion) and the negative feedback will then reduce secretion. However, the acute effect of FA is not addressed in this manuscript. In order to postulate a feedback loop, the authors would need to test the individual nodes of this loop.

Significance

I think that the basic finding of this manuscript is highly novel, by showing the impact of the ECM and focal adhesions on COPII-dependent trafficking. I think that this will not only appeal to people from the trafficking community, but also to people working on cell migration and on mechanobiology. The work in its current form does not require much extra efforts (max. 3 month). However, if the authors would decide to increase the breadth of data, they would require 3-6 months.

Referees cross-commenting

I went through the comments of the two other reviewers and agree with their verdict. Some extra work on the characterization of the early secretory pathway would be good. Both reviewers provided a nice catalogue of possible experiments to choose from.

Note: This rebuttal was posted by the corresponding author to Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Reply to the reviewers

Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

1. General Statements

We want to thank all three reviewers for their positive feedback, constructive comments, and suggestions for clarity and improvement. We are delighted to find their consensus that the manuscript represents a contribution to the field.

Accordingly, we made changes in the text (all highlighted in blue in the revised manuscript) and added a new figure as detailed in the point-by-point response.

2. Point-by-point description of the revisions

Reviewer #1 (Evidence, reproducibility and clarity (Required)):

The authors describe results of the comprehensive analysis of the prevalence and functionality of intrinsically disordered regions of the pathogen-encoded signaling receptor Tir, which serves as an illustrative example of the bacterial effector proteins secreted by Attaching and Effacing (A/E) pathogens. This is an interesting and important study that represents an impressive amount of data generated computationally and using a broad spectrum of biophysical techniques. The work serves as a model of the well-designed and perfectly conducted study, where intriguing conclusions are based on the results of the comprehensive experiments. The manuscript is well-written and concise, and I have a real pleasure reading it. The text and figures are clear and accurate.

We thank the Reviewer for these positive comments on our work.

Although, in general, prior studies are referenced appropriately, the authors should mention that the pre-formed structural elements they found in Tir are in line with the concept of "PreSMos" (pre-structured motifs) previously introduced and described in several important studies from the laboratory of Kyou-Hoon Han.

We thank the Reviewer for this suggestion. We have added a sentence to acknowledge the presence of “PreSMos” in the target-free state of Tir as putative signatures for target-binding, referring to a review article summarizing several local structural elements in unbound IDPs:

“This supports the presence of pre-structured motifs (PreSMos) as pre-existing signatures for target binding and function within target-free Tir (72)**.”

Please, note that we decided to keep this discussion to a minimum, as we cannot rule out the contribution of the induced fit model to the binding mechanism (i.e., disorder-to-order transition upon binding).

Reviewer #1 (Significance (Required)):

Solid evidence is provided that structural disorder and short linear motifs represent common features of A/E pathogen effectors. In fact, using a set of bioinformatics tools, the authors first show that although prokaryotic proteins typically contain significantly less intrinsic disorder than eukaryotic proteins, A/E pathogen effectors are as disordered as eukaryotic proteins. Using the translocated intimin receptor (Tir) as a subject of focused study, the authors then utilized a number of biophysical techniques to draw an impressive picture of disorder-based functionality. This study clearly represents a major advancement in the field of functional intrinsic disorder in general and in disorder-based functionality of proteins expressed by pathogenic bacteria. This was adds significantly to the field and will have a noticeable impact.

Again, reading this manuscript was a real joy. Finally, this work perfectly fits in the area of my expertise, since for the past 25 years or so I am working on the different aspects of intrinsically disordered proteins.

Thank you for this encouraging assessment.

**Referee Cross-commenting**

I agree with the amended recommendation of reviewer #3 to add in the manuscript EPEC O127.

According to the suggestion of Reviewer #3, we have now included EPEC O127:H6 in the manuscript.

I completely agree with comments of reviewer #2 and partially agree with reviewer #3. In my view, comparison of various strains as references for EPEC represents an interesting but independent project. It can be recommended to the authors as one of the potential future developments of their work.

Thanks for the suggestion. We are pursuing that line of research.

Reviewer #2 (Evidence, reproducibility and clarity (Required)):

The general impression is that this is an excellent study that establishes

The C-terminal intracellular region of Tir called C-Tir spanning residues 338 to 550 is largely disordered, however, observe helical structural elements involved with lipid interactions; multi-phosphorylation. The intracellular N-terminal part of Tir called N-Tir spanning residues 1 to 233 is also partially disordered but include a folded domain that is shown to assemble into a dimer

The only major concern is that no SDS-PAGE gels or size exclusion chromatograms have been included to verify purity and monodispersed of the various constructs worked on. In particular, the SAXS and CD measurement is highly sensitive to purity, and the level of degradation as IDPs are notorious for being difficult to handle in solution. it would strengthen the arguments made based that

We produced N-Tir and C-Tir as fusion proteins with a cleavable N-terminal thioredoxin tag (Trx-His6) and C-terminal Strep-tag. The latter allowed us to purify them via Strep-tag affinity chromatography as indicated by SDS-PAGE (please see Fig. S1).

We agree with the Reviewer that even small amounts of impurities (i.e., higher oligomers/degradation) can interfere with the data analysis and make interpretation of the resulting data difficult and potentially misleading. So, to avoid such problems, all samples were purified in monodispersed forms by size-exclusion chromatography (SEC) before any biophysical study.

Following the Reviewer's suggestion, we added a new supplementary figure (Fig. S5) showing the SEC-SAXS chromatogram profiles of C-Tir, N-Tir, and NS-Tir. Briefly, in the inline SEC-SAXS experiment, the sample eluates from an HPLC system directly and continuously into a BioSAXS flow cell for subsequent X-ray interrogation. Under our experimental conditions, C-Tir elutes as a single peak with Rg-values and mass compatible with a disordered monomeric protein, providing an excellent fit to the experimental SAXS curves. For N-Tir and NS-Tir, by SEC-SAXS, we separated the dimer from small amounts of high-order oligomers to yield the experimental SAXS curves of the pure dimers.

“Fig. S5. SEC-SAXS chromatograms of (A) C-Tir, (B) N-Tir, and (C) NS-Tir. Each plane shows normalized total scattering intensity I(s), over the entire s range, from each frame acquired along elution volume and respective Rg-value (black circles). The flat variation of Rg reflects a pure monodisperse sample. The column type for size exclusion chromatography and sample concentrations are on the top left of each panel. For reference, the retention volume for monomeric BSA (66.4 kDa) is displayed by red triangles.”

**Minor Comments**

Read through the manuscript to remove passages with spoken language

We thank the Reviewer for this suggestion. We went through the manuscript and improved the writing to reduce passages with spoken language.

Line 263, "To do so", should be removed

Line 290 "Our data thus" replaced with "this"

We have amended the manuscript accordingly.

Line 292 "lipid bilayers that might potentially fine-tune Tir's activity in the host cell." Weak sentence and the word fine-tune is slang. Rewrite the sentence. The interaction with lipids is fascinating!

Thanks for the suggestion. The sentence has now been changed to “**This shows that C-Tir can undergo multivalent and tunable electrostatic interaction with lipid bilayers via pre-structured elements, suggesting that membrane-protein interplay at the intracellular side might control the activity and interactions of Tir in host cells.**”

We also reinforce this fascinating message in the abstract by adding the sentence: “Membrane affinity is residue-specific and modulated by lipid composition, suggesting a previously unrecognized mechanism for interaction with the host.”

Line 192 "In figure Fig. 3A," remove the Fig

Fixed.

Line 326, "In a similar fashion," is redundant. Rewrite the sentences below.

We have modified the sentence as follows: “We evaluated whether the N-terminal cytosolic region of Tir (N-Tir; Fig S1) was also intrinsically disordered ...”

Line 342 add spaces between digit and SI unit "52kDa" there are more cases of this.

Thank you for pointing this out. This has now been corrected to 52 kDa.

Reviewer #2 (Significance (Required)):

I expect this study to have broad relevance to microbiologists working with the intimin and translocated intimin receptor, in particular the lipid interaction is likely to be followed up by the community.

We thank the reviewer for this comment. Indeed, we believe that further studies on Tir's lipid-binding ability as a novel molecular strategy in host-pathogen interactions, will potentially provide new insights on virulence, transmembrane signaling in general, and disorder-mediated functions.

**Referee Cross-commenting**

What reviewer 3 suggested in the comments sounds like added value and should be included.

I agree with reviewer 1, that the strain comparison potentially is beyond the scope presented in this manuscript.

We have now included EPEC O127:H6 in the manuscript.

Reviewer #3 (Evidence, reproducibility and clarity (Required)):

**Summary:**

This interesting manuscript look at the structure of the Nter and Cter of the effector Tir from enteropathogenic E. coli. The authors confirmed previous study highlighting the "disordered" part of the Cter. However, the extended experimental work (NMR, Small-angle X-ray scattering and CD spectroscopy) from this study also reveals the connection between different area of Tir and its implication during Tir phosphorylation and its interactions with SH2 domain.

We thank the Reviewer for this positive remark. Indeed, in our work, we highlight the structural features of the SH2-mediated interaction between Tir and host SHP-1 protein, and we also show that C-Tir is capable of lipid interaction via pre-structured motifs and that N-Tir is disordered but assembled into a dimer. Overall, we provide an updated and wide picture of Tir's intracellular side that goes beyond the scrutiny of previously described disorder features.

**Major Comments:**

The authors used E2348/69 (O127:H7) strain as a reference for EPEC. However, this strain are the least effectors of all the EPEC sequences and may over estimated the PDR in EPEC. It would be wiser to use a strain like B171 as a reference for EPEC to be able to conclude "Disordered Proteins (PDR) with long disordered regions occur in EPEC effectors similar to the human proteome". I believe that the PDR in EPEC is similar to EHEC and CR. I do not have any major concern for the rest of the work.

We thank the Reviewer for this comment. So, to clarify, we amended “EPEC” with “EPEC O127:H6” in text and figures.

We also added a paragraph at the beginning of the Discussion section to acknowledge that our prediction analysis concerns EPEC O127:H6 and two additional representative A/E bacteria strains:

“Among the enteropathogenic Escherichia coli strains EPEC O127:H6 (E2348/69) is commonly used as a prototype strain to study EPEC biology, genetics, and virulence (69). Here, we have determined the structural disorder propensity of EPEC O127:H6 sequences and two additional representatives of A/E bacteria: EHEC O157:H7 and CR ICC168.”

Finally, the Reviewer suggests to include EPEC strain B171 (serotype O111:NM) in our analysis. We agree that considering additional strains would be of value, however we believe that this is beyond the scope of this manuscript, which mainly focuses on the characterization of the structural features of the E2348/69 Tir effector. We are currently working on a broader comparative analysis among different Escherichia coli pathogenic strains, including B171, and we hope to share our findings with the community in the near future.

**Minor comments**

Statistic problem: Mann Whitney U Test (Wilcoxon Rank Sum Test) is a comparison of two independent samples with the underlying assumption is normally distributed or that the samples were sufficiently large. It is not certain that any of this assumption is correct. In addition, the effector are part of the whole proteome. Can it be then considered that both groups are independent?

We thank the Reviewer for this remark, which allows us to clarify the choice of this particular test. Indeed the Mann Whitney U-test is a non-parametric test to compare two samples with the alternative hypothesis being that one of the two samples is stochastically greater than the other. As it is a nonparametric test samples are not required to be normally distributed, as it is for the Student t-test.

Regarding the independence of the samples, when comparing the effectors collections to their corresponding proteomes, we did exclude the effectors sequences from the latter. We have clarified this point in the Supplementary Material and Methods section.

Line 120 and 442: O127 not H127

Thank you for pointing this out. It has now been corrected to O127.

Line 212: positions 409 or 405?

Yes, it should be 405. Thank you.

Reviewer #3 (Significance (Required)):

**Nature and significance:**

Tir plays a major role during EPEC infection. It is a signalling platform that has been reported to interact with multiple proteins. Whereas the extracellular part has been well characterised and crystallised, the intracellular part has been proven so far to be difficult to study. Over the last decade, no progress has been made to explain how Tir works. This manuscript provides interesting information that shade some light on how the protein could work.

**Existing literature:**

The last research manuscript trying to highlight the structural function of Tir dates from 2007 (PMC1896257). This study is far more extended and in depth than any other previous work done.

**Audience:**

the Audience may probably limited to researcher working on the field of cellular microbiology and the function associated with bacterial effector in the host. This study could be also a useful tool to identify new effectors base on their "disorder".

We thank the Reviewer for recognizing the importance of this study. We agree that our work highlights the pivotal role of disordered regions in bacterial effectors, thus enabling a better understanding of the molecular mechanisms used by pathogens to subvert the host-cell processes. We indeed believe that our work can stimulate further research on the characterization of intrinsically disordered effectors, and also beyond the cellular microbiology field, in order to gain a broader knowledge on the molecular dialogue at the host-pathogen interface, which is essential to design better therapeutic strategies.

**Expertise:**

I have been working on A/E pathogens for the last 15 years with a particular interest in Tir signalling. My domain of expertise is more in relation to cell signalling than crystallography or structural study.

**Referee Cross-commenting**

I agree with both reviewers. My comment about EPEC is more about the conclusion for some of the figures. I don't think they should conclude for the whole EPEC. The Tir variation among EHEC O157:H7 is low, but it is far more diverse for EPEC. Simply adding in the manuscript EPEC O127 should be enough.

We thank the Reviewer for this comment. As mentioned above, we now state in the manuscript, in both Results and Discussion sections, that we used E2348/69 as a representative strain for EPEC.

Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Referee #3

Evidence, reproducibility and clarity

Summary:

This interesting manuscript look at the structure of the Nter and Cter of the effector Tir from enteropathogenic E. coli. The authors confirmed previous study highlighting the "disordered" part of the Cter. However, the extended experimental work (NMR, Small-angle X-ray scattering and CD spectroscopy) from this study also reveals the connection between different area of Tir and its implication during Tir phosphorylation and its interactions with SH2 domain.

Major Comments:

The authors used E2348/69 (O127:H7) strain as a reference for EPEC. However, this strain are the least effectors of all the EPEC sequences and may over estimated the PDR in EPEC. It would be wiser to use a strain like B171 as a reference for EPEC to be able to conclude "Disordered Proteins (PDR) with long disordered regions occur in EPEC effectors similar to the human proteome". I believe that the PDR in EPEC is similar to EHEC and CR. I do not have any major concern for the rest of the work.

Minor comments

Statistic problem: Mann Whitney U Test (Wilcoxon Rank Sum Test) is a comparison of two independent samples with the underlying assumption is normally distributed or that the samples were sufficiently large. It is not certain that any of this assumption is correct. In addition, the effector are part of the whole proteome. Can it be then considered that both groups are independent?

Line 120 and 442: O127 not H127

Line 212: positions 409 or 405?

Significance

Nature and significance:

Tir plays a major role during EPEC infection. It is a signalling platform that has been reported to interact with multiple proteins. Whereas the extracellular part has been well characterised and crystallised, the intracellular part has been proven so far to be difficult to study. Over the last decade, no progress has been made to explain how Tir works. This manuscript provides interesting information that shade some light on how the protein could work.

Existing literature:

The last research manuscript trying to highlight the structural function of Tir dates from 2007 (PMC1896257). This study is far more extended and in depth than any other previous work done.

Audience:

the Audience may probably limited to researcher working on the field of cellular microbiology and the function associated with bacterial effector in the host. This study could be also a useful tool to identify new effectors base on their "disorder".

Expertise:

I have been working on A/E pathogens for the last 15 years with a particular interest in Tir signalling. My domain of expertise is more in relation to cell signalling than crystallography or structural study.

Referee Cross-commenting

I agree with both reviewers. My comment about EPEC is more about the conclusion for some of the figures. I don't think they should conclude for the whole EPEC. The Tir variation among EHEC O157:H7 is low, but it is far more diverse for EPEC. Simply adding in the manuscript EPEC O127 should be enough.

Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Referee #2

Evidence, reproducibility and clarity

The general impression is that this is an excellent study that establishes The C-terminal intracellular region of Tir called C-Tir spanning residues 338 to 550 is largely disordered, however, observe helical structural elements involved with lipid interactions; multi-phosphorylation. The intracellular N-terminal part of Tir called N-Tir spanning residues 1 to 233 is also partially disordered but include a folded domain that is shown to assemble into a dimer

The only major concern is that no SDS-PAGE gels or size exclusion chromatograms have been included to verify purity and monodispersed of the various constructs worked on. In particular, the SAXS and CD measurement is highly sensitive to purity, and the level of degradation as IDPs are notorious for being difficult to handle in solution. it would strengthen the arguments made based that

Minor Comments

Read through the manuscript to remove passages with spoken language

Line 263, "To do so", should be removed

Line 290 "Our data thus" replaced with "this"

Line 292 "lipid bilayers that might potentially fine-tune Tir's activity in the host cell." Weak sentence and the word fine-tune is slang. Rewrite the sentence. The interaction with lipids is fascinating!

Line 192 "In figure Fig. 3A," remove the Fig

Line 326, "In a similar fashion," is redundant. Rewrite the sentences below.

Line 342 add spaces between digit and SI unit "52kDa" there are more cases of this.

Significance

I expect this study to have broad relevance to microbiologists working with the intimin and translocated intimin receptor, in particular the lipid interaction is likely to be followed up by the community.

Referee Cross-commenting

What reviewer 3 suggested in the comments sounds like added value and should be included.

I agree with reviewer 1, that the strain comparison potentially is beyond the scope presented in this manuscript.

Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Referee #1

Evidence, reproducibility and clarity

The authors describe results of the comprehensive analysis of the prevalence and functionality of intrinsically disordered regions of the pathogen-encoded signaling receptor Tir, which serves as an illustrative example of the bacterial effector proteins secreted by Attaching and Effacing (A/E) pathogens. This is an interesting and important study that represents impressive amount of data generated computationally and using a broad spectrum of biophysical techniques. The work serves as a model of the well-designed and perfectly conducted study, where intriguing conclusions are based on the results of the comprehensive experiments. The manuscript is well-written and concise, and I have a real pleasure reading it. The text and figures are clear and accurate.

Although, in general, prior studies are referenced appropriately, the authors should mention that the pre-formed structural elements they found in Tir are in line with the concept of "PreSMos" (pre-structured motifs) previously introduced and described in several important studies from the laboratory of Kyou-Hoon Han.

Significance

Solid evidence is provided that structural disorder and short linear motifs represent common features of A/E pathogen effectors. In fact, using a set of bioinformatics tools, the authors first show that although prokaryotic proteins typically contain significantly less intrinsic disorder than eukaryotic proteins, A/E pathogen effectors are as disordered as eukaryotic proteins. Using the translocated intimin receptor (Tir) as a subject of focused study, the authors then utilized a number of biophysical techniques to draw an impressive picture of disorder-based functionality. This study clearly represents a major advancement in the field of functional intrinsic disorder in general and in disorder-based functionality of proteins expressed by pathogenic bacteria. This was adds significantly to the field and will have a noticeable impact.

Again, reading this manuscript was a real joy. Finally, this work perfectly fits in the area of my expertise, since for the past 25 years or so I am working on the different aspects of intrinsically disordered proteins.

Referee Cross-commenting

I agree with the amended recommendation of reviewer #3 to add in the manuscript EPEC O127.

I completely agree with comments of reviewer #2 and partially agree with reviewer #3. In my view, comparison of various strains as references for EPEC represents an interesting but independent project. It can be recommended to the authors as one of the potential future developments of their work.

Note: This rebuttal was posted by the corresponding author to Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Reply to the reviewers

Author Response to Reviewer Comments

Review Commons

Manuscript number: RC-2021-00979

Corresponding author(s): Horvitz, H Robert

Reviewer #1:

Major comments: The manuscript is very well written and results have been very clearly presented. The key conclusions drawn by the authors are convincing. However, one of the claims by the authors is not supported by the data. In lines 206-215 the authors discuss experiments where they visualized the morphology of the AIAs in ctbp-1 mutants where ctbp-1 expression is restored temporally in the L4-young adult stage using a heat-shock promoter construct. The authors conclude that "ctbp-1 can act ... in older worms to maintain aspects of AIA morphology in a manner similar to AIA gene expression." However, the data presented in Fig. 3I-L show no statistically significant difference between ctbp-1 mutants and mutants with the HS-construct, either with and without heat shock. Thus, although there seems to be some effect of the heat shock, this is not significant and thus does not support the conclusion of the authors. In addition, an important control is missing. How does the heat shock affect the morphology of AIAs in wt or ctbp-1 animals, without the hs-construct?

We agree with this comment and have updated the manuscript to clarify that suggestion of the activity of CTBP-1 in preventing further disruption of AIA morphology is speculative. We will conduct the suggested control experiment and include the results in a revised version of the manuscript.

Apart from the above, all strong claims by the authors are valid. In addition, the authors suggest a mechanism, where CTBP-1 regulates the function of the EGL-13 transcription factor in AIA and that overexpression of CEH-28 in AIA contributes to the olfactory adaptation defect observed in the ctbp-1 mutant animals. These mechanistic speculations could be relatively easily strengthened by two additional experiments. One, does ctbp-1 loss of function affect egl-13 expression? The model presented in Fig 8 suggests that egl-13 expression levels are not affected, but from the data in the paper it is not even clear of egl-13 is expressed in AIA. Whether egl-13 is expressed in AIA, and if its expression levels are affected by mutation of ctbp-1 could be tested using egl-13::gfp expressing animals.

This is an excellent suggestion and experiments we had been attempting already. We will include findings from these experiments once they are complete in a revised version of the manuscript.

Two, does overexpression of ceh-28 cause an olfactory adaptation defect? This could be tested by cell specific overexpression of ceh-28 in AIA.

This is also a great suggestion. We will conduct this experiment and include the findings in a revised manuscript.

The data and the methods have been presented in such a way that they can be reproduced. I do have some doubts with regard to the statistical analysis. The authors report that statistical analysis involved unpaired t-tests. But as all results involve the analysis of data from 3-5 different strains, a multiple sample analysis should be used. To correct for the number of samples, one should first use an ANOVA to test for statistical differences, followed by a post hoc analysis to identify those that are significantly different.

We agree with this criticism. We have replaced instances of multiple sample analyses with a one-way ANOVA test followed by Tukey’s multiple test correction. The current version of the manuscript reflects these changes in figures, figure legends and in the Materials and Methods.

Reviewer #2: \*Major comments:**

1. The paper is well written and figures are clearly organized. The authors made suitable conclusions based on the data provided. Materials and methods are appropriately described for reproductivity.*

We agree and are currently attempting such experiments. Meaningful results from these experiments will be included in a revised manuscript.

Reviewer #3 (Evidence, reproducibility and clarity (Required)): \*Major comments:**

• The key conclusions of this manuscript are highly convincing and are supported by multiple mutant alleles and rescue experiments.*

*

• There are certain claims in the manuscript that need to be clarified (detailed below).*

*

• No additional experiments are essential to support the claims of the paper.*

- Most of the data and the methods presented well - however a Table listing genes identified in the AIA-specific RNA Seq is required. The GEO accession number has been made available for the RNA Sequencing data however listed the genes identified would aid the reader. Were ctbp-1 and egl-13 shown to be expressed in the AIAs using this approach?

We have included such a table, replacing Fig. S6 (which previously showed only ceh-28 expression) with a table listing expression of all confirmed hits from the scRNA-Seq experiment. ctbp-1 and egl-13 were also found to be expressed in the AIA neurons in this scRNA-Seq experiment.

- No evidence is presented that EGL-13 is expressed in the AIAs?

As noted above, the scRNA-Seq experiment showed egl-13 expression in the AIAs. We also will assay egl-13 expression in the AIAs using a GFP reporter and include the results in a revised manuscript.

- Can the authors comment and include in the manuscript information regarding whether the promoters of AIA-expressed genes that are regulated by EGL-13 contain EGL-13 binding sites? Also, are the promoters of AIA-expressed genes not regulated by EGL-13 missing these sites?

We have added such information to the manuscript. Briefly, our analysis identified no promising candidates for EGL-13 binding sites in the promoter regions of either ceh-28 or acbp-6, suggesting that regulation of these by EGL-13 is likely indirect. Further, no previous work has indicated that either of these genes is regulated directly by EGL-13, although in the case of acbp-6 little is known about this gene or the ways in which it is regulated. However, the claim that EGL-13 regulates expression of acbp-6 and ceh-28 indirectly is speculative and is not a conclusion of this current work.

- Experiments and statistical analysis are adequate.

Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Referee #3

Evidence, reproducibility and clarity

Summary:

In this manuscript, Saul et al. found that the CTB-1 transcriptional co-repressor acts cell-autonomously to maintain aspects of AIA neuronal fate, morphology and function. They found that CTBP-1 utilizes the Sox transcription factor EGL-13 to transcriptionally repress specific genes in the AIA neurons. This work proposes that CTBP-1 and other co-repressors play critical roles in selectively maintaining or repressing expression of specific genes.

Major comments:

• The key conclusions of this manuscript are highly convincing and are supported by multiple mutant alleles and rescue experiments.
• There are certain claims in the manuscript that need to be clarified (detailed below).
• No additional experiments are essential to support the claims of the paper.
• Most of the data and the methods presented well - however a Table listing genes identified in the AIA-specific RNA Seq is required. The GEO accession number has been made available for the RNA Sequencing data however listed the genes identified would aid the reader. Were ctbp-1 and egl-13 shown to be expressed in the AIAss using this approach?
• No evidence is presented that EGL-13 is expressed in the AIAs?
• Can the authors comment and include in the manuscript information regarding whether the promoters of AIA-expressed genes that are regulated by EGL-13 contain EGL-13 binding sites? Also, are the promoters of AIA-expressed genes not regulated by EGL-13 missing these sites?
• Experiments and statistical analysis are adequate.

Minor comments:

I list below a number of changes and typographical errors that will improve the manuscript.

Page 11 Line 235 - the authors state that ctbp-1 L4s have an increased attraction to butanone. As the chemotaxis index is 0 for the ctbp1- mutant compared to -0.5 in WT I understand what the authors mean hear but the statement of "increased attraction" suggests that ctbp1- mutants are attracted to butanone when they are actually ambivalent to it.

Page 12 Line 248 - change functioning to functional

Page 18 Line 397 - it would be helpful to the reader if the authors referred back to the ctbp1- mutant data (Figure 5) for comparison in Fig 7D.

Page 19 Line 404 - remove the word causally

Page 19 Line 414 "However, while conditioned ctbp-1 ceh-28 double mutants appeared similar to both the wild type and ctbp-1 single mutants at the L1 stage (Fig. 7I-J), these double mutants displayed an intermediate phenotype between wild-type and ctbp-1 animals for adaptation at the L4 larval stage (Fig. 7K-L).

This sentence is confusing as the ctbp-1 ceh-28 phenotype is not significant different to the ctbp-1 single mutant.

Page 50 Line 1001 - change mlg-1 to mgl-1

Figure 7A-C - please label with the genotype examined.

Significance

• This work identifies a function for the transcriptional corepressor CTBP-1 in controlling the expression of a subset of genes in the AIA neurons. It suggests that CTBP-1 may play a similar role in controlling subsets of gene expression in diverse neuronal classes. This would be interesting to examine in single cell sequencing experiments of all C. elegans neurons.
  • This work adds to the literature that describes CTBP-1 functions in the C. elegans nervous system. It also speculates that other transcriptional co-repressors play similar functions in other cells and tissues in other organisms.
  • An audience with interests in cell fate determination and the function of specific gene regulatory modules that control subsets of genes within a cell.
  • My field of expertise is C. elegans neurobiology (axon guidance and cell fate) and I am therefore well-qualified to review this manuscript.

Referee Cross-commenting

Comments from other reviewers are fair. I am happy with the overall conclusions.

Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Referee #2

Evidence, reproducibility and clarity

In this paper, the authors identified several mutations from a forward genetic screen in the transcriptional corepressor gene ctbp-1 that cause mixexpression of a M4 neuronal marker in the two AIA interneurons in C. elegans. ctbp-1 mutant AIA neurons also display a defect in morphology and sensory function. The penetrance and severity of these defects in gene expression, morphology, and function progressively increase with age. Their data suggests that ctbp-1 acts cell-autonomously and in older worms to maintain gene expression, morphology, and function in AIA neurons. Single-cell RNA sequencing was performed to identify changes in AIA transcriptional profiles between wild type and ctbp-1 mutants. Using the data from AIA transcriptional profiles, they showed that ctbp-1 mutant AIA neurons lose the expression of two genes characteristic of the adult AIA while misexpress at least two genes uncharacteristic of AIA. Taken together, their findings demonstrate that ctbp-1 acts to maintain the AIA identity at the level of gene expression, morphology, and function, while ctbp-1 does not act to establish the AIA cell identity. Furthermore, the authors identified a few mutations of a SOX family transcription factor gene egl-13 from a froward genetic screen that suppress the ctbp-1 mutant phenotype. The authors conclude their results that ctbp-1 maintains AIA function and some aspects of AIA gene expression by antagonizing egl-13 function and that ctbp-1 maintains AIA morphology through pathways independently of egl-13.

Major comments:

1. The paper is well written and figures are clearly organized. The authors made suitable conclusions based on the data provided. Materials and methods are appropriately described for reproductivity.
2. It would strengthen the model (Figure 8) by testing physical interaction between CTBP-1 and EGL-13 in AIA using BiFC.

Minor comments:

1. The authors mentioned a previous finding that the mammalian ortholog of EGL-13, SOX6, interacts with the mammalian ortholog of CTBP-1, CtBP2. The authors should also discuss the function of interacting SOX6 and CTBP-1 in mammalian systems.
2. It would be good to increase the font size of some figures and tables for easier reading.

Significance

This study identifies roles of conserved transcriptional corepressor CTBP-1 and a SOX family transcription factor gene egl-13 from unbiased forward genetic screens in the maintenance of AIA interneurons in C. elegans.

Since CTBP-1 and EGL-13 have mammalian orthologs, although the roles of their mammalian orthologs were not discussed, this study may have broad implications for development in a range of organisms.

The findings of this study will be of interest to a broad audience in the field of developmental biology, particularly in transcriptional regulation of cell identity maintenance.

I have expertise in transcriptional regulation of sensory neuron diversification using C. elegans as a model. I am comfortable about evaluating this manuscript.

Referee Cross-commenting

I agree with the comments from reviewers 1 and 3.

Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Referee #1

Evidence, reproducibility and clarity

In this manuscript, Saul et al identify the transcriptional corepressor ctbp-1 as a regulator of ceh-28::gfp expression in the AIA neurons of the nematode C. elegans. They find 18 independent mutants in this gene, including several presumptive null alleles. Using cell specific rescue and temporal expression of the ctbp-1 gene, the authors show that ctbp-1 acts cell autonomously in AIA to regulate ceh-28 expression, and can do so in young adult animals. Next, using various reporters they show that the AIAs do not transdifferentiate to M4-like cells, but the AIAs do show morphological defects, which increase with age of the animal. Using behavioral experiments the authors next determine the functionality of the AIAs in the ctbp-1 mutant animals. They find that loss of ctbp-1 in AIA affects the function of the AIAs and that ctbp-1 does so on young adult animals. The authors conclude the characterization of the AIAs of ctbp-1 mutant animals by identifying several other genes whose expression is misregulated in ctbp-1 animals, using a single cell RNAseq experiment, confirmed using gfp-fusion constructs. These experiments identity one other gene, acbp-6, that is misexpressed in the AIAs of L4 ctbp-1 animals and 2 genes, sra-11 and glr-2 that are normally expressed in AIA, but not in ctbp-1 animals.

To find out how ctbp-1 regulates gene expression in AIA, the authors perform a genetic suppressor screen and show that loss of function of egl-13 suppresses the ceh-28::gfp misexpression in AIA in ctbp-1 mutants. They show that egl-13 functions cell-autonomously in the AIAs. They find it does not suppress the morphological defects of the AIAs in ctbp-1 mutants, but it does suppress the effect of ctbp-1 loss of function on olfactory adaptation. In addition, mutation of egl-13 suppressed the misexpression of acbp-6, but not that of sra-11 and glr-2. Finally, the authors show that the olfactory adaptation defect observed in ctbp-1 mutant animals can be partially suppressed by inactivating ceh-28 suggesting that the behavioral defect is caused in part by overexpression of ceh-28.

The manuscript is very well written and results have been very clearly presented. The key conclusions drawn by the authors are convincing. However, one of the claims by the authors is not supported by the data. In lines 206-215 the authors discuss experiments where they visualized the morphology of the AIAs in ctbp-1 mutants where ctbp-1 expression is restored temporally in the L4-young adult stage using a heat-shock promoter construct. The authors conclude that "ctbp-1 can act ... in older worms to maintain aspects of AIA morphology in a manner similar to AIA gene expression." However, the data presented in Fig. 3I-L show no statistically significant difference between ctbp-1 mutants and mutants with the HS-construct, either with and without heat shock. Thus, although there seems to be some effect of the heat shock, this is not significant and thus does not support the conclusion of the authors. In addition, an important control is missing. How does the heat shock affect the morphology of AIAs in wt or ctbp-1 animals, without the hs-construct?

Apart from the above, all strong claims by the authors are valid. In addition, the authors suggest a mechanism, where CTBP-1 regulates the function of the EGL-13 transcription factor in AIA and that overexpression of CEH-28 in AIA contributes to the olfactory adaptation defect observed in the ctbp-1 mutant animals. These mechanistic speculations could be relatively easily strengthened by two additional experiments. One, does ctbp-1 loss of function affect egl-13 expression? The model presented in Fig 8 suggests that egl-13 expression levels are not affected, but from the data in the paper it is not even clear of egl-13 is expressed in AIA. Whether egl-13 is expressed in AIA, and if its expression levels are affected by mutation of ctbp-1 could be tested using egl-13::gfp expressing animals.

Two, does overexpression of ceh-28 cause an olfactory adaptation defect? This could be tested by cell specific overexpression of ceh-28 in AIA.

These are relatively simple experiments that would not take much time or investments, but would strengthen or clarify the model presented.

The data and the methods have been presented in such a way that they can be reproduced. I do have some doubts with regard to the statistical analysis. The authors report that statistical analysis involved unpaired t-tests. But as all results involve the analysis of data from 3-5 different strains, a multiple sample analysis should be used. To correct for the number of samples, one should first use an ANOVA to test for statistical differences, followed by a post hoc analysis to identify those that are significantly different.

Minor comments:

Page 7, in the heat shock rescue experiment that authors conclude that ctbp-1 acts "in older worms" to prevent expression of ceh-28 in AIA. "Older" is quite unspecific. Please be specific, i.e. in L4-young adult animals. The same applies to various other phrases where "older" worms are mentioned. Line 229, the authors state that animals were "briefly starved". Please be precise and indicate how long the animals were starved.

Significance

Most studies that address cell fate, focus on the first phase where cell fate is determined. How cell fate is maintained is far less well understood. This manuscript convincingly identifies two transcription regulators that are important for cell fate maintenance, both a transcriptional repressor and an activator. The manuscript provides first clues as to how this process functions, and as such provides important conceptual insights. These not only apply to the worm, C. elegans, but as these are strongly conserved proteins, probably also provide a firm basis for our understanding of cell fate maintenance mechanisms in higher organisms including mammals. In addition, this study reports an excellent model that can be used to further unravel this mechanism. As such, I expect that this manuscript will be of interest to a broad range of scientists, interested in cell fate determination and maintenance and transcriptional control.

My expertise lies in C. elegans behavior, where we focus on identification of the molecular and cellular mechanisms that allow C. elegans to respond to its environment even in changing circumstances. In addition, we study the mechanisms of cell fate determination and maintenance in C. elegans sensory neurons.

Referee Cross-commenting

I agree with the comments of reviewers 2 and 3.

Transparent Peer Review

---

Download the complete Review Process [PDF] including:

• reviews
• authors' reply
• editorial decisions

</br>

Transparent Peer Review

---

Download the complete Review Process [PDF] including:

• reviews
• authors' reply
• editorial decisions

</br>

Transparent Peer Review

---

Download the complete Review Process [PDF] including:

• reviews
• authors' reply
• editorial decisions

</br>

Note: This rebuttal was posted by the corresponding author to Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Reply to the reviewers

Reviewer #1 (Evidence, reproducibility and clarity (Required)):

**Summary:**

Provide a short summary of the findings and key conclusions (including methodology and model system(s) where appropriate).

Authors developed a novel primer/probe set for detection of subgenomic (sgE) transcripts for SARS-CoV-2 with the aim to develop a system that may predict the presence of infectious virus in patient samples. After studying the specificity and sensitivity of their system, they compared it with already validated/published systems for diagnostic of SARS-CoV-2 infection. Interestingly, they also studied the effect of the conditions of isolation. They showed Vero E6 expressing TMPRSS2 (Vero E6-TMPRSS2) to be more sensitive to infection than Vero E6, allowing a higher number of isolation from patient samples. They also showed their system to be more sensitive than a previously published sgE system as well as than a negative-strand RNA assay but less sensitive than the WHO/Charité primer/probe set. Anyway, all samples containing infectious particles (successful virus isolation on Vero E6-TMPRSS2) were detected with their primer/probe system contrary to the other tested sgE assay. They showed the negative strand assay to be unlikely to detect virus genetic material in samples which nevertheless contain infectious particles.

**Major comments:**

Are the key conclusions convincing?

I salute the intention of the authors to try to fix cut-off values for infectious patients but I would be more careful on the assertion of "using a total viral RNA Ct cut-off of >31 or specifically testing for sgRNA can serve as an effective rule-out test for viral infectivity". It is true that in this study, virus was not isolated from any of the samples below a Ct of 31 or negative in the developed sgE assay but all those assays are done on cell culture. We do not know how the transmission could occur for those samples from human to human. Being able to fix a cut-off in Ct value for a define PCR/RT-PCR system would be a great improvement for SARS-CoV-2 infected patient having to stay in quarantine. It is even more important for Ebola positive patients in Africa who has to stay in quarantine in precarious conditions under tents, warm temperatures and without privacy for long period because they still positive by RT-PCR. Unfortunately, fix those values would need a very high number of experiments, including animal experiment.

We appreciate the reviewer’s acknowledgment of the significance of this issue. We agree that in vivo animal experiments to more precisely determine the lowest infectious or transmissible dose would be valuable. But such experiments are outside the scope of the current study. To acknowledge the reviewer’s important point regarding the unavoidable limitations of cell culture systems, we have modified the abstract (line 51) to say “an effective rule out test for the presence of culturable virus,” a conclusion that is fully supported by our data.

Should the authors qualify some of their claims as preliminary or speculative, or remove them altogether?

No

Would additional experiments be essential to support the claims of the paper? Request additional experiments only where necessary for the paper as it is, and do not ask authors to open new lines of experimentation.

No

Are the suggested experiments realistic in terms of time and resources? It would help if you could add an estimated cost and time investment for substantial experiments.

Yes.

Are the data and the methods presented in such a way that they can be reproduced?

Kinetic of SARS-CoV-2 (figure 2):

The method is not detailed in the Methods part and is not clear in the figure legend. When supernatant are collected, is it all the supernatant that is remove? An aliquot? If aliquot, do you replace with new medium?

We apologize for this omission and have included the requested details in the methods. We seed a separate well for each time point and collected the entire supernatant for a given time point, rather than replacing media. We added the following text to the methods section (lines 402-412): “Viral growth kinetics were measured in Vero E6 or Vero E6 TMPRSS2 cells at an MOI of 0.001. Separate wells were seeded for each time point, and growth curves were conducted in technical duplicates for each biological experiment. Supernatants and cell lysates were collected twice daily 1 & 2 dpi, and again on 3, 4, 7 and 8 dpi (Vero E6 TMPRSS2 cells were harvested for the final time at day 7 due to faster growth kinetics in this cell type). For each time point, the supernatant was removed and clarified to remove cellular debris, before being split into separate aliquots for RNA extraction (mixed 1:1 with AVE lysis buffer) and viral titration (by focus assay). Dead cells/debris that was pelleted after clarifying supernatants was combined with cells scraped from each well into PBS and spun again to obtain a pellet of all cell material from each timepoint. This pellet was then lysed in AVE viral lysis buffer for RNA extraction.”

Stability of infectious SARS-CoV-2:

I am very surprise by your results on stability of cultured virus, knowing we observed a decreased of SARS-CoV-2 titer in our lab after freezing/thawing steps. Do you freeze cell supernatant directly or do you prepare your samples another way? Please state it in the Methods part

We measured the stability after freeze/thaw for our normal high concentration viral stocks. Our viral stocks are grown in DMEM with 10% FBS, 1% HEPES, 1% pen/strep, and clarified before use. It is possible that lab-lab variation in the media components or HEPES concentration used to prepare viral stocks explains the differences seen in our work vs the reviewer’s lab. We have added the following additional detail to the methods section (lines 415-418) of the manuscript to clarify how these experiments were performed: “High concentration viral stocks (prepared as above in DMEM, 10% FBS, 1% HEPES, 1% pen/strep) were used to measure viral stability over time and after multiple freeze-thaw cycles. Stocks were stored at the indicated temperatures in the dark and aliquots were removed at the indicated days or after each freeze-thaw cycle for measuring infectious virus by focus assay.”

Are the experiments adequately replicated and statistical analysis adequate?

Yes

**Minor comments:**

Specific experimental issues that are easily addressable.

Figure 2C and D: Instead of Ct values in cells, it would be more relevant to normalize these results with an endogenous gene and present results as fold change to mock-infected cells. Because you affirm that the level of RNA decline than stay stable over the time but you also note there is CPE. If you have less cells but same level of viral RNA, it means you have an increase in the RNA level in alive cells.

We have measured the GAPDH level in these cells over time, and that data is included as gray lines in Fig 2 C&D (see new figure 2). As we are combining the cell pellet from clarified supernatants with the cells that remain adherent to the dish for each harvested timepoint we expect to be harvesting the majority of cells/cell debris for each time point. The levels of GAPDH remain broadly similar over the viral growth curve, with no drop in RNA levels.

It would have been interesting to have the results of isolation at different time-point of treatment for patient samples (figure 3A and B) to see if the virus is stable in samples

We have access to only limited volume (several hundred µl) of residual patient sample which would make it technically challenging to compare multiple days of storage conditions/ temperatures. Unfortunately, we do not have any remaining sample volume for the specimens used in this study, and so we are unable to perform additional isolations at other times/temperatures. While we agree this would be an interesting line of future inquiry, we feel it is outside the scope of the current study.

Are prior studies referenced appropriately? Yes

Are the text and figures clear and accurate?

Yes.

Line 140: "this delay in virus and RNA production". You do not talk about RNA yet...

We have removed “and RNA” from this sentence and replaced with “infectious virus production”.

Line 156 to 163: sgE RNA detected in cell free supernatant. Can't it come from lysed cells?

We have replaced “cell-free” with “clarified”.

Line 167: "...virus in cell culture time course experiment in TMPRRS2 expressing cells (fig.2)"

We have modified this text to read according to the Reviewer’s suggestion.

Ligne 258: Fig 6A and B

We have added the missing reference to Fig 6B as requested.

-Do you have suggestions that would help the authors improve the presentation of their data and conclusions? No

Reviewer #1 (Significance (Required)):

Describe the nature and significance of the advance (e.g. conceptual, technical, clinical) for the field.

This new primer/probe system will participate to the accurate diagnostic of SARS-CoV-2. The comparison with the existing methods is relevant to highlight the strengths and weaknesses of each system. Comparison of isolation of SARS-CoV-2 on commonly used Vero E6 with Vero E6-TMPRSS2 will lead to a great improvement of the isolation method for SARS-CoV-2.

We appreciate the Reviewer’s assessment of the significance of our study and the improvement in our isolation method compared to the existing standard of using Vero E6 cells.

Place the work in the context of the existing literature (provide references, where appropriate).

Properly done in the introduction of the paper.

State what audience might be interested in and influenced by the reported findings.

Diagnostic laboratories

Define your field of expertise with a few keywords to help the authors contextualize your point of view. Indicate if there are any parts of the paper that you do not have sufficient expertise to evaluate.

Virology, Molecular Biology, cell biology

Not enough expertise to evaluate ROC data/analysis

Reviewer #2 (Evidence, reproducibility and clarity (Required)):

**Summary:**

Bruce et al present a new RT-PCR assay with primer sets that specifically detect sgE RNA from SARS-CoV2 samples. The authors compare this assay to other diagnostic assays in an effort to identify assays capable of correlating RNA detection with culturable virus (i.e. infectious virus). While this new assay identified 100% of culturable isolates, only 56% of isolates testing positive actually had culturable virus. Compared with other assays, the WHO total E RNA assay had better parameters when used at a cutoff Ct value of 31 (PPV of 61%). Overall, this manuscript provides a novel primer probe set for RT-PCR diagnostic assay and conducted comparisons with other assays on the same clinical samples. There are some areas that the authors should address prior to publication.

**Major comments:**

The authors repeatedly tout VeroE6 TMRSS2 cells as supporting higher viral infection. Therefore, the authors should address why one clinical isolate (E16) was culturable in VeroE6 but not VeroE6 TMRSS2. Was this experiment repeated multiple times? What are the reasons for this discrepancy?

We did not have sufficient residual sample volume to repeat isolation attempts of any clinical specimen, so we are limited to a single data point for each cell line. It is possible that this sample had levels of infectious virus at the limit of detection, and stochastic probability meant infectious virus was only present in the aliquot used to infect the Vero E6 (rather than Vero E6-TMPRSS2) cells. It is also possible that viral adaptation/evolution occurred in the VeroE6 well that allowed this virus to successfully grow, but we do not have sequencing data or remaining nucleic acids to test this theory.

The authors' argument at lines 166-169 is not supported by the data in Fig. 2. The levels of viral RNA between VeroE6 and VeroE6 TMRSS2 appear to show similar trends in the supernatant across the time course but the infectious viral levels are dramatically different. This discordance between FFU levels and RNA levels cannot be explained by instability of viral particles alone. Have the authors looked into differences in viral particles produced from these two cell lines? The authors should collect virus particles from these two cell lines and conduct the stability experiment in Fig 2D to directly test the hypothesis that indeed the drop seen in FFU in VeroE6 TMRSS2 is due to instability.

We apologize for the confusion. We did not intend to make claims about differences in particle stability as a result of the cell line used for viral production, but rather to highlight a general observation that RNA was more stable than infectious virus. This is more obvious in the TMRPSS2 cell line, as replication is faster and more synchronized than in Vero E6 cells (the TMRPSS2 cells are largely dead by day 4, whereas infection progresses more slowly in Vero E6 cells so that new virions continue to be produced during the measured time period). We have added clarifying text at line 167-169, “We observed that SARS-CoV-2 RNA species persist for much longer than infectious virus in cell culture time course experiments, a feature that was most obvious in Vero E6 TMRPSS-2 cells due to their viral kinetics but is likely not cell specific (Fig 2).”

The evidence for the packaging of sgE RNA into virions is weak. GAPDH detection by PCR is not a proof that the concentration process did not pellet RNA nonspecifically. First, the authors should provide ample information about viral isolation process at line 379 including rotor, centrifuge and speed utilized. In addition, ribosomes typically stay intact following viral lysis (and can be found in supernatant after release from dead cells). Actively translating ribosomes can contain sgE RNA as well. The authors should consider detecting ribosomal RNAs in their samples to rule out the possibility of contaminating ribosomes. In addition, the authors should strongly consider repeating the experiment with high EDTA concentration to break up ribosomes and only pellet virions.

We have added additional experimental details (rotor, centrifuge and speed) describing how the viral concentration step was performed (line 389-394), “Viral RNA (courtesy of David Bauer, The Francis Crick Institute, UK) from concentrated SARS-CoV-2 (England02 strain, B lineage ‘Wuhan-like’) was obtained by clarifying viral supernatants (2 x 4000 rpm for 30 mins at 4°C in a Beckman Allegra X-30R centrifuge with a SX4400 rotor), overlaying clarified media onto a 30% sucrose/PBS cushion (1/4th tube volume) and concentrating by ultracentrifugation in a Beckman ultra XPN-90 centrifuge with SW32TI rotor for 90 min at 25,500 rpm at 4°C. Pellets were then resuspended in buffer and extracted with TRIzol LS.” We thank the reviewer for their suggestion of including an additional control, and we have added an 18S primer-probe set (see new Figure 8). This data, while not as pronounced as the GAPDH control, suggests that the ultracentrifugation step has removed significant amounts of 18S RNA (though the clarified supernatants retain similar amounts of 18S RNA as the cells, suggesting that clarification alone is not sufficient to remove contaminating ribosomes). While we agree that repeating the ultracentrifuge concentration with high concentrations of EDTA is an interesting line of inquiry we feel it is outside the scope of this manuscript (and we face additional technical restrictions to pursue this as we currently lack access to an ultracentrifuge at BSL-3). We have updated the discussion to include the possibility of residual ribosome-protected fragments of sgE as a potential alternative interpretation (line 350-352).

**Minor comments:**

At line 197, the authors refer to "viruses" with lower levels of SARS-CoV2 RNA. This is incorrect and should be changed to "isolates" as the SARS-CoV2 virus particle does not package variable amount of genomic RNA.

We have changed this to “clinical specimens” for clarity.

The authors statement on lines 210-212 does not seem to be supported clearly by Fig. 5. The authors should consider including trendlines as well as other analyses that help show the correlation between viral RNA vs FFU. In addition, the authors should label the Y-axis clearly for Fig. 5.

We have added clarifying labels to both the X and Y axes. Due to the limited sample volume we were unable to directly measure the infectious titers from the clinical samples used in this study, and thus the FFU/mL represents the titer post-isolation while the CT represents the amount of RNA pre-isolation. Nonetheless, we do see broad trends (ie, the colored dots are generally arranged in rainbow order from left to right, though we agree there is variation within this trend). We have also modified the text at lines 212-217 to reflect the reviewer’s concern- “Greater initial viral RNA levels was broadly associated with faster viral growth in both cell lines (seen in the progression of colors from left to right), however we saw significant variation within these trends. Our data suggests that when standard SARS-CoV-2 RNA RT-PCR values are the only available data for patient or population-level viral loads, they are useful in gauging the presence of infectious virus in patient NP samples (Fig 5).”

The authors should expand on the methodology for creating ROC curves at line 467.

We have included the following text in the methods section for ROC curve analysis:

“ROC curves were generated using R and plotted with the ggplot2 package

[43]. For each potential scoring marker (CT_e, CT_sge1, CT_sge2, neg_e,) samples were ordered by that marker, followed by culturable status. The false-positive rate was calculated as the cumulative count of culturable samples (after ordering by marker intensity) divided by the total count of culturable samples; the true positive rate was calculated as the cumulative count of non-culturable samples (after ordering) divided by the total count of non-culturable samples. The false positive rate was plotted on the X axis of the ROC curves and the true positive rate on the Y axis.”

Reviewer #2 (Significance (Required)):

This study is significant because it assesses the utility of several clinical assays for the measurement of viral RNA and correlating it with culturable virus. This is important in the field because it helps to identify methods whereby infectivity can be predicted from a simple diagnostic test. This is important to know as a virologist working in the SARS-CoV2 field. It is also important from a public health perspective to better define quarantine requirements for persons testing positive. While the study provided a new primer probe set, it appears that the already available WHO total E RNA assay is superior in predicting infectivity and this study provides further evidence to support this notion.

We appreciate the Reviewer’s assessment that this study is significant and provides information of high interest to SARS-CoV-2 virologists that also has important public health implications.

Reviewer #1 (Evidence, reproducibility and clarity (Required)):

**Summary:**

Provide a short summary of the findings and key conclusions (including methodology and model system(s) where appropriate). Authors developed a novel primer/probe set for detection of subgenomic (sgE) transcripts for SARS-CoV-2 with the aim to develop a system that may predict the presence of infectious virus in patient samples. After studying the specificity and sensitivity of their system, they compared it with already validated/published systems for diagnostic of SARS-CoV-2 infection. Interestingly, they also studied the effect of the conditions of isolation. They showed Vero E6 expressing TMPRSS2 (Vero E6-TMPRSS2) to be more sensitive to infection than Vero E6, allowing a higher number of isolation from patient samples. They also showed their system to be more sensitive than a previously published sgE system as well as than a negative-strand RNA assay but less sensitive than the WHO/Charité primer/probe set. Anyway, all samples containing infectious particles (successful virus isolation on Vero E6-TMPRSS2) were detected with their primer/probe system contrary to the other tested sgE assay. They showed the negative strand assay to be unlikely to detect virus genetic material in samples which nevertheless contain infectious particles.

**Major comments:**

Are the key conclusions convincing?

I salute the intention of the authors to try to fix cut-off values for infectious patients but I would be more careful on the assertion of "using a total viral RNA Ct cut-off of >31 or specifically testing for sgRNA can serve as an effective rule-out test for viral infectivity". It is true that in this study, virus was not isolated from any of the samples below a Ct of 31 or negative in the developed sgE assay but all those assays are done on cell culture. We do not know how the transmission could occur for those samples from human to human. Being able to fix a cut-off in Ct value for a define PCR/RT-PCR system would be a great improvement for SARS-CoV-2 infected patient having to stay in quarantine. It is even more important for Ebola positive patients in Africa who has to stay in quarantine in precarious conditions under tents, warm temperatures and without privacy for long period because they still positive by RT-PCR. Unfortunately, fix those values would need a very high number of experiments, including animal experiment.

We appreciate the reviewer’s acknowledgment of the significance of this issue. We agree that in vivo animal experiments to more precisely determine the lowest infectious or transmissible dose would be valuable. But such experiments are outside the scope of the current study. To acknowledge the reviewer’s important point regarding the unavoidable limitations of cell culture systems, we have modified the abstract (line 51) to say “an effective rule out test for the presence of culturable virus,” a conclusion that is fully supported by our data.

Should the authors qualify some of their claims as preliminary or speculative, or remove them altogether? No

Would additional experiments be essential to support the claims of the paper? Request additional experiments only where necessary for the paper as it is, and do not ask authors to open new lines of experimentation. No

Are the suggested experiments realistic in terms of time and resources? It would help if you could add an estimated cost and time investment for substantial experiments. Yes.

Are the data and the methods presented in such a way that they can be reproduced?

Kinetic of SARS-CoV-2 (figure 2): The method is not detailed in the Methods part and is not clear in the figure legend. When supernatant are collected, is it all the supernatant that is remove? An aliquot? If aliquot, do you replace with new medium?

We apologize for this omission and have included the requested details in the methods. We seed a separate well for each time point and collected the entire supernatant for a given time point, rather than replacing media. We added the following text to the methods section (lines 402-412): “**Viral growth kinetics were measured in Vero E6 or Vero E6 TMPRSS2 cells at an MOI of 0.001. Separate wells were seeded for each time point, and growth curves were conducted in technical duplicates for each biological experiment. Supernatants and cell lysates were collected twice daily 1 & 2 dpi, and again on 3, 4, 7 and 8 dpi (Vero E6 TMPRSS2 cells were harvested for the final time at day 7 due to faster growth kinetics in this cell type). For each time point, the supernatant was removed and clarified to remove cellular debris, before being split into separate aliquots for RNA extraction (mixed 1:1 with AVE lysis buffer) and viral titration (by focus assay). Dead cells/debris that was pelleted after clarifying supernatants was combined with cells scraped from each well into PBS and spun again to obtain a pellet of all cell material from each timepoint. This pellet was then lysed in AVE viral lysis buffer for RNA extraction.”

Stability of infectious SARS-CoV-2: I am very surprise by your results on stability of cultured virus, knowing we observed a decreased of SARS-CoV-2 titer in our lab after freezing/thawing steps. Do you freeze cell supernatant directly or do you prepare your samples another way? Please state it in the Methods part

We measured the stability after freeze/thaw for our normal high concentration viral stocks. Our viral stocks are grown in DMEM with 10% FBS, 1% HEPES, 1% pen/strep, and clarified before use. It is possible that lab-lab variation in the media components or HEPES concentration used to prepare viral stocks explains the differences seen in our work vs the reviewer’s lab. We have added the following additional detail to the methods section (lines 415-418) of the manuscript to clarify how these experiments were performed: “High concentration viral stocks (prepared as above in DMEM, 10% FBS, 1% HEPES, 1% pen/strep) were used to measure viral stability over time and after multiple freeze-thaw cycles. Stocks were stored at the indicated temperatures in the dark and aliquots were removed at the indicated days or after each freeze-thaw cycle for measuring infectious virus by focus assay.”

Are the experiments adequately replicated and statistical analysis adequate? Yes

**Minor comments:**

Specific experimental issues that are easily addressable.

Figure 2C and D: Instead of Ct values in cells, it would be more relevant to normalize these results with an endogenous gene and present results as fold change to mock-infected cells. Because you affirm that the level of RNA decline than stay stable over the time but you also note there is CPE. If you have less cells but same level of viral RNA, it means you have an increase in the RNA level in alive cells.

We have measured the GAPDH level in these cells over time, and that data is included as gray lines in Fig 2 C&D (see updated figure). As we are combining the cell pellet from clarified supernatants with the cells that remain adherent to the dish for each harvested timepoint we expect to be harvesting the majority of cells/cell debris for each time point. The levels of GAPDH remain broadly similar over the viral growth curve, with no drop in RNA levels.

It would have been interesting to have the results of isolation at different time-point of treatment for patient samples (figure 3A and B) to see if the virus is stable in samples

We have access to only limited volume (several hundred µl) of residual patient sample which would make it technically challenging to compare multiple days of storage conditions/ temperatures. Unfortunately, we do not have any remaining sample volume for the specimens used in this study, and so we are unable to perform additional isolations at other times/temperatures. While we agree this would be an interesting line of future inquiry, we feel it is outside the scope of the current study.

Are prior studies referenced appropriately? Yes

Are the text and figures clear and accurate? Yes.

Line 140: "this delay in virus and RNA production". You do not talk about RNA yet...

We have removed “and RNA” from this sentence and replaced with “infectious virus production”.

Line 156 to 163: sgE RNA detected in cell free supernatant. Can't it come from lysed cells?

We have replaced “cell-free” with “clarified”.

Line 167: "...virus in cell culture time course experiment in TMPRRS2 expressing cells (fig.2)"

We have modified this text to read according to the Reviewer’s suggestion.

Ligne 258: Fig 6A and B

We have added the missing reference to Fig 6B as requested.

-Do you have suggestions that would help the authors improve the presentation of their data and conclusions? No

Reviewer #1 (Significance (Required)):

Describe the nature and significance of the advance (e.g. conceptual, technical, clinical) for the field.

This new primer/probe system will participate to the accurate diagnostic of SARS-CoV-2. The comparison with the existing methods is relevant to highlight the strengths and weaknesses of each system. Comparison of isolation of SARS-CoV-2 on commonly used Vero E6 with Vero E6-TMPRSS2 will lead to a great improvement of the isolation method for SARS-CoV-2.

We appreciate the Reviewer’s assessment of the significance of our study and the improvement in our isolation method compared to the existing standard of using Vero E6 cells.

Place the work in the context of the existing literature (provide references, where appropriate). Properly done in the introduction of the paper.

State what audience might be interested in and influenced by the reported findings. Diagnostic laboratories

Define your field of expertise with a few keywords to help the authors contextualize your point of view. Indicate if there are any parts of the paper that you do not have sufficient expertise to evaluate. Virology, Molecular Biology, cell biology Not enough expertise to evaluate ROC data/analysis

Reviewer #2 (Evidence, reproducibility and clarity (Required)):

**Summary:**

Bruce et al present a new RT-PCR assay with primer sets that specifically detect sgE RNA from SARS-CoV2 samples. The authors compare this assay to other diagnostic assays in an effort to identify assays capable of correlating RNA detection with culturable virus (i.e. infectious virus). While this new assay identified 100% of culturable isolates, only 56% of isolates testing positive actually had culturable virus. Compared with other assays, the WHO total E RNA assay had better parameters when used at a cutoff Ct value of 31 (PPV of 61%). Overall, this manuscript provides a novel primer probe set for RT-PCR diagnostic assay and conducted comparisons with other assays on the same clinical samples. There are some areas that the authors should address prior to publication.

**Major comments:**

The authors repeatedly tout VeroE6 TMRSS2 cells as supporting higher viral infection. Therefore, the authors should address why one clinical isolate (E16) was culturable in VeroE6 but not VeroE6 TMRSS2. Was this experiment repeated multiple times? What are the reasons for this discrepancy?

We did not have sufficient residual sample volume to repeat isolation attempts of any clinical specimen, so we are limited to a single data point for each cell line. It is possible that this sample had levels of infectious virus at the limit of detection, and stochastic probability meant infectious virus was only present in the aliquot used to infect the Vero E6 (rather than Vero E6-TMPRSS2) cells. It is also possible that viral adaptation/evolution occurred in the VeroE6 well that allowed this virus to successfully grow, but we do not have sequencing data or remaining nucleic acids to test this theory.

The authors' argument at lines 166-169 is not supported by the data in Fig. 2. The levels of viral RNA between VeroE6 and VeroE6 TMRSS2 appear to show similar trends in the supernatant across the time course but the infectious viral levels are dramatically different. This discordance between FFU levels and RNA levels cannot be explained by instability of viral particles alone. Have the authors looked into differences in viral particles produced from these two cell lines? The authors should collect virus particles from these two cell lines and conduct the stability experiment in Fig 2D to directly test the hypothesis that indeed the drop seen in FFU in VeroE6 TMRSS2 is due to instability.

We apologize for the confusion. We did not intend to make claims about differences in particle stability as a result of the cell line used for viral production, but rather to highlight a general observation that RNA was more stable than infectious virus. This is more obvious in the TMRPSS2 cell line, as replication is faster and more synchronized than in Vero E6 cells (the TMRPSS2 cells are largely dead by day 4, whereas infection progresses more slowly in Vero E6 cells so that new virions continue to be produced during the measured time period). We have added clarifying text at line 167-169, “We observed that SARS-CoV-2 RNA species persist for much longer than infectious virus in cell culture time course experiments, a feature that was most obvious in Vero E6 TMRPSS-2 cells due to their viral kinetics but is likely not cell specific (Fig 2).”

The evidence for the packaging of sgE RNA into virions is weak. GAPDH detection by PCR is not a proof that the concentration process did not pellet RNA nonspecifically. First, the authors should provide ample information about viral isolation process at line 379 including rotor, centrifuge and speed utilized. In addition, ribosomes typically stay intact following viral lysis (and can be found in supernatant after release from dead cells). Actively translating ribosomes can contain sgE RNA as well. The authors should consider detecting ribosomal RNAs in their samples to rule out the possibility of contaminating ribosomes. In addition, the authors should strongly consider repeating the experiment with high EDTA concentration to break up ribosomes and only pellet virions.

We have added additional experimental details (rotor, centrifuge and speed) describing how the viral concentration step was performed (line 389-394), “Viral RNA (courtesy of David Bauer, The Francis Crick Institute, UK) from concentrated SARS-CoV-2 (England02 strain, B lineage ‘Wuhan-like’) was obtained by clarifying viral supernatants (2 x 4000 rpm for 30 mins at 4°C in a Beckman Allegra X-30R centrifuge with a SX4400 rotor), overlaying clarified media onto a 30% sucrose/PBS cushion (1/4th tube volume) and concentrating by ultracentrifugation in a Beckman ultra XPN-90 centrifuge with SW32TI rotor for 90 min at 25,500 rpm at 4°C. Pellets were then resuspended in buffer and extracted with TRIzol LS.” We thank the reviewer for their suggestion of including an additional control, and we have added an 18S primer-probe set (see new Figure 8). This data, while not as pronounced as the GAPDH control, suggests that the ultracentrifugation step has removed significant amounts of 18S RNA (though the clarified supernatants retain similar amounts of 18S RNA as the cells, suggesting that clarification alone is not sufficient to remove contaminating ribosomes). While we agree that repeating the ultracentrifuge concentration with high concentrations of EDTA is an interesting line of inquiry we feel it is outside the scope of this manuscript (and we face additional technical restrictions to pursue this as we currently lack access to an ultracentrifuge at BSL-3). We have updated the discussion to include the possibility of residual ribosome-protected fragments of sgE as a potential alternative interpretation (line 350-352).

**Minor comments:**

At line 197, the authors refer to "viruses" with lower levels of SARS-CoV2 RNA. This is incorrect and should be changed to "isolates" as the SARS-CoV2 virus particle does not package variable amount of genomic RNA.

We have changed this to “clinical specimens” for clarity.

The authors statement on lines 210-212 does not seem to be supported clearly by Fig. 5. The authors should consider including trendlines as well as other analyses that help show the correlation between viral RNA vs FFU. In addition, the authors should label the Y-axis clearly for Fig. 5.

We have added clarifying labels to both the X and Y axes. Due to the limited sample volume we were unable to directly measure the infectious titers from the clinical samples used in this study, and thus the FFU/mL represents the titer post-isolation while the CT represents the amount of RNA pre-isolation. Nonetheless, we do see broad trends (ie, the colored dots are generally arranged in rainbow order from left to right, though we agree there is variation within this trend). We have also modified the text at lines 212-217 to reflect the reviewer’s concern- “Greater initial viral RNA levels was broadly associated with faster viral growth in both cell lines (seen in the progression of colors from left to right), however we saw significant variation within these trends. Our data suggests that when standard SARS-CoV-2 RNA RT-PCR values are the only available data for patient or population-level viral loads, they are useful in gauging the presence of infectious virus in patient NP samples (Fig 5).”

The authors should expand on the methodology for creating ROC curves at line 467.

We have included the following text in the methods section for ROC curve analysis:

“ROC curves were generated using R [43]. For each potential scoring marker (CT_e, CT_sge1, CT_sge2, neg_e,) samples were ordered by that marker, followed by culturable status. The false-positive rate was calculated as the cumulative count of culturable samples (after ordering by marker intensity) divided by the total count of culturable samples; the true positive rate was calculated as the cumulative count of non-culturable samples (after ordering) divided by the total count of non-culturable samples. The false positive rate was plotted on the X axis of the ROC curves and the true positive rate on the Y axis.”

Reviewer #2 (Significance (Required)):

This study is significant because it assesses the utility of several clinical assays for the measurement of viral RNA and correlating it with culturable virus. This is important in the field because it helps to identify methods whereby infectivity can be predicted from a simple diagnostic test. This is important to know as a virologist working in the SARS-CoV2 field. It is also important from a public health perspective to better define quarantine requirements for persons testing positive. While the study provided a new primer probe set, it appears that the already available WHO total E RNA assay is superior in predicting infectivity and this study provides further evidence to support this notion.

We appreciate the Reviewer’s assessment that this study is significant and provides information of high interest to SARS-CoV-2 virologists that also has important public health implications.

Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Referee #2

Evidence, reproducibility and clarity

Summary:

Bruce et al present a new RT-PCR assay with primer sets that specifically detect sgE RNA from SARS-CoV2 samples. The authors compare this assay to other diagnostic assays in an effort to identify assays capable of correlating RNA detection with culturable virus (i.e. infectious virus). While this new assay identified 100% of culturable isolates, only 56% of isolates testing positive actually had culturable virus. Compared with other assays, the WHO total E RNA assay had better parameters when used at a cutoff Ct value of 31 (PPV of 61%). Overall, this manuscript provides a novel primer probe set for RT-PCR diagnostic assay and conducted comparisons with other assays on the same clinical samples. There are some areas that the authors should address prior to publication.

Major comments:

-The authors repeatedly tout VeroE6 TMRSS2 cells as supporting higher viral infection. Therefore, the authors should address why one clinical isolate (E16) was culturable in VeroE6 but not VeroE6 TMRSS2. Was this experiment repeated multiple times? What are the reasons for this discrepancy?

-The authors' argument at lines 166-169 is not supported by the data in Fig. 2. The levels of viral RNA between VeroE6 and VeroE6 TMRSS2 appear to show similar trends in the supernatant across the time course but the infectious viral levels are dramatically different. This discordance between FFU levels and RNA levels cannot be explained by instability of viral particles alone. Have the authors looked into differences in viral particles produced from these two cell lines? The authors should collect virus particles from these two cell lines and conduct the stability experiment in Fig 2D to directly test the hypothesis that indeed the drop seen in FFU in VeroE6 TMRSS2 is due to instability.

-The evidence for the packaging of sgE RNA into virions is weak. GAPDH detection by PCR is not a proof that the concentration process did not pellet RNA nonspecifically. First, the authors should provide ample information about viral isolation process at line 379 including rotor, centrifuge and speed utilized. In addition, ribosomes typically stay intact following viral lysis (and can be found in supernatant after release from dead cells). Actively translating ribosomes can contain sgE RNA as well. The authors should consider detecting ribosomal RNAs in their samples to rule out the possibility of contaminating ribosomes. In addition, the authors should strongly consider repeating the experiment with high EDTA concentration to break up ribosomes and only pellet virions.

Minor comments:

-At line 197, the authors refer to "viruses" with lower levels of SARS-CoV2 RNA. This is incorrect and should be changed to "isolates" as the SARS-CoV2 virus particle does not package variable amount of genomic RNA.

-The authors statement on lines 210-212 does not seem to be supported clearly by Fig. 5. The authors should consider including trendlines as well as other analyses that help show the correlation between viral RNA vs FFU. In addition, the authors should label the Y-axis clearly for Fig. 5.

-The authors should expand on the methodology for creating ROC curves at line 467.

Significance

This study is significant because it assesses the utility of several clinical assays for the measurement of viral RNA and correlating it with culturable virus. This is important in the field because it helps to identify methods whereby infectivity can be predicted from a simple diagnostic test. This is important to know as a virologist working in the SARS-CoV2 field. It is also important from a public health perspective to better define quarantine requirements for persons testing positive. While the study provided a new primer probe set, it appears that the already available WHO total E RNA assay is superior in predicting infectivity and this study provides further evidence to support this notion.

Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Referee #1

Evidence, reproducibility and clarity

Summary:

Provide a short summary of the findings and key conclusions (including methodology and model system(s) where appropriate). Authors developed a novel primer/probe set for detection of subgenomic (sgE) transcripts for SARS-CoV-2 with the aim to develop a system that may predict the presence of infectious virus in patient samples. After studying the specificity and sensitivity of their system, they compared it with already validated/published systems for diagnostic of SARS-CoV-2 infection. Interestingly, they also studied the effect of the conditions of isolation. They showed Vero E6 expressing TMPRSS2 (Vero E6-TMPRSS2) to be more sensitive to infection than Vero E6, allowing a higher number of isolation from patient samples. They also showed their system to be more sensitive than a previously published sgE system as well as than a negative-strand RNA assay but less sensitive than the WHO/Charité primer/probe set. Anyway, all samples containing infectious particles (successful virus isolation on Vero E6-TMPRSS2) were detected with their primer/probe system contrary to the other tested sgE assay. They showed the negative strand assay to be unlikely to detect virus genetic material in samples which nevertheless contain infectious particles.

Major comments:

-Are the key conclusions convincing?

I salute the intention of the authors to try to fix cut-off values for infectious patients but I would be more careful on the assertion of "using a total viral RNA Ct cut-off of >31 or specifically testing for sgRNA can serve as an effective rule-out test for viral infectivity". It is true that in this study, virus was not isolated from any of the samples below a Ct of 31 or negative in the developed sgE assay but all those assays are done on cell culture. We do not know how the transmission could occur for those samples from human to human. Being able to fix a cut-off in Ct value for a define PCR/RT-PCR system would be a great improvement for SARS-CoV-2 infected patient having to stay in quarantine. It is even more important for Ebola positive patients in Africa who has to stay in quarantine in precarious conditions under tents, warm temperatures and without privacy for long period because they still positive by RT-PCR. Unfortunately, fix those values would need a very high number of experiments, including animal experiment.

-Should the authors qualify some of their claims as preliminary or speculative, or remove them altogether? No

-Would additional experiments be essential to support the claims of the paper? Request additional experiments only where necessary for the paper as it is, and do not ask authors to open new lines of experimentation. No

-Are the suggested experiments realistic in terms of time and resources? It would help if you could add an estimated cost and time investment for substantial experiments. Yes.

-Are the data and the methods presented in such a way that they can be reproduced?

-Kinetic of SARS-CoV-2 (figure 2): The method is not detailed in the Methods part and is not clear in the figure legend. When supernatant are collected, is it all the supernatant that is remove? An aliquot? If aliquot, do you replace with new medium? -Stability of infectious SARS-CoV-2: I am very surprise by your results on stability of cultured virus, knowing we observed a decreased of SARS-CoV-2 titer in our lab after freezing/thawing steps. Do you freeze cell supernatant directly or do you prepare your samples another way? Please state it in the Methods part

-Are the experiments adequately replicated and statistical analysis adequate? Yes

Minor comments:

• Specific experimental issues that are easily addressable.

Figure 2C and D: Instead of Ct values in cells, it would be more relevant to normalize these results with an endogenous gene and present results as fold change to mock-infected cells. Because you affirm that the level of RNA decline than stay stable over the time but you also note there is CPE. If you have less cells but same level of viral RNA, it means you have an increase in the RNA level in alive cells. It would have been interesting to have the results of isolation at different time-point of treatment for patient samples (figure 3A and B) to see if the virus is stable in samples

-Are prior studies referenced appropriately? Yes

-Are the text and figures clear and accurate? Yes.

Line 140: "this delay in virus and RNA production". You do not talk about RNA yet...

Line 156 to 163: sgE RNA detected in cell free supernatant. Can't it come from lysed cells?

Line 167: "...virus in cell culture time course experiment in TMPRRS2 expressing cells (fig.2)"

Ligne 258: Fig 6A and B

-Do you have suggestions that would help the authors improve the presentation of their data and conclusions? No

Significance

-Describe the nature and significance of the advance (e.g. conceptual, technical, clinical) for the field.

This new primer/probe system will participate to the accurate diagnostic of SARS-CoV-2. The comparison with the existing methods is relevant to highlight the strengths and weaknesses of each system. Comparison of isolation of SARS-CoV-2 on commonly used Vero E6 with Vero E6-TMPRSS2 will lead to a great improvement of the isolation method for SARS-CoV-2.

-Place the work in the context of the existing literature (provide references, where appropriate). Properly done in the introduction of the paper.

-State what audience might be interested in and influenced by the reported findings. Diagnostic laboratories

-Define your field of expertise with a few keywords to help the authors contextualize your point of view. Indicate if there are any parts of the paper that you do not have sufficient expertise to evaluate. Virology, Molecular Biology, cell biology Not enough expertise to evaluate ROC data/analysis

Note: This rebuttal was posted by the corresponding author to Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Reply to the reviewers

We thank the reviewer for their input. Our response to their comments is in the attached preliminary revision plan.

Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Referee #2

Evidence, reproducibility and clarity

Summary:

• Provide a short summary of the findings and key conclusions (including methodology and model system(s) where appropriate). Please place your comments about significance in section 2.

This manuscript provides a detailed and very clear description of multiSero, which is an open source multiplex-ELISA platform for analyzing antibody responses to SARS-CoV-2 infection. This tool is a very promising step towards fully open-source multiplex testing. Using terrific visualizations the different steps involved in measuring the antibody levels is carefully explained. It starts with a clear explanation of the principle of printed antigen arrays, the usage of developed and opensource software Pysero to analyse the colorimetric signal of each spot associated with a different antigen. The colorimetric signal was read using both a commercial reader and an inexpensive, open plate reader. The comparison between the two proved that the open plate reader is as good as the commercial reader is.

Major comments:

• Are the key conclusions convincing? • Should the authors qualify some of their claims as preliminary or speculative, or remove them altogether? • Would additional experiments be essential to support the claims of the paper? Request additional experiments only where necessary for the paper as it is, and do not ask authors to open new lines of experimentation. • Are the suggested experiments realistic in terms of time and resources? It would help if you could add an estimated cost and time investment for substantial experiments. • Are the data and the methods presented in such a way that they can be reproduced? • Are the experiments adequately replicated and statistical analysis adequate?

The authors provide a new method to measure antibody levels. A comparison with an exisisting ELISA for anti-spike IgG would be worthwhile.

A gradient boosting tree was used to combine the signal from multiple antigens. However, this did not lead to any notable improvements in classification performance. This result could be due to a property of the data or the algorithm. Two things that would be very useful here would be to plot the data (e.g. anti-Spike vs anti-N) and use a much simpler algorithm such as a logistic regression.

The performance of the tool is based on one positive and one negative pool. And as the the authors mention, antibody levels are highly dependent on severity and time since infection. The performance of the classifier therefore strongly depends on the characteristics of the positive pool. It would improve the manuscript by providing additional information, if possible. If not, I think this should be mentioned as a short-coming in the discussion. Possibly, having serum panel with more asymptomatic infections or longer time since infection, would result in a poorer performance from the classifier.

Related to the point above is what is written in line 248-249. The direction of the performance of the tool with additional samples depends on the characteristics (time since infection, age, severity) of the currently used samples and the samples to be added. The assumption that the performance can only increase is in my opinion not correct.

The authors compared three normalization methods to circumvent using a standard curve. The normalization of ODs by the mean of anti-IgG Fc ODs is most promising as shown in Fig. S5. A comparison between this normalization method and using a standard curve is not given. It would be worthwile to look at the distribution of a serum panel from different plates, in relative antibody units as well as normalized ODs. Is the captured antibody distribution by normalized ODs as good as relative antibody concentrations derived from the standard dilution.

In the abstract, the reader is told that the multiSero tool could be used with up to 48 antigens. I assume that at this number of antigens, the use of duplicate/triplicate antigens is not possible anymore? Also, the layout and spacing of the antigen array with more antigens would introduce more experimental artificats like comets and debris ?

In FigS3, and line 146/147 the authors state that they find the that the presence of comets odes not cause observable bias or variance. This strikes me as rather subjective, and my impression of FigS3 B3 is that there is some bias due to comets?

Minor comments:

• Specific experimental issues that are easily addressable. • Are prior studies referenced appropriately? • Are the text and figures clear and accurate? • Do you have suggestions that would help the authors improve the presentation of their data and conclusions?

Important reason for developing the multiSero tool according to the authors is the deployment of high content, multiplex serology platforms across the world and this paper makes a huge step towards this goal. Main hurlde of implementing the multisero tool in low-resource settings is its dependency on printed antigen arrays, which are produced by machines costing around 100,000-300,000 $ as mentioned by the authors. The authors also realize this and acknowledge this bottleneck in the discussion. I think it would possibly be good to elaborate a little further on why this is a limitation. Less freedom with the user what they want to test because dependent on producer of printed 96 well-plates?

In line 47, I suppose the word are is missing.

The overall language use is very clear. An improvement in my opinion would be to replace words such as cognate (line 46) and « in lieu of » (line 227) by easier alternatives, such as associated and instead of.

Comets and debris are first mentioned in line 129/130 but require more explanation. An explanation of what is meant with comets only became clear to me after reading the discussion. I would use the explanation mentioned in line 250/250 right after the first time mentioning comets. What debris means, remains unclear to me.

In line 174, I suppose that the word points should be line.

Pysero sometimes starts with a capital P, sometimes with a lower case p, see for example line 108 and 109.

Authors find using a standard curve as labor-intensive (line 190), I find this too strongly put.

In line 281-283 the authors mention they are unaware of examples of classifiers distinguishing positive from negative samples based on more than antigen. Examples could be the classification of cholera using 2-6 antigens by : Azman et al, 2019 in Sci. Transl. Med.

In Fig S1 the Nauttilus plate reader is shown. The costs of this reader are estimated to be less than 1500$. These are the costs without the motorized

Significance

With this manuscript, the authors show that multiplex serology platforms can become more accessible to low and medium income countries due to their development of a new open source tool. This means that multiplex serology seems to be becoming more accessible in low-resource settings. Next step is to use this multiSero tool in a low-source setting.

Specific audience potentially interested are computational biologists involved in the analysis, visualization, and interpretation of the results of techniques and microbiologists quantitating and measuring antibodies. A broader audience that could be interested are infectious disease epidemiologists, especially those that are involved in serosurveillance and are keen to pick up new methods to potentially improve epidemiologal descriptions of immunity to several infections in low-and medium-resource settings.

My field of expertise is limited to field-epidemiology and sero-epidemiology. Techniques such as the detection of spots and registering grids with multiSero are outside of my expertise. The construction of the Nautilus reader is new to me, and therefore hard to assess how easy it would be set up such a system in low-resource settings. I also feel my expertise regarding the choice of classifiers is limited, as I have not used gradient boosting before. Further, I am not an expert in the field of new developments in multipex assays and therefore not up-to-date with the latest literature in this field.

Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Referee #1

Evidence, reproducibility and clarity

Summary:

There is a need for multiplex serological tests，and ELISA is the most applicable platform. However, the current ELISA based multiplex serological tests are heavily dependent on expensive and sophisticated instruments and softwares, and this hinders the wide application. To address this challenge, by incorporating open-resourced instruments, developing new analysis software, the authors proposed an integrated platform for multiplex serological test. To test the platform, SARS-CoV-2 was included as the example. Overall, this study is more technical oriented. The major contents are the establishment and optimization of the platform. The aim is focused and clear, the design of the experiments are comprehensive. The conclusions could be supported by the data.

Major comments:

• Are the key conclusions convincing? Yes
• Should the authors qualify some of their claims as preliminary or speculative, or remove them altogether? No
• Would additional experiments be essential to support the claims of the paper? Request additional experiments only where necessary for the paper as it is, and do not ask authors to open new lines of experimentation. No
• Are the suggested experiments realistic in terms of time and resources? It would help if you could add an estimated cost and time investment for substantial experiments. N/A
• Are the data and the methods presented in such a way that they can be reproduced? Yes
• Are the experiments adequately replicated and statistical analysis adequate? Yes

Minor comments:

• Specific experimental issues that are easily addressable.
• Are prior studies referenced appropriately? Yes
• Are the text and figures clear and accurate? Yes
• Do you have suggestions that would help the authors improve the presentation of their data and conclusions? No

Significance

• Describe the nature and significance of the advance (e.g. conceptual, technical, clinical) for the field.

This study is more technical centered. The major contribution is the development of an ELISA-based platform for multiplex serological test. The authors intended to make their platform applicable at resource limited regions. However, the problem here is the current platform is still too complicate for wide application in real world. For a platform which may could be widely applied, especially at poor regions, it needs to meet several key features: 1. Low cost; 2. Standardized; 3. Simple (reduce operation to as few as possible). The major focus of this study is the first feature, and the other two features were bared touched. But, even "low cost" is still valuable and worth publication. The reviewer suggest the author to modify the manuscript to better reflect the fact.

• Place the work in the context of the existing literature (provide references, where appropriate).

The existing literatures were well referenced.

• State what audience might be interested in and influenced by the reported findings.

Researchers who are interested in assay development.

• Define your field of expertise with a few keywords to help the authors contextualize your point of view. Indicate if there are any parts of the paper that you do not have sufficient expertise to evaluate.

Protein microarray technology. Assay development. SARS-CoV-2 antibody response analysis.

The reviewer is not familiar with the software part.

Other specific points:

1. The authors mentioned that the multiplex serological test could be applied to differentiate infection and vaccination, in the case of SARS-CoV-2, how could this be possible if there is no specific biomarker?
2. Have the authors also tested IgM?
3. To simplify the normalization, the authors have tried several strategies, however, none works well. The results need to be further explained. Is there any other strategy could be attempted?
4. What's the definition of the "background"?
5. What's the rationale to select the two concentrations? Will more concentrations be better?
6. The authors stated "open source analysis tools can be adapted for multiplexed detection of pathogens by printing pathogen-specific antibodies, instead of antigens". This is true, however, highly specific antibodies are required.

Transparent Peer Review

---

Download the complete Review Process [PDF] including:

• reviews
• authors' reply
• editorial decisions

</br>

Transparent Peer Review

---

Download the complete Review Process [PDF] including:

• reviews
• authors' reply
• editorial decisions

</br>

Note: This rebuttal was posted by the corresponding author to Review Commons. Content has not been altered except for formatting.

Learn more at Review Commons

---

Reply to the reviewers

1. General Statements [optional]

This section is optional. Insert here any general statements you wish to make about the goal of the study or about the reviews.

We thank the reviewers for their constructive and helpful comments on our manuscript and are delighted to find their consensus that the manuscript represents an important contribution to the field. We provide a detailed response to specific points below. In addition, we propose to include new data showing that our method can be applied to experimentally infected lung tissue. Namely, we show highly sensitive detection of SARS-CoV-2 RNA in infected hamster lung section.

2. Description of the planned revisions

Insert here a point-by-point reply that explains what revisions, additional experimentations and analyses are planned to address the points raised by the referees.

Reviewer #1 **Major comments:**

The authors used approaches provided in FISH-quant (Mueller et al, Nat Methods 2013) and big-fish. However, these tools to analyze RNA aggregates were not designed and validated for such massive aggregations as observed by SARS-Cov-2. They were developed for cases such as transcription sites with much smaller aggregations, with a few tens to a hundred molecules. With a regular spot detection approach, usually a few thousand spots can be detected in a cell (e.g. King et al, J Virol 2018), but this depends also on the used microscope and the available cellular volume. Higher RNA concentrations cannot be resolved with a standard approach, because RNA spots start to overlap. Decomposing RNA aggregations can help but will not work reliably for the high RNA densities observed for SARS-Cov-2, especially at later infection time-points. The tools will then not provide accurate estimates anymore. To my knowledge, there is currently not accurate quantification method for such massive RNA levels in smFISH. What has been done in the past, is using cellular intensity as an approximation and perform calibrations with cells having lower and thus still resolvable RNA counts (Raj et al., PLO Biology; https://doi.org/10.1371/journal.pbio.0040309.sg003). The authors proposed three expression regimes (partially resistant, permissive, and super permissive). My concerns here apply mainly to the category super-permissive, where an accurate estimation can't be performed. Here a more cautious quantification should be applied. __To a lesser extent, this will also apply to some of quantifications of gRNAs per factory, with counts exceeding 100s of molecules. As mentioned above, this does not affect any of the conclusions, but would reflect more accurately what kind of reliable information can be drawn from such experiments.__

We agree with the reviewer that approaches like FISH-quant and Big-FISH cannot reliably quantify RNA spots with high spatial density such as our examples of “super-permissive” cells. Single molecule quantitation of such cases is likely to underestimate RNA expression as noted by us and King et al 2018 (doi: 10.1128/JVI.02241-17). Therefore, we integrated the combined smFISH signal intensity within entire cellular volumes and compared to the median intensity of single molecules in cells with lower infection density. We will (i) revise the methods and results sections to explain more carefully and explicitly the quantification of RNA in super-permissive cells. (ii) Provide a calibration plot for the quantitation as previously reported (Raj et al 2006, doi: 10.1371/journal.pbio.0040309).

We agree that high local RNA density has the potential to interfere with quantification of gRNAs within viral factories. We have used the “cluster.decomposition()” function of Big-FISH to quantify viral factories, which is conceptually similar to the “Integrated intensity” mode of FISH-quant. Applying this algorithm to non-super permissive cells allows us to use the mean intensity of a reference single-molecule spot to estimate the number of molecules in a cluster. We are confident such estimates are reliable in the majority of viral factories, which contain less than or equal to 200 single gRNA molecules. We will revise the methods section to clarify this method of analysis.

Reviewer #1 __**Minor comments:**__

1.Page 6; the authors state that "smFISH identifies ... cellular distribution .... within ER-like membranous structures". However, the authors didn't directly show such a localization, could they provide an experiment with an ER stain?

This text was based on previous light microscopy and EM studies that reported SARS-CoV-2 RNA in ER-derived membranes (termed Double Membrane Vesicles - DMVs) or co-localisation of anti-dsRNA (J2) with ER-markers (Cortese et al 2020; Hackstadt et al 202; Mendonca et al 2021)*. We propose to clarify the text on page 6 including the citation of these publications and to tone down our claim that the virus is located in ER-like membranous structures.

*Cortese et al 2020, doi: 10.1016/j.chom.2020.11.003

Hackstadt et al 2021, doi: 10.3390/v13091798

Mendonca et al 2021, doi: 10.1038/s41467-021-24887-y

2.It might be worthwhile pointing out that the probe-sets can be used in different host organisms (Vero - African green monkey; human cell lines).

We propose to revise the text to emphasise more clearly the applicability of SARS-CoV-2 probes for the study of many different host organisms.

3.I really liked the experiment, where the authors showed absence of signal when infecting with another virus & elegant control with the J2 AB. Maybe the authors could explain more clearly that the used a different coronavirus & that based on their sequence alignment no/little signal would be expected.

Thank you for this supportive comment. We plan to follow the reviewer’s suggestion and expand our explanation of the rationale of this experiment in the text.

7.The experiment with the isolated virions shows nicely that the smFISH approach has single-virus sensitivity. Did the authors compare the intensity of these isolated virions with the signal in Fig 1B? This might be a question of personal taste, but to me, this section might actually fit better in the first paragraph of page 4/5, where the authors describe single virions in cells.

Thank you for the interesting question. We have not performed a direct comparison of the spot intensities of intracellular genomic RNA molecules and those from the isolated virions, because isolated SARS-CoV-2 requires poly-L-lysine coating for the coverslip attachment while our infection strategy utilises cells growing on uncoated glass. Nonetheless, the isolated virion spot intensities follow a unimodal distribution, and their shape approximates to the point-spread function of the microscope. Since spots at 2 hpi are largely derived from non-replicative viral genomes and they are measured in the intracellular environment with the same background (autofluorescence), they are a better ‘single RNA molecule’ reference.

We also thank the reviewer for suggesting rearranging the text section. To address this point we plan to move the relevant text to the second paragraph of the Results section.

8.Page 6. The authors state "+ORF-N and +ORF-S single labelled spots, corresponding to sgRNAs, were more uniformly distributed throughout the cytoplasm than dual labelled gRNA". This is difficult to appreciate from the image. Is this something the authors could quantify, e.g. with the metrics proposed by Stueland et al, Scientific Reports 2019?

To address this point, we plan to: (i) present an alternative image illustrating a clearer example of differential spatial localisation of gRNA and sgRNA, and (ii) perform quantification of spatial dispersion indices for gRNA and sgRNA using the suggested method for our revision.

9.Page 6. The authors perform a FISH/IF experiment including a co-localization analysis, where a "limited overlap" with sgRNAs was observed. I was wondering if this overlap could actually be simply due to rather high density of the sgRNAs. Maybe a control analysis by slightly changing the RNA positions could provide insight here, and give a threshold for what's to be expected randomly at a given RNA density.

The reviewer’s comment is correct, in that a high density of sgRNAs and nucleocapsid protein could lead to signal overlap due to chance. This is why we excluded “super-permissive” cells from this analysis. Our co-localisation data showed that gRNA spots had a bimodal nucleocapsid immunofluorescence intensity distribution (data not shown), suggesting nucleocapsid-associated and “free” gRNAs, providing a threshold for this analysis. Nevertheless, we agree with the reviewer that the analysis of randomly positioned transcripts of the same density would provide a valuable control. In our revised MS we will include: (i) a random distribution analysis comparing the overlap between sgRNA and nucleocapsid in the “Observed” and a “Randomised” simulation, and (ii) a plot showing a full distribution of co-localised nucleocapsid immunofluorescence intensity for both genomic and sub-genomic viral RNAs.

10.I don't fully follow the argument about stability on page 8. The authors also see an increase in the RNA levels. Couldn't this increase compensate for loss of RNA due to degradation? Would it be possible to perform an experiment at a very high REMDESIVIR concentrations which would blocks transcription?

Remdesivir is a nucleoside analogue that inhibits viral RNA polymerase activity. While this drug inhibits viral replication, the inhibition is incomplete and using higher concentrations results in cellular toxicity. At the present time there are no stronger polymerase inhibitors available, so these experiments are the best approximation possible to assess viral RNA stability. We propose to revise the text to discuss the limitations of Remdesivir for modelling RNA stability.

12.How did the authors define/detect replication factories? I couldn't find information about this in the methods.

This is a good point raised by both the reviewers. Please see [Reviewer 2 General comment #1] for our response.

Reviewer #2 **General comments:**

1.The authors' definition of viral factories, in part as foci with at least 4 gRNA molecules, comes across as arbitrary. Perhaps a clearer explanation of this cutoff would be helpful to the readers' understanding of this definition. Additionally, confirmation of the functionality of such factories by immunofluorescence with anti-RdRp, for example, in addition to identifying staining of gRNAs and (-) sense viral RNAs at each focus could provide valuable support to the authors' conclusions.

We thank both reviewers for requesting further information on our explanation of viral factories. We defined viral factories as smFISH signals with spatially extended foci that exceed the size of the point spread function of the microscope and the intensity of a reference single molecule. We then filtered these candidate factories based on the radius of the signal foci with EM-measured radii of double-membrane vesicles and single-membrane vesicles formed by SARS-CoV-2 (150 nm pre-8hpi and 200 nm post-8hpi) (Cortese et al 2020; Mendoca et al 2021). Our terminology encompasses both replication and viral assembly sites. The threshold of 4 genomic RNA molecules was selected as a technical threshold to limit an over-estimation of viral factories at later timepoints. For our spinning-disk confocal imaging system, we found the threshold of 3-7 RNA molecules provided satisfactory results. We propose to revise both the Results and Methods sections to clarify our rationale for defining and quantifying viral factories.

As the reviewer mentioned, we have shown a partial overlap of positive sense genomic RNAs with negative sense genomic RNAs (Figure 2D, S2C), suggesting these viral factories represent double membrane vesicles. The use of antibodies against the viral polymerase (nsp12) is also a possibility to detect replication centres. However, replication centres are not the only ‘viral factories’ as there are also double-membrane structures where viral particles assemble (Mendoca et al 2021) and they, in principle, lack negative sense RNA and replication machinery, so neither smFISH probes against the negative strand nor a nsp12 antibody will comprehensively detect viral factories. We appreciate the valuable suggestion, but the classification of viral factories into replication and assembly sites would be challenging due to reagent availability and is beyond the scope of this manuscript.

2.The random distribution of super-permissive cells in each cell line was demonstrated early in the infection, primarily at 8 hpi. The authors do not show how this pattern changes over time (8, 10, 12, 16, 24 hpi, for example). Do clusters of super-permissive cells appear at later time points, or does the pattern of 'highly' infected cells remain random for each virus? Any strain-specific differences identified from such patterns may be important for understanding infection progression. Finally, the authors do acknowledge this point, but it cannot be overstated that these data were taken from cell culture systems that have limited similarities to the human respiratory epithelium. A better model for such studies might be primary cultured human bronchial epithelial cells, but of course, these cells are not as readily accessible as the cell lines used in this manuscript.

We share the same view that the presence and the spatial distribution of “super-permissive” cells can provide unique insights into SARS-CoV-2 infection dynamics. Our findings suggest that even at 24 hours post infection (hpi), not all cells become “super-permissive” and the culture maintains a heterogenous population of “partially resistant”, “permissive” and “super-permissive” cells (Figure 3C, S3C-D). We agree with the reviewer that the spatial distribution of “super-permissive” cells at later timepoints is of interest. To address this point, we plan to: (i) analyse the spatial distribution of “super-permissive” cells at 24 hpi, and (ii) compare the distribution of “super-permissive” cells at 24 hpi between VIC and B.1.1.7 strains.

We appreciate the comment on the limitations of the cell culture systems to the human respiratory tract. However, Calu-3 and A549-ACE2 lung epithelial cells have been used in many studies over the last year and we feel it is important to publish single cell quantitation with these models to enable comparison with the published literature. We believe our results provide valuable information on the intrinsic nature of host cell susceptibility to support viral replication. During the review of this manuscript, we applied our smFISH probes to detect SARS-CoV-2 RNA in infected Golden Syrian hamster lung sections, which show an uneven distribution of infected cells. While the identification and spatial characterisation of susceptible cell types in the lung are beyond the scope of this manuscript, we are excited to include this data in our revised paper to demonstrate the utility of this sensitive approach to track spatiotemporal viral infection dynamics.

3.The difference in early replication kinetics between the VIC and B.1.1.7 strains is an exciting finding that may have implications for clinical outcomes and transmissibility of these viruses. However, the authors did not clearly demonstrate how these differences in RNA production correlate to infectious viral load released from these cells (in bulk) at each time point. An explanation of this omission would be helpful.

We will provide data on the level of infectious virus secreted from VIC and B.1.1.7 infected cells at all time points in the revised paper.

In my opinion, findings related to specific cell lines are of much less importance (and are much less biologically relevant) that identification of replicative differences among strains. Such differences could be used, in part, to aid prediction of the transmissibility of VOC, for example. I think this point gets a bit 'lost in the weeds' of the rest of the paper.

To address this comment, we will revise text on the differential replication kinetics of the SARS-CoV-2 strains to make this more prominent in our paper.

3. Description of the revisions that have already been incorporated in the transferred manuscript

Please insert a point-by-point reply describing the revisions that were already carried out and included in the transferred manuscript. If no revisions have been carried out yet, please leave this section empty.

Reviewer #1 __**Minor comments:**__

4.I might have missed this, but they authors could also mention the positive control data about but Calu3 infected with SARS-COv2. One thing I was wondering: why did the authors use two different cell lines for this experiment?

To address this point, we have added a sentence about a positive control visualising SARS-CoV-2 in Calu-3 cells using our probe set (page 5 – line 17).

The experiments with HCoV-229E were done in Huh-7.5 cells because SARS-CoV-2 and HCoV-229E have distinct cell preferences. Using the J2 antibody we show that the levels of the dsRNA derived from viral replication are similar in the two cell lines and with the two viruses. Therefore, the lack of smFISH signal in HCoV-229E infected cells supports the high specificity of the probe set.

5.Fig 1E. Would be nice to have the intensity scale for all time-points to permit a comparison of image intensities along the different time-points.

6.Fig 3B. Would be important to have intensity scale bars to judge the signal intensities across the different time-points.

The fluorescence intensity scale in Figure 1E is applicable to all timepoints, except for the lower panel at 24 hpi, which was intended to show wider dynamic contrast range. To address this point, we have provided intensity scales for all time-points studied in this figure and also Figure 3B.

11.Fig 3C. maybe indicate the two groups with dashed lines.

We have added a dashed line at the 102 mark in Figure 3C to visually differentiate “partially resistant” and “permissive” cells.

4. Description of analyses that authors prefer not to carry out

Please include a point-by-point response explaining why some of the requested data or additional analyses might not be necessary or cannot be provided within the scope of a revision. This can be due to time or resource limitations or in case of disagreement about the necessity of such additional data given the scope of the study. Please leave empty if not applicable.