This doesn't feel quite fair, as UMAP and tSNE were designed to handle datasets of this size and have been widely used to generate embeddings for single-cell datasets of this size and larger. Also, I believe at least UMAP is sub-quadratic in the number of samples, as it uses an approximate kNN algorithm that is n log n.

Minor comment: using a log-log scale for these plots would be helpful, as it would prevent the reference methods (UMAP, tSNE, PHATE) from appearing as a flat line.

It would be helpful to clarify what this claim is based on, as I can't see anything in Figure 2 that indicates a 99% change in any of the metrics between CPM and PM.

I agree that this analysis sounds promising, but I think it would be more convincing with a comparison to more traditional methods (e.g. random forest after PCA or NMF)

Since several different datasets were used here, what preprocessing was necessary to obtain spectra with this consistent range and resolution? Was it necessary to upsample or downsample the spectra from some datasets?

I would be very cautious about drawing this (or any) conclusions on the basis of a UMAP, as UMAP embeddings can be strongly dependent on hyperparameter choices.

I'm not sure I follow this: just because some CTCs will eventually become metastatic doesn't necessarily mean that they will look similar or "overlap" in the present.

I think more details abour how the data were split into train and test sets would be important to include here. From reading the supmat of [20], it sounds like the 172k spectra were obtained from 11 hyperspectral images, implying that many of the individual spectra are likely to be highly correlated, so I think care would be needed when splitting into train/test sets to avoid data leakage (e.g. perhaps by splitting at the level of the original hyperspectral images)

This is missing a citation - presumably this dataset was from [20]?

Could you comment on this was value was chosen? It seems like it could be a really important hyperparameter...

If the spectra are 400-dimensional and the patch size is 16, there would be 20 patches per spectrum, not 16.

I think it would be helpful to briefly describe how these 2D lines are constructed, as they may be an unfamiliar way of visualizing sequence data for many biologists.

Could it instead be that larger models are simply less prone to getting stuck in repeats?

From the figure, it doesn't look like the models are simply recapitulating the true sequence. I think a more direct comparison like a sequence alignment between the true and generated sequence would be more informative here.

This feels pretty vague and hand-wave-y. The term "Lyapunov stable point" is mentioned without establishing or discussing the mathematical conditions or assumptions that would justify using it. Also, phrases like "system-wide energy expenditure" are not defined; what is meant by "energy", and why would viruses adapt in order to minimize it?

This is so succinct that it may be hard for readers to understand what is being referred to here. If these constraints are important, I think it would be worth elaborating on them a little bit.

I appreciate that this presentation is intentionally abstract, but I think it would be helpful to explain more explicitly what omega represents here. In other words, what is the nature of the "characteristics" z_i? I think readers may assume that these characteristics refer to phenotypes, but later on, it becomes clear that these "characteristics" actually refer to something like "genomic state" (if not simply a genotype)

In the absence of any evidence favoring one factor or another, I think it would be fair to include other possible factors here (especially as, imo, a lack of data science education is not the most plausible one)

This is a bit of a nitpick, but I'm not sure it's fair to equate "visualization errors" with "data misinterpretation", either on the part of authors or readers, because visualization errors can be made for other reasons by authors and won't necessarily lead to data misinterpretation by readers (even if they do make it more likely)

fwiw, another possible explanation that strikes me as more plausible is that there are very strong incentives for authors to publish in prestigious journals, which in turn incentives them to present their results in the most compelling possible light.

Perhaps a simpler explanation is that papers with more authors tend to be longer and to have more figures, increasing the chances that at least one figure contains a mistake.

I don't think it's quite accurate to say that the ground truth is known here unless the meaning of "in almost every cell" can be quantified. I agree that the fact that SpotMAX finds six foci in 97% of nuclei is qualitatively consistent with the biology of COSA-1 foci, but there's no way to quantify this performance without first quantifying the true prevalence of nuclei with more or less than six foci (and I think we do know it can't be 100%).

It's also important to specify the meiotic stage (that is, the position of the nuclei within the gonad) at which foci are counted, since (if I recall correctly) COSA-1 foci appear gradually, so I think in mid-pachytene there will be many more nuclei with fewer than six foci than at late pachytene.

Is it known that the presence of RAD-51 foci alone is a reliable proxy for the activation of the DNA damage checkpoint in late meiotic prophase? (I'm not sure, but I wouldn't be surprised if it is not)

Does SpotMAX support a range of expected spot sizes, or does it require one specific size?

I think it'd be helpful to briefly explain what Cell-ACDC is and why it is important that SpotMAX is integrated into it, as some readers may not be familiar with it.

If I recall correctly, COSA-1 foci aren't always diffraction-limited, so I'm not sure that it's really possible to determine the true number of foci in the two examples shown, at least from the 2D images shown in the figure (though I agree that the example on the left does look a lot like two barely-separated distinct foci)

Does "significantly shorter" mean that the difference was statistically significant? If not statistically significant, I'm not sure it's worth mentioning. If it is, could this data be added to Figure 5?

The phrase "total body" here implies the intensity is a sum, which would mean it depends on worm length. I think it'd be important to clarify this.

Is this the average intensity? (so that the intensity is independent of worm length)

If the differences were not statistically significant, I don't think these conclusions can be drawn.

Is an increase in bend frequency known to indicate neuroexcitation? It seems like it could also result from metabolic changes, perturbation of feeding behavior, or direct effects on muscle cells.

If the trend wasn't statistically significant, I'm not sure this is worth commenting on.

Is it possible to make a more more precise statement than "basically consistent"? This is a little confusing, especially since there was a statistically significant difference between the control and the higher dose.

How was worm death measured?

For the 1mM MCP condition, it looks like most worms were still alive after 24 hours, but the bend frequency of all worms was zero. Does this make sense?

It looks like this figure is missing the number of worms analyzed for each condition. Also, the survival curves in Panel A are in discrete steps of the same size, suggesting a smaller number of worms than there are dots in Figures B and C.

Again, I think a baseline here would make this claim more convincing. In other words, what aspects of the differentiation dynamics described here could only be captured by ESPRESSO?

It would be helpful to compare this result to some baseline obtained from an established method like cell painting. In other words, can existing techniques also readily distinguish these cell types?

it would be helpful to also provide some absolute measures of throughput here, such as how many FOVs of a given size and resolution can be imaged per unit time.

It sounds like the clustering was done after the embedding step; that is, using the low-dimensional embeddings from PacMAP, rather than the original feature matrix. If so, I'm worried that this will result in inaccurate clusters, as PacMAP (like all such methods) does not perfectly preserve the relationships between the original high-dimensional feature vectors.

This is a bit confusing; it's so vague and general that it sounds like it could describe almost any protein.

I'm not sure that two examples can constitute evidence for or against filtering. Is it possible to use a ground-truth dataset to make this kind of filtering/no-filtering decision with more confidence?

This is a bit confusing, because protein language models are a kind of deep learning model. It would help to clarify what "deep-learning-based models" refers to in this context.

What does a reliability index value of "1" mean?

The phrase "within the first 50 hits" feels tantalizing. What else appeared among the top hits? Were there hits that were surprising or potentially false positives? And were there proteins that should have appeared among the top hits, but didn't?

Again, it would be really great to quantify what "highly similar" means here.

It would be helpful to mention details about the hardware here, as the time cost is hard to interpret without that information.

Again, it would be great to quantify this and/or to discuss some examples of proteins for which this is a real problem.

I think these claims would be more convincing if they could be quantified and if the performance of AlphaFind could be compared to other existing tools, if possible.

What is the relationship between this approach and approaches to indexing or similarity-based lookup used by common vector databases?

It would be helpful to explain in more detail how this is done, since it seems like a crucial step.

It's surprising to me that chromosomes are randomly ordered; this feels a bit like the equivalent of randomly shuffling the clauses of a sentence. It would be helpful to explain this choice or discuss reasons why it might or might not be a concern.

This feels confusing: if start tokens are unique to species, how is UCE able to generate embeddings for datasets from species it was not trained on?

This feels awfully hand-wavy. I can understand that a leveling off is expected at some distance, by why at 5 hops in particular?

It would be helpful to mention here what these three species were and how distantly related they are to the eight species on which UCE was trained.

This result feels hard to interpret without a comparison to other approaches or models. In other words, are embeddings from UCE uniquely able to capture the information required for this classification task?

This feels a bit subjective; I think this claim would be more convincing if it were grounded in a quantitative measure of clustering accuracy.

If possible, it would be helpful to contextualize these relative increases in performance, particular given that none of the models listed in Supp Table 1 appear to significantly outperform using the log-normalized raw data. (the "overall score" is 0.74 for UCE and 0.72 for "log-normalized expression"). Without more context, it's hard to know what this means, whether it should be surprising, whether it reflects limitations of the metrics or of the models, etc.

Also, I think it would be more transparent to mention here that there are two metrics for which UCE does not outperform other models (the ARI score and the "ASW (batch) score").

It would be helpful to understand the context and motivation for this design decision. In other words, what aspects of UCE's performance depend (or are suspected to depend) on including information about genomic position?

It would be good to clarify here if "training data" refers to the data used to train the protein language model or UCE itself.

Again, it's great to see something like this quantified so carefully!

It seems surprising that there is such a big difference from 1x to 1.4x. Is this by design? (was the 1x intensity chosen from prior experience or experiments to be as high as possible without inhibiting cell division?)

It's super helpful that an absolute measure of intensity is provided here, but it would be great to also include the wavelength (or range of wavelengths) of the excitation light.

Is it possible to correct this for the fact that, as the colony itself expands, cells near the edge necessarily must move more than cells in the center (which will not move at all, if the colony as a whole is stationary)?

It's great to see a subtle detail like this quantified so carefully! Is this consistent with prior work (if there is any)?

This is a bit hard to understand. How is distance in time measured? (i.e. the difference between the time of mitosis onset in the manual annotations and the segmentation results)

how were these weights chosen? And is it correct to think of these weights as a kind of correction for the class imbalance between non-mitotic and mitotic nuclei?

how were these numbers of frames chosen?

This is a little confusing, especially the "thresholded by 2" part, and I didn't find the caption in Supp Fig 1 to be that much clearer. It would help to explain the origin of the variability in the predictions (in other words, what is an "instance of the same model"?)

It would be great to mention what kind of images the pre-trained model trained on. Do you have a sense for how important it is that pre-training be done on similar images? (and what kinds of similarity are most important: cell type, imaging modality, magnification, etc)

Why 0.55 and not 0.5?

would it be possible (and meaningful) to mention how many GPU hours this required? Also, some more details would be helpful for non-ML experts; e.g., why the choice of 100 epochs, was a stopping criterion used, which epoch was used for the final analysis/results, etc.

This equation is a bit opaque; it would be helpful to explain what the superscripts and subscripts of theta mean.

Does the 1-1.5m figure mean single-cell images? or FOVs? It would also be super helpful to comment on how this dataset size was chosen. Was it the minimum amount of data required for this level of performance? More generally, did you do any experiments varying the quantity or diversity of the training data?

I think to make this claim more convincing, it would be important to show how many genes in the 1640 library are very similar to (rather than merely identical to) genes in the 124 PoC library ("very similar" is obviously subjective but I'm thinking of homologs/paralogs or genes that are components of the same complex or pathway)

it would be helpful to mention here what cellular structures of features the pS6 antibody labels, and also (for the non-biologists among us) what mTORC1 is

It's awesome to see such an explicit and direct comparison of classic feature engineering with modern unsupervised ML models!

If possible it would be great to quantify how much better the DINO-based approach is; Figures 4a-d are a bit hard to understand at first and obscure the relative differences; Fig 4d in particular doesn't give the impression that DINO is that much better than the CellStats approach (even though the 0.12 of DINO vs the 0.09 of CellStats is actually a 30% improvement!). Also, some measure of statistical significance would be helpful; in particular, how likely is it that the 0.09 vs 0.12 in Fig 4d is reproducible?

It feels like there's a typo here somewhere, since genes don't really have a "mechanism of action" and the screen here does not involve compounds but rather gene KOs. Is the idea to use the phenotype of the KOs to cluster genes by the MoA of the compounds that target them? In any case, the reference to MoAs here is doubly confusing because the clustering shown in Fig 4E appears to capture cellular localization (and also pathway membership?), but I couldn't see any discussion of the clustering relative to the MoAs of the compounds used to select the 300 genes