On 2022-10-11 18:59:18, user Willow K. Coyote wrote:
General assessment:<br /> D’Costa, Hinds et al. develop a framework to infer protein fitness based on laboratory evolution data using DHFR as a test case. This framework is based on a generalized Pott’s model, which infers parameters from successive rounds of selections using sequencing data. This model works well to predict fitness and epistasis within the experimental data’s bounds but cannot predict beyond it. Overall, we enjoyed the exploration of the model and the data, we had a difficult time understanding what major insights were gleaned from the work, and we found the manuscript was written for perhaps a bit too specialized of an audience. From our perspective, the manuscript could be greatly improved by focusing on the novelty of the models and including additional context for more general audiences.
Strengths:<br /> 1)The manuscript is a rigorous and model driven exploration of how data can be used for training <br /> 2) The authors transparently present the performance of there models even when it does not work as planned.
Weaknesses:<br /> 1) It was difficult to identify what the key takeaways of the paper were. Perhaps working on the narrative would make it easier to identify these takeaways.
2) From our reading we found the manuscript was written at an expert level. We found this made it difficult to interpret some of the results.
High-level Recommendations:
1) The quality of a high throughput assay is largely determined based on the quality of the input library. However, the authors do not show the level of coverage gleaned from the library that could result in some parts of the gene not being well-represented within the model. We would love to see more discussion of the raw sequencing data and its analysis to get a sense of the overall quality/coverage of the DHFR libraries and the signal-to-noise within the selection.
2) From our reading the overall framing of the manuscript appeared to be for a very technical audience. As readers we are very familiar with genotype-phenotype assays and to a limited degree models to represent the data from this assays. However, as we are not deeply familiar with Potts models and their novelty in the context of fitness landscapes, it was difficult for us to understand portions of the manuscript. In addition as using Pott’s model to represent the directed evolution experiments appeared to be a major novelty for the manuscript it would be useful to provide more background for why this specifically is novel.
3) From our reading the interpretation and exploration of the model was focused on describing the smooth or ruggedness of the fitness landscape and testing whether one could extrapolate from this fitness landscape to future predicted generations of sequences. We found it admirable the authors shared that the extrapolation did not work. Fot the fitness landscape exploration more background would likely benefit a non-expert reader. For example, the fitness landscape was described as being ‘Fuji-like’. For those familiar with fitness landscapes, this means a smooth landscape with a single peak. However, for those unfamiliar, this could be somewhat confusing. Furthermore, perhaps it would be worth exploring what we can learn about fitness landscapes with the model, as this could yield further insight.
Specific discussions of results and figures:<br /> Figure 1: Shows the directed evolution experiment using error-prone PCR on DHFR to search the fitness landscape for diverse sequences encoding DHFR activity. A: In the text the authors could explain why they stopped the experiment at round 15, but not earlier or later. B: This panel shows clearly that round 15 is distant from other rounds. Within D the authors show a plot of what fraction of mutations are accumulated throughout the experiment at a given position. A high score here likely implies that there is a beneficial impact of mutating a residue at a position. In the text position, 20 is discussed widely but is not noted on the figure while the figure notes positions 71 and 117 are noted but not discussed until later during the interactions discussion mostly focused on Figure 2. We found this somewhat confusing and it would perhaps be beneficial to note where specifically residue 20 is (presumably within the greyed out ‘Nucleotide Binding’ part of the plot). Perhaps more discussion of why mechanistically N20D is beneficial would be useful for aiding in interpreting the result.
Figure 2:<br /> A: Shows a heatmap of how the model predicts fitness for mutations at each position. Error-prone PCR was used to make libraries for this experiment as in classic directed evolution experiments and is well known to not result in full coverage. Within the model it appears that most positions have a predicted fitness with many being 0 (based on the greyish color). We would like to know the percentage of variants present for the predictions. As illustrated later in the manuscript, the model does not perform well when extending beyond the data. Perhaps if positions with low confidence or without frequent observed mutations at the beginning of the experiment were withheld the performance of the model would improve.Within the text the authors mention how the heatmap highlights the importance of the catalytic residues but as presented it is difficult to see this as we did not know where these are within the heatmap. It would be useful to have annotations for where the catalytic/functional residues are. It appears that the N20D that is enriched with the library is not positive within the model. Why could that be? <br /> B: The authors mapped the average predicted mutational effect onto the structure of DHFR. The discussion of this data is rather limited to negative impacts within the core. Perhaps it would be useful to talk more about what can be learned or compare to previous library studies of DHFR such as recent work done in Kimberly Reynold’s lab (McCormick, J., 2021 eLife).<br /> C-D: The authors then explore interactions within the model as a way to look at epistasis by looking at interactions across residues. Within D the authors show specific examples mapped onto the structure focusing on either side of the NADPH binding site. It would be useful within C to highlight interactions that appear important and highlighted in D. <br /> E-F: The authors compare how well the model can be used to identify contacting residues based on interactions scores. The authors show that including the 15th round improves model performance. For many interactions contacts are incorrectly predicted, however within the text the authors discuss how the model performs well compares to other approaches. For many regions of the contact map especially the end of the gene there are few predictions. Why is that? Overall, we find this exploration of the models to be difficult to interpret as from our view it appears the model is frequently predicting incorrect contacts or no contacts at all. Perhaps this is expected and that should likely be discussed further within the text.
Figure 3: Using their model trained on experimental data, they simulated several evolutionary trajectories of DHFR until a fitness peak was reached. Each simulation converged on the same peak, relatively close to the WT sequence. Following this, simulations were performed starting from the last experimental time point; however, mutants after this point were found to be inactive. In the discussion of these results, the authors note several reasons for this shortcoming of their model, all of which seem reasonable and highlight the need for more data to effectively use such models. Key suggestions include a discussion of how/where the inactivating mutations occur in the enzyme. Are these concentrated in specific hotspots, or are they in areas well sampled by the preceding libraries?
Suggested citation: “Deciphering protein evolution and fitness landscapes with latent space models”, Xinqiang Ding, Zhengting Zou & Charles L. Brooks III, Nature Communications 2019
Format of review: We reviewed this paper as part of our weekly paper discussion. We read a paper in depth weekly and discuss it as a group. In contrast to most journal clubs, we do not make slides, and everyone is an active participant in presenting a paper in which we typically each will describe each figure within the paper. As we are a group with diverse backgrounds, the person with the most experience in that area will bring everyone up to speed if anything is confusing. We focus on going through a paper from figure to figure. For these reasons, most of our comments and suggestions are about the figures. Beyond that, we try to understand the major points a paper makes and whether the data presented in the figures supports it. Different people wrote a description for each figure they wrote.
Authors of Review: <br /> Willow Coyote-Maestas<br /> Matthew Howard<br /> Patrick Rockefeller Grimes<br /> Christian Macdonald<br /> Donovan Trinidad<br /> Arthur Melo
Relevant expertise: Scientifically, we come from many different backgrounds but most people within our group have experience with membrane proteins, high throughput genotype-phenotype assays, and developing assays for measuring how mutations break membrane protein.