Reviewer #2 (Public review):
Summary:
Chakravarty et al. propose a 'synchronized framework' for studying perceptual decision-making (DM) across species -namely humans, rats, and mice. Although all species shared hallmarks of evidence accumulation, the results highlighted species-specific differences. Humans were the slowest and most accurate, rats optimized the speed-accuracy tradeoff to maximize the reward rate and mice were the fastest but least accurate. In addition, while humans were better fit by a classic DDM with fixed bounds, rodents were better fit by a DDM with collapsing bounds. While comparing behavioral strategies in evidence accumulation tasks across species is an important and timely question, some of the presented differences across species lack a clear interpretation and could be simply caused by differences in the task design. There is important information and analyses missing about the DDM and the other models used, which lowers the confidence and enthusiasm about the results.
Strengths:
The comparison of behavior across species, including humans and commonly used laboratory species like rats and mice, is a fundamental step in neuroscience to establish more informed links between animal experiments and human cognition. In this work, Chakravarty et al. analyze and model the behavior of three species during the same evidence accumulation task. They draw conclusions about the different strategies used in each case.
Weaknesses:
Novelty:<br /> While quite relevant, some parts of the work presented are more novel than others. That EA drives choice behavior and these choices can be described with a DDM have been shown before (see e.g. (Kane et al. 2023; Brunton et al. in 2013; Pinto et al 2018)). The novelty here mostly lies in the comparison of three species in the same task and in fitting the same exact model (close quantitative comparison of behavioral strategies). However, some of the differences lack a clear interpretation. For instance, the values of some of the DDM fitted parameters between the three species are not ordered "as expected" (e.g. non-decision time or DDM BIC). Other comparison results completely lack an explanation (e.g. rats' RT are near optimal while humans and mice are not). The aspect that I found most novel and exciting is the application of HMMs to each of the species. However, this part comes at the end of the paper and has been done without sufficient depth. There is almost no explanation for the results. I would suggest the authors bring up this part and move back to other aspects which are, in my opinion, less novel or interpretable (e.g. results around the optimality of RT).
Task design:<br /> Since there is no fixation, the response time (RT) reflects both the evidence integration time plus the motor time (stimuli are played until a response is given). This design makes it hard to compare RTs between species. While humans just had to press a button, rodents had to move their whole bodies from a central port to a side port. When comparing rats and mice, their difference in size relative to port distance could explain different RTs. This could for example explain the large difference in non-decision time (ndt) in Figure 3F between mice and rats. Are the measurements of the rat and the mouse boxes comparable? The authors should explain this difference more openly and discuss its implications when interpreting the results. The Methods should also provide information about the distance between ports for each species. I also strongly recommend including a few videos of rats and mice performing the task to have a sense of the movements involved in the task in each species.
(1) DDM
Goodness of fit:<br /> The authors conclude that the three species use an accumulation of evidence strategy because they can fit a DDM. However, there is little information about the goodness of these fits. They only show the RT distributions for one example subject (too small to distinguish whether the fit of the histograms is good or not). We suggest they make a figure showing in more detail the match of the RT distributions across subjects (e.g. they can compare RT quartiles for data and model for the entire group of subjects). Then they provide BIC which is a measure that depends on the number of trials. Were the number of trials matched across subjects/species? Could the authors provide a measure independent of the number of trials (e.g. cross-validated log-likelihood per trial)? Moreover, is this BIC computed only on the RTs, mouse responses, or both?
Overparameterization:<br /> The authors chose to include as DDM parameters the variability of the initial offset, the variability in non-decision time, and the variability of the drift rate. Having so many parameters with just one stimulus condition (80:20 ratio of flashes) may lead to unidentifiability problems as recognized previously (e.g. see M. Jones (2021) here osf.io/preprints/psyarxiv/gja3u). Their parameter recovery Supplementary Figure 3 shows that at least two of these variability parameters can not be recovered. I also couldn't find the values of these parameters for the fitted DDM. So I was wondering the extent to which adding these parameters improves the fits and is overall necessary.
Tachometric curves:<br /> The authors show increasing tachometric curves (i.e. Accuracy vs RT) and use this finding as proof of accumulation. They fit these curves using a GAAM with little justification or detail (in fact the GAAM seems to over-fit the data a bit). The authors do not say, however, that the other model used, i.e. the DDM, may not reproduce these increasing tachometric curves because "in its basic form", the DDM gives flat tachometric curves. Does the DDM fitted to the individual RT and choice data capture the monotonic increase observed in the tachometric curves?
Correct vs Error trials:<br /> In a similar line, the authors do not test the fitted DDM separately in correct vs error trials, which is a classical distinction that most DDMs can't capture. It would be good to know if: (1) the RT in the data of correct vs error responses are similar (quantified in panel Figure 2B because in 2E it is not clear) and (2) the same trend between correct and error RTs are observed in the fitted DDMs.
Urgency model:<br /> It is not clear how the urgency model used works. The authors cite Ditterich (2006), but in that paper, the urgency signal was applied to a race model with two decision variables: the urgency signal "accelerated" both DVs equally and sped up the race without favoring one DV versus the other. In a one-dimensional DDM, it is not clear where the urgency is applied. We assume it is applied in the direction of the stimulus, but then it is unclear how the urgency knows about the stimulus, which is what the DDM is trying to estimate in the first place. The authors should explain this model in greater detail and try to resolve this question.
Despite finding differences between species, the analyses seem mostly exploratory instead of hypothesis-driven. There is little justification for why differences in some DDM parameters across species would be expected.
(2) GLM and HMM
The GLM fits show nicely that humans, rats, and mice weigh differently the total provided evidence (Figures 6C-D). This may be because the internal noise in the accumulation of evidence is higher but also it could simply be because animals do not weigh the evidence that is presented when they are already moving towards the side ports. A parsimonious alternative to the "more noisy" species is simply that they only consider the first part of the stimulus. Extending the GLM to capture the differential weighting of each sequential sample (what is called the Psychophysical kernel, PK) should be straightforward and would provide a more fair comparison between species (i.e. perhaps the slope of the psychometric curves is not that different, once evidence is weighted in each species with its corresponding PK.
Choice Bias:<br /> Panel 3G (DDM starting point) shows that both rats and mice are slightly but systematically biased to the Left (x0 < 0.5). Panel 6D "Bias" seems to be showing the absolute value of the GLM bias parameter. It would be nice to (i) show the signed GLM bias parameter. (ii) Compare that the biases computed in the DDM and GLM are comparable across species and subjects; it looks like from the GLM they are comparable in magnitude across species whereas the in DDM they weren't (mice had a much bigger |x0| in the DDM), (iii) explain (or at least comment) on why animals show a systematic bias to one side.