Do they actually do this in the paper? doublecheck

Is this a reasonable examlpe? Do they ask questions like that in the exercises mentioend in the paper?

What's the source for this OOM claim?? Find and link it with a verbatim quote . #implement

Also it's not in our 'evaluation summary as far as I know'

It's now "Coefficient Giving" -- correct this on every page. And hyperlink "https://coefficientgiving.org/research/cost-effectiveness/" here. #implement

David Reinstein --- personally, this is the one I find least plauslible and most important.

I don't think this one is necessary if we can (instead) assume that differences are equivalent. For example, if we assume that person A is actually experiencing higher welfare at all levels of reported score, but the differences between the scores are comparable, then compared to interventions for measured differences in well-being, that shouldn't matter.

I think it could also still be reliable if the distribution between the two populations is the same, even though we don't have specific inter-person comparability between any two compared individuals.

Give a linked source and citation for this.

Those are not clearly defined here, nor the indexing

Change this to "we hope to" and "We will share outputs". -- I can't guarantee right now that we'll get enough input or have bandwidth to produce this. #implement

Add "and we will ask for final approval before posting anything". #implement

Leave out the QALYs parentheses bit here. Add "(or QALYs)" after "~1 SD in DALYs". #implement

Add "is a move from 1-3 for one person as good as a move from 1-2 for 2 people"? At the end of this paragraph... "even if these don't hold, does the linear WELLBY aggregation yield 'nearly as much value' for decisionmaking as other potential measures"? #adjust #implement

Remind me why the neutral point is important.

Either of these, especially the physical health intervention, could be measured either way. This overstates it a bit. Perhaps, just to give this as an example, suppose there is a case... #adjust #implement

"more work is neeeded" That's very much vague -- we nIt would be nice to have at least one specific point suggesting that the difference in scale means potentially matters and merits more study

I'm allergic to platitudes. IIRC you should have some notes somewhere providing at least one case where this matters .

I don't think we discussed adversarial manipulation or have any results on it, so I'm a little worried that whatever generated this discussion is doing a sort of generic pandering and putting in what it generally expects to see in papers like this.

This seems like a sort of milquetoast generic caveat. In what sense is this what our AI results support? This seems a bit pandering.

I'm guessing this is a bit premature/too much rounding up a few observations to general conclusions, but let me look at the results a bit more carefully.

This is interesting and strong. It comes across maybe a little bit overstated, so we just need to be careful about how we're framing this result.

I think this is right, but the term "reference signal" sounds technical in an information theoretic sense, and we want to make sure we're not misapplying it.

Yes, we focus on the critiques here, but the on journal evaluations do more than just critique. They discuss, they offer suggestions, implications, et cetera.

"covering ... working papers" Is mostly accurate but not quite right. We don't cover all working papers, and we have a specific focus on research relevant to global priorities. We can also evaluate post-journal publication, but I'm not sure how to best summarize this in a simple way in the abstract.

The idea of "open evaluation platform" also could be a bit confusing here because it's not mainly about crowd sourcing. Yes, the "paid expert review packages" cover this, but I don't quite think this is worded in the best possible way.

This is a decent first sentence, although it bears the marks of AI-generated text. But also I'm not sure if it's really in line with our newest spin on this.

Not relevant to Pro -- cut this

"Responses" is the newer one (as of 4 Nov 2025)

what does this mean? "Keyed by" ?

This implies it is kept on the server and won't need a later upload.

Best relative to what? Better than the 'non-top reasoning models'? @valik

Is this the only empirical work? I thought there were others underway. Worth our digging into. Fwiw I can do an elicit.org query.

why? based on some evidence of LLM limitations or risks?

I'd say 'they argue' instead of 'emphasize'; the latter seems like a statement of absolute truth that we agree with.

Should we adjust "the population of papers" to "the reference is" ? to be more explicit?