Peasgood et al. (unpublished)

Unit-change comparability

📚 Further Reading: Unjournal Evaluations The Unjournal has commissioned independent evaluations of papers relevant to this debate: → StrongMinds & Friendship Bench Evaluation — Critical assessment of HLI's meta-analysis and cost-effectiveness claims → Long-Run Effects of Psychotherapy on Depression — Cuijpers et al. meta-analysis on therapy durability → Cash Transfers vs Psychotherapy: Comparative Impact — McGuire et al. direct comparison in Liberia → Mental Health Therapy as a Core Strategy (Ghana) — Barker et al. on scaling community-based therapy

mortality-focused interventions

Recent work has attempted to estimate the neutral point empirically. The estimates vary widely (0.6 to 6.0 on a 0-10 scale) depending on the elicitation method and sample. HLI is advising the UK Green Book guidelines and has compiled the following estimates:[*]Table compiled by Samuel Dupret (HLI), shared March 2026. Note that different questions elicit very different values—asking about "life no longer worth living" yields lower estimates than asking about "minimally acceptable" levels. Source Value Method Sample Samuelsson et al. (2023) - HLI pilot 1.26 Asked when life is no longer worth living on 0-10 LS scale N=79, UK Samuelsson et al. (2023) - HLI pilot 5.30 Asked where balance between satisfied/dissatisfied on 0-10 LS N=128, UK Peasgood et al. (unpublished) 2.00 Time trade-offs (QALY method rather than wellbeing scale) N=75, UK IDinsight Beneficiary Survey (2019) 0.56 "At what point on the ladder is it worse than dying?" N=70, Ghana & Kenya Moss (Rethink Priorities, unpublished) 2.49 Asked level preferring alive to dead (converted 0-100 → 0-10) N=35, likely UK Moss (Rethink Priorities, unpublished) 6.05 Asked minimally acceptable level to live an extra year N=101, likely UK Jamison et al. (forthcoming) 2.39 Policy comparison: saving people from dying (0-100 → 0-10) N=1800 (Brazil, China, UK) Jamison et al. (forthcoming) 2.54 Policy comparison: saving people from non-existence (0-100 → 0-10)

Practical guidance for funders now Given the uncertainties above, what should funders actually do? This section offers a decision-oriented framework, not a single prescription.

paper develops methods using calibration questions[5]Survey items with objectively correct answers—the same for all respondents. Benjamin et al. use visual calibration (e.g., "How dark is this circle?") to reveal individual scale-use tendencies without text-interpretation issues. and vignette exercises[6]Unlike traditional anchoring vignettes (rating hypothetical people), Benjamin et al.'s vignettes ask respondents to imagine situations in their own life and rate dimensions of well-being for themselves. Part of an "ideal approach" that also requires multi-dimensional wellbeing questions and stated-preference surveys. to detect

We're organizing the discussion around four key questions:

The neutral point is the life satisfaction level representing neither positive nor negative welfare—essentially the boundary between "life worth living" and "suffering." Estimates range from 2-5 on the 0-10 scale. Peasgood et al. (2018) tentatively estimate ~2.

evaluation summary

QALYs (quality-adjusted life years)

instruments like EQ-5D

Other measures include QALYs (quality-adjusted life years), income-equivalent measures, and multi-dimensional poverty indices. QALYs are similar to DALYs but measure health gained rather than lost.

Confirmed Participants

—and what would change their minds?

Unlike WELLBYs, DALYs are based on expert-derived disability weights rather than self-reported wellbeing—weights are constructed through surveys of health professionals rating hypothetical health states.

Zoom chat for quick reactions;

Segment structure is set; timing may adjust slightly. Updated March 11, 2026

calibrated

Consider the value obtained when using the best feasible measure for cross-intervention comparison in contexts like the focal context. What share of this value is obtained, in expectation, from using the simple linear WELLBY measure (as defined above) for all interventions?

Consider the value obtained

Essential

1. WELLBY Reliability and Value

See the canonical formulations on Coda.

Vignette exercises: respondents rate hypothetical people's life satisfaction based on descriptions, revealing how individuals anchor the scale and enabling cross-person calibration.

Calibration questions ask respondents to rate well-defined scenarios (e.g., "How satisfied would you be if you won $1,000?"). By observing how people rate the same reference points, researchers can estimate individual differences in scale use.

Cost-effectiveness estimates vary by an order of magnitude depending on how WELLBYs are valued relative to DALYs.

Open Philanthropy

Each scale point represents an equal welfare increment. If violated, summing is invalid and interventions targeting different baselines become incomparable.

nterpersonal Comparability LSA = 7 ≈ LSB = 7 implies UA ≈ UB When two people report the same score, they experience similar welfare. Scale-use heterogeneity violates this assumption.

equires four implicit assumptions

1 WELLBY = 1-point increase on a 0-10 life satisfaction scale × 1 person × 1 year W = Σi Σt LSit

We'll produce a practitioner-focused summary document, belief elicitation results with confidence intervals, and structured notes.

Participants can opt out of recording for specific segments if needed

(Note: QALYs may be more directly comparable than DALYs for this purpose.)

scale?

Where is the "neutral point" on the scale?

When comparing a mental health intervention (measured in WELLBYs) to a physical health intervention (measured in DALYs)

but more work is needed.

Each has strengths and limitations—and how they relate to each other, and whether either reliably captures what matters for human welfare, directly affects which interventions get prioritized.

adversarial manipulation.

Our results support AI as structured screening and decision support rather than full automation,

xhibiting consistent failure modes: compressed rating scales, uneven criterion coverage, and variable identification of expert-flagged concerns.

often approach the ceiling implied by human inter-rater variability on several criteria,

high-quality but noisy reference signal

narrative critiques

overing economics and social-science working papers

Peer review is strained, and AI tools generating referee-like feedback are already adopted by researchers and commercial services—yet field evidence on how reliably frontier LLMs can evaluate research remains scarce.

“high” reasoning effort

OpenAI Responses API

returned file id keyed by path, size, and modification time.

d the best performance from top reasoning models

Zhang and Abernethy (2025) propose deploying LLMs as quality checkers to surface critical problems instead of

but still recommend human oversight.

emphasize

The population of papers