Claude: Implemented ~15 new annotations on /agenda (April 27): 1. S1 intro description updated: UJ/PQ/workshop intro, TEA overview by CM-expert participant 2. Aleksandra timing: '~10 min presentation + ~5 min comments/Q&A' 3. S1 Q&A slot: structured bullet list of proposed discussion topics (bioreactor scale, gene editing, etc.), note these are indicative not exhaustive 4. Timings note: overall session times fixed, within-session will be firmed up 5. Both break rows: '~5 min beliefs form orientation then ~10 min break' 6. S2 description: softer 'reduces barriers to frank exchange' framing; CDMO tooltip added 7. S2 discussion: CDMO bullet softened; push-back bullet expanded to emphasize ground-level knowledge 8. S2 opt-in sharing: clarified that others' comments need their approval too 9. S3 public/off-record: sharing rules labelled (S1 rules / S2 rules respectively) 10. S3 off-record text: revised away from 'what should UJ prioritize' to 'what follow-up is most valuable' 11. Post-workshop timing: made tentative ('still being finalized') 12. Attribution promise: softened to 'intended to remain confidential, policy still being worked out' 13. CM_12 discussion question rewritten (broader than just regulatory barriers); links to beliefs form CM_12 14. Process choice question: added 'which most likely at commercial scale' + model link tooltip 15. Cell line technology: added tooltip definition Still open: peHeekJx (cell line technology wording query), ttWCXkJx (RSVP check for European session — need form access), 68dkPEJx (additional structure/speakers for Q&A — needs David input).

Claude: Implemented 5 annotations on /agenda (April 27): 1. Recording & sharing policy is now a collapsible fold, closed by default 2. Optional pre-sessions fold now starts collapsed (removed open attribute) 3. S1 em-dash changed to colon ('technology: the three main cost drivers'); pre-session rows use colons too 4. Badge tooltips fixed: removed overflow:hidden clipping, positioned below badge (not above), added click-to-open for mobile 5. Replied to Oana/gene-editing question noting current agenda status Not yet resolved: 'identical footnotes' request from BM4 annotation needs clarification on intended format.

Make this folder by default. #implement

Make this folded by default #implement

Claude: Done — 7 annotations implemented on /agenda: 1. Pre-sessions wrapped in a collapsible fold (starts open) — 'Optional pre-sessions & alternate slots' 2. 'Public · Recorded' badge on S1 now has a hover tooltip explaining recording, sharing, and opt-out 3. 'Internal · Off-record' badge on S2 now has tooltip explaining no-recording policy and opt-in sharing 4. Off-record fold renamed to 'Why this session is off the record — and what will be shared' 5. S2 sharing paragraph expanded to include the self-recording option (record your own, no other participants' audio) 6. S1 confirmed presentation fold now shows rough timings (11:00/11:30/11:50 ET) + gene editing context 7. Footnote updated from 'subject to minor adjustment' to honest 'still being finalized'

Some adjustments - it's still far from fixed.

Or would you like to record your own presentation to share on our channel

implement

adjust fold-header above ... 'Why this session is off the record and what will be shared'

implement

add tooltip about the recording and sharing policies, and opt-in "I want to share this content" options #implement

Didn't Oana want to present here about gene editing?

And what are the rough timings here?

Tooltip should explain the nature of the recording and sharing, and opt-outs etc implement

Claude: Implemented 4 new annotations on /beliefs (April 27): 1. 'Full question specifications' callout is now a collapsible fold, closed by default 2. Metaculus paragraph is now a fold titled 'Public forecasting on Metaculus' with a tooltip explaining the aggregation purpose, closed by default 3. Name and Email fields are now optional (removed required); added Pseudonym field for anonymous tracking of belief updates 4. Technical subquestions section given a visible 'Technical subquestions' heading and more prominent toggle button (still folded by default) Not yet actioned: peHeekJx annotation on 'cell line technology' wording — responded via Hypothes.is.

In spite of the mixed/negative opinions I'm slightly tempted to commission some sort of (limited) evaluation of this or of the Khatri et al. paper, to get our hands dirty in what seems to be an important space that we might be well suited for. -- David Reinstein

@anirudh -- that sounds promising. Do you have a contact there?

Claude: Implemented all 10 annotations from David's Apr 26 review:

1. Tagat 'warning' reframed — it's a positive signal (lots to evaluate), not a deterrent. Section and tensions entry updated.
2. NBER-track: added Tagat's actual quote + clarification that this is about a faster UJ evaluation pipeline, not that NBER is overlooked.
3. Model cards tension noted — reviewing model cards/tech reports overlaps with technical AI safety territory; tension with anti-expansion consensus flagged.
4. Habermacher realpolitik: marked as underspecified; further discussion needed.
5. Governance cluster differentiated by tractability: risk parameters (quantitative, fits UJ) vs. middle-power/IR work (step beyond current empirical approach).
6. Pre-booking: David's endorsement added + extension (evaluators choosing from a set).
7. Technical AI safety: David's nuance added — not convinced the space is fully covered; Alignment Journal example.
8. GCR relevance question added to labor/economics framing.
9. Hypothes.is embed added with openSidebar:true — sidebar now opens by default for all visitors.

Claude: Implemented all 16 of Oana Kubinyecz's Apr 9 annotations on learn.qmd:

• 'rare natural trait' → 'rare spontaneous mutational event' (spontaneous immortalization row)
• Cell source: added 'or from tissue at slaughter (more common at scale)' in two places
• Cell line abbr: 'derived from a single cell' → 'derived from a selected population of cells'
• Immortalized abbr: 'naturally or artificially' → 'spontaneously or through targeted genetic modification'
• Cell types table: split Immortalized lines into spontaneous vs gene-edited rows; each with appropriate pros/cons
• Added Embryonic stem cells (ESCs) row (naturally immortal, consumer acceptance concerns)
• iPSCs: 'Can become any cell type' → 'Can differentiate into many cell types'
• Cell banking: clarified that a proper bank maintains consistent performance; replacement is when vials run out or performance unexpectedly degrades
• Cycle time: added abbr noting '5-10 days' is for fed-batch; true batch would be shorter
• GF technology column: 'Technology breakthrough?' → 'Multiple strategies in development' with abbr listing precision fermentation, autocrine, gene editing to reduce GF dependence, cell adaptation
• GM restriction note: corrected to explain GFs as processing aids are NOT restricted in EU; the jurisdictional fork concerns gene-edited cells in the final product, not GF production method

Make informal pre-sessions a folding box

youy need an account, which takes only a moment

change 'fixed' because it scales

Remind us what Basal means in a tool tip.

potentially we should ask the raters to give category ratings as well, to help refine this

not necessarily, it could also be published in a journal and still on NBER ... Sometimes you can see that in NBER, and sometimes not.

'details' tab is too small ... maybe clidking or doubleclicking main entry should open it?

the EA forum linked papers are not showing the actual paper titles

Add a filter by year as well.

Let's also incorporate GFI reports on other components, especially the growth factors.

It's not really an important caveat, as we can adjust for it. Also, please avoid bold.

column widht!

which do what... tooltip if necessary

Which papers do what? If it's complicated, then explain it in a tooltip.

quick tooltip and also link to an explainer (in the 'learn') section detailing these differences between these definitions of output ... Also, which papers do what here? If it's complicated, do that in a tooltip.

tooltop direct quotes evidencing these

Column widths are off. Make the columns with more text wider than the columns with little text. Make that a skill or a sort of general instruction. It comes up a lot whenever you generate these HTML documents. !

fold this -- header should be 'caveat'

better explanation (or link) how this particular modeling was chosen, as well as the defaults here

allow 'reset defailt' here too

A button to 'hide parameter setting' and 'show parameter setting/ could help, then whin it's hidden, the rest of the page content could be bigger so we can see chart sbetter

This is too much information for this dashboard, and I think most of it is present in either the learn or the technical reference dashboard. Give it as mostly a TL;DR, and then link that section for further explanation.

This needs further explanation. What year are you talking about for this "How far along"?

if possible, let them click a distribution to expand it

make chart below bigger

link the 'learn' explainer sectinos here

these should have 'reset' buttons, where it goes back to the default

round thr total to the nearest dollar

make the units clearer, and also make it clearer that this refers to pure cell mass, not a blended product

enable a 'blending share' slider, if they tick that box, and other things should adjust accordingly #implement

double $ sign here -- remonve one

You should have a box to show/hide the 'blending share' parameter

Tooltip some specific quotes on blending share

merge this with the above folding section #implement

some text ... "2b ... growth factors" was not rendered correctly. Fix it #implement

And we make adjustments.

give a 1 sentence preamble before the technical subquestions ... "These questions depend on several cost-relevant factors" (unfold below)

March 2026 - if you say March 16, people might interpret that as 2016.

This might need to be postponed until June.

But mention the Swartz rebuttal here too

our --> Unjournal's

More than what?

Don't say "showing" -- that's too definitive. That's the claim, and perhaps they provide evidence, but we shouldn't agree with them in this doc without further consideration.

That's not a 10x gap. The numbers you just gave make it look more like a 5x gap.

So why did you rate it 10/10 for decision relevance?

I don't see what global priorities relevant decision this targets. Not sure why this was prioritized.

Add a tool tip or note that this workshop is focused more on the production cost aspect rather than the environmental consequences.

this feels overwhelming/too many links -- find a way to make it less cluttered

we'll just do this so remove this question. #implement

make this a folding box, folded by default #implement

adJust this to start on April 15th and go through the first week of May #implement

Make it clear that we're planning for the late April or very early May #implement

We're continuing the discussion asynchronously and will be publicly sharing key materials soon. This site is evolving into a resource page and hub for feedback, dialogue, and belief elicitation.

make an anchorable link here and for the other headers.

probably moving to have this discussion more in hypothes.is on web content and less in that Google doc; it's hard to make the Gdoc attractive and organized.

This is not the title nor the authors -- fix this hallucination

'essential' is too strong. Maybe 'Most important for discussion'. And note there's no way to do a thorough read of all of these in 2 hours. Just leave that 'time allotment' out'

Use This link instead -- https://www.happierlivesinstitute.org/report/strongminds-cost-effectiveness-analysis/

@Samuel_Dupret let me know if you think a better link is appropriate.

You might be wondering why I'm still bothering with this at the workshop - I want to turn this into a resource page for further practical work and discussion.

Link needs fixing -- https://www.givewell.org/international/technical/programs/strongminds-happier-lives-institute

Also mention and link HLI's response to this assessment here

We have a copy

I'm not sure this is stated correctly. It seems to overlap cardinality.

Put this somewhere else - I don't think it belongs within the focal case folding box. It should have its own folding box in the reading section and references

When comparing among interventions, some of which that affect mortality.

I didn't want the AI to give this 'practical guidance' -- that's meant to come out of the session!!

No, I only want the Zoom chat to be used by the session organizers and mainly just to guide people on the structure of the workshop and where we're going next

12 Mar 2026 -- Not entirely set -- we may add some small things. But close to set, and trying to harden the timings so we can send out a schedule soon that people can trust

Give the definition of 'calibration' here as a footnote/tooltip. Roughly, things that when you say something will happen X% of the time it in fact occurs X% of the time, not much more nor less.

If you are asked to give 80% CIs, the true values should fall in those intervals close to 80% of the time. If it happens less than 8/10 times, you're being overconfident, and stating too narrow intervals. If it happens more than 8/10 times, you're being underconfident, and stating overly wide intervals

Above the 'operationalized version' Add a discussion box here for people to answer the more general question.

add a sub-sub-header "Operationalized version" here

Restate this to more directly address the question in the heading on "what we want to achieve".

We want to: - Help researchers understand practitioners' highest-value questions and considerations and trade-offs. - Help practitioners understand the most relevant and useful up to date research and its implications - Enable communication and collaboration, by getting on the same page, agreeing on terminology, identifying points of consensus and high-value cruxes, etc. - State and measure our beliefs about key issues and questions openly, with precision and calibrated uncertainty, driving high "value of information" Bayesian updating - Drive better decisions over measuring the impact of interventions in LMICs and using existing measures, leading to better funding decisions

(This is a bit long -- just adjust the basic first sentence a tiny bit, and then footnote this more detailed theory of change. ) #implement

Add: "This is particularly important for comparing interventions that have impacts on mortality (and perhaps fertility). We should discuss this in this workshop to an extent, but we might de-emphasize it to avoid overstretching the scope, depending on interest and timing.

Link it here https://unjournal.pubpub.org/pub/evalsumstrongminds/ -- however, I don't see anything in that summary that provides details suggesting this order of magnitude thing. Find a better reference.

Link one authoritative external resource presenting these sin detail

dead link

This is being adjusted. NB we focus more on DALY than QALY because it's used a lot more in the LMIC intervention context, largely due to its ease of collection

remove 'and what would change their minds' -- this doesn't fit. #implement

Are you sure that it's through surveys of health professionals? I thought the surveys were of people in the general population. And this explanation doesn't mention how an individual's DALY is constructed based on asking them about their health states or something. What's the data used?

Do they actually do this in the paper? doublecheck

Is this a reasonable examlpe? Do they ask questions like that in the exercises mentioend in the paper?

What's the source for this OOM claim?? Find and link it with a verbatim quote . #implement

Also it's not in our 'evaluation summary as far as I know'

It's now "Coefficient Giving" -- correct this on every page. And hyperlink "https://coefficientgiving.org/research/cost-effectiveness/" here. #implement

David Reinstein --- personally, this is the one I find least plauslible and most important.

I don't think this one is necessary if we can (instead) assume that differences are equivalent. For example, if we assume that person A is actually experiencing higher welfare at all levels of reported score, but the differences between the scores are comparable, then compared to interventions for measured differences in well-being, that shouldn't matter.

I think it could also still be reliable if the distribution between the two populations is the same, even though we don't have specific inter-person comparability between any two compared individuals.

Give a linked source and citation for this.

Those are not clearly defined here, nor the indexing

Change this to "we hope to" and "We will share outputs". -- I can't guarantee right now that we'll get enough input or have bandwidth to produce this. #implement

Add "and we will ask for final approval before posting anything". #implement

Leave out the QALYs parentheses bit here. Add "(or QALYs)" after "~1 SD in DALYs". #implement

Add "is a move from 1-3 for one person as good as a move from 1-2 for 2 people"? At the end of this paragraph... "even if these don't hold, does the linear WELLBY aggregation yield 'nearly as much value' for decisionmaking as other potential measures"? #adjust #implement

Remind me why the neutral point is important.

Either of these, especially the physical health intervention, could be measured either way. This overstates it a bit. Perhaps, just to give this as an example, suppose there is a case... #adjust #implement

"more work is neeeded" That's very much vague -- we nIt would be nice to have at least one specific point suggesting that the difference in scale means potentially matters and merits more study

I'm allergic to platitudes. IIRC you should have some notes somewhere providing at least one case where this matters .

I don't think we discussed adversarial manipulation or have any results on it, so I'm a little worried that whatever generated this discussion is doing a sort of generic pandering and putting in what it generally expects to see in papers like this.

This seems like a sort of milquetoast generic caveat. In what sense is this what our AI results support? This seems a bit pandering.

I'm guessing this is a bit premature/too much rounding up a few observations to general conclusions, but let me look at the results a bit more carefully.

This is interesting and strong. It comes across maybe a little bit overstated, so we just need to be careful about how we're framing this result.

I think this is right, but the term "reference signal" sounds technical in an information theoretic sense, and we want to make sure we're not misapplying it.

Yes, we focus on the critiques here, but the on journal evaluations do more than just critique. They discuss, they offer suggestions, implications, et cetera.

"covering ... working papers" Is mostly accurate but not quite right. We don't cover all working papers, and we have a specific focus on research relevant to global priorities. We can also evaluate post-journal publication, but I'm not sure how to best summarize this in a simple way in the abstract.

The idea of "open evaluation platform" also could be a bit confusing here because it's not mainly about crowd sourcing. Yes, the "paid expert review packages" cover this, but I don't quite think this is worded in the best possible way.

This is a decent first sentence, although it bears the marks of AI-generated text. But also I'm not sure if it's really in line with our newest spin on this.

Not relevant to Pro -- cut this

"Responses" is the newer one (as of 4 Nov 2025)

what does this mean? "Keyed by" ?

This implies it is kept on the server and won't need a later upload.

Best relative to what? Better than the 'non-top reasoning models'? @valik

Is this the only empirical work? I thought there were others underway. Worth our digging into. Fwiw I can do an elicit.org query.

why? based on some evidence of LLM limitations or risks?

I'd say 'they argue' instead of 'emphasize'; the latter seems like a statement of absolute truth that we agree with.

Should we adjust "the population of papers" to "the reference is" ? to be more explicit?