colore on black text here is very hard to read

Would not attend live but offered to follow up asynchronously.

Fix this. It takes too long to generate.

how many use it now -- put this in context

check and annotate -- what does e.g., 5 wellbys per DALY mean in context, and how does it compare with what people currently do?

This seems like a valid objection, but I think we still phrase the question such that you would give a meaningful answer, or you could give a meaningful answer in this case in terms of the value generated if you were forced to use a single conversion.

The discussion might be more valuable, or I would say is likely to be more valuable than the response, particularly for this question.

let's do better to differentiate this from calibrated well-being. It's not fully clear to someone glancing at this briefly.

this is not a 'subquestion' ... label this page header differently

Add backlink to the workshop and to the elicitation form here.

Make sure you can access the literal question from this interface to know exactly what the respondents are answering. If these things get very long, you can use tooltips.

some visulaizations (bar or donut or mosaic graph) would be helpful here

I don't htink this is a stochastic model? Perhaps an extension of this should give these (correlated?) distributions. ... Squiggle-type modeling

There should also be discplay of the actual equation behind the model, and a folding box or linked page explaining it in more detail

Maybe put this model at the top, after a short preamble?

any prediction markets deal with the fundraising/donation/nonprofit askpets of this?

why? is this just a truism?

this advice seems on the overly generic side?

what is this -- things like this need definitions, in the text with a link and in tooltips

title font too big, taking upd too much screen saace

I'll try to respond / adapt to hypothes.is comments, especially if you flag @daaronr

Unjournal data

There's some text overlap in the diagram.

what 'costs of negative findings'? This is a bit of a vague sentence.

also state and interpret the equation in terms of 'benefits vs cost' -- easier to interpret I think

don't use "+" and "-" as variables ... we can do better

links don't need to be bolded

overlapping text in diagram below

remove 'or policy relevance' perhaps -- The Unjournal prioritizes research with global impact potential (although that's not what we mainly rate the research on)

not sure I understand the logic behind the latter part

how would someone know this?

This is a possible set of considerations -- I don't want to state it as definitive

maybe the 'gatekeeper' or 'editor' instead of the 'reader'?

Is it a truly 'public' signal, or only one the evluator sees after reading the paper? Or should we have both?

notation needs improvement, and it should be explained more -- how derived, how to interpret it? Tooltips and expanding sections could help

I suspect another paper has dealt with this question ... 'when noisy signals help the seller' or some such

this needs clarification, I don't quite see why this is the case. Isn't it possible that the author's signal is positive so they submit, but the evaluator reading the paper gets a negative signal?

relevant to whom?

what is 'acts' and what is 'c' -- underexplained

is this 'binary' rhe relevant threshold? Where did it com from? is it sort of generalizable? Consider if it misses some important nuance

who is the 'reader'? I think we mean the later journal evaluator, or career gatekeeper

base

simple

why?

this should be a fold or footnote -- give a quick statistic and footnote yow it was captured

too strong. And takes the 'high dowside' idea for granted too much IMO

too much 'not this but that' AI speak. And 'publicity' is vague here

more like 'editors and peer reviewers'

too much use of bold within paragraphs and prose

the latex/math is not rendering as well as before we moved this to Codex for editing. Can we recover the better formats and get the best of both worlds?

this needs further explanation and clarification -- 'usual channels' should already encompass clarification

this is the 'AI language of dichotomy' overused

You're using em dashes too much.

The language of this is a bit unclear. Try to make it easier to understand.

Don't use bold here.

I'm not sure if the fact that evaluators are paid here is relevant to this question - rather than giving these details, you could just say "The Unjournal is the focal example"

this is meta -- I don't want meta, or at least put that into an 'opt-in' list

We can swap in here the legal prioritization prototype -- https://uj-prioritization-prototype.netlify.app/legal/ -- please do this -- and note that we're looking for feedback and examples to help improve and train this. Note that we don't envision this prioritization to be mainly driven by AI models -- humans will be making the ultimate decisions -- but these tools can be very helpful in the process.

Give people the option to suggest/add content.

Show/hide

Proposed and in consideration.

Let us know if you have any questions about this.

this is not Swartz! adjust!

Make this a folding box - and the header should say AI/human collaboration in some way

Another folding box should have the standard call out about how we want feedback, and you can use the hypothesis tool for that.

Still considering how to frame this workshop, and it depends on interest and participation. One frame is directly targeting what we know about plant-based products, who consumes them, and what it suggests for potential substitution and animal welfare. However, that evidence seems to be rather thin, inconclusive, and premature, perhaps. (See links to EA forum posts, etc.) Furthermore, our evaluation of Bray et al. on experimental versus standard quantitative marketing/I.O. estimates of own price elasticities suggests perhaps deep uncertainty. and lack of ability to be confident in these parameters, not to mention cross-price effects and substitution patterns. This potentially motivates a pivot towards focusing on these methodological questions, as well as framing it in terms of "what can we know and what research is worth pursuing."

Change the date here. The date is still undetermined, probably late summer 2026.

Remove this page for now because it makes it seem like the workshop already happened.

This is not his full evaluation. He gave a very in-depth evaluation, and you've only taken one paragraph here.

1. Firstly, the header does not fully describe the critiques here. It's only one of the critiques.
2. Secondly, even in this scrollytelling depiction, we probably want a bit more about what the evaluators are saying, going into more than one theme very briefly, because this is the core of The Unjournal's value add.

Why "scene"? Are we making a video? Does it even need numbering

these tooltips are not coming up after seconds of hover. -- I only see the "?" -- please fix

We probably want a little bit of a transition here between the issue and the issue of measuring individuals' well-being through self-reports and what the Unjournal is now doing in terms of rating the paper, which is also on certain scales that may have subjective components themselves. Funnily enough. Make the distinction clearer here

This probably needs a little bit more context on why we're trying to measure people's well-being and happiness through self-reports.

the diagram is not fully explained? what does each dot represent? Should we be giving 'names of people' (or IDs, or types of people) to make that clearer?

Make it clear that the number line here is supposed to represent some fixed measure of true well-being.

I know this is meant for a public audience, but it's a little bit oversimplified. Perhaps we can say it in an equally concise and appealing way, but without making the absolute claims like "for one stubborn reason..." there may have been other reasons too. (Note to AI -- try to make this a persistent pattern in your writing. )

Individual "pubs" in the Unjournal PubPub interface

Make it clear that this is what we're calling the "hub" layout.

this is taking up a lot of space -- make it folded by default

--> evaluation package

--> evaluation package on one page

let's use actual content and structure from the paper!

This makes me think you're looking at this from a "deontological" standpoint.

In one sense, that's obviously true, as if the risk of causing harm is high enough, the expected value goes negative.

But if you're saying "We should not make even known positive expected value bets if the downside risk is too large"that's a judgment call, and it depends on your moral/ethical worldview.

I'd like to see ambitious research initiatives independent from traditional university/journal processes. If we have the funding, we can build these fields.

I don't fully agree that we need "EA AW community" hands-on involvement in the intermediate and technical steps.

I think it's more a matter of providing funding and incentives and clearly communicating the goals, priorities, and need for rigor to researchers, at enabling coordination.

But I agree that academic incentives on their own are not enough to ensure high-quality, credible work focused on animal welfare implications.

And it would indeed help if the researchers intrinsically cared about animal welfare and thus about producing useful and accurate results. This makes the incentive alignment easier, but I think it can still work even if much of the work is done by people who aren't intrinsically interested in animal welfare or don't think about effectiveness in the same way - as long as they can understand and embody the priorities in their work.

At The Unjournal, we are trying to bring together researchers, practitioners, and funders to do this sort of prioritization and coordination to be able to generate, communicate, powerful, useful evidence, robustly assessing its credibility and improving. Something we're trying to implement through our pivotal questions and workshops, e.g., https://uj-cm-workshop.netlify.app/summary ... and I think we're having some successes.

But I'm not saying we're necessarily the best positioned to do this, and I'd love to work with others or see others move forward on this.

I agree. I strongly believe that some coordination is necessary, and we shouldn't just rely on academic incentives. Large-scale, ambitious evidence and collaborations seem high-value to me.

I would say that this depends on whether you're willing to rely on what might be considered fairly "common sense" priors about the substitution effect in an environment where we have little evidence and it's very hard to collect reliable evidence. Perhaps for any intervention there will be some aspect of the model that we might need to take for granted as just a common sense implication.

I can appreciate that you might not agree that the price and taste equivalent plant-based meat would substantially crowd out the consumption of conventional meat. I find it harder to imagine that the same would hold for cell-based meat, but this is obviously a judgment call.

That said, there's a bit of a chicken-and-egg problem here in that I think it's plausible it's reasonable to assume that until cell-based meat is actually in restaurants and supermarket shelves, we won't have a good sense of whether people will buy it as a replacement for conventional animal meat. But the only way for it to actually get on the shelves is by there being substantial investment. So requiring this high degree of certainty might make it impossible for us to consider potentially high-value but risky investments in this sort of innovation.

The evidence is, in fact, all over the map. -- see e.g., the survey here -- https://forum.effectivealtruism.org/posts/3Eh8MbqLwFBsD7GK2/how-much-do-plant-based-products-substitute-for-animal#Existing_Research

But there are also doubts about whether we can reliably collect evidence in this domain. See https://uj-pba-workshop.netlify.app/context/pbm_fuller_report/ for an (AI-aggregated) synthesis ... Bray et al cast doubt on the reliability of even own-price-elasticity estimates in fairly standard settings -- see our evaluation of this paper here -- https://unjournal.pubpub.org/pub/evalsumbraybray/, and we're working on further discussion about the implications of this for substitution.

(FWIW, intuitively, I have a strong prior that there would be a fairly strong substitution, even if not one that would completely end animal farming. As Bayesians, our prior beliefs should count for a lot in a domain without substantial evidence.)

And they are funding more -- see their most recent post.

Confirmed. There does seem to be deep uncertainty here substantial lack of measurement

I'd like to see more justification of this bolded claim that it's indeed very sensitive to the assumptions. Can you provide a link or a footnote about this? Are there reasonable specifications under which it goes the other way?

The USA and WOR ones would seem to also, although the difference may not attain statistical significance in a conventional sense. But it still contributes to the evidence in that direction.

I would still admit that the evidence, as you presented it, seems to go the other way, though.

This table is somewhat hard to read and somewhat hard to see a synthesis of. I think something like a forest plot could be helpful here. And of course, it would be helpful to just report on the actual meta-analytical results.

What seems important to me is the expected magnitude of the difference in mortality between systems and the implications for the difference in suffering, factoring in the other differences in life quality and health.

In fact, in this case, the aviary mortality is actually lower than conventional cage mortality.

Should correct for multiple comparisons, we'll stop. At the same time, I disagree with drawing conclusions from a lack of statistical significance. To make a conclusion about "it's very unlikely that there's any difference large enough to be meaningful", you need to do something like an equivalence test, or report Bayesian posteriors over the difference.

What is the double arc sign transformation here, and why is it used? AI please remind us

This part certainly suggests that, but we should really present the net with some adding up and weighting for the magnitude of the suffering of each event.

These things and their consequent animal suffering/animal welfare burden could presumably be weighted and aggregated and compared. "More" could be slightly more in one case and far more in another case. What are the aggregate differences for reasonable assumptions here?

AI -- please look this up/clarify

I guess that's my take on what you shared too, but how do authors and experts in the field (other than you) interpret it? A bit of steelmanning+feedback could be useful imo.

AI: look this up, including in comments below, provide sources for other research.

if you want to rewrite or rework a bit, maybe lead with this, so we can understand what the evidence is getting at.

ok this informally suggests important heterogeneity to me, suggesting the need for nontrivial sample sizes.

have you tried to contact the authors? I think that would be high value -- both directly and in terms of field building-- and happy to help facilitate it if I can be of help. They might be particular eager to clarify and happy to hear how their research is valued ... and may also see this as a route to potential future funding.

Is that really how they presented it? Let's double check. That is extremely vague.

a fair comparison to what?

Again, I'm really not sure whether these are good or bad things. Why do we care if they recover their ability to move later if we're normally killing them with this?

OK this is explained a bit further below .... because they may wake up again before being killed

Okay, following the above, I guess the point is that you kind of see similar things happen from ice lorry and electric shock, so it's not clear why one followed by the other is the best. ??

Confused about what this is supposed to mean. Are we considering cutting their head off, or are you saying that if this happens, even though you cut their head off, that means that it probably doesn't indicate anything meaningful, just a sort of knee-jerk response?

are these things good or bad? Not immediately obvious to me.

I found your presentation below on this rather convincing. This also comports with what I've heard from other EAs (although perhaps the same circle of conversation). We need better evidence on which of these AW improving technologies actually reduce animal suffering. I'm in some discussions about possibly building and funding an evaluation service for specific tools and approaches (maybe something between The Unjournal and a fast-review journal, also inspired by Rapid Reviews Infectious Diseases).

Very small sample sizes do not always mean lack of inference. For instance, in very predictable contexts without a lot of noise, like, let's say, Newtonian physics, even a few data points could help us narrow our beliefs substantially. I like Richard McElreath's example about how you can substantially update on the share of a planet that is water by simply, even in the first few random samples, choosing a single point on a planet's sphere.

More intuitive -- if I ask 4 people to taste a drink and they all wince deeply in pain and disgust, I'm going to be highly confident it tastes bad. If all 4 smile and praise it, I'll be fairly confident that it's at least tolerable.

But I don't know that that is the case here. There might in fact be a lot of uncertainty and heterogeneity. What I wonder is whether the sample sizes observing the behavior and bioindicators of these fish are very expensive, or whether it could easily be scaled up with just a small amount of money. as a non-biologist, it seems intuitive that it should be cheap, but I might be missing something

I.e., 6 animals killed with each. See previous comments about sample size. ... what's the within treatment variation etc., and how costly is it per animal? How does this sample size compare with typical measurements in these domains? If the measurements taken are costly, could we get more reliability with cheaper measurements and a larger sample?

I think I would indeed lean in this direction, and I'd suggest that in fact most grants I've seen go the other way. (But I may have some biases here, this goes in line with what we're trying to do at the Unjournal, etc. )

[Consider -- does this post make a clear case for the 'should', demonstrating that the VOI of research here will exceed the expected (?) impact of the best or current 'existing work'?]

v speculative -- this may be highly timely if we think the Anthropic IPO will be driving some money to AW , and donors are evidence driven and 'difference-making risk-averse'

In terms of the 'expected impact'/'uncertainty of impact' tradeoff heard of Animal Welfare as being in between GH&D and X-risk/GCR/AI Risk.

Your concerns may be driven by "Difference-making risk aversion" -- see discussion here, for example https://forum.effectivealtruism.org/s/WdL3LE5LHvTwWmyqj/p/9EENSGhiQiKFaRh4t

Or you might be driven by something more deontological, related to 'do no harm', perhaps?

Maybe to be fair to note that this is along a specific dimension of transition -- conventional caged to regulatory mandated cage-free. Perhaps better evidence for other certified standards of free-range etc?

This is something Unjournal is trying to make happen. Come working with https://www.aw-econ.org/about and others. I think, given the very limited investment in this space, I think there are high marginal returns, although some of the value of information will be learning about what is or is not "epistemically possible".

I think what we have is a substantial lack of evidence in this domain. This is something the Unjournal is trying to remedy. (Unjournal.org, see https://uj-pba-workshop.netlify.app/ for a link to some of our efforts, and https://forum.effectivealtruism.org/posts/3Eh8MbqLwFBsD7GK2/how-much-do-plant-based-products-substitute-for-animal or an earlier take).

Evidence in this domain is very hard to come by, and there are substantial doubts about the extent to which we can even reliably measure these things. (The aforementioned post gets at this. Also see our an evaluation of the Bray et al. paper, which casts substantial doubt on the ability to even measure simpler things like own price elasticity with conventional methods -- https://unjournal.pubpub.org/pub/evalsumbraybray/ -- I'm working to follow up this evaluation package with more detailed evaluation managers' discussion and dissemination, focusing on the applied issues.

Following up on this further -- see https://app.notion.com/p/Validation-Evidence-for-Food-Demand-Elasticities-PBM-Pivotal-Question-376e97e2ad3381a898d3ceb589b265f2 and links within for a preview.

At the same time, there's also just not a lot of economics, social science, and marketing work done that focuses on animal welfare implications (or alternative proteins).

The aforementioned workshop on associated pivotal questions will be focused not only on what we know in this domain (~substitution effects within and between animal and plant-product consumption) , but also on what we can know with given data, what we might be able to reliably learn with more ambitious data collection exercises, experiments, etc., and what (and what is sort of fundamentally unknowable and does not merit further research investment).

I've been hearing this for a while. It's something that organizations like Animal Charity Evaluators are trying to address, but they don't have the dedicated resources and funding that, say, GiveWell has. Furthermore, there's no comparable academic/national resource base for animal-welfare-relevant research. For example, in the economics animal product space, it seems most of the "agricultural economics" is oriented towards supporting the farm industry. (Perhaps with climate change a secondary priority in some countries )

Note -- we'd probably make this additional to PubPub, as the latter goes into standard bibliometrics and information standards.

(Which in turn, brings the risk of divided attention)

Add a folding box presenting and explaining the equations behind this!

Give these clear anchors to hyperlink to, and allow people to extract hyperlinks

Let's try to use some referenced values as a starting point, linking them/tooltips. I don't think there's a lot of good research, but still good to start right

Just confirming this is indeed the status

document this better -- is red negative, green positive, and grey close to 0?

maybe we don't want to anchor too much on this --- NB that paper does not involve substitution. "Anchor paper" could be misinterpreted. This is The Unjournal evaluation package that is most strongly connected atm

evan in headers, let's have specific links (perhaps linking section below) for claims

this tracks

doublecheck -- if it's 50 everywhere just take it out of the data set -- it's just distraction and probably our testing the form

give response dates

'the possibility of'

what are these buttons meant to do? Are they supposed to be links?

Or should we focus mainly on the US context because of the greater 'review gap' as well as the greater role for court jurisprudence in the US. On the other hand US legal scholars may be paid more, overcommitted to lucrative and influential work, and thus less willing to do the evaluations.

--> are highly credible to legal audiences, useful to practitioners, and show potential for global impact.

reword this. A 'narrow pilot' is what we will do, that's not a precondition

this is a bit vague, needs clarification

why not?

This itself would be a useful public good, if we curated it well, with feedback from organizations that wanted to use this.

Naturally we will do this in a human-AI collaboration, with AI doing much of the initial search and filtering. (see https://uj-prioritization-prototype.netlify.app/ for a prototype for our main stream)

We did this about 8 months ago, but most of these will probably be stale

relevant yes. But how do we know it's important for these questions?

Fix the tooltip -- it shouldn't be in all caps

these are subquestions, but we're missing some of the 'goal oriented' questions here

add hyperlinks/headers back to the workshop here

give the full question language, and hyperlink the question in context (tooltip if long)

just say 'anonymous 2' .... "H" is confusing

Note -- I did not intent to have CIs over probabilities. This was an artifact of a changed question and vibe coding. Also investigate whether this was the wording of the question when participants answered it

this seems to need more digging into!

this seems worth highlighting, even if it's a rough calculation

state this as percentage

Pre-worskhop discussion !!

there is too much overlap between this and the paragraph beginning "The sceptical concerns"!

We need more of a TOC and sections/navigation on this page. Longer content in folding boxes

shorten this a bit

Rewrite to "extends Beyond pba"

This is a "for example"

"allready eating less meat" is. Vague. And. I'm not clear what the argument is here

The lower animal welfare burden of beef vs chicken may not be known to all readers

replace: "corporate animal welfare campaigns"

Also mention other questions, such as "will meat taxes improve or worsen animal welfare?" and "Will innovative products such as PBA and cultured meat substitute for farmed animal consumption, or will they mainly be taken up by (existing) vegans and vegetarians"

This is kind of captured above, but I would do something more here with belief elicitation, interactive updating, and aggregating knowledge.

This isn't something we're trying to achieve. This is just a path we're thinking of taking with the workshop.

and/or

Colon instead of dash.

Rather than "conclude" something like "do currently available methods and data even yield useful insight?"

Conclude is too strong here. I would say, what can we reasonably say about substitution effects and with what confidence?

Also industrial organization, quantitative demand estimation, economists, and quantitative marketing economists

Aims to - we don't have a confirmed guest list yet.

funders and industry and charity practitioners

Economists interested in animal welfare

Tool tip with some reference discussion of the limitations here.

Mention the use of instrumental variables and other strategies here, perhaps in a tooltip. Give specific references in that tooltip.

Add a tooltip here, discussing some of the strengths and limitations of each, using the context and explanations discussed elsewhere . Let me know if you need more context on this.

... (tooltip) Note this is in part due to the aforemationed point that elasticity is not likely to be constant across an individual or market demand curve, and there will also be heterogeneity thus, it matters what parts of the curve you are looking at, and which markets, times, etc.

rephrase this -- it's not quite right, and confusing

Also be clear: these are estimates of own price elasticity, although it seems unlikely that cross-price elasticities would be more consistent or robust. And these are price-shifting field experiments. But also note, in a tooltip, some of the critiques of these experiments themselves. Ask me if you need context.

I don't see what this part of the sentence adds. If the data is available in later years, we can focus on that later data. Maybe just leave this out, or mention something like "partly because of the limited availability of these products, and lags in releasing data for research use." -- But That's tooltip details. Also, I want you to ground some of these statements with references and links, mainly in tooltips.

More when they expect demand to be more price sensitive --- have pro or counter-cyclical pricing; Put the details in a tooltip

These explanations are taking up too much space and will take up even more when you consider a wider range of approaches.

Use folding boxes and tooltips more.

These are key issues with ~traditional econometric (IO and quant. marketing) methods.

Field experiments (supermarket-level or at school cafeterias etc.) have less of an endogeneity issue, but some of these issues are still present (e.g., short term vs long term), and these are hard to implement at scale and cleanly, and have issues of their own (see the notes/discussion, and sketch these).

Hypothetical and small-value choice experiments and hypothetical discrete choice surveys have other important limitations (mention these, from the sources and discussion).

That suggests

moderate this. This is vague. and there are a few kinds of field experiments in addition to this, including price shift experiments (esp. Bray et al), although few if any involving PBA

not "though"

this is a bit generic, maybe not necessary

likely --> "may" ... tooltip some other possible explanations for this. (discussed in the notes).

This is probly too strong ... needs caveating and referencing and tooltips.

just "seem to have serious limitations, and existing estimates are 'all over the map'" -- hyperlink or tooltip and link https://forum.effectivealtruism.org/posts/3Eh8MbqLwFBsD7GK2/how-much-do-plant-based-products-substitute-for-animal#Existing_Research

--> Observational demand estimation (using scanner, retail, and macro data)

rephrase. More like an "revisiting an underlying question"

make this 'between different animal products' and 'e.g., chicken vs. beef vs. eggs...' -- relevant for AW when considering issues like the AW impact of meat taxes -- which might shift consumption from beef to chicken, with a higher AW burden -- mention this briefly with further details in a tooltip

Skip this one

Research and state this. Also for impossible and. Beyond vs conventionally ground beef

The correct term is hamburgers

Penetration might not be the right term. I think we just mean market share here

Whatts. Breaded. Center plate?

Evidence for this claim? Otw State as ,,we. SpeculTE that,,

Claude and various openai models

mention the insights here? I'm not sure we'll put out htis video either; it's not something interesting to watch , I guess. It was mostly preparation and broad discussion.

this is a strong claim ("most under-modeled") -- what's it based on? Reasoning transparency please. Provide support and links to this, tooltips etc. I want to make sure this is well-backed before I post and ~"co-sign" it !

all claims need more direct supporting evidence ... quotes, links, etc.; tooltips are your friend

wait -- are you sharing the beliefs here? we didn't wnt to do that yet!

skip/remove this -- no one showed up