Identify the fragile assumptionIn your preferred bottom-up TEA, which single assumption would break the headline result if it were wrong by one order of magnitude? Which interactions …
where did this (and the other questions ) come from? Tooltip the derivation, of course w/o identifying anyonw who wanted anonymity
workpapers
what are 'workpapers'? Also, this header takes up too much vertical screen
traceable draft calculations
'traceable draft calculations' -- this is not clear -- more normal language or tooltip definitions please
Stop and obtain specialist review
the tool should be able to do some of this, or at least provide advice? Wwhy not?
The PDF output is a draft field packet, not a filled official IRS PDF.
highish value if do-able ... but rather good if it jus tprovides instrutions on exactly what to imput
PDF/XLSX extraction and an AJ Bell-specific parser are not yet automated.
would dbe highish value, please ctnu
QEF, mark-to-market, purging elections, foreign-tax-credit allocation, and notional accumulation-unit determinations are not implemented.
let's implement those -- these seem high value
The calculator refuses to guess a missing rate or cumulative interest factor.
this warning is confusing -- what does it mean
Reset to evaluation lens Current to
why 'evaluation lens'?
Job-matching and recommendation systems affect labor-market frictions, unemployment duration, and match quality, all of which have major welfare and policy consequences.
But isn't this mostly relevant to rich country job markets? Is it reflective of an AI-impacted job market? Are recommended systems responsible for a large share of the job-match and productivity value?
Decision relevance
there should be a link amnd/or tooltip to the full description of each of these
Crux/PQ match update:
is this auto/updated?
API cost: not shown in this export
why not recover the API cost here as well as implied processing cost from headless runs
Human ratings visible here:
should we/can we reeasonably incoprporate Unjournal prioritization ratings for this haere? Need to check with the team to understand the terms under which those were given/shared. NB David Reinstein gives full permission to share/report any of his prioritization ratings, but would want to consider before releasing his discussion content in Coda.
Adjust these sliders to create your own priority score, then sort by Custom weights. This changes only your browser view; it does not change The Unjournal scorer.
are these reflecting global social value in the way we usually prefer?
Human ratings visible here:
"UJ historical" should contain human raitings, no?
The useful question now is not whether the old paper should be revived unchanged. It is what, if anything, is worth testing under current technology and policy.
another annoying AI dichotomy. Just list the thing that IS important
gument in an Essex Economics discussion paper and an unfinished theory project
That paper was related to the project, but it doesn't matter that it was "Essex economics", just say "discussion paper".
The strongest warning comes from take-up: Ofcom reports 532,000 UK social-tariff customers in June 2025, only 8.6% of a conservative proxy for eligible households.
what does this indicate? I don's see it as evidence of a lack of a welfare gain. Perhaps a friction (~transactions cost) or a stigma issue?
so "free redistribution" is too strong. The more defensible claim is narrower: this could create additional purchasing power for some low-income consumers with less public expe
this is AI-speak ... the not this buut that juxtaposition
lower pric
Maybe this missses the usual 'what's in it for the retailer'? and the answer is 'it helps them price discriminate by the (likely) single most indicative measure of willingness to pay -- the income (or adjusted income', helping them increase their profits
The OpportunityCard, ten years on
it seems a bit pretentious because 'the opportunity card' was never actually a thing, just an idea I had
::: {.callout-note collapse=“true”} ## Mo
rendering issue
Calibration anchors are a small set of real example papers that teach The Unjournal's AI prioritization scorer what “value of evaluation” looks like in each area — including the boundary cases where a good paper is not a good candidate. Each anchor's rating is the team's real prioritization rating; the proposed lesson is an AI inference about the calibration takeaway. Use this page to confirm or correct each rating and lesson, suggest new anchors, and track the discussion.
This anchoring page should allow people to specify the different dimensions of quality, that is, the different ratings.
And the form should be more quantitative rather than "too high, too low." -- or maybe I'm missing the point?
To be honest, I'm not sure what was intended by the question "not by the answer", "not a good anchor". ... What is the " lesson" here? That's confusing.
Research training
We should label 'what kind of research training' here. I'm not sure the right term ... social science, quantitative modeling of cost/benefit forecasting, etc., economics, statistics, etc.
genuinely
"I'd genuinely like" sounds like AI slop
decent evidence AI tutors teach content well
link/tooltip evidence
s a possible consolidation of things I already do at a smaller scale: modeling workshops on contested quantitative questions, Fermi-estimation and parameter-elicitation sessions, and supervising early-career research
The Fermi estimation session thing is a bit of an overstatement -- something we're considering running soon
PREDOC's
add hyperlinks to these, and tooltip explanations
Intensive research training
The proposal is more than just intensive research training. The idea is that it's outside of a university institution and provides more direct hands-on experience and one-to-one or small group feedback, sort of replacing grades, degrees, and accreditation it's proven work and personal recommendations.
For a few years I've had a half-formed idea for an intensive research-training program: a few weeks of courses, a supervised project, then a conference where the work actually gets read closely. I mostly kept it in my head — I think I've floated it to one or two people. My original sketch was a fairly conventional mix of statistics, data science, economics, and behavioral science, plus supervised research. What's changed my thinking is AI: less of the mechanical work you can now hand to a machine, more judgment and understanding what results actually mean. So I'm writing it up to see whether it holds together.
This takes up a bit too much space on the page.
Training researchers for the part AI can't do for you.
This looks too much like a marketing slogan. It's more of a question as to whether this is worthwhile and, if so, what direction to take it in.
Foie gras (luxury/unstructured)
consider a log scale or a 'discontinuity here -- with Foi Gras all the others appear minute
How to read each column: the conventional product's retail price; the cultivated cost the model delivers to retail (your biomass slider + scaffold for cuts + markup); their ratio R; and the estimated share of that product's own market cultivated would capture (not a share of all meat). Share-bar colour: green > 30%, amber 8–30%, red < 8%.
give some column headers instead
One caveat to this species-by-species framing (our own intuition, not in his model): especially early on, a cultivated “chicken” or “shrimp” product may be received as its own distinct food rather than competing head-to-head only with the conventional product it imitates, so cross-category substitution could matter more than a per-species contest implies.
"my" not "our" -- and make this a tootip
tated expert
(and already collected 10+)
What drives the answer.
I want to check this -- make it clear that this synthesis is your (Claude Opus atm) analysis.
On agreements: we draw on the same source literature and read it similarly. Humbird’s 2021 pessimism was driven mostly by amino-acid/media cost, which is not a hard thermodynamic constraint and which Pasitka’s 2024 empirical work (hydrolysate at $0.63/L) pushed down sharply. Both of us read this as suggesting cultivated meat likely lands a few-fold above conventional meat — not at parity, but not orders of magnitude off either.
I still don't want to suggest that I am 'reading this' in a way suggesting a particular conclusion!
$/kg →
make middle column narrower. Also less whitespace on the right within each column
Cost
Example hypothes.is comment. Flag @daaronr on any other comments to be sure I am notified
Share colour: green > 30%, amber 8–30%, red < 8%. Ordering across species is driven almost entirely by the price ratio R (cost fixed, conventional price varies), plus the per-tier authenticity offset — reproducing Pablo's inversion: cheap chicken and pork resist, expensive beef, seafood, and luxury foie gras are penetrable.
label this better -- what actually are the columns/outcomes here? And maybe put in whole shrimp and shrimp paste too
may capture little chicken or pork share
my intuitions -- at least in early stages, even if they see it as 'real meat' the chicken, fish, etc. imitating products may still be seen as distinct and not compete only with the product theory imitate
validates out-of-sample against plant-based milk
was it calibrated on this though?
and its 89% mainstream split,
clarify what this means
it fills the half we skip;
rephrase -- it's not necessarily 'half' -- there are other issues too
an ad hoc dependence structure
tooltip what this means
weak
language should be more tentative -- this is an AI judgement; it's not entirely credible to 'judge ourselves'
h caution
add to header -- "annotation tool"
price ratio R
what is "R" again?
His is the piece we have explicitly not built.
--> His work could be seen as a 'missing piece' for our analysis.
['the piece' suggests there is only one missing piece']
Both of us conclude
too strong -- we do not "conclude" anything
Our model
Explain more carefully ... we are not just a 'model' but also a calculator allowing you to provide different assumptions, and a template/example for future modelers
Takes cost as exogenous. Cost → retail price ratio → discrete-choice (logit) market share by species, product tier, and geography → diffusion over time.
make wider, use the space generated
and connect end-to-end:
could be connected
The two models are complementary halves of one pipeline
--> The models focus on different aspects -- Production cost (TEAs) and Demand-side
AI-assisted prioritization for Unjournal evaluation — Prototype, March 2026
should prominently show last tool update, last scan, and (tooltip or hidden) API costs. Also 'number of human ratings/raters'
This dashboard helps identify research of interest for Unjournal evaluation. Internal prioritization currently uses our Coda interface. (March 2026)
I don't like the persistent header
Early prototype (March 2026). Coverage and scoring depth will improve as we expand sources and incorporate human feedback. Scores are AI-generated suggestions to help identify candidates for evaluation.
Put most of this into a tooltip
What is The Unjournal? We commission and publish independent, public evaluations of research that can inform high-stakes global decisions. We focus on economics, quantitative social science, forecasting, and policy-relevant research—including development economics, global health, animal welfare, AI governance, climate policy, and catastrophic risks. Learn more →
make this a folding box ... with just 'What is The Unjournal cisible pre-fold'
Precision fermentation and plant molecular farming will reduce growth factor costs substantially
Parentheses and tooltips should note how many people said/voted on these things
100% probability (CM_17) to this before 2036.
Tooltip -- Side note: We discourage forecasters/experts from expressing these "degenerate" probabilities, as they are somewhat implausible from a Bayesian point of view, and not easily integrated.
Before mapping disagreements, it helps to note what workshop participants broadly converged on. These are not fully settled — but they represent positions where the workshop did not surface active disagreement.
Shorten this paragraph. There's too much filler here.
The workshop participant group
Hyperlink and tooltip the list of actual participants from the workshop page
2–10× below
"2 - 10x below" is unclear. Do you mean "1/10 to 1/2 the price?"
A "Key questions for optimists" section will be added as input from more skeptical voices is gathered.
'we plan to add' ... italics not bold ... "as we gather"... Avoid using passive.
Talk: The Unjournal (Exeter)
presented at exeter, but slides are generic. And it's the slides not the 'talk' that we link
An interactive note
and ~microeconomic model
Modeling approach →
it's a proposed collaboration, not an 'approach'. the model sketch is at https://unjournal.github.io/cm_pq_modeling/ -- for CM -- you can link that too
Why several of these look rough.
remove sentence, put the tooltip on the previous bit
up and do the grant-writing.
this suggests we take up all ideas -- not the case. But I'll be the one doing the large majority grant-writing work, although their input is also welcomed and solicited
plenty
I mean, if they want to say more, that's good
A short snapshot of what we've built recently, and directions we could take with new funding. Made to give our team a sense of what's already underway — and an easy way to say what they'd like to work on next.
leave this out or make it a tooltip
Email us your idea →
slack is good too -- just dm me or post if you're ok sharing it more widely
round what you'd want to do
that's a bit too strong -- it's a key factor though
for The Unjournal
not 'for The Unjournal' -- more 'decision-relevant for global priorities'
Post or comment URL *
although this is mostly about EA Forum/LW, we should allow people to submit suggested cruxes here untied to posts if they (although links to further discussion is v helpful)
Cleanup mode
tooltip -- what exactly does this adjust?
Cadence
tooltip explanations. Needs more options (like 1x/fortnight) and maybe manual text-entry options
Meeting purpose, agenda, or decision
here the 'pre-filled text' should be empty, or an example. Otherwise it just adds clutter.
Also (separate fields) allow convener to ask questions/poll participants
Use after enough responses are in
have it email the createor (they should be able to enter their own email) when more than X responses are in (they can choose that, defaults to 4)
Shared Availability
probably needs more room on the page ... expanding leaves little room ot se eit
copying the participant link sends invitees to the response-only page.
i don't understand this sentence
oordinator return link
what is this? tooltip. Why is it needed here?
Meeting-room link
not sure this 'regular link' works -- please check
Offered start times ?
give more eas y options (or adjust based on a description given
Meeting duration
enable shorter meetings. could it work to just allow you to enter the number of minutes?
Candidate dates ?
I should be able to give a verbal description here and it will propogate. I should also be able to check against my Google calendar here (or at least The Unjournal's Gcal
Request ID
this seems to default to 'unjournal-management' -- ideally it should be auto-adjusted based on the poll title set above
Poll title
The text you enter in the boxes should be a lighter font than the headers, and maybe a serif font -- it looks like entering bold font here.
Set the meeting purpose, dates, times, and links here. Drafts save in this browser while you work. Copy or preview the participant link when the request looks right.
this should be a tooltip
Claim Highlighter
this is a dead link. What happened here?
Research Prioritization Dashboard
link the legal one separately (maybe a subfield here)
Complete Unjournal evaluations with guided metrics, 90% credible intervals, calibration practice, and multiple export formats (JSON, CSV, Markdown).
this links the academic evaluation form only -- also link the applied one
Evaluation Form
Separate the "Evaluation, assessment, meta-research tools" from the "Research, questions, and cruxes" databases
Interactive Tools
interactive tools and databases
Searchable database of explicit disagreements, uncertainties, and "what would change my mind" statements from EA Forum and LessWrong posts, to inform research prioritization.
put this further down
AI-assisted dashboard surfacing high-impact research candidates for Unjournal evaluation — papers discovered from multiple sources and scored for evaluation priority.
Actiony bullets work better here
Prototype
more than a prototype now
AW cruxes / Pivotal Questions
Don't call the EA forum list "pivotal questions." Just refer to them as cruxes.
Research database Browse candidate papers; filter to animal welfare and food systems. AW cruxes / Pivotal Question
Pivotal questions should probably have its own set of boxes.
https://coda.io/d/Unjournal-Public-Pages_ddIEzDONWdb/Cell-cultured-meat-PQ_su7KnL0o#_lu-eSUA3 and https://coda.io/d/Unjournal-Public-Pages_ddIEzDONWdb/Plant-based-substitution-PQ_su7lhgBI#_lu87soYb and https://coda.io/d/Unjournal-Public-Pages_ddIEzDONWdb/The-Unjournal-Blog-post-updates-and-opportunities_suAkrsSS#_luyuj6My should be incorporated as appropriate.
links
Maybe also link our https://info.unjournal.org/pivotal-questions.html page somewhere here.
What The Unjournal adds in animal welfare
Engages with animal welfare research
Cultivated-meat costs Techno-economic cruxes, cost trajectories, forecasts, and synthesis. Join / advise / evaluate Get involved, nominate evaluators, or join the broader Unjournal network.
This links the workshop with a range of resources.
Published AW evaluations Public Unjournal evaluations relevant to animal welfare and adjacent topics. Plant-based substitution PQ Demand-estimation evidence, operational questions, and workshop planning. Cultivated-meat costs Techno-economic cruxes, cost trajectories, forecasts, and synthesis. Join / advise / evaluate Get involved, nominate evaluators, or join the broader Unjournal network.
Make the published AW evaluations more prominent... These are evaluation packages, some with author responses. Perhaps the folding box could list and link this individual one.
See AW cruxes
I wouldn't make this one of the top links.
evaluations
Mention the evaluation packages first. ... Those should be perhaps the most prominent.
Concrete ask: Link the public outputs, suggest papers and speakers, forecast tournament questions, annotate pages with corrections, and point us to drafts or slides when authors are comfortable sharing them.
Rephrase this or get rid of it. It seems weird to have an ask like this on a public page.
Featured work and tools
There's quite some repetition on this page. Let's make it more concise. People will be overwhelmed here.
Animal-welfare economics has many high-value questions where evidence is thin, contested, or scattered across academic papers, industry reports, forecasts, and practitioner knowledge. Conventional journal review is often too slow or too detached from the decisions funders and policymakers face.
It's also difficult to get rigorous feedback and evaluation on animal welfare relevant work from the methodologically strongest, economics researchers, psychologists, etc. We provide an opportunity to directly connect and directly reach out to top mainstream academics
Find people and collaboration points
Here's the Get Involved page. https://www.unjournal.org/getting-involved and here's the Join Us form. https://coda.io/form/Join-the-Unjournal_dc3NLlpa-eq
Send a lead Suggest papers, drafts, speakers, evaluators, or pivotal research questions.
This is just an email box to contact us. Instead, why not share https://coda.io/form/Author-Submission-Form_dZ4MMytY7aI
For people who want to submit research and people who want to suggest research can do so here: https://uj-prioritization-prototype.netlify.app/
Research candidates Find animal-welfare relevant quantitative social science and economics research. AW cruxes EA Forum cruxes filtered to animal welfare and alt-protein topics. Animal Futures Tournament Forecast concrete animal-welfare, alt-protein, policy, and AI questions. Plant-based substitution Pivotal Question on demand estimation and substitution away from animal products.
This is actually linking the workshop page. (For an upcoming potential workshop )
Research candidates Find animal-welfare relevant quantitative social science and economics research.
Don't call this research candidates. Call it something like Research Database ... Note that they will need to select the correct filters to focus on animal welfare.
AW overview Short public framing of The Unjournal's animal-welfare evaluation work.
Instead of outlinking that page, which is basically a grant application summary, why not recap that somewhere within this document, perhaps in a folding box, and then link that instead
These are the most shareable animal-welfare economics links. The rest of the page adds context and routing.
This is too wordy and probably obvious.
Copy share blurb
Why would a user want to copy a share blurb?
Send research leads
This is underexplained, and this is just really a contact us form, so if you just want to contact us, put it there.
Animal welfare overview
You don't really need to share it as a grant here. We need to put it in a different format.
Live preview Collaborate
Live preview doesn't make sense for something that we're sharing around.
Unjournal tools and outputs for animal-welfare economics
Too much vertical space.
What The Unjournal does for animal-welfare research
This line is taking up too much vertical space. Let's make it a bit more efficient.
Pick a time and prepare calendar, Slack, and email drafts. 1. ReadCheck what the meeting is for and any linked context. 2. Mark timesClick individual slots, whole days, or whole time columns. 3. SubmitWait for the saved message before closing the page.
i don't understand 'prepare drafts' here ... does the app do that?
1. Describe
make the description tooltips
I'm available for Room: main Reach link ready: Phone People will get a one-click call button on your card.
Agent pre-plans calls too -- you can share your calendars and it makes suggested times/matches
I'm available for
Async voice prompts
tooltip what the point of htis is let people upvote or downvote responses ,and then sorted by this or in chronological order.
Preview my card Current general viewer preview. Group-specific previews can be added later. Visibility preview What people see Close This preview only shows the card for someone who is allowed to see you. It does not reveal future group visibility rules. Free now
it should automatically highlight the matches, e.g., if you're both up for a drive time call ... Tell you how many minutes you could have overlapping
Free
don't show yourself in the thingbelow
Discussion responses (open text)
make this a folding box/boxes
Mean: 13.5 $/kg · Geometric mean: $6.8/kg · n=8
Make the aggregation more prominent in each case -- aggregations first, and bold, perhaps a relevant plot, then (unfold to see ) individual statements and discussion.
By the way why is the geometric mean interesting here? Remind me and explain in a tooltip. And why not use it for the probabilities -- that's where I'm more used to seeing it
Interactive model: public test after selection into evaluation
Where is the 'pre-selection into evaluation' probability in this diagra (Po)
record
what does 'adverse record' mean below? I guess we are talking about the evaluation rating here?
Evaluator anonymity in current Unjournal public data. A bit over half of deduplicated evaluator-paper rating pairs use anonymous or generic public identifiers: 65/113, or 57.5%. This matters because later gatekeepers see the public report, but often not a named evaluator.
interesting but you haven't really connected this to the discussion or model. Why does evaluator anonymity matter to the author's decision?
That the premium persists once public evaluation is common.
also, as with these signaling games, there's usually multl equilibria, including a sort of 'babbling' one iirc
Author Benefits, With Limits
this section is not written well. As far as I know only the things linked to Propositions 1 and 2 even engage the model ... you can mention other possible costs and benefits not considered by the model, which are largely empirical questions
Non-exclusivity: the public record can stack on a journal path.
I don't see how this is a 'result' -- it's just an obvious construction
A calibrated causal effect of public evaluation on citations or journal acceptance.
simplfy this ... that's obviously an empirical issue.
The Proposition 2 signal, made interpretable by benchmarked ratings, journal-tier equivalents, and uncertainty intervals.
link and tooltip "Proposition 2" -- it hasn't been introduced yet
backs
'backs' or 'focuses on'
third category
what 'third category'?
not formal publication by The Unjournal.
remove 'not formal publication' -- already noted, but the 'formal publication' is not a benefit per se
downside
'downside risk'
visible
The diagram below is underexplained. Also note we geve evaluators the opportunity to revise if the author points out clear omissions or errors
supports conditional timing
this needs clarification or a tooltip. I think you are talking about the 'conditional embargos'?
Pr(favorable | H) = Pr(adverse | L)
The probability of why should the probability of each of these be the same? That's weird and seems very limiting to me. The model should be made more general.
favorable
Use variable names for these also.
The public signal Y
This was the first time that you mentioned the variable y - you need to define it better.
A Simple Base Model
This could be fleshed out more carefully and slowly with more explanation.
author response
encouraging authors to respond, and evaluators to update their evaluations if authors find clear mistakes or oversight.
Tooltip: We're also working to build and coach our evaluator pool, and hope to provide paid calibration workshops in future.
The Unjournal's design
"The Unjournal works to limit the noise component through..."
as accuracy q
is accuracy a 'false negative and positive rate' or what -- give a tooltip on what q means
Rejection bias
Flesh out -- why would this be a particular concern for public evaluation? (Explain briefly, tooltip a longer explanatino)
Concern
wait -- 'concern with what', 'relucatance to do what'? Needs clarification, seems somewhat inconsistnet
Yes. Peer review is noisy: the 2014 NeurIPS experiment had committee disagreement on 43 of 166 duplicated papers
link please, and give details in tooltips
where The Unjournal's structure changes the mechanism from cases where private author risk remains.
this is a bit too stark. I don't know if we can be sure that UJ fully changes these situations, nor whether in the 'risky' cases the risk is substantial. State it more tentatively or diretionally
elevant bar
'publication tier bar'
visa
sklip 'visa'
Treat this as a set of considerations, not a decision rule.
We discuss a series of reasonable considerations
plus author response
and, evaluator updates, and evaluation manager synthesis.
evaluation
quantified evaluation and a detailed report
remaining
'mistakenly being connected'
trongest private case is a paper already ready for serious expert scrutiny whose true quality exceeds its default credibility.
Should we also mention the 'submitting your work for public evaluation can also be a strong signal of your confidence in the credibility of your work?'
A public commitment — and a signal. “I’m willing to have this evaluated openly.” Feedback now, a public signal now — journal path still open.
Can we have a 'separating equilibrium' or other image here. Let's focus this slide on the "willing to recieve and respond to public criticism signals research strength' part ... and then the 'immediate underdog benefit thing' is the next slide (which you can tease)
The Pivotal Questions project
use the space on this slide bette
runs
1st and second box ... thought bubbles or callouts (0-100 percentile relative to pool); Both suggestor and assessor writes a motivating explanation/discussion.
"Whole team votes" -- 5 point approval scale (strong/weak/neutral)
What does open (Unjournal) evaluation provide? Now: faster, useful feedback + a credible public signal, and useful inputs to practitioners and funders. Soon: it starts to carry career value. Eventually: it can replace much or all of what we ask the journal stamp to do. Which of these would actually help your work?
the text on this 'all green slide' is a bit hard to read. Make it more readable and clear.
outputs
prioritization and evaluation outputs
quality
and usefullness, providing multodimensional ratings and discussion -- not just "which venue published it".
For research leaders & managers encouraging engagement signals a commitment to rigour, transparency & innovation — and opens the research-impact channel (our funder & practitioner network, incl. Pivotal Questions). Two audiences. Individuals/committees: a strong public evaluation should count as evidence of quality in its own right — hiring, promotion, REF narratives, grants, esteem. Research managers / those setting direction for a group or department: encouraging engagement is a visible demonstration of research rigour, transparency and openness to innovation — and brings more, faster, more transparent feedback and signals than standard peer review. Exeter needn’t lead, but it could position itself as open to this innovation. The research-impact channel is real: strong connections to funders and nonprofit practitioners, including via Pivotal Questions. If useful, happy to discuss a light next step. span.MJX_Assistive_MathML { position:absolute!important; clip: rect(1px, 1px, 1px, 1px); padding: 1px 0 0 0!important; border: 0!important; height: 1px!important; width: 1px!important; overflow: hidden!important; display:block!important; }
space this slide out more
Each
add a vertical space before this sentence
pool
in the prioritized pool
research
I think I wanted more images, more faces, here or below. I had a comment about that
with apologies to a certain lager
remove the disclaimer
certain
center the image.
3.9
adjust these examples to have some with wider and others w narrower bands
and importance if true
leave this off; that's not quite what we ask, we ask about implications
expressed
--> with quantified uncertainty
Full “model” (v. preliminary, ~Fable-generated with human feedback): unjournal-reluctance-note.netlify.app Reframed per your screening/sorting logic: the value is highest when you’re strong-but-under-credited or just below a bar — exactly where an extra credible signal can move you. And if committing to open evaluation becomes a positive signal in a sorting equilibrium, you want to be an early mover. The case for waiting is narrow: work that already clears the bar AND a genuinely sensitive moment — then the extra signal adds little. I’d be less worried about about “harmful criticism”: our evaluations are constructive and you get a public response; public scrutiny isn’t a bogeyman. For timing/embargo, people talk to us. span.MJX_Assistive_MathML { position:absolute!important; clip: rect(1px, 1px, 1px, 1px); padding: 1px 0 0 0!important; border: 0!important; height: 1px!important; width: 1px!important; overflow: hidden!important; display:block!important; }
This bit is cut off at the bottom of the page. Need to use the vertical space a little more conservatively here. Have smaller fonts or more use of the horizontal space.
Watch the 2-minute explainer
still not embedding
We’re working to be highly visible — so evaluations & ratings are seen before conventional journals/reviewers weigh in
Add image of our Google Scholar visibility here : https://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&q=source%3Aunjournal&btnG= as one example
The strongest private case is a paper already ready for serious expert scrutiny whose true quality exceeds its default credibility
what about the case that 'willingness to make all work available for evaluation could be a strong signal of your confidence and credibility'?
good
More like, "How does AI evaluation compare to humans?"
And I'd frame this more as an open question, one we're exploring, but at the moment the general attitude seems to be that there needs to be a human in the loop, at the very least, making the final judgment calls, prioritization, and communication
Questions for you
I'd add things like: - How could this invigorate teaching and research training? - How could it help with building agendas, attracting funding, and demonstrating value for exercises like the REF?
What would make an evaluation count as evidence of quality?
This is perhaps the most important question here - maybe put this one first. ... What would make it reliable, meaningful, and valued?
Where would faster public evaluation be most useful in economic
Not quote in economics. That's asking too much. Leave the last bit out, but presumably they'll understand that we're asking about what would be useful to them.
From papers to decision-relevant uncertainty
This slide should have the header "Pivotal questions project" Not
LLM vs. human ratings: modest correlation — not aligned enough to substitute
We don't have such strong evidence on this to say it's not a line enough to substitute, to be honest. We only have one trial that we attempted. This slide overstates things, and it would be better to have it link and show some of our output, just so people know what was done in our trial.
Human judgment still central
This seems like a claim, but what backs it up?
evaluation
"efficient and transparent evaluation" ... "nd connection to 'real stakeholders and impact'"
papers
Generated by AI tools, which may or may not actually be correct or useful
Forecasts via Metaculus; partners incl. Institute for Replication, Center for Open Science
Those are not the relevant partners. The relevant partners are: - Founders Pledge - Animal Charity Evaluators We've had participants in workshops from Coefficient Giving and many other organizations..
Metaculus is not really at the center of this at the moment. ... It's more on our own pages and platforms. https://uj-wellbeing-workshop.netlify.app/beliefs the Metaculus thing is sort of an extension.
Questions
shoulf hyperlink https://info.unjournal.org/pivotal-questions.html
~9 management · 15+ advisory board · 40+ field specialists · 180+ evaluators (over half economists, over half doctorates). The point isn’t celebrity endorsement — it’s that we have enough disciplinary coverage, advisory oversight, evaluator depth, and process experience to be taken seriously. Usually 2–3 evaluators per package (not “everyone behind every evaluation”). Field specialists across eight areas help prioritise and recruit. Two of those field specialists are here at Exeter (next section). Full team at unjournal.org/team. span.MJX_Assistive_MathML { position:absolute!important; clip: rect(1px, 1px, 1px, 1px); padding: 1px 0 0 0!important; border: 0!important; height: 1px!important; width: 1px!important; overflow: hidden!important; display:block!important; }
Make this bit larger and more visible. Maybe include some of the visuals on the composition of the evaluator pool.
markets
I don't think labor markets get at it here. We're talking about the impact of transformative technological change on labor markets, not labor markets on their own.
certified
Accepted after substantial revisions ... I don't know why we would use the word "certified" here.
R&R
At best
The
Versus a traditional journal
problem
Add the image of the fish banding together on this slide. You can find that on the previous PowerPoint/Google slide presentations.
It’s a coordination problem
Add "funding and grantmaker incentives will help".
And maybe replace this with "solving the coordination problem".
Replace Fear of Standing Out with Fear of Missing Out
Also a bullet about how we're making ourselves prominent in the ecosystem so that the evaluations and ratings will be seen before the paper is reviewed by conventional journals.