If we took one task out of our task suite or added another task to our task suite, potentially instead of measuring this Claude Opus 4.6 time horizon of, I think, 14 and a half hours, we'd be measuring it at something like eight or 20 hours.
增减一道题，测量结果从 8 小时变成 20 小时——这意味着整个 METR 时间地平线排行榜，本质上是由极少数「关键任务」撑起来的脆弱测量。当一个评测体系对单点数据如此敏感，它的「精确数字」就不应该被当作事实引用，而应该被当作噪声分布的一次采样。而目前，媒体和公众正是在拿这些数字做严肃决策。
high-level behavioral patterns like uncertainty management and self-verification are fragile and can be suppressed by irrelevant context
「高级行为模式是脆弱的」——这句话揭示了推理模型的一个深层结构性弱点：自我验证不是一种稳健的、内置的能力，而是一种在特定条件下才会激活的脆弱涌现行为。这与人类认知科学的发现高度吻合：人在高负荷环境下，最先退化的是「元认知」能力（对自己思维的监控）。模型复现了这个人类弱点，却没有人类的生理疲劳触发机制——而是用「上下文长度」代替了「疲劳度」。
Johnson, S. (2021, June 7). Spat at, abused, attacked: Healthcare staff face rising violence during Covid. The Guardian. http://www.theguardian.com/global-development/2021/jun/07/spat-at-abused-attacked-healthcare-staff-face-rising-violence-during-covid
A good library is filled with mostly unread books. That’s the point. Our relationship with the unknown causes the very problem Taleb is famous for contextualizing: the black swan. Because we underestimate the value of what we don’t know and overvalue what we do know, we fundamentally misunderstand the likelihood of surprises. The antidote to this overconfidence boils down to our relationship with knowledge. The anti-scholar, as Taleb refers to it, is “someone who focuses on the unread books, and makes an attempt not to treat his knowledge as a treasure, or even a possession, or even a self-esteem enhancement device — a skeptical empiricist.” My library serves as a visual reminder of what I don’t know.
I prefer the positive interpretation, of how much more there is to know. Quantifying anything in terms of how much we do not have is limited because we have finite knowledge out of an infinite set.
Each book instead of referencing something we do not know is a portal into things we have yet to know.
I think Nassim Taleb also mentioned elsewhere that having a lot of books you haven't read shows interest in a topic.
ReconfigBehSci. (2021, July 6). RT @mvankerkhove: I’m struggling with how best to stress how fragile the global situation is, so I’ll be blunt: Each week >2.6 million cas… [Tweet]. @SciBeh. https://twitter.com/SciBeh/status/1412416348676820992
By making the storage and organization of information everyone’s responsibility and no one’s, the internet and web could grow, unprecedentedly expanding access, while making any and all of it fragile rather than robust in many instances in which we depend on it.
Bartik, A. W., Bertrand, M., Cullen, Z. B., Glaeser, E. L., Luca, M., & Stanton, C. T. (2020). How Are Small Businesses Adjusting to COVID-19? Early Evidence from a Survey (Working Paper No. 26989; Working Paper Series). National Bureau of Economic Research. https://doi.org/10.3386/w26989
Robinson, A. (2020). How did ancient cities weather crises? Nature, 583(7816), 349–350. https://doi.org/10.1038/d41586-020-02070-5
Dureab, F., Al-Awlaqi, S., & Jahn, A. (2020). COVID-19 in Yemen: Preparedness measures in a fragile state. The Lancet Public Health, 0(0). https://doi.org/10.1016/S2468-2667(20)30101-8