reasoning models tend to produce much shorter reasoning traces (up to 50%) for the same problem under different context conditions compared to the traces produced when the problem is presented in isolation.
令人震惊的发现：同一道题，仅仅因为周围塞入了无关上下文，推理模型的思考链长度就缩短了最多 50%——而题目本身一字未改。这意味着我们以为在评估模型「解题能力」，实际上评估的是「在特定上下文包装下的解题能力」。所有在孤立问题上测得的推理 benchmark，都可能严重高估了模型在真实 Agent 场景中的实际推理深度。
Nature Portfolio. (2021, December 8). Estimates of COVID-19 vaccine uptake in the US based on large surveys that are used to guide policy-making decisions tend to overestimate the number of vaccinated individuals, according to research published in @Nature. Https://go.nature.com/3EBQPOh https://t.co/rSoclzWIdg [Tweet]. @NaturePortfolio. https://twitter.com/NaturePortfolio/status/1468633979364560899
Gugerty, L., Shreeves, M., & Dumessa, N. (2021). Biased belief updating in causal reasoning about COVID-19. PsyArXiv. https://doi.org/10.31234/osf.io/bfw76
And asking them if they think they know what they are doing will not help, because many people will overestimate their knowledge, making the support even more complicated as the tech guy may at first believe them and only find out later that they told wrong things because they do not actually know what they are pretending to know.
Dr Ellie Murray. (2021, May 7). I’m seeing a lot of “these people are over-estimating risk” chatter that doesn’t acknowledge that the probability you die if you get covid is always less than the probability anyone dies if you get covid. It’s not “over-estimation” to consider community impacts. [Tweet]. @EpiEllie. https://twitter.com/EpiEllie/status/1390792624777334797
Register to read | Financial Times. (n.d.). Retrieved 5 March 2021, from https://www.ft.com/content/879f2a2b-e366-47ac-b67a-8d1326d40b5e