81 Matching Annotations
  1. Apr 2026
    1. The depth of recursion becomes a tunable compute axis at inference time, requiring no retraining. A small model, by reading itself, can iterate toward answers that neither it nor any of its workers could reach in a single pass.

      文章描述了一种递归推理机制,称小模型通过自我迭代可以达到单次推理无法达到的结果,但未提供具体的性能提升数据或实验证据。这一断言缺乏量化依据,需要更多实验数据支持。

    2. The depth of recursion becomes a tunable compute axis at inference time, requiring no retraining.

      大多数人认为AI模型的计算能力主要受限于其架构和训练数据,一旦训练完成,其推理能力基本固定,但作者提出Fugu模型可以通过调整递归深度在推理时动态扩展计算能力,这挑战了传统AI模型的固定计算范式,暗示未来AI系统可能具有前所未有的灵活性。

    3. The depth of recursion becomes a tunable compute axis at inference time, requiring no retraining. A small model, by reading itself, can iterate toward answers that neither it nor any of its workers could reach in a single pass.

      大多数人认为模型性能提升需要更大的参数规模或重新训练,但作者提出了一种反直觉的方法:通过递归调用自身,小模型可以在推理时自我迭代,达到单次推理无法达到的答案质量。这挑战了我们对模型规模与能力关系的传统认知。

    1. The question it forces is not which model is best. It is who owns the inference layer your organization depends on, what happens when the economics of that layer stop being subsidized, and whether the thing in your pocket turns out to matter more than the thing in the datacenter.

      大多数人关注AI模型本身的性能和优势,但作者认为真正关键的是谁拥有推理层以及其经济可持续性。这挑战了当前AI行业的主流关注点,暗示未来竞争的核心将从模型本身转向推理层的控制和成本结构,这是一个反直觉的视角转换。

    1. a lightweight surrogate trained on them can absorb a significant portion of future traffic at near-zero marginal inference cost

      大多数人认为模型替换会带来明显的质量下降或需要持续监督。但作者提出轻量级代理模型可以'吸收大量未来流量'且'边际推理成本接近零',这种近乎零成本的替代方式颠覆了传统模型替换的质量-成本权衡观念。

    1. TPU 8i is designed with more memory bandwidth to serve the most latency-sensitive inference workloads, which is critical because interactions between agents at scale magnify even small inefficiencies.

      通常认为内存带宽是通用硬件的需求,但作者提出TPU 8i针对低延迟推理进行了优化,这与通用硬件设计追求平衡的常规做法不同。

    1. We see continued gains from inference scaling on larger projects, suggesting they may be solvable given enough tokens.

      这一发现揭示了AI性能与推理计算资源之间的正相关关系,暗示了通过增加计算预算可能解决更复杂的编程任务。这为AI能力的边界提供了重要线索,也引发了关于计算资源投入与AI能力提升之间关系的深刻思考。

    1. Training on fields themselves forces the model to learn the physics that produces S-parameters, rather than learning to approximate the mapping directly.

      这是文章最深刻的洞见之一。仅基于S参数训练模型会使其寻找统计捷径,导致在分布外产生自信但错误的预测。而基于场训练,则是让模型学习产生S参数的底层物理原因,而非仅拟合表象映射。这种从“果”到“因”的范式转移,是实现泛化的关键。

    1. 19世紀の経済学者ジェヴォンズは、蒸気機関の効率向上によって石炭の消費効率が上がると、かえって全体の消費量が増えることを見出しました。

      用「杰文斯悖论」解释推理时间扩展(inference scaling)——这是一个绝妙的框架选择。效率提升→整体消耗增加,这正是 o1/R1 类推理模型出现后发生的事:单次推理更贵,但人们愿意为更难的问题付出更多算力。Sakana 用一个 19 世纪的经济学悖论,为 2026 年的 AI 产品战略提供了令人信服的理论背景——在技术营销中,历史类比是建立认知可信度的最有效工具之一。

    1. Using vLLM high-throughput LLM serving on DGX Spark provides a high-performance platform for the largest Gemma 4 models

      大多数人认为运行最大的Gemma 4模型需要专门的硬件和复杂的部署流程。但作者声称vLLM可以在DGX Spark上高效运行这些大型模型,暗示推理优化技术可能已经达到了一个临界点,使得复杂模型部署变得更加简单和高效。

  2. Sep 2025
    1. I'm using this logic as as to build spacetime. But I think it's going to give an even more powerful approach. I don't have to minimize some free energy principle. I I have a more direct computational way

      for - future project - building a model to explain spacetime using Active Inference - Donald Hoffman - use Active Inference to minimise surprise using Markov chains - this model assumes consciousness is fundamental - this is going to be a model of intelligence based entirely from a model which takes consciousness as fundamental. - it goes back to game theory again. - back to the idea of a simulation - If you're able to create a piece of software that - is able to replicate and - is built on the fundamentals of consciousness. - Then it's potentially, it's going to think it's conscious

  3. Jul 2025
    1. Inter-node communication stalls: high batching is crucial to profitably serve millions of users, and in the context of SOTA reasoning models, many nodes are often required. Inference workloads then resemble more training.

      Oh, so to get the highest throughout, the inference servers also batch operations making it look a bit like training too

  4. Dec 2024
  5. Nov 2024
    1. What’s missing obviously is a viable third option that would disrupt and transform the status quo by leaning into and operating from an awareness of the emerging future

      Active inference is natural (and mathematical) way to describe and model the predicted landscape and adapt to the emergent surprise

  6. Oct 2024
  7. Jan 2024
    1. Kevin Mitchell says in one of his books free agents he talks about I 00:27:10 move therefore I am is that yeah yeah no that's that's that's that's exactly right and all the work on um uh uh active inference

      for - definition - consciousness - active inference

      definition - consciousness - active inference - In Levin's opinion, one important aspect of defining consciousness that seems generally overlooked is outputs - actions - active inference is a field that deals with the actions that result from intelligence - currently, there is a greater focus on the input / perception side of consciousness but not as strong a focus on the output / action side

  8. Dec 2023
    1. The thing most obvious about the type systems of Java, C, C++, Pascal, and many other widely-used “industry” languages is not that they are statically typed, but that they are explicitly typed.In other words, they require lots of type declarations. (In the world of less explicitly typed languages, where these declarations are optional, they are often called “type annotations”.) This has nothing to do with static types. continued
  9. Oct 2023
    1. Kallus, N. (2020). DeepMatch: Balancing deep covariate representations for causal inference using adversarial training. In I. H. Daumé, & A. Singh (Eds.), Proceedings of the 37th international conference on machine learning. In Proceedings of Machine Learning Research: vol. 119 (pp. 5067–5077). PMLR

    1. "Causal Deep Learning" Authors:Jeroen Berrevoets, Krzysztof Kacprzyk, Zhaozhi Qian, Mihaela van der Schaar

      Very general and ambitious approach for representing the full continuous conceptual spectrum of Pearl's Causal Ladder, and ability to model and learning parts of this from Data.

      Discussed by Prof. van der Shaar at ICML2023 workshop on Counterfactuals.

    1. Performing optimization in the latent space can more flexibly model underlying data distributions than mechanistic approaches in the original hypothesis space. However, extrapolative prediction in sparsely explored regions of the hypothesis space can be poor. In many scientific disciplines, hypothesis spaces can be vastly larger than what can be examined through experimentation. For instance, it is estimated that there are approximately 1060 molecules, whereas even the largest chemical libraries contain fewer than 1010 molecules12,159. Therefore, there is a pressing need for methods to efficiently search through and identify high-quality candidate solutions in these largely unexplored regions.

      Question: how does this notion of hypothesis space relate to causal inference and reasoning?

    1. To avoid such bias, a fundamental aspect in the research design of studies of causalinference is the identification strategy: a clear definition of the sources of variation in the datathat can be used to estimate the causal effect of interest.

      To avoid making false conclusions, studies must identify all the sources of variation. Is this is even possible in most caes?

    2. Matching: This approach seeks to replicate a balanced experimental design usingobservational data by finding close matches between pairs or groups of units andseparating out the ones that received a specified treatment from those that did not, thusdefining the control groups.

      Matching approach to dealing with sampling bias. Basically use some intrinsic, or other, metric about the situations to cluster them so that "similar" situations will be dealt with similiarly. Then analysis is carried out on those clusters. Number of clusters has to be defined, some method, like k-means, if often used. Depends a lot on the similarity metric, the clustering approach, other assumptions

  10. May 2023
  11. Apr 2023
    1. The Delta Method, from the field of nonlinear regression. The Bayesian Method, from Bayesian modeling and statistics. The Mean-Variance Estimation Method, using estimated statistics. The Bootstrap Method, using data resampling and developing an ensemble of models.

      Four methods to compute prediction intervals.

    1. A novel method for estimating prediction uncertainty using machine learning techniques is presented. Uncertainty is expressed in the form of the two quantiles (constituting the prediction interval) of the underlying distribution of prediction errors. The idea is to partition the input space into different zones or clusters having similar model errors using fuzzy c-means clustering. The prediction interval is constructed for each cluster on the basis of empirical distributions of the errors associated with all instances belonging to the cluster under consideration and propagated from each cluster to the examples according to their membership grades in each cluster. Then a regression model is built for in-sample data using computed prediction limits as targets, and finally, this model is applied to estimate the prediction intervals (limits) for out-of-sample data. The method was tested on artificial and real hydrologic data sets using various machine learning techniques. Preliminary results show that the method is superior to other methods estimating the prediction interval. A new method for evaluating performance for estimating prediction interval is proposed as well.

      Prediction intervals using quantiles. Use clustering.

  12. Feb 2023
    1. collecting and checking the content of declarations of private interests, of personal data that are liable to disclose indirectly the political opinions, trade union membership or sexual orientation of a natural person constitutes processing of special categories of personal data, for the purpose of those provisions.

      Second question: If you collect it, can you infer from it?

    2. those provisions cannot be interpreted as meaning that the processing of personal data that are liable indirectly to reveal sensitive information concerning a natural person is excluded from the strengthened protection regime prescribed by those provisions, if the effectiveness of that regime and the protection of the fundamental rights and freedoms of natural persons that it is intended to ensure are not to be compromised.

      And here's the key element for indirect/inferred data. In order for Article 9 to matter, it must also include data that infers SCD.

  13. Jan 2023
  14. Nov 2022
  15. Jul 2022
    1. And I declare, on my word of honour, that what I am now about to write is, strictly and literally, the truth.

      Sounds reminiscent of the oath witnesses take on the stand: "tell the truth, the whole truth, and nothing but the truth." Since The Moonstone is described as a detective novel, this kind of language and the conflict with the narrator's cousin suggest that the plot may be concerned with finding out the truth from the narrator and cousin's (presumably) conflicting accounts of events.

    1. so now finally we get to active inference all this discussion and we're finally getting to the point here right for his lab so um i had and i had already touched on 01:47:35 some of this before but um it would you know today if you're going to develop a really good ai system you're and you're going to have a you have a robot saying the robot has to act 01:47:47 in some environment it is pretty well understood that that if you program that robot to you give it a you give it a i mean traditionally you'll give it a a a fitness function or some kind of 01:47:59 valuation function and it's for example it's good if it it you know you lose points if you fall through a trap door and is and you get points if you uh you know whatever 01:48:11 find find the piece of cake or something well that's uh that's fine for extremely simple universes that your robot might work in but as soon as you get beyond you know as soon as you get to any kind of more realistic uh 01:48:24 universe that your robot has to work in that pre-programming pre-programming concept just kind of falls apart it is you you it would require the the the practitioner to think ahead of all the 01:48:37 things that the robot might encounter and then how to value certain you know value those situations in certain ways uh and that is really uh what active inference 01:48:49 offers is a is a kind of a cognitive understanding or a mechanism by which an organism will uh uh where its 01:49:00 fitness score is in a sense involves both uh you know achieving goals and exploring its world to for for for epistemic gain so 01:49:16 um that's what we would like the that's how we would like to program the robot in a sense so that it can learn from it can learn on the fly from its experiences it can it can alter its actions and 01:49:30 goals as it be as it becomes clear as it gathers more information from its universe as it as it meets new situations that were never never conceived of by the by the 01:49:42 programmer that it through through an active inference or an active inference like uh you know mechanism it can learn and explore and and critically balance exploration with 01:49:54 exploitation and then we come right back to that whole concept of criticality so you know what you would really like your robot to do is remain at that critical uh phase between 01:50:06 exploring what's out there and making use and gold directed behavior of what's in front of it and um and uh you know that's how you could program this world this robot to act in the world and be pretty good at 01:50:20 it you know if you if you build it well so that's what the systems of a society can help a society to do you you don't you it's worth talking about building new systems i think it would not be wise to say 01:50:32 this checklist of like we wanted this level of education we want to want this you know to react this way in this situation react this way in this situation and this level of uh you know whatever money and this level of this and this 01:50:45 level of that while those kinds of preferences can be a useful start society has to be alive in its moment you know in the moment as society is alive it's cognating it's 01:50:57 it's it's it's actively uh you know comparing what it's the result of its actions to the model that is in its head and uh so active inference offers this way 01:51:09 to uh to balance uh exploration and and uh and uh exploitation and remain critical and remain optimally cognitive right so that's part of it 01:51:24 uh and then part of it i mean and for me this the the the idea of the embodied uh you know the three four e's uh this is what i really am attracted to in 01:51:46 active inference is in a sense it's kind of a simple concept it's not really very complicated you know if you've studied bayesian uh theory it all it's kind of straight you know in a way it's kind of straightforward 01:51:58 but the the you know the way fristen has connected the dots and and and and uh extended that into the bigger picture of life kind of it it to me it is uh it is rich 01:52:11 there's a there's a lot yet to be learned and gained and explored in this umbrella of active inference

      Active inference is exemplified using a robot, but is really a model of how humans learn, process information and make decisions in the world.

  16. Jun 2022
  17. Apr 2022
    1. ReconfigBehSci. (2021, February 2). @MichaelPaulEdw1 @islaut1 @ToddHorowitz3 @richarddmorey @MaartenvSmeden as I just said to @islaut1 if you want to force the logical contradiction you move away entirely from all of the interesting cases of inference from absence in everyday life, including the interesting statistical cases of, for example, null findings—So I think we now agree? [Tweet]. @SciBeh. https://twitter.com/SciBeh/status/1356530759016792064

    1. ReconfigBehSci. (2021, February 1). @islaut1 @richarddmorey I think of strength of inference resting on P(not E|not H) (for coronavirus case). Search determines the conditional probability (and by total probability of course prob of evidence) but it isn’t itself the evidence. So, was siding with R. against what I thought you meant ;-) [Tweet]. @SciBeh. https://twitter.com/SciBeh/status/1356216290847944706

  18. Mar 2022
  19. Jan 2022
  20. Dec 2021
    1. Tom Moultrie. (2021, December 12). Given the comedic misinterpretation of the South African testing data offered by @BallouxFrancois (and many others!) last night ... I offer some tips having contributed to the analysis of the testing data for the @nicd_sa since April last year. (1/6) [Tweet]. @tomtom_m. https://twitter.com/tomtom_m/status/1469954015932915718

  21. Oct 2021
  22. Aug 2021
  23. May 2021
  24. Mar 2021
  25. Feb 2021
  26. Jan 2021
    1. While no serious climate scientist doubts the fact that human activities are causing climate change, this can’t be proved through experimentation on another Earth.

      In both cases, the answers should be clear when looking at the evidence and the mechanisms at play without an ideological bias

  27. Dec 2020
    1. “provenance” — broadly, where did data arise, what inferences were drawn from the data, and how relevant are those inferences to the present situation? While a trained human might be able to work all of this out on a case-by-case basis, the issue was that of designing a planetary-scale medical system that could do this without the need for such detailed human oversight.

      Data Provenance

      The discipline of thinking about:

      (1) where did the data arise? (2) what inferences were drawn (3) how relevant are those inferences to the present situation?

  28. Oct 2020
  29. Aug 2020
  30. Jul 2020
  31. Jun 2020
  32. Mar 2020
    1. They reflect an area of science known as biological taxonomy, the classification of organisms into different groups.

      But these facts bear little consequence in day-to-day interactions hence their exotic status. People confuse less and fewer because using one or the other rarely changes the interaction. Calling a cashew a seed or a nut really doesn't change much.

  33. Jul 2018
  34. Sep 2016
  35. May 2015
    1. McNally‟s knowledg e that Brady prefers footballs inflat ed at the low end of the permissible range and his express request that the referee set the balls at a 12.5 psi level

      If there have been instances of balls being inflated by referees to 16, it is plausible that Brady would instruct the guy who gives the balls to the officials to make sure they stay at 12.5.

    2. Brady and Jastremski shortly after suspicions of ball tampering became public on January 1

      This is another inference to the negative. If you are implicated in something with someone who works with/for you is it a natural reaction to stop communicating? Is it more natural to speak with that person? How does behavior change when the entire global media is involved?