Inter-node communication stalls: high batching is crucial to profitably serve millions of users, and in the context of SOTA reasoning models, many nodes are often required. Inference workloads then resemble more training.
Oh, so to get the highest throughout, the inference servers also batch operations making it look a bit like training too
What is heavy should be local, and what is light should be global and shared.
Overly simple? Rather than a binary local - cosmo, some biological models of self-organization have nested levels of optimization based on the context
What’s missing obviously is a viable third option that would disrupt and transform the status quo by leaning into and operating from an awareness of the emerging future
Active inference is natural (and mathematical) way to describe and model the predicted landscape and adapt to the emergent surprise
An Active Inference Model of Collective Intelligence
Short explanation of a Model for collective intelligence based upon the socio-cognitive traits of individual members Active Inference
Kevin Mitchell says in one of his books free agents he talks about I 00:27:10 move therefore I am is that yeah yeah no that's that's that's that's exactly right and all the work on um uh uh active inference
for - definition - consciousness - active inference
definition - consciousness - active inference - In Levin's opinion, one important aspect of defining consciousness that seems generally overlooked is outputs - actions - active inference is a field that deals with the actions that result from intelligence - currently, there is a greater focus on the input / perception side of consciousness but not as strong a focus on the output / action side
The thing most obvious about the type systems of Java, C, C++, Pascal, and many other widely-used “industry” languages is not that they are statically typed, but that they are explicitly typed.In other words, they require lots of type declarations. (In the world of less explicitly typed languages, where these declarations are optional, they are often called “type annotations”.) This has nothing to do with static types. continued
Kallus, N. (2020). DeepMatch: Balancing deep covariate representations for causal inference using adversarial training. In I. H. Daumé, & A. Singh (Eds.), Proceedings of the 37th international conference on machine learning. In Proceedings of Machine Learning Research: vol. 119 (pp. 5067–5077). PMLR
Using adversarial deep learning approaches to get a better correction for causal inference from observational data.
"Causal Deep Learning" Authors:Jeroen Berrevoets, Krzysztof Kacprzyk, Zhaozhi Qian, Mihaela van der Schaar
Very general and ambitious approach for representing the full continuous conceptual spectrum of Pearl's Causal Ladder, and ability to model and learning parts of this from Data.
Discussed by Prof. van der Shaar at ICML2023 workshop on Counterfactuals.
Performing optimization in the latent space can more flexibly model underlying data distributions than mechanistic approaches in the original hypothesis space. However, extrapolative prediction in sparsely explored regions of the hypothesis space can be poor. In many scientific disciplines, hypothesis spaces can be vastly larger than what can be examined through experimentation. For instance, it is estimated that there are approximately 1060 molecules, whereas even the largest chemical libraries contain fewer than 1010 molecules12,159. Therefore, there is a pressing need for methods to efficiently search through and identify high-quality candidate solutions in these largely unexplored regions.
Question: how does this notion of hypothesis space relate to causal inference and reasoning?
[ Bengio, The Consciousness Prior, Arxiv, 2018]
Causal Deep Learning Authors:Jeroen Berrevoets, Krzysztof Kacprzyk, Zhaozhi Qian, Mihaela van der Schaar
Very general and ambitious approach for representing the full continuous conceptual spectrum of Pearl's Causal Ladder, and ability to model and learning parts of this from Data.
Discussed by Prof. van der Shaar at ICML2023 workshop on Counterfactuals.
(Cousineau,Verter, Murphy and Pineau, 2023) " Estimating causal effects with optimization-based methods: A review and empirical comparison"
To avoid such bias, a fundamental aspect in the research design of studies of causalinference is the identification strategy: a clear definition of the sources of variation in the datathat can be used to estimate the causal effect of interest.
To avoid making false conclusions, studies must identify all the sources of variation. Is this is even possible in most caes?
Matching: This approach seeks to replicate a balanced experimental design usingobservational data by finding close matches between pairs or groups of units andseparating out the ones that received a specified treatment from those that did not, thusdefining the control groups.
Matching approach to dealing with sampling bias. Basically use some intrinsic, or other, metric about the situations to cluster them so that "similar" situations will be dealt with similiarly. Then analysis is carried out on those clusters. Number of clusters has to be defined, some method, like k-means, if often used. Depends a lot on the similarity metric, the clustering approach, other assumptions
Terwiesch, 2022 - "A review of Empircal Operations Managment over the Last Two Decades" Listed as an important review of methods for addressing biases in Operations management by explicitly addressing causality.
Shayan Shirahmad Gale Bagi, Zahra Gharaee, Oliver Schulte, and Mark Crowley Generative Causal Representation Learning for Out-of-Distribution Motion Forecasting In International Conference on Machine Learning (ICML). Honolulu, Hawaii, USA. Jul, 2023.
"Causal Triplet: An Open Challenge for Intervention-centric Causal Representation Learning" Yuejiang Liu1, 2,* YUEJIANG.LIU@EPFL.CH Alexandre Alahi2 ALEXANDRE.ALAHI@EPFL.CH Chris Russell1 CMRUSS@AMAZON.DE Max Horn1 HORNMAX@AMAZON.DE Dominik Zietlow1 ZIETLD@AMAZON.DE Bernhard Sch ̈olkopf1, 3 BS@TUEBINGEN.MPG.DE Francesco Locatello1 LOCATELF@AMAZON.DE
Emphasizing lifetime-polymorphism can also make type inference untenable, a design choice that wouldn’t fit OCaml.
References or sources? Why? Presumably there's some research into this?
Nooria never goes for water, nor does Mother.Maryam doesn't, either. She doesn't have to doanything!
This sounds very childish, like something a 11-year old will say.
The Delta Method, from the field of nonlinear regression. The Bayesian Method, from Bayesian modeling and statistics. The Mean-Variance Estimation Method, using estimated statistics. The Bootstrap Method, using data resampling and developing an ensemble of models.
Four methods to compute prediction intervals.
A novel method for estimating prediction uncertainty using machine learning techniques is presented. Uncertainty is expressed in the form of the two quantiles (constituting the prediction interval) of the underlying distribution of prediction errors. The idea is to partition the input space into different zones or clusters having similar model errors using fuzzy c-means clustering. The prediction interval is constructed for each cluster on the basis of empirical distributions of the errors associated with all instances belonging to the cluster under consideration and propagated from each cluster to the examples according to their membership grades in each cluster. Then a regression model is built for in-sample data using computed prediction limits as targets, and finally, this model is applied to estimate the prediction intervals (limits) for out-of-sample data. The method was tested on artificial and real hydrologic data sets using various machine learning techniques. Preliminary results show that the method is superior to other methods estimating the prediction interval. A new method for evaluating performance for estimating prediction interval is proposed as well.
Prediction intervals using quantiles. Use clustering.
collecting and checking the content of declarations of private interests, of personal data that are liable to disclose indirectly the political opinions, trade union membership or sexual orientation of a natural person constitutes processing of special categories of personal data, for the purpose of those provisions.
Second question: If you collect it, can you infer from it?
those provisions cannot be interpreted as meaning that the processing of personal data that are liable indirectly to reveal sensitive information concerning a natural person is excluded from the strengthened protection regime prescribed by those provisions, if the effectiveness of that regime and the protection of the fundamental rights and freedoms of natural persons that it is intended to ensure are not to be compromised.
And here's the key element for indirect/inferred data. In order for Article 9 to matter, it must also include data that infers SCD.
Hermeneutic circle In traditional humanities scholarship, the hermeneutic circle refers to the way in which we understand some part of a text in terms of our ideas about its overall structure and meaning -- but that we also, in a cyclic fashion, update our beliefs about the overall structure and meaning of a text in response to particular moments.
As I think today microservice can do much more than just gives predictions using a single model, like:
List of differences between a microservice and inference service.
(see bullet points below annotation)
And I declare, on my word of honour, that what I am now about to write is, strictly and literally, the truth.
Sounds reminiscent of the oath witnesses take on the stand: "tell the truth, the whole truth, and nothing but the truth." Since The Moonstone is described as a detective novel, this kind of language and the conflict with the narrator's cousin suggest that the plot may be concerned with finding out the truth from the narrator and cousin's (presumably) conflicting accounts of events.
so now finally we get to active inference all this discussion and we're finally getting to the point here right for his lab so um i had and i had already touched on 01:47:35 some of this before but um it would you know today if you're going to develop a really good ai system you're and you're going to have a you have a robot saying the robot has to act 01:47:47 in some environment it is pretty well understood that that if you program that robot to you give it a you give it a i mean traditionally you'll give it a a a fitness function or some kind of 01:47:59 valuation function and it's for example it's good if it it you know you lose points if you fall through a trap door and is and you get points if you uh you know whatever 01:48:11 find find the piece of cake or something well that's uh that's fine for extremely simple universes that your robot might work in but as soon as you get beyond you know as soon as you get to any kind of more realistic uh 01:48:24 universe that your robot has to work in that pre-programming pre-programming concept just kind of falls apart it is you you it would require the the the practitioner to think ahead of all the 01:48:37 things that the robot might encounter and then how to value certain you know value those situations in certain ways uh and that is really uh what active inference 01:48:49 offers is a is a kind of a cognitive understanding or a mechanism by which an organism will uh uh where its 01:49:00 fitness score is in a sense involves both uh you know achieving goals and exploring its world to for for for epistemic gain so 01:49:16 um that's what we would like the that's how we would like to program the robot in a sense so that it can learn from it can learn on the fly from its experiences it can it can alter its actions and 01:49:30 goals as it be as it becomes clear as it gathers more information from its universe as it as it meets new situations that were never never conceived of by the by the 01:49:42 programmer that it through through an active inference or an active inference like uh you know mechanism it can learn and explore and and critically balance exploration with 01:49:54 exploitation and then we come right back to that whole concept of criticality so you know what you would really like your robot to do is remain at that critical uh phase between 01:50:06 exploring what's out there and making use and gold directed behavior of what's in front of it and um and uh you know that's how you could program this world this robot to act in the world and be pretty good at 01:50:20 it you know if you if you build it well so that's what the systems of a society can help a society to do you you don't you it's worth talking about building new systems i think it would not be wise to say 01:50:32 this checklist of like we wanted this level of education we want to want this you know to react this way in this situation react this way in this situation and this level of uh you know whatever money and this level of this and this 01:50:45 level of that while those kinds of preferences can be a useful start society has to be alive in its moment you know in the moment as society is alive it's cognating it's 01:50:57 it's it's it's actively uh you know comparing what it's the result of its actions to the model that is in its head and uh so active inference offers this way 01:51:09 to uh to balance uh exploration and and uh and uh exploitation and remain critical and remain optimally cognitive right so that's part of it 01:51:24 uh and then part of it i mean and for me this the the the idea of the embodied uh you know the three four e's uh this is what i really am attracted to in 01:51:46 active inference is in a sense it's kind of a simple concept it's not really very complicated you know if you've studied bayesian uh theory it all it's kind of straight you know in a way it's kind of straightforward 01:51:58 but the the you know the way fristen has connected the dots and and and and uh extended that into the bigger picture of life kind of it it to me it is uh it is rich 01:52:11 there's a there's a lot yet to be learned and gained and explored in this umbrella of active inference
Active inference is exemplified using a robot, but is really a model of how humans learn, process information and make decisions in the world.
inference to increase the coverage of Wikidata data considerably
so wikidate can create knowledge by 'thinking' inferring in software.
ReconfigBehSci [@SciBeh]. ‘RT @CAUSALab: Interested in #causalinference? Learn from Top Experts in the Field. Summer Courses Offered at the Harvard T.H. Chan Schoo…’. Tweet. Twitter, 20 December 2021. https://twitter.com/SciBeh/status/1483138177837715464.
ReconfigBehSci. (2021, February 2). @MichaelPaulEdw1 @islaut1 @ToddHorowitz3 @richarddmorey @MaartenvSmeden as I just said to @islaut1 if you want to force the logical contradiction you move away entirely from all of the interesting cases of inference from absence in everyday life, including the interesting statistical cases of, for example, null findings—So I think we now agree? [Tweet]. @SciBeh. https://twitter.com/SciBeh/status/1356530759016792064
ReconfigBehSci. (2021, February 1). @islaut1 @richarddmorey I think of strength of inference resting on P(not E|not H) (for coronavirus case). Search determines the conditional probability (and by total probability of course prob of evidence) but it isn’t itself the evidence. So, was siding with R. against what I thought you meant ;-) [Tweet]. @SciBeh. https://twitter.com/SciBeh/status/1356216290847944706
assumes that the function identifier f has a particular type
How is the initial assumption choosen? Does it start with a completely generic type and then tries to narrow it down?
Nast, C. (2022, January 15). Do the Omicron Numbers Mean What We Think They Mean? The New Yorker. https://www.newyorker.com/magazine/2022/01/24/do-the-omicron-numbers-mean-what-we-think-they-mean
Tom Moultrie. (2021, December 12). Given the comedic misinterpretation of the South African testing data offered by @BallouxFrancois (and many others!) last night ... I offer some tips having contributed to the analysis of the testing data for the @nicd_sa since April last year. (1/6) [Tweet]. @tomtom_m. https://twitter.com/tomtom_m/status/1469954015932915718
Tran, V.-T., Perrodeau, E., Saldanha, J., Pane, I., & Ravaud, P. (2021). Efficacy of COVID-19 Vaccination on the Symptoms of Patients With Long COVID: A Target Trial Emulation Using Data From the ComPaRe e-Cohort in France (SSRN Scholarly Paper ID 3932953). Social Science Research Network. https://papers.ssrn.com/abstract=3932953
Are 7 French fries too many?. A causal inference explainer | by Ellie Murray | Medium. (n.d.). Retrieved August 22, 2021, from https://medium.com/@EpiEllie/are-7-french-fries-too-many-d6226e78dc1f
Dunn, E. W., Chen, L., Proulx, J. D. E., Ehrlinger, J., & Savalei, V. (2021). Can Researchers’ Personal Characteristics Shape Their Statistical Inferences? Personality and Social Psychology Bulletin, 47(6), 969–984. https://doi.org/10.1177/0146167220950522
Rohrer, J. M., Schmukle, S., & McElreath, R. (2021). The Only Thing That Can Stop Bad Causal Inference Is Good Causal Inference. PsyArXiv. https://doi.org/10.31234/osf.io/mz5jx
Stephen Senn. (19:12:01 UTC). De Finetti meets Popper [Data & Analytics]. https://www.slideshare.net/StephenSenn1/de-finetti-meets-popper
Blakely, Tony, John Lynch, Koen Simons, Rebecca Bentley, and Sherri Rose. ‘Reflection on Modern Methods: When Worlds Collide—Prediction, Machine Learning and Causal Inference’. International Journal of Epidemiology 49, no. 6 (1 December 2020): 2058–64. https://doi.org/10.1093/ije/dyz132.
Freedman, D. A. (n.d.). Ecological Inference and the Ecological Fallacy. 7.
Lakens, D. (2020, November 29). The 20% Statistician: Why I care about replication studies. The 20% Statistician. https://daniellakens.blogspot.com/2020/11/why-i-care-about-replication-studies.html
Haug, N., Geyrhofer, L., Londei, A., Dervic, E., Desvars-Larrive, A., Loreto, V., Pinior, B., Thurner, S., & Klimek, P. (2020). Ranking the effectiveness of worldwide COVID-19 government interventions. Nature Human Behaviour, 4(12), 1303–1312. https://doi.org/10.1038/s41562-020-01009-0
Cousins, R. D. (2017). The Jeffreys–Lindley paradox and discovery criteria in high energy physics. Synthese, 194(2), 395–432. https://doi.org/10.1007/s11229-014-0525-z
Perez Santangelo, A., & Solovey, G. (2020, November 9). Time to Shine: Reliable Response-Timing Using R-Shiny for Online Experiments. https://doi.org/10.31234/osf.io/nuxdg
While no serious climate scientist doubts the fact that human activities are causing climate change, this can’t be proved through experimentation on another Earth.
In both cases, the answers should be clear when looking at the evidence and the mechanisms at play without an ideological bias
“provenance” — broadly, where did data arise, what inferences were drawn from the data, and how relevant are those inferences to the present situation? While a trained human might be able to work all of this out on a case-by-case basis, the issue was that of designing a planetary-scale medical system that could do this without the need for such detailed human oversight.
Data Provenance
The discipline of thinking about:
(1) where did the data arise? (2) what inferences were drawn (3) how relevant are those inferences to the present situation?
Kunin, D. (n.d.). Seeing Theory. Retrieved October 27, 2020, from http://seeingtheory.io
The Alan Turing Institute: Causal Inference, Causal Decision Making Under Uncertainty | CogX 2020. (2020, June 25). https://www.youtube.com/watch?v=JAGRHbDLvUs
Speelman, C., & McGann, M. (2020). Statements about the Pervasiveness of Behaviour Require Data about the Pervasiveness of Behaviour [Preprint]. PsyArXiv. https://doi.org/10.31234/osf.io/bxzm4
Young, J.-G., Cantwell, G. T., & Newman, M. E. J. (2020). Robust Bayesian inference of network structure from unreliable data. ArXiv:2008.03334 [Physics, Stat]. http://arxiv.org/abs/2008.03334
Manski, C. F., & Molinari, F. (2020). Estimating the COVID-19 Infection Rate: Anatomy of an Inference Problem (Working Paper No. 27023; Working Paper Series). National Bureau of Economic Research. https://doi.org/10.3386/w27023
Leininger, A., & Schaub, M. (2020). Voting at the dawn of a global pandemic [Preprint]. SocArXiv. https://doi.org/10.31235/osf.io/a32r7
Méndez, P. F. (2020). Blue uncertainty: Warding off systemic risks in the Anthropocene – Lessons from COVID-19 [Preprint]. SocArXiv. https://doi.org/10.31235/osf.io/z2br5
Zhang, L., & Peixoto, T. P. (2020). Statistical inference of assortative community structures. ArXiv:2006.14493 [Cond-Mat, Physics:Physics, Stat]. http://arxiv.org/abs/2006.14493
Causal inference isn’t what you think it is. (n.d.). LSHTM. Retrieved June 26, 2020, from https://www.lshtm.ac.uk/newsevents/events/causal-inference-isnt-what-you-think-it
Marshall, B. D. L., & Galea, S. (2015). Formalizing the Role of Agent-Based Modeling in Causal Inference and Epidemiology. American Journal of Epidemiology, 181(2), 92–99. https://doi.org/10.1093/aje/kwu274
They reflect an area of science known as biological taxonomy, the classification of organisms into different groups.
But these facts bear little consequence in day-to-day interactions hence their exotic status. People confuse less and fewer because using one or the other rarely changes the interaction. Calling a cashew a seed or a nut really doesn't change much.
discover how these mountain people identified relatives and friends
tried to discover and learn from the people form the culture
Rather than studying people, ethnography means learning from people
involves making inferences and knowing back round knowledge
without Brady‟ s knowledge and approval
Brady needed no knowledge of this activity. Safe to assume he told them that he likes 12.5 and that he is upset when they are inflated higher e.g. 16 psi.
“You good Jonny boy? ”; “You doing good?
Again, this is another negative inference that can easily be considered normal behavior in this situation.
speaking by telephone three times in the hours after the game for a total of 37 minutes and 11 seconds
Seemingly normal whether guilty or innocent.
McNally‟s knowledg e that Brady prefers footballs inflat ed at the low end of the permissible range and his express request that the referee set the balls at a 12.5 psi level
If there have been instances of balls being inflated by referees to 16, it is plausible that Brady would instruct the guy who gives the balls to the officials to make sure they stay at 12.5.
Brady and Jastremski shortly after suspicions of ball tampering became public on January 1
This is another inference to the negative. If you are implicated in something with someone who works with/for you is it a natural reaction to stop communicating? Is it more natural to speak with that person? How does behavior change when the entire global media is involved?