341 Matching Annotations
  1. Jun 2024
    1. we are on course for AGI by 2027 and that these AI 00:19:25 systems will basically be able to automate basically all all cognitive jobs think any job that can be done remotely

      for - AI evolution - prediction - 2027 - all cognitive jobs can be done by AI

    2. suppose that GPT 4 training took 3 months in 2027 a leading AI lab will be able to train a GPT 4 00:18:19 level model in a minute

      for - stat - AI evolution - prediction 2027 - training time - 6 OOM decrease

      stat - AI evolution - prediction 2027 - training time - 6 OOM decrease - today it takes 3 months to train GPT 4 - in 2027, it will take 1 minute - That is, 131,400 minutes vs 1 minute, or - 6 OOM

    3. by 2027 rather than a chatbot you're going to have something that looks more like an agent and more like a coworker

      for - AI evolution - prediction - 2027 - AI agent will replace AI chatbot

    4. it is strikingly plausible that by 2027 models 00:03:36 will be able to do the work of an AI researcher SL engineer that doesn't require believing in sci-fi it just requires in believing in straight lines on a graph

      for - quote - AI prediction for 2027 - Leopold Aschenbrenner

      quote - AI prediction for 2027 - Leopold Aschenbrenner - (see quote below) - it is strikingly plausible that by 2027 - models will be able to do the work of an AI researcher SL engineer - that doesn't require believing in sci-fi - it just requires in believing in straight lines on a graph

    1. In this sense, humor would help maintain a good emotional state while avoiding solving the problem at hand; and this strategy has been proposed to be particularly effective with low controllable or uncontrollable stressors, such as the COVID-19 pandemic scenario [18].

      humor as a coping mechanism on uncontrollable situations would be hedonistic disengagement because we have no control over the situation nor can we anticipate what's gonna happen next

    2. Humor can be defined as a hedonistic escapism strategy that would work better in the presence of unpredictable or uncontrollable stressors, such as the spread of the COVID-19 pandemic and its related confinement measures.

      interesting. would humor then work on situations that are predictable or congruous?

    1. "humor involves an idea, image, text or event that is in some sense incongruous, odd, unusual, unexpected, surprising, or out-of-the-ordinary. In addition, the humor stimulus must be accompanied by cues that signal us to appraise the stimulus in a playful, nonserious, nonliteral frame of mind in which people temporarily abandon rules of logic and expectations of common sense and congruity (e.g. Apter, 1982, Berlyne, 1972; Cohen, 1999; Cundall, 2007; McGhee, 192; Morreall, 1987; Mulkay, 1988).

      pg.4

  2. Apr 2024
  3. Dec 2023
    1. next year we we'll know whether your your your numbers are right in your pipeline paper around May of next year 01:46:30 and then it's going to be a very warm year it's going to be a lot of Destruction then we need we need to see how far the temperature Falls with the elino with the linia that follows but I 01:46:42 I expect it's not going to fall as much as you would otherwise have expected because of the large planetary energy balance there's more energy coming in than going out so it's hard for the 01:46:55 linia to cool it off as much as it used to
      • for:May 2024 - James Hansen prediction, extreme weather event - May 2024 - Hansen 2023 paper, prediction - extreme weather 2024
  4. Aug 2023
    1. T9 (text prediction):generative AI::handgun:machine gun

      Link to: https://hypothes.is/a/n6wXvkeNEe6DOFexaCD-Qg

    2. Some may not realize it yet, but the shift in technology represented by ChatGPT is just another small evolution in the chain of predictive text with the realms of information theory and corpus linguistics.

      Claude Shannon's work along with Warren Weaver's introduction in The Mathematical Theory of Communication (1948), shows some of the predictive structure of written communication. This is potentially better underlined for the non-mathematician in John R. Pierce's book An Introduction to Information Theory: Symbols, Signals and Noise (1961) in which discusses how one can do a basic analysis of written English to discover that "e" is the most prolific letter or to predict which letters are more likely to come after other letters. The mathematical structures have interesting consequences like the fact that crossword puzzles are only possible because of the repetitive nature of the English language or that one can use the editor's notation "TK" (usually meaning facts or date To Come) in writing their papers to make it easy to find missing information prior to publication because the statistical existence of the letter combination T followed by K is exceptionally rare and the only appearances of it in long documents are almost assuredly areas which need to be double checked for data or accuracy.

      Cell phone manufacturers took advantage of the lower levels of this mathematical predictability to create T9 predictive text in early mobile phone technology. This functionality is still used in current cell phones to help speed up our texting abilities. The difference between then and now is that almost everyone takes the predictive magic for granted.

      As anyone with "fat fingers" can attest, your phone doesn't always type out exactly what you mean which can result in autocorrect mistakes (see: DYAC (Damn You AutoCorrect)) of varying levels of frustration or hilarity. This means that when texting, one needs to carefully double check their work before sending their text or social media posts or risk sending their messages to Grand Master Flash instead of Grandma.

      The evolution in technology effected by larger amounts of storage, faster processing speeds, and more text to study means that we've gone beyond the level of predicting a single word or two ahead of what you intend to text, but now we're predicting whole sentences and even paragraphs which make sense within a context. ChatGPT means that one can generate whole sections of text which will likely make some sense.

      Sadly, as we know from our T9 experience, this massive jump in predictability doesn't mean that ChatGPT or other predictive artificial intelligence tools are "magically" correct! In fact, quite often they're wrong or will predict nonsense, a phenomenon known as AI hallucination. Just as with T9, we need to take even more time and effort to not only spell check the outputs from the machine, but now we may need to check for the appropriateness of style as well as factual substance!

      The bigger near-term problem is one of human understanding and human communication. While the machine may appear to magically communicate (often on our behalf if we're publishing it's words under our names), is it relaying actual meaning? Is the other person reading these words understanding what was meant to have been communicated? Do the words create knowledge? Insight?

      We need to recall that Claude Shannon specifically carved semantics and meaning out of the picture in the second paragraph of his seminal paper:

      Frequently the messages have meaning; that is they refer to or are correlated according to some system with certain physical or conceptual entities. These semantic aspects of communication are irrelevant to the engineering problem.

      So far ChatGPT seems to be accomplishing magic by solving a small part of an engineering problem by being able to explore the adjacent possible. It is far from solving the human semantic problem much less the un-adjacent possibilities (potentially representing wisdom or insight), and we need to take care to be aware of that portion of the unsolved problem. Generative AIs are also just choosing weighted probabilities and spitting out something which is prone to seem possible, but they're not optimizing for which of many potential probabilities is the "best" or the "correct" one. For that, we still need our humanity and faculties for decision making.


      Shannon, Claude E. A Mathematical Theory of Communication. Bell System Technical Journal, 1948.

      Shannon, Claude E., and Warren Weaver. The Mathematical Theory of Communication. University of Illinois Press, 1949.

      Pierce, John Robinson. An Introduction to Information Theory: Symbols, Signals and Noise. Second, Revised. Dover Books on Mathematics. 1961. Reprint, Mineola, N.Y: Dover Publications, Inc., 1980. https://www.amazon.com/Introduction-Information-Theory-Symbols-Mathematics/dp/0486240614.

      Shannon, Claude Elwood. “The Bandwagon.” IEEE Transactions on Information Theory 2, no. 1 (March 1956): 3. https://doi.org/10.1109/TIT.1956.1056774.


      We may also need to explore The Bandwagon, an early effect which Shannon noticed and commented upon. Everyone seems to be piling on the AI bandwagon right now...

  5. Apr 2023
    1. The Delta Method, from the field of nonlinear regression. The Bayesian Method, from Bayesian modeling and statistics. The Mean-Variance Estimation Method, using estimated statistics. The Bootstrap Method, using data resampling and developing an ensemble of models.

      Four methods to compute prediction intervals.

    1. A novel method for estimating prediction uncertainty using machine learning techniques is presented. Uncertainty is expressed in the form of the two quantiles (constituting the prediction interval) of the underlying distribution of prediction errors. The idea is to partition the input space into different zones or clusters having similar model errors using fuzzy c-means clustering. The prediction interval is constructed for each cluster on the basis of empirical distributions of the errors associated with all instances belonging to the cluster under consideration and propagated from each cluster to the examples according to their membership grades in each cluster. Then a regression model is built for in-sample data using computed prediction limits as targets, and finally, this model is applied to estimate the prediction intervals (limits) for out-of-sample data. The method was tested on artificial and real hydrologic data sets using various machine learning techniques. Preliminary results show that the method is superior to other methods estimating the prediction interval. A new method for evaluating performance for estimating prediction interval is proposed as well.

      Prediction intervals using quantiles. Use clustering.

  6. Mar 2023
    1. It is safe to predict that in the near future intelligence tests will bring tens of thousands of these high-grade defectives under the surveillance and protection of society. This will ultimately result in curtailing the reproduction of feeble-mindedness and in the elimination of an enormous amount of crime, pauperism, and industrial inefficiency. It is hardly necessary to emphasize that the high-grade cases, of the type now so frequently overlooked, are precisely the ones whose guardianship it is most important for the State to assume.

      I think it is interesting how they say it is safe to predict in the future intelligence tests will bring thousands of high-grade defectives. The result they have predicted is interesting because they think the results will eliminate crime, pauperism, and industrial inefficiency. This relates to the history of psychology because they predicted the future. Since we are the future, I don't think there has been a decrease in crime, pauperism which is poverty, and industrial inefficiency. I think we are on the rise of crime and pauperism.

    1. Good protocols tend to form persistent Schelling points in spaces of problems worth solving, around solutions good enough to live with – for a while. And surprisingly often, they manage to induce more complex patterns of voluntary commitment and participation than are achieved by competing systems of centralized coordination.

      "inducing more complex patterns" reminds me a little bit of Peter Cotton's Microprediction, does a market form around an idea? Is a protocol one of these ideas?

      Appreciate the 'problems worth solving'

      Because it's fresh in memory, I like this as illustrated by Chasing Venus

  7. Jan 2023
  8. Nov 2022
  9. Oct 2022
    1. Intellectual readiness involves a minimumlevel of visual perception such that the child can take in andremember an entire word and the letters that combine to formit. Language readiness involves the ability to speak clearly andto use several sentences in correct order.

      Just as predictive means may be used on the level of letters, words, and even whole sentences within information theory at the level of specific languages, does early orality sophistication in children help them to become predictive readers at earlier ages?

      How could one go about testing this, particularly in a broad, neurodiverse group?

  10. Aug 2022
    1. Doing this, we can confidentlyconclude that by the year 2035 it is more likelythan not that quantum technology will have ad-vanced sufficiently to be able to break RSA2048efficiently. This conclusion is shared by well es-tablished researchers (see, e.g.[2, 3])

      Here, author uses other researcher's conclusions and states that by the year 2035 it is a fact that quantum technology will have advance sufficiently to be able to break RSA2048 efficiently.

    Tags

    Annotators

  11. Mar 2022
    1. Eric Topol. (2022, February 28). A multimodal #AI study of ~54 million blood cells from Covid patients @YaleMedicine for predicting mortality risk highlights protective T cell role (not TH17), poor outcomes of granulocytes, monocytes, and has 83% accuracy https://nature.com/articles/s41587-021-01186-x @NatureBiotech @KrishnaswamyLab https://t.co/V32Kq0Q5ez [Tweet]. @EricTopol. https://twitter.com/EricTopol/status/1498373229097799680

  12. Feb 2022
    1. Adele Groyer. (2022, January 8). Friday report is now out. Https://covidactuaries.org/2022/01/07/the-friday-report-issue-58/ I am struck that perception of a “mild” Covid situation is relative. In SA natural deaths were >30% higher than predicted in Dec. The last time weekly death rates in E&W were more than 30% above 2015-19 levels was in Jan 2021. Https://t.co/S9fkn2WFVk [Tweet]. @AdeleGroyer. https://twitter.com/AdeleGroyer/status/1479760460589191170

  13. Jan 2022
    1. Zimmerman, M. I., Porter, J. R., Ward, M. D., Singh, S., Vithani, N., Meller, A., Mallimadugula, U. L., Kuhn, C. E., Borowsky, J. H., Wiewiora, R. P., Hurley, M. F. D., Harbison, A. M., Fogarty, C. A., Coffland, J. E., Fadda, E., Voelz, V. A., Chodera, J. D., & Bowman, G. R. (2021). SARS-CoV-2 simulations go exascale to predict dramatic spike opening and cryptic pockets across the proteome. Nature Chemistry, 13(7), 651–659. https://doi.org/10.1038/s41557-021-00707-0

    1. Keeling, M. J., Brooks-Pollock, E., Challen, R. J., Danon, L., Dyson, L., Gog, J. R., Guzman-Rincon, L., Hill, E. M., Pellis, L. M., Read, J. M., & Tildesley, M. (2021). Short-term Projections based on Early Omicron Variant Dynamics in England. (p. 2021.12.30.21268307). https://doi.org/10.1101/2021.12.30.21268307

  14. Dec 2021
    1. Eric Feigl-Ding. (2021, December 2). A rise in possible #Omicron in England—Tripling (0.1 to 0.3) of S-Gene dropout PCR signal, which is a proxy for Omicron (before 🧬 sequencing confirms). @_nickdavies estimates this represents around ~60 cases in 🏴󠁧󠁢󠁥󠁮󠁧󠁿. Still early—But it displacing #DeltaVariant is not good sign. 🧵 https://t.co/4aIiqiVsqH [Tweet]. @DrEricDing. https://twitter.com/DrEricDing/status/1466234026843205637

  15. Nov 2021
    1. Benjamin Veness. (2021, November 2). Singapore’s 🇸🇬 Senior Minister of State for Health, Dr Janil Puthucheary, told Parliament on 1 November: “I hope my explanation has helped members understand why although we say we are living with COVID-19, we cannot just open up, and risk having the number of cases shoot up.” [Tweet]. @venessb. https://twitter.com/venessb/status/1455396047765733376

  16. Oct 2021
  17. Sep 2021
    1. Hippisley-Cox, J., Coupland, C. A., Mehta, N., Keogh, R. H., Diaz-Ordaz, K., Khunti, K., Lyons, R. A., Kee, F., Sheikh, A., Rahman, S., Valabhji, J., Harrison, E. M., Sellen, P., Haq, N., Semple, M. G., Johnson, P. W. M., Hayward, A., & Nguyen-Van-Tam, J. S. (2021). Risk prediction of covid-19 related death and hospital admission in adults after covid-19 vaccination: National prospective cohort study. BMJ, 374, n2244. https://doi.org/10.1136/bmj.n2244

    1. Bracher, J., Wolffram, D., Deuschel, J., Görgen, K., Ketterer, J. L., Ullrich, A., Abbott, S., Barbarossa, M. V., Bertsimas, D., Bhatia, S., Bodych, M., Bosse, N. I., Burgard, J. P., Castro, L., Fairchild, G., Fuhrmann, J., Funk, S., Gogolewski, K., Gu, Q., … Xu, F. T. (2021). A pre-registered short-term forecasting study of COVID-19 in Germany and Poland during the second wave. Nature Communications, 12(1), 5173. https://doi.org/10.1038/s41467-021-25207-0

  18. Aug 2021
  19. Jul 2021
  20. Jun 2021
    1. Your brain is a prediction machine.

      See also Agrawal, A., Gans, J., & Goldfarb, A. (2018). Prediction Machines: The Simple Economics of Artificial Intelligence. Harvard Business Review Press.

    1. V Shah, A. S., Gribben, C., Bishop, J., Hanlon, P., Caldwell, D., Wood, R., Reid, M., McMenamin, J., Goldberg, D., Stockton, D., Hutchinson, S., Robertson, C., McKeigue, P. M., Colhoun, H. M., & McAllister, D. A. (2021). Effect of vaccination on transmission of COVID-19: An observational study in healthcare workers and their households [Preprint]. Public and Global Health. https://doi.org/10.1101/2021.03.11.21253275

  21. May 2021
    1. ReconfigBehSci on Twitter: “this is utterly bizarre: How would one conceptually even begin to determine a number by which the model overestimated unmitigated deaths. What is the comparison unmitigated ‘prediction’ to what actually happened supposed to mean?” / Twitter. (n.d.). Retrieved May 1, 2021, from https://twitter.com/SciBeh/status/1384070393514790918

  22. Apr 2021
    1. Eric Topol on Twitter: “The variants of concern/interest fall into a spectrum of immune evasiveness, w/ B.1.351 being most; B.1.1.7, B.1.429 least. This property pertains to potential for reinfection & some reduction in vaccine efficacy My prelim estimates based on publications/preprints, subject to Δ https://t.co/fQZwBCUEGS” / Twitter. (n.d.). Retrieved April 28, 2021, from https://twitter.com/EricTopol/status/1380203664317456385

    1. ReconfigBehSci. (2021, April 19). @ToddHorowitz3 it could be meaningful only vis a vis certain qualitative constraints: E.g., ‘look, model predicts fewer deaths for unmitigated than observed even with lockdown’ => model underpredicts.... But that’s very much not the scenario here [Tweet]. @SciBeh. https://twitter.com/SciBeh/status/1384146492609372177

    1. ReconfigBehSci. (2021, April 19). @ToddHorowitz3 so, given that no one can know the ‘unmitigated number’ what they seem to be calculating is in difference deaths given lockdown and model prediction without lockdown and calling that the ‘overestimate’—Which seems truly bizarre [Tweet]. @SciBeh. https://twitter.com/SciBeh/status/1384147188180082692

    1. Jeremy Faust MD MS (ER physician) on Twitter: “Let’s talk about the background risk of CVST (cerebral venous sinus thrombosis) versus in those who got J&J vaccine. We are going to focus in on women ages 20-50. We are going to compare the same time period and the same disease (CVST). DEEP DIVE🧵 KEY NUMBERS!” / Twitter. (n.d.). Retrieved April 15, 2021, from https://twitter.com/jeremyfaust/status/1382536833863651330

    1. Bleuler defined schizophrenia with his four ‘A’s’, referring to the blunted Affect (diminished emotional response to stimuli); loosening of Associations (by which he meant a disordered pattern of thought, inferring a cognitive deficit), Ambivalence (an apparent inability to make decisions, again suggesting a deficit of the integration and processing of incident and retrieved information) and Autism (a loss of awareness of external events, and a preoccupation with the self and one’s own thoughts)

      I stumbled upon this accidentally. I was going to add to my prediction that schizophrenia might be related to autism, but now that I've found this I need to publish my draft.


      Edit: Here was the prediction I wrote. Copied unmodified, ensuring transparency.

      March 28, 6:15pm Prediction: Some cases of schizophrenia are being misdiagnosed as autism. I recently took a Coursera.org course on schizophrenia. The negative symptoms look similar to some autism symptoms.

      Before I look it up, there are a few other predictions I should make. Do I think schizophrenia and autism will be linked? If there’s cross-diagnosis, will this link be artificial or real? Last time I looked, people with aspergers had (more or less) normal sleep EEGs. In contrast, schizophrenia is associated with disrupted sleep spindles. I already know that schizophrenia and bipolar are genetically linked, but I don’t know what the bipolar sleep EEG looks like. That is to say, I don’t know if the lack of sleep abnormality in autism is evidence against a link to schizophrenia. All in all, I predict that there will be a real link (for example, genetic), but I have a low confidence in this prediction. The reason is that I expect there is little EEG sleep changes in bipolar, implying that there is a supra-mechanism causing all these effects; somewhat like metabolic syndrome, the same cause may manifest in different ways.

    1. Using this data, a large international team was able to pinpoint 114 specific loci – locations in the human genome – that contribute to risk of both schizophrenia and bipolar disorder, and four genome regions that contribute to differences in the biology of the two disorders.

      This is exactly what I expected. In fact, I would have been extremely surprised if this weren't the case. I just google "schizophrenia bipolar genes" expecting this result.

      I had the thought a few minutes ago, and google it right away. This means that I wasn't able to write it down as a prediction. Nonetheless, I think this points in favor of my prediction abilities. My confidence was inordinately high (i.e. on the order of 90%) even before collecting any evidence. Compare that to other high confidence beliefs (e.g. CFS is caused partly by blood volume), for which I have confidence on the order of 95%, but I have good evidence for that belief. Thus, this instance provides data that my confidence meter is reliable. I'll continue to make an effort to write down predictions ahead of time (to eliminate publication bias).

      There are several reasons I suspected this would be the case. Firstly, personal subjective experience; that's what gave me the first inkling. Secondly, the connection of mania with long periods of sleeplessness. If the sleep deprivation causes the mania, then bipolar may be a sleep disorder. This is backed up by the sleep deprivation therapy for depression. Additionally, the connection of depression to sleep disturbance implies that sleep may also be causal in low mood. Furthermore, given that schizophrenia is associated with disrupted sleep spindles, it follows that the two sleep disorders, namely schizophrenia and bipolar, may be closely related genetically (via sleep regulating genes). Moreover, I knew that schizophrenia and bipolar were two of the most heritable psychological conditions; given that both are highly genetic and both involve sleep, it follows that they would likely be closely linked. Finally, I know mania can be associated with delusions, so there are several symptom crossovers. All in all, it is highly surprising that I have not seen this discussed before. Neither documentaries on schizophrenia nor documentaries on manic depression/bipolar have mentioned a link. Nor have studies I've read (admittedly few on this particular topic) mentioned anything of the sort. I shall have to look through the literature to see if this idea has been around for long.

  23. Mar 2021
    1. Nick Barrowman. (2021, March 26). Throughout the pandemic, a widespread inability to reason counterfactually has been on display. For example, some people apparently think lockdowns don’t work. They seem unable to imagine the situation had there not been a lockdown. Lockdowns are costly, but they work! [Tweet]. @nbarrowman. https://twitter.com/nbarrowman/status/1375240312264740870

    1. This NFT world is likely the greatest unlock of artist opportunity in 100+ years. This isn't a suboptimal or fringe version of the real-world art economy, it is a vastly improved one.

      its fascinating how this may be simulatenously true with the fact that NFTs in their current actual implementation are a scam.. eg regarding copyright, minting on other chains, NFT frozen to a specific URL, NFTs can be trasnferred off chain / OTC such that the 10% royalty doesnt happen, etc etc)

      what ive come to see is that the reality is it doesnt matter - people making decisions on their career or specific investments, this matters a lot

      .. but before people thinking of diving into this cuz it may be the next big thing for the next 20 years, then ya that's probably correct

      this is massive cuz a lot of the problems will be solved one by one methodically even if its crappy today, itll be improved steadily cuz we are good at developing, what were bad at is blank sheet

      cant believe im agreeing with this belsky guy

    1. Stefan Simanowitz. (2020, November 14). “Sweden hoped herd immunity would curb #COVID19. Don’t do what we did” write 25 leading Swedish scientists “Sweden’s approach to COVID has led to death, grief & suffering. The only example we’re setting is how not to deal with a deadly infectious disease” https://t.co/azOg6AxSYH https://t.co/u2IqU5iwEn [Tweet]. @StefSimanowitz. https://twitter.com/StefSimanowitz/status/1327670787617198087

    1. ReconfigBehSci. (2020, December 8). I’ve been pondering failed predictions today. A spectacular error of mine: In the early media rush to listen to scientists and doctors, I actually thought Western societies might be seeing the end of the “influencer” and a renewed interest in people who did stuff 1/2 [Tweet]. @SciBeh. https://twitter.com/SciBeh/status/1336383952232308736

  24. Feb 2021
  25. Jan 2021
  26. Dec 2020
  27. Nov 2020
  28. Oct 2020
    1. Complexity, interconnectivity, novelty, & creation is beyond any single entity's ability to effectively forecast.
  29. Sep 2020
    1. Blokland, I. V. van, Lanting, P., Ori, A. P., Vonk, J. M., Warmerdam, R. C., Herkert, J. C., Boulogne, F., Claringbould, A., Lopera-Maya, E. A., Bartels, M., Hottenga, J.-J., Ganna, A., Karjalainen, J., Study, L. C.-19 cohort, Initiative, T. C.-19 H. G., Hayward, C., Fawns-Ritchie, C., Campbell, A., Porteous, D., … Franke, L. H. (2020). Using symptom-based case predictions to identify host genetic factors that contribute to COVID-19 susceptibility. MedRxiv, 2020.08.21.20177246. https://doi.org/10.1101/2020.08.21.20177246

  30. Aug 2020
    1. Menni, C., Valdes, A. M., Freidin, M. B., Sudre, C. H., Nguyen, L. H., Drew, D. A., ... & Visconti, A. (2020). Real-time tracking of self-reported symptoms to predict potential COVID-19. Nature Medicine, 1-4.

    1. Wittgenstein writes: “The limits of my language mean the limits of my world”. Maybe he was trying to make a restrictive statement, one about how we can’t know the world beyond our language. But the reverse is also true; language and the world have the same boundaries. Learn language really well, and you understand reality. God is One, and His Name is One, and God is One with His Name. “Become good at predicting language” sounds like the same sort of innocent task as “become good at Go” or “become good at Starcraft”. But learning about language involves learning about reality, and prediction is the golden key. “Become good at predicting language” turns out to be a blank check, a license to learn every pattern it can.

      Because language is an isomorphic mapping to the world, learning to predict language means you're learning to predict patterns that occur in the world.

    1. In “the eggplant is a fruit,” probably what is meant is that all eggplants are fruits. In “the dog is a Samoyed,” probably what is meant is that some dog is a Samoyed. We can reasonably assume these meanings from our background understanding of their topics. This knowledge is nowhere in the sentence. The meaning depends on its parts—but not only on them.

      It's common in speech coding (e.g. a vocoder) to rely on a thing that reconstructs the 'meaning' of a signal by predicting its 'full' representation.

      This act of predicting then is also a form of compressing, by predicting the full representation from its lossy analogue you require less bandwidth to transmit messages just like if you'd used a non-stochastic compression technique.

  31. Jul 2020