917 Matching Annotations
  1. Last 7 days
    1. Shanahan, Murray. "Talking About Large Language Models." arXiv, (2022). https://doi.org/10.48550/arXiv.2212.03551.

      Found via Simon Wilson.

      Abstract

      Thanks to rapid progress in artificial intelligence, we have entered an era when technology and philosophy intersect in interesting ways. Sitting squarely at the centre of this intersection are large language models (LLMs). The more adept LLMs become at mimicking human language, the more vulnerable we become to anthropomorphism, to seeing the systems in which they are embedded as more human-like than they really are. This trend is amplified by the natural tendency to use philosophically loaded terms, such as "knows", "believes", and "thinks", when describing these systems. To mitigate this trend, this paper advocates the practice of repeatedly stepping back to remind ourselves of how LLMs, and the systems of which they form a part, actually work. The hope is that increased scientific precision will encourage more philosophical nuance in the discourse around artificial intelligence, both within the field and in the public sphere.

    2. LLMs are generative math-ematical models of the statistical distributionof tokens in the vast public corpus of human-generated text, where the tokens in question in-clude words, parts of words, or individual char-acters including punctuation marks. They aregenerative because we can sample from them,which means we can ask them questions. Butthe questions are of the following very specifickind. “Here’s a fragment of text. Tell me howthis fragment might go on. According to yourmodel of the statistics of human language, whatwords are likely to come next?”

      LLM definition

    1. The breakthroughs are all underpinned by a new class of AI models that are more flexible and powerful than anything that has come before. Because they were first used for language tasks like answering questions and writing essays, they’re often known as large language models (LLMs). OpenAI’s GPT3, Google’s BERT, and so on are all LLMs. But these models are extremely flexible and adaptable. The same mathematical structures have been so useful in computer vision, biology, and more that some researchers have taken to calling them "foundation models" to better articulate their role in modern AI.

      Foundation Models in AI

      Large language models, more generally, are “foundation models”. They got the large-language name because that is where they were first applied.

    2. An AI model that can learn and work with this kind of problem needs to handle order in a very flexible way. The old models—LSTMs and RNNs—had word order implicitly built into the models. Processing an input sequence of words meant feeding them into the model in order. A model knew what word went first because that’s the word it saw first. Transformers instead handled sequence order numerically, with every word assigned a number. This is called "positional encoding." So to the model, the sentence “I love AI; I wish AI loved me” looks something like (I 1) (love 2) (AI 3) (; 4) (I 5) (wish 6) (AI 7) (loved 8) (me 9).

      Google’s “the transformer”

      One breakthrough was positional encoding versus having to handle the input in the order it was given. Second, using a matrix rather than vectors. This research came from Google Translate.

    1. taken her last breath

      The phrase 'last breath' indicates someone taking their last breath before dying.

    2. gave a star

      Idiom: to move suddenly from surprise or fright

    3. His eyes fell on his coat

      Metaphor. His eyes didn't literally fall on his coat, rather he turned his eyes downwards to see his coat.

    4. Her dead hands undidher salwar and lowered it

      She is described to have returned back from death. Play on 'dead' hands being able to perform action.

    5. Suddenly, the roomlit up.

      Metaphor for being lit up as in the energy lit up or lifted after recognizing his daughter's body, but also literally in that a doctor came and turned the light on.

    6. There was no one in the dark room, just a stretcher witha body on it.

      Possibly implying the lack of staff due to the sheer bloodshed of this situation. Mention of dark room.

    7. young volunteers

      'Young' + 'Volunteers' sets a tone for the dedication of these men in fighting for their nation.

    8. Sirajuddin wanted to cry, but his eyes would not cooperate. Who knewwhere all the tears had gone?

      Just like Sakina, his tears were gone. Rhetorically asking where both had gone. Once again, sensory imagery/personification of the eyes as they were not cooperating. Additionally, this shows depth to Sirajuddin in how we wants to express emotion.

    9. But as he racked his brains, his mind fixed onSakina’s mother’s body, her intestines spilled out, then he could think nofurther

      Sirajuddin, old and tired, is putting himself through 'hell' to find his loved ones. In doing so, his mind is fogged with a traumatic event occured recently. Describing the sequence of events shows how defenseless he is right now.

    10. Loot. Fire. Stampede. Station.Bullets. Night. And Sakina.

      Listing and punction creates emphasis on all subjects discussed.

    11. he felt as if he were floating in a void

      Similie

    12. There was chaos all roundhim, but he heard nothing, as if his ears were blocked

      Describing a moment of pure awe using the sensory imagery.

    13. swelling sea of men, women and children

      Interesting use of swelling. What is the meaning behind it?

    Tags

    Annotators

  2. Jan 2023
    1. https://polygloss.app/posts/scottish-gaelic-and-language-jam/

      Some useful resources for Scottish Gaelic.

      Also interesting to see how someone else approaches self-learning a language and what they compiled.

    1. Having thiscapacity,however,doesnotalwaysguaranteeitsuse (Sefman, Beardslee, Schultz, Krupa,&Podorefsky1986). For example,although mature individualsmayknowthat itisrudetointerruptanother speaker whoisstrugglingtoexpress anidea,they maynonethelessdosoduringapassionate debate or discussion.

      I think this is so important and often times not thought of. Especially when educating a classroom full of 20-30 students, it could become frustrating but it is important to remember that every brain is wired differently.

    1. The hypothesis of linguistic relativity, also known as the Sapir–Whorf hypothesis /səˌpɪər ˈwɔːrf/, the Whorf hypothesis, or Whorfianism, is a principle suggesting that the structure of a language influences its speakers' worldview or cognition, and thus people's perceptions are relative to their spoken language.

      https://en.wikipedia.org/wiki/Linguistic_relativity


      link to Toki Pona as a conlang


      Link to https://hypothes.is/a/6Znx6MiMEeu3ljcVBsKNOw We shape our tools and thereafter they shape us.

    1. a common technique in natural language processing is to operationalize certain semantic concepts (e.g., "synonym") in terms of syntactic structure (two words that tend to occur nearby in a sentence are more likely to be synonyms, etc). This is what word2vec does.

      Can I use some of these sorts of methods with respect to corpus linguistics over time to better identified calcified words or archaic phrases that stick with the language, but are heavily limited to narrower(ing) contexts?

    1. Siwan Clark, a Welsh speaker and MSc candidate in Social Research Methods at University College London, said Wales is environmentally suffering because of centuries of British colonialism, particularly during the Industrial Revolution, which had a devastating global environmental impact.Clark said her mother, who grew up on a farm in North Wales, understands critical Welsh words—a language Clark said is deeply agricultural—that “have no context or meaning” for her.“Industrialized farming is inextricable from empire,” Clark said. “If those small farms fail, then the language won’t truly survive.” 

      !- Welsh indigneous language : bio-cultural worldview - researcher Siwan Clark, University College London, claims Wales is suffering environmentally since British colonialism during Industrial Revolution - North Wales - critical Welsh words are deeply agricultural. The fall of the small farms also erodes the language they evovled

    2. “a huge amount of scientific information in Irish was passed down orally, but was lost to the trauma of the famine and immigration.” Today, only 1.7 percent of the Irish population is fluent. Ó Séaghdha, who said many words in Irish are like “one-word poems,” finds himself fascinated by this bio-cultural knowledge. “The word for swallow is fáinleog,” he said. “It means little wanderer, because the bird migrates during winter. In our languages, we understood bird migration, thousands and thousands of years ago. We had words for phosphorus magnetism before they had scientific books.”

      !- Irish indigenous language : bio-cultural worldview - Irish word for "swallow" is "fainleog" which means little wanderer because the bird migrates during winter. - Irish language was rich in bird migration words from thousands of years earlier. - The Irish had a word for phosphorus magnetism before science discovered it

    3. Learning a New Language Can Help Us Escape Climate Catastrophe

      !- Title : Learning a New Language Can Help Us Escape Climate Catastrophe !- Author : Nylan Burton !- comment : summary - while I agree with the analysis, the futures-related question I ask is this: what does a desirable hybridized linguistic landscape look like that integrates English, evolved into a post-colonialist lingua franca and reconstituted indigenous languages with their rich bio-cultural heritage?

    1. Feng, 2022. "Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesis"

      Shared and found via: Gowthami Somepalli @gowthami@sigmoid.social Mastodon > Gowthami Somepalli @gowthami StructureDiffusion: Improve the compositional generation capabilities of text-to-image #diffusion models by modifying the text guidance by using a constituency tree or a scene graph.

    1. Fried-berg Judeo-Arabic Project, accessible at http://fjms.genizah.org. This projectmaintains a digital corpus of Judeo-Arabic texts that can be searched and an-alyzed.

      The Friedberg Judeo-Arabic Project contains a large corpus of Judeo-Arabic text which can be manually searched to help improve translations of texts, but it might also be profitably mined using information theoretic and corpus linguistic methods to provide larger group textual translations and suggestions at a grander scale.

    1. When such consumers therefore mistake the meaning attributed tothe MT output as the actual communicative intent of the originaltext’s author, real-world harm can ensue.

      Harm from Machine Translation (MT) models

      MT models can create fluent and coherent blocks of text that mask the meaning in the original text and the intent of the original speaker.

    1. Considerations

      What about chained dotted access? foo.bar.baz is probably okay as bar.baz @ (the Foo) (or even @the Foo), but probably not if it takes the form bar.baz from the Foo. (It just doesn't look reasonable to me.)

      Alternatively, what about @bar.baz for the Foo?

    1. In Lua you can write raw, multiline strings with [[]]: [[ Alice said "Bob said 'hi'". ]]

      This is indeed very good (for the reasons stated here).

    1. how important is the concrete syntax of their language in contrast to

      how important is the concrete syntax of their language in contrast to the abstract concepts behind them what I mean they say can someone somewhat awkward concrete syntax be an obstacle when it comes to the acceptance

  3. Dec 2022
    1. Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT '21). Association for Computing Machinery, New York, NY, USA, 610–623. https://doi.org/10.1145/3442188.3445922

    1. When I started working on the history of linguistics — which had been totally forgotten; nobody knew about it — I discovered all sorts of things. One of the things I came across was Wilhelm von Humboldt’s very interesting work. One part of it that has since become famous is his statement that language “makes infinite use of finite means.” It’s often thought that we have answered that question with Turing computability and generative grammar, but we haven’t. He was talking about infinite use, not the generative capacity. Yes, we can understand the generation of the expressions that we use, but we don’t understand how we use them. Why do we decide to say this and not something else? In our normal interactions, why do we convey the inner workings of our minds to others in a particular way? Nobody understands that. So, the infinite use of language remains a mystery, as it always has. Humboldt’s aphorism is constantly quoted, but the depth of the problem it formulates is not always recognized.

      !- example : permanent mystery - language - Willhelm von Humboldt phrase "infinite use" - has never been solved - Why do decide to say one thing among infinitely many others?

    2. The miracle that so amazed Galileo and Arnauld — and still amazes me, I can’t understand it — is how can we, with a few symbols, convey to others the inner workings of our mind? That’s something to really be surprised about, and puzzled by. And we have some grasp of it, but not a lot.

      !- example : permanent mystery - language! This is what constantly amazes me!

    1. There are different kinds of information, some of which don't make sense being recorded at all. I was struggling with what to record and what not to record for a long time. For example, I took notes on programming syntax that are just useless (most of these things can be googled in seconds and they are usually decently documented already).

      How was this not obvious from the jump? Was the author of the essay so distracted by shiny object syndrome they failed to see the obvious?

      It's like taking notes in a language class... the goal is to read and write with fluency, so you practice these things regularly and gain fluency over time. Taking notes about the grammar and syntax of a language is highly unlikely to get you to fluency. Note taking is the wrong tool for a number of processes and the user should very quickly get a gut feeling for what's useful and what is not.

      This author obviously missed the boat here.

    1. natural-language processing is going to force engineers and humanists together. They are going to need each other despite everything. Computer scientists will require basic, systematic education in general humanism: The philosophy of language, sociology, history, and ethics are not amusing questions of theoretical speculation anymore. They will be essential in determining the ethical and creative use of chatbots, to take only an obvious example.
    1. Houston, we have a Capability Overhang problem: Because language models have a large capability surface, these cases of emergent capabilities are an indicator that we have a ‘capabilities overhang’ – today’s models are far more capable than we think, and our techniques available for exploring the models are very juvenile. We only know about these cases of emergence because people built benchmark datasets and tested models on them. What about all the capabilities we don’t know about because we haven’t thought to test for them? There are rich questions here about the science of evaluating the capabilities (and safety issues) of contemporary models. 
    1. While we are still a while away from a Google Translate equivalent for animal languages that can decode the nuances of intra-species communication, technology, especially machine learning, is keeping this hope alive. The ability to understand animal languages could open up a realm of possibilities, potentially shaping conservation efforts, determining our future relationship with other species, and even offering insights into the evolution of human language itself. 
    2. The scientific community is, thus, increasingly using technological tools including drones, recorders, robots and AI to study the calls of a range of species, from chickens and rodents, to cats and lemurs.
    1. “Digital technologies, so often associated with our alienation from nature, are offering us an opportunity to listen to nonhumans in powerful ways, reviving our connection to the natural world,”
    2. Researchers are using drones, AI, and digital recorders to create a “zoological version of Google Translate.”
    1. Frisch referred to honeybee dances as a “magic well”: The more he studied them, the more complex they turned out to be. Every species, Frisch argued, has its own magic well. Humans have verbal language. Whales have echolocation, which endows them with the ability to visualize their entire environment via sound. Honeybees have spatial, embodied language: We now recognize some of the subtle differences in their body movements and vibrations, which include waggling, knocking, stridulating, stroking, jerking, grasping, piping, trembling and antennation, to name just a few. 
    2. Human verbal language is largely based on the noises we make with our vocal cords and mouths, the expressions we make with our faces, and the way we hold and move our bodies. In contrast, bee language is mostly spatial and vibrational. Its syntax is based on something very different from human language: the type, frequency, angle and amplitude of vibrations made by the bees’ bodies, including their abdomens and wings, as they move through space. 
  4. Nov 2022
    1. locally-based staff and carries out its programs in conjunction with local partners. Teams of international instructors and volunteers support the programs through projects year-round.

      So many good features in your project!

      Employing local staff that know the setting and can be role models for the kids.

      Supporting mentoring by volunteers to scale.

      Working with bodies to get a visceral experience that change is possible.

      Mentoring in groups to build a community.

      Spotlighting diversity and building bridges beyond the local community.

      Some related resources: Ballet dancer from Kibera

      Fighting poverty and gang violence in Rio's favelas with ballet

    1. “pattern language” to describe the show’s plot formulas, which they and ultimately other users would then apply to a variety of programs.

      Tropes are shorthand storytelling methods that rely on a common storytelling grammar or pattern language to quickly relay information to the viewer or listener.

    1. Robert Amsler is a retired computational lexicology, computational linguist, information scientist. His P.D. was from UT-Austin in 1980. His primary work was in the area of understanding how machine-readable dictionaries could be used to create a taxonomy of dictionary word senses (which served as the motivation for the creation of WordNet) and in understanding how lexicon can be extracted from text corpora. He also invented a new technique in citation analysis that bears his name. His work is mentioned in Wikipedia articles on Machine-Readable dictionary, Computational lexicology, Bibliographic coupling, and Text mining. He currently lives in Vienna, VA and reads email at robert.amsler at utexas. edu. He is currenly interested in chronological studies of vocabulary, esp. computer terms.

      https://www.researchgate.net/profile/Robert-Amsler

      Apparently follow my blog. :)

      Makes me wonder how we might better process and semantically parse peoples' personal notes, particularly when they're atomic and cross-linked?

    1. there is no single perfect universal programming language. Until I came to that point, I wasted a lot of time thinking that GW-BASIC QBASIC QB 4.5 VB4 Delphi Java C++ C# 1.0 was the only language I would ever need
    1. Guy de Maupassant, was no lawyer,but his advice can help guide lawyerswho seek precision in their writing.“Whatever you want to say,” he assert-ed, “there is only one word to expressit, only one verb to give it movement,only one adjective to qualify it. Youmust search for that word, that verb,that adjective, and never be contentwith an approximation, never resortto tricks, even clever ones, and neverhave recourse to verbal sleight-of-hand to avoid a difficulty.”11

      11 Guy de Maupassant, Selected Short Sto- ries 10-11 (Roger Colet ed., 1971) (Maupassant quoting French writer Gustave Flaubert).

  5. Oct 2022
    1. a little flaw (Google translation can not find the translation of the word "瑕疵", so can only use the word "flaw" instead)

      annotation meta: may need new tag: no exact translation in other language

    1. https://www.explainpaper.com/

      Another in a growing line of research tools for processing and making sense of research literature including Research Rabbit, Connected Papers, Semantic Scholar, etc.

      Functionality includes the ability to highlight sections of research papers with natural language processing to explain what those sections mean. There's also a "chat" that allows you to ask questions about the paper which will attempt to return reasonable answers, which is an artificial intelligence sort of means of having an artificial "conversation with the text".

      cc: @dwhly @remikalir @jeremydean

    1. grammar Parser { rule TOP { I <love> <lang> } token love { '♥' | love } token lang { < Raku Perl Rust Go Python Ruby > } } say Parser.parse: 'I ♥ Raku'; # OUTPUT: 「I ♥ Raku」 love => 「♥」 lang => 「Raku」 say Parser.parse: 'I love Perl'; # OUTPUT: 「I love Perl」 love => 「love」 lang => 「Perl」
    2. Definable grammars for pattern matching and generalized string processing

      annotation meta: may need new tag: "definable __"?

    1. @1:10:20

      With HTML you have, broadly speaking, an experience and you have content and CSS and a browser and a server and it all comes together at a particular moment in time, and the end user sitting at a desktop or holding their phone they get to see something. That includes dynamic content, or an ad was served, or whatever it is—it's an experience. PDF on the otherhand is a record. It persists, and I can share it with you. I can deliver it to you [...]

      NB: I agree with the distinction being made here, but I disagree that the former description is inherent to HTML. It's not inherent to anything, really, so much as it is emergent—the result of people acting as if they're dealing in live systems when they shouldn't.

    1. Machines understand languages, that are formal and rigid, with unique and unambiguous instructions that are interpreted in precisely one way. Those formal, abstract languages, and programming languages in general, are hard to understand for humans like ourselves. Primarily, they are tailored towards the requirements of the machine. The user is therefore forced to adapt to the complexity of the formal language.

      .

    2. Instead of forcing humans to understand the complex inner workings of machines, we should construct machines in a way, so they better understand us humans!

      .

    1. https://ymlaenwelsh.com/2018/11/11/on-word-field-farming/

      Creating world fields (groups of words related to a particular area or field of knowledge) can be helpful for acquiring vocabulary in a new language. There's no research here to back up the claim, but it's an interesting word game and method for familiarize oneself with a small area and acquire new words related to an area or various related stem words.

    1. Method and theory are like thelanguage of the country you live in: it is nothing to bragabout that you can speak it, but it is a disgrace, as well asan inconvenience, if you cannot.
  6. Sep 2022
    1. we never 00:32:28 we never say all that we mean and we never mean all that we say you wouldn't be speaking human language if you did so when politicians tell you I say what I mean and I mean what they're lying already because language doesn't work 00:32:41 that way we we leave a lot of things unspecified and we let the cultural context fill it in so if I say she sat down Who am I talking about you only know who she is if you saw her 00:32:53 come in or I've referred to her earlier and you can follow it so actually why do divorces happen in modern societies because language doesn't really work 00:33:06 well I mean that's a you can tell that language of all because it doesn't work very well there just doesn't communicate as well as we would like it my wife and I have conversations sometimes and we 00:33:19 realized that we didn't know what the other one was talking about we clearly didn't know what the other was talking about we have different cultural assumptions and although if I do figure out what she's talking about then I pretended that's what I was talking 00:33:31 about

      !- for : incompleteness of language - much is not said outside of what is said - context is required to fill it in

    2. ow many sounds do you need to have a language well 00:30:33 think about a computer what can you say on a computer anything right I mean you can type anything that's why people get addicted to Facebook and everything but how many letters does a computer have it 00:30:44 has two zero and one you have a binary digit language and those I would like to call the sounds of the computer zero and one that's how it interprets everything or that's how it presents information 00:30:58 that is interpreted by the program that was created by a person with language you don't really need more than two sounds

      !- for : language evolution - how many symbols do you need for a language? - no more than 2, like a computer with "0" and "1"

    3. let's just 00:29:52 talk a little bit about their vocal apparatus what kinds of sounds could they have made very often when linguists are talking about the evolution of speech they talk about sounds were they capable of making sounds Homo erectus 00:30:05 would have been roughly a talking gorilla they had the vocal apparatus that is much more similar to a gorilla they couldn't have made all the sounds we made the sounds they made would have sound more muffled does that mean they 00:30:19 couldn't have language no it doesn't mean that at all there are a lot of people today that have speech impediments that can't make the same range of sounds we make but they certainly have language

      !- for : language evolution homo erectus vocal cords

    4. language is much more 00:12:18 complicated than is often presented it's to me the primary unit of language is not the sentence or the word it's the conversation I think that we didn't have language in the archaeological record 00:12:31 until we had a conversation symbols were the beginning but symbols would never have arisen outside of conversations they could have only come about in conversations so we have meaning and 00:12:44 lexical meaning phonetics the history of languages grammar psychology culture on top of all of that so language is quite a complicated thing which is one reason I'm not worried about robots learning 00:12:57 language anytime soon they would have to at least be able to learn culture at the same time

      !- key insight : the primary unit of language is conversation - conversation is what gives rise to symbols, words, sentences

    5. the greatest technology ever invented was language

      Language is the greatest technology ever invented

    1. note that this will show the historical content within the current template – so not necessarily exactly the same as the original page
    1. The LISP part, though, is not going well. Porting clever 1970s Stanford AI Lab macros written on the original SAIL machine to modern Common LISP is hard. Anybody with a knowledge of MACLISP want to help?
    1. not designed to be secure

      "secure" is used in this context to mean, roughly, "work with adversarial input"

    1. XBL is a proprietary technology developed by Mozilla

      Example of when "proprietary" is not an antonym of "open source".

  7. Aug 2022
    1. Obnoxious.

      As someone recently pointed out on HN, it's very common nowadays to encounter the no-one-knows-else-what-they're-doing-here refrain as cover—I don't have to feel insecure about not understanding this because not only am I not alone, nobody else understands it either.

      Secondly, if your code is hard to understand regarding its use of this, then your code his hard to understand. this isn't super easy, but it's also not hard. Your code (or the code you're being made to wade into) probably just sucks. The this confusion is making you confront it, though, instead of letting it otherwise fly under the radar.* So fix it and stop going in for the low-effort, this-centric clapter.

      * Not claiming here that this is unique; there are allowed to be other things that work as the same sort of indicator.

    1. The use ofhyphens in compound words is becoming less frequent exceptwhen essential for clarity of meaning. The customary prac-tice is to write such words as coordinate with the dieresisrather than the hyphen.
    1. and free of globals

      Ah! This remark highlights a fundamental difference in understanding between two camps, which I have been (painfully) aware of, but the source of this confusion has eluded me until only just right now. (Really, this is a source of frustration going back years.)

      In one camp, the advice "don't use global variables" is a way of attacking a bunch of things endemic to their use, most notably unnecessary coupling to spooky state. In another camp "no global variables" is understood to mean literally that and taken no further—so you can have as much spookiness as you like, and so long as the value is not directly accessible (visible) from, say, another given piece of code appearing at the top-level ("global") context, as with the way i is bound to the activation record in this example but is not accessible outside the scope of getGetNext, then you're good.

      That is, there are two aspects to variables: visibility and extent, and the first interpretation seeks to avoid the negative effects on both dimensions, while the second is satisfied by narrowly prohibiting direct visibility across boundaries.

      I find the latter interpretation bizarre and completely at odds with the spirit of the exhortation for avoiding globals in the first place.

      (What's worse is the the second interpretation usually goes hand in hand with the practice of making extensive use of closures, which because they are propped up as being closely associated with functions, then leads people to regretfully refer to this style as functional programming. This is a grave error—and, to repeat, totally at odds with the spirit of the thing.)

    1. "Why have a locked wiki when you can instead just post static Web pages?"

      What even is a locked wiki insofar as the ways it differs from traditional (pre-wiki) content publishing pipelines? Where's the wiki part of it?

  8. Jul 2022
    1. Later in life and irrespective to the character of the relationship held, the good enough approachinforms how communication between people can be practiced. One of the widest known formulasfor that is called Nonviolent Communication, subtitled as the ‘language of life’ [ 39]. The subtitle seemsparticularly appropriate to our case, as it describes a method of communication that does not servesocial programming and allow humans to author and own their speech. A nonviolent communicatordoes not reinforce the boundary cuts and refrains from installing the personware-shaping doublebinds.

      !- definition : nonviolent communication, language of life * a method of communication that does not prioritize social programming over an individual's right to articulate and own their own speech.

    1. The trouble with redefining "REST" to mean "not REST" is that the first step in learning known techniques to solve a problem is learning the terminology that people use to explain the techniques. If you think you know the terminology, but you have the wrong definition in your mind, you will not be able to understand the explanations, and you will not be able to figure out why you can't understand them, until you finally figure out that the definition you learned was wrong.
    1. 1. Focus on items that occur with high frequency in the language as awhole (see Table 3.1 for examples). Such items will occur often inmany different texts.2. Focus on strategies that can be used with most texts (see Table 3.1for examples).

      .c1

    1. Let's call this style of API pseduoREST or JSON-RPC.

      What the re-education around REST needs is a catchy label for what people call REST that works well as a light pejorative. Two-Bit History gave it a shot, coining the ad hoc acronym "FIOH", but it doesn't have the desired properties.

    1. what happens um when we're thinking about our inner states one of the things that we need to recognize is that our introspection when 00:22:54 we we become aware of our beliefs our desires and our hopes and our fears and so forth is all done through language and on the model of language when i decide that i believe that john dunn 00:23:07 gave a great talk this morning when i believe that hal roth is a great scholar of zen and when i believe that alan wallace gave us a beautifully inspirational talk about the role of practice and contemplation in the 00:23:19 understanding of the self and i introspect that way i'm using those sentences alan gave that great talk john gave us a great talk about pramana and so forth as models for my inner states and i'm not 00:23:32 doing that because i looked inside and saw little english sentences in my brain i'm looking i'm doing that by using language as a kind of introspective model that's a matter of self-interpretation 00:23:44 it's easy to forget that because it feels so immediate so language gives us the concepts that we use to think about the world but it is also the model for the concepts of our propositional attitudes like belief 00:23:58 desire knowledge and so forth and as a model we have to recognize that the model the map isn't the reality to go back to what john uh reminded us of he reminded us of earlier introspection in 00:24:11 terms of language gives us an interpretation it doesn't give us an independent reality that is being interpreted and when we think about the madhyamaka 00:24:23 of nagarjuna and chandrakiri we remember that to be empty is to be empty of any intrinsic nature and if we follow chandra charity as i suggested earlier that means that it is to exist only 00:24:37 dependent on conceptual imputation and what i am suggesting now is that all of our inner cognitive states that we introspect we encounter only through a conceptual imputation only through 00:24:50 interpretation only through language and that is they exist conventionally not intrinsically even though they might appear to us to exist just as we see 00:25:02 them and to do so intrinsically

      Another key point:

      Language is the tool we use for introspection and as Nagarjuna and Chandrakirti hold, are empty of intrinsic nature. All inner cognitive states that we introspect are attained only through linguistic conceptual imputation so can only exist conventionally and not intrinsically.

      This underscores the importance of the symbolosphere, of symbols and language.

  9. Jun 2022
    1. Interleaving is a learning technique that involves mixing together different topics or forms of practice, in order to facilitate learning. For example, if a student uses interleaving while preparing for an exam, they can mix up different types of questions, rather than study only one type of question at a time.Interleaving, which is sometimes referred to as mixed practice or varied practice, is contrasted with blocked practice (sometimes referred to as specific practice), which involves focusing on only a single topic or form of practice at a time.

      Interleaving (aka mixed practice or varied practice) is a learning strategy that involves mixing different topics, ideas, or forms of practice to improve outcomes as well as overall productivity. Its opposite and less effective strategy is blocking (or block study or specific practice) which focuses instead on working on limited topics or single forms of practice at the same time.


      This may be one of the values of of the Say Something In Welsh method which interleaves various new nouns and verbs as well as verb tenses in focused practice.

      Compare this with the block form which would instead focus on lists of nouns in a single session and then at a later time lists of verbs in a more rote fashion. Integrating things together in a broader variety requires more work, but is also much more productive in the long run.

    1. the more effort they had to put into the study strategy, the less they felt they were learning.

      misinterpreted-effort hypothesis: the amount of effort one puts into studying is inversely proportional to how much one feels they learn.


      Is this why the Says Something In Welsh system works so well? Because it requires so much mental work and effort in short spans of time? Particularly in relation to Duolingo which seems easier?

    1. As my colleague Robin Paige likes to say, we are also social beings in a social world. So if we shift things just a bit to think instead about the environments we design and cultivate to help maximize learning, then psychology and sociology are vital for understanding these elements as well.

      Because we're "social beings in a social world", we need to think about the psychology and sociology of the environments we design to help improve learning.

      Link this to: - Design of spaces like Stonehenge for learning in Indigenous cultures, particularly the "stage", acoustics (recall the ditch), and intimacy of the presentation. - research that children need face-to-face interactions for language acquisition

    1. vulnerable

      So we are currently testing the hypothesis tool on our own website, and the first annotation goes to the problematic terminology such as "vulnerable"

    1. New insights on infant word learning

      infant word learning

    2. "The idea is that over long periods of time, traces of memory for visual objects are being built up slowly in the neocortex," Clerkin said. "When a word is spoken at a specific moment and the memory trace is also reactivated close in time to the name, this mechanism allows the infants to make a connection rapidly." The researchers said their work also has significant implications for machine learning researchers who are designing and building artificial intelligence to recognize object categories. That work, which focuses on how names teach categories, requires massive amounts of training for machine learning systems to even approach human object recognition. The implication of the infant pathway in this study suggests a new approach to machine learning, in which training is structured more like the natural environment, and object categories are learned first without labels, after which they are linked to labels.

      visual objects are encoded into memory over a long period of time until it becomes familiar. When a word is spoken when the memory trace associated with the visual object is reactivated, connection between word and visual object is made rapidly.

    1. The dominant idea is one of attention, by which a representation at a position is computed as a weighted combination of representations from other positions. A common self-supervision objective in a transformer model is to mask out occasional words in a text. The model works out what word used to be there. It does this by calculating from each word position (including mask positions) vectors that represent a query, key, and value at that position. The query at a position is compared with the value at every position to calculate how much attention to pay to each position; based on this, a weighted average of the values at all positions is calculated. This operation is repeated many times at each level of the transformer neural net, and the resulting value is further manipulated through a fully connected neural net layer and through use of normalization layers and residual connections to produce a new vector for each word. This whole process is repeated many times, giving extra layers of depth to the transformer neural net. At the end, the representation above a mask position should capture the word that was there in the original text: for instance, committee as illustrated in Figure 1.
    1. we see by learning to see. The brain evolved the mechanisms for finding patterns, finding relationships in information, 00:04:38 and associating those relationships with a behavioral meaning, a significance, by interacting with the world. We're very aware of this in the form of more cognitive attributes, like language. I'm going to give you some letter strings, and I want you to read them out for me, if you can. Audience: "Can you read this?" "You are not reading this." "What are you reading?" Beau Lotto: "What are you reading?" Half the letters are missing, right? 00:05:04 There's no a priori reason why an "H" has to go between that "W" and "A." But you put one there. Why? Because in the statistics of your past experience, it would have been useful to do so. So you do so again. And yet you don't put a letter after that first "T." Why? Because it wouldn't have been useful in the past. So you don't do it again.

      Being journey 3 Linguistic BEing journey - filling in missing letters in incomplete sentence is based on our past experience with specific sentences that have those letters. This becomes compelling when we can demonstrate with multiple languages, including ones we are not familiar with. Those people in the other cultures will fill in missing letters in their words in their language that we would be completely clueless about.

    1. This trick of using a one-hot vector to pull out a particular row of a matrix is at the core of how transformers work.

      Matrix multiplication as table lookup

    1. “So I’m supposed to ask the Lakota Language Consortium if I can use my own Lakota language,” Taken Alive asked in one of many TikTok posts that would come to define his social media presence. 

      Based on some beyond the average knowledge of Indigenous cultures, I'm reading some additional context into this statement that is unlikely to be seen or understood by those with Western-only cultural backgrounds who view things from an ownership and capitalistic perspective.

      There's a stronger sense of identity and ownership of language and knowledge within oral traditions than can be understood by Westerners who didn't grow up with it.

      He obviously feels like we're stealing from him all over again. We need better rules and shared definitions between Indigenous peoples and non before embarking on these sorts of projects.

    2. “No matter how it was collected, where it was collected, when it was collected, our language belongs to us. Our stories belong to us. Our songs belong to us,” Taken Alive, who teaches Lakota to elementary school students, told the tribal council in April. 
    1. A recent book that advocates for this idea is Range: Why Generalists Triumph in a Specialized world by David Epstein. Consider reading Cal Newport’s So Good They Can’t Ignore You along side it: So Good They Can’t Ignore You focuses on building up “career capital,” which is important for everyone but especially people with a lot of different interests.1 People interested in interdisciplinary work (including students graduating from liberal arts or other general programs) might seem “behind” at first, but with time to develop career capital these graduates can outpace their more specialist peers.

      Similar to the way that bi-lingual/dual immersion language students may temporarily fall behind their peers in 3rd and 4th grade, but rocket ahead later in high school, those interested in interdisciplinary work may seem to lag, but later outpace their lesser specializing peers.

      What is the underlying mechanism for providing the acceleration boosts in these models? Are they really the same or is this effect just a coincidence?

      Is there something about the dual stock and double experience or even diversity of thought that provides the acceleration? Is there anything in the pedagogy or productivity research space to explain it?

  10. May 2022
    1. ART. 2. - La Société n'admet aucune communication concernant, soit l'origine du langage~ soit la création d'une langue universelle.
    1. an acknowledgement of network effects: LP is unlikely to ever catch on enough to be the majority, so there needs to be a way for a random programmer using their preferred IDE/editor to edit a "literate" program

      This is part of the reason why I advocate for language skins for comparatively esoteric languages like Ada.

    1. memory usage and (lack of) parallelism are concerns

      Memory usage is a concern? wat

      It's a problem, sure, if you're programming the way NPMers do. So don't do that.

      This is a huge problem I've noticed when it comes to people programming in JS—even, bizarrely, people coming from other languages like Java or C# and where you'd expect them to at least try to continue to do things in JS just like they're comfortable doing in their own language. Just because it's there (i.e. possible in the language, e.g. dynamic language features) doesn't mean you have to use it...

      (Relevant: How (and why) developers use the dynamic features of programming languages https://users.dcc.uchile.cl/~rrobbes/p/EMSE-features.pdf)

      The really annoying thing is that the NPM style isn't even idiomatic for the language! So much of what the NodeJS camp does is so clearly done in frustration and the byproduct of a desire to work against the language. Case in point: the absolutely nonsensical attitude about always using triple equals (as if to ward off some evil spirits) and the undeniable contempt that so many have for this.

  11. www.mindprod.com www.mindprod.com
    1. local a (e.g. aPoint) param p (e.g. pPoint) member instance m (e.g. mPoint) static s (e.g. sPoint)

      This is really only a problem in languages that make the unfortunate mistake of allowing references to unqualified names that get fixed up as if the programmer had written this.mPoint or Foo.point. Even if you're writing in a language where that's possible, just don't write code like that! Just because you can doesn't mean you have to.

      The only real exception is distinguishing locals from parameters. Keep your procedures short and it's less of a problem.

    1. This can get much worse than the above example; the number of \’s required is exponential in the nesting depth. Rc fixes this by making the backquote a unary operator whose argument is a command, like this: size=‘{wc -l ‘{ls -t|sed 1q}}
    1. Giants that prefer the hyphenated spelling—Merriam-Webster, The Chicago Manual of Style, and The New Yorker, have a good reason for doing so. E-mail is a compound noun, made out of two words—“electronic” and “mail.” The e in e-mail is an abbreviation for “electronic,” and it’s used in a lot of other words as well—e-commerce, e-learning, and e-business, for example. There are also other compound nouns formed from an abbreviation and a noun, like the H-bomb, which is short for hydrogen bomb. The general rule of hyphenation in compound words that combine a single letter (or a number) and a word is to hyphenate them. So, based on tradition, e-mail is the correct way to do it.
    1. List of Italian language learning resources from language learning forum.