LLMs are generative math-ematical models of the statistical distributionof tokens in the vast public corpus of human-generated text, where the tokens in question in-clude words, parts of words, or individual char-acters including punctuation marks. They aregenerative because we can sample from them,which means we can ask them questions. Butthe questions are of the following very specifickind. “Here’s a fragment of text. Tell me howthis fragment might go on. According to yourmodel of the statistics of human language, whatwords are likely to come next?”

The breakthroughs are all underpinned by a new class of AI models that are more flexible and powerful than anything that has come before. Because they were first used for language tasks like answering questions and writing essays, they’re often known as large language models (LLMs). OpenAI’s GPT3, Google’s BERT, and so on are all LLMs. But these models are extremely flexible and adaptable. The same mathematical structures have been so useful in computer vision, biology, and more that some researchers have taken to calling them "foundation models" to better articulate their role in modern AI.

An AI model that can learn and work with this kind of problem needs to handle order in a very flexible way. The old models—LSTMs and RNNs—had word order implicitly built into the models. Processing an input sequence of words meant feeding them into the model in order. A model knew what word went first because that’s the word it saw first. Transformers instead handled sequence order numerically, with every word assigned a number. This is called "positional encoding." So to the model, the sentence “I love AI; I wish AI loved me” looks something like (I 1) (love 2) (AI 3) (; 4) (I 5) (wish 6) (AI 7) (loved 8) (me 9).

taken her last breath

gave a star

His eyes fell on his coat

Her dead hands undidher salwar and lowered it

Suddenly, the roomlit up.

There was no one in the dark room, just a stretcher witha body on it.

young volunteers

Sirajuddin wanted to cry, but his eyes would not cooperate. Who knewwhere all the tears had gone?

But as he racked his brains, his mind fixed onSakina’s mother’s body, her intestines spilled out, then he could think nofurther

Loot. Fire. Stampede. Station.Bullets. Night. And Sakina.

he felt as if he were floating in a void

There was chaos all roundhim, but he heard nothing, as if his ears were blocked

swelling sea of men, women and children

Having thiscapacity,however,doesnotalwaysguaranteeitsuse (Sefman, Beardslee, Schultz, Krupa,&Podorefsky1986). For example,although mature individualsmayknowthat itisrudetointerruptanother speaker whoisstrugglingtoexpress anidea,they maynonethelessdosoduringapassionate debate or discussion.

The hypothesis of linguistic relativity, also known as the Sapir–Whorf hypothesis /səˌpɪər ˈwɔːrf/, the Whorf hypothesis, or Whorfianism, is a principle suggesting that the structure of a language influences its speakers' worldview or cognition, and thus people's perceptions are relative to their spoken language.

a common technique in natural language processing is to operationalize certain semantic concepts (e.g., "synonym") in terms of syntactic structure (two words that tend to occur nearby in a sentence are more likely to be synonyms, etc). This is what word2vec does.

Siwan Clark, a Welsh speaker and MSc candidate in Social Research Methods at University College London, said Wales is environmentally suffering because of centuries of British colonialism, particularly during the Industrial Revolution, which had a devastating global environmental impact.Clark said her mother, who grew up on a farm in North Wales, understands critical Welsh words—a language Clark said is deeply agricultural—that “have no context or meaning” for her.“Industrialized farming is inextricable from empire,” Clark said. “If those small farms fail, then the language won’t truly survive.” 

“a huge amount of scientific information in Irish was passed down orally, but was lost to the trauma of the famine and immigration.” Today, only 1.7 percent of the Irish population is fluent. Ó Séaghdha, who said many words in Irish are like “one-word poems,” finds himself fascinated by this bio-cultural knowledge. “The word for swallow is fáinleog,” he said. “It means little wanderer, because the bird migrates during winter. In our languages, we understood bird migration, thousands and thousands of years ago. We had words for phosphorus magnetism before they had scientific books.”

Learning a New Language Can Help Us Escape Climate Catastrophe

wer lesen kann, ist klar im Vorteil

Fried-berg Judeo-Arabic Project, accessible at http://fjms.genizah.org. This projectmaintains a digital corpus of Judeo-Arabic texts that can be searched and an-alyzed.

When such consumers therefore mistake the meaning attributed tothe MT output as the actual communicative intent of the originaltext’s author, real-world harm can ensue.

They are foldable in IDEs

Considerations

In Lua you can write raw, multiline strings with [[]]: [[ Alice said "Bob said 'hi'". ]]

how important is the concrete syntax of their language in contrast to

When I started working on the history of linguistics — which had been totally forgotten; nobody knew about it — I discovered all sorts of things. One of the things I came across was Wilhelm von Humboldt’s very interesting work. One part of it that has since become famous is his statement that language “makes infinite use of finite means.” It’s often thought that we have answered that question with Turing computability and generative grammar, but we haven’t. He was talking about infinite use, not the generative capacity. Yes, we can understand the generation of the expressions that we use, but we don’t understand how we use them. Why do we decide to say this and not something else? In our normal interactions, why do we convey the inner workings of our minds to others in a particular way? Nobody understands that. So, the infinite use of language remains a mystery, as it always has. Humboldt’s aphorism is constantly quoted, but the depth of the problem it formulates is not always recognized.

The miracle that so amazed Galileo and Arnauld — and still amazes me, I can’t understand it — is how can we, with a few symbols, convey to others the inner workings of our mind? That’s something to really be surprised about, and puzzled by. And we have some grasp of it, but not a lot.

There are different kinds of information, some of which don't make sense being recorded at all. I was struggling with what to record and what not to record for a long time. For example, I took notes on programming syntax that are just useless (most of these things can be googled in seconds and they are usually decently documented already).

natural-language processing is going to force engineers and humanists together. They are going to need each other despite everything. Computer scientists will require basic, systematic education in general humanism: The philosophy of language, sociology, history, and ethics are not amusing questions of theoretical speculation anymore. They will be essential in determining the ethical and creative use of chatbots, to take only an obvious example.

Houston, we have a Capability Overhang problem: Because language models have a large capability surface, these cases of emergent capabilities are an indicator that we have a ‘capabilities overhang’ – today’s models are far more capable than we think, and our techniques available for exploring the models are very juvenile. We only know about these cases of emergence because people built benchmark datasets and tested models on them. What about all the capabilities we don’t know about because we haven’t thought to test for them? There are rich questions here about the science of evaluating the capabilities (and safety issues) of contemporary models. 

While we are still a while away from a Google Translate equivalent for animal languages that can decode the nuances of intra-species communication, technology, especially machine learning, is keeping this hope alive. The ability to understand animal languages could open up a realm of possibilities, potentially shaping conservation efforts, determining our future relationship with other species, and even offering insights into the evolution of human language itself. 

The scientific community is, thus, increasingly using technological tools including drones, recorders, robots and AI to study the calls of a range of species, from chickens and rodents, to cats and lemurs.

“Digital technologies, so often associated with our alienation from nature, are offering us an opportunity to listen to nonhumans in powerful ways, reviving our connection to the natural world,”

Researchers are using drones, AI, and digital recorders to create a “zoological version of Google Translate.”

Frisch referred to honeybee dances as a “magic well”: The more he studied them, the more complex they turned out to be. Every species, Frisch argued, has its own magic well. Humans have verbal language. Whales have echolocation, which endows them with the ability to visualize their entire environment via sound. Honeybees have spatial, embodied language: We now recognize some of the subtle differences in their body movements and vibrations, which include waggling, knocking, stridulating, stroking, jerking, grasping, piping, trembling and antennation, to name just a few. 

Human verbal language is largely based on the noises we make with our vocal cords and mouths, the expressions we make with our faces, and the way we hold and move our bodies. In contrast, bee language is mostly spatial and vibrational. Its syntax is based on something very different from human language: the type, frequency, angle and amplitude of vibrations made by the bees’ bodies, including their abdomens and wings, as they move through space. 

locally-based staff and carries out its programs in conjunction with local partners. Teams of international instructors and volunteers support the programs through projects year-round.

🌟 Highlight words as they are spoken (karaoke anybody?). 🌟 Navigate video by clicking on words. 🌟 Share snippets of text (with video attached!). 🌟 Repurpose by remixing using the text as a base and reference.

“pattern language” to describe the show’s plot formulas, which they and ultimately other users would then apply to a variety of programs.

Robert Amsler is a retired computational lexicology, computational linguist, information scientist. His P.D. was from UT-Austin in 1980. His primary work was in the area of understanding how machine-readable dictionaries could be used to create a taxonomy of dictionary word senses (which served as the motivation for the creation of WordNet) and in understanding how lexicon can be extracted from text corpora. He also invented a new technique in citation analysis that bears his name. His work is mentioned in Wikipedia articles on Machine-Readable dictionary, Computational lexicology, Bibliographic coupling, and Text mining. He currently lives in Vienna, VA and reads email at robert.amsler at utexas. edu. He is currenly interested in chronological studies of vocabulary, esp. computer terms.

there is no single perfect universal programming language. Until I came to that point, I wasted a lot of time thinking that GW-BASIC QBASIC QB 4.5 VB4 Delphi Java C++ C# 1.0 was the only language I would ever need

Guy de Maupassant, was no lawyer,but his advice can help guide lawyerswho seek precision in their writing.“Whatever you want to say,” he assert-ed, “there is only one word to expressit, only one verb to give it movement,only one adjective to qualify it. Youmust search for that word, that verb,that adjective, and never be contentwith an approximation, never resortto tricks, even clever ones, and neverhave recourse to verbal sleight-of-hand to avoid a difficulty.”11

a little flaw (Google translation can not find the translation of the word "瑕疵", so can only use the word "flaw" instead)

grammar Parser { rule TOP { I <love> <lang> } token love { '♥' | love } token lang { < Raku Perl Rust Go Python Ruby > } } say Parser.parse: 'I ♥ Raku'; # OUTPUT: ｢I ♥ Raku｣ love => ｢♥｣ lang => ｢Raku｣ say Parser.parse: 'I love Perl'; # OUTPUT: ｢I love Perl｣ love => ｢love｣ lang => ｢Perl｣

Definable grammars for pattern matching and generalized string processing

Note that one extra type that is accepted by convention is the Boolean type, which represents both the TrueClass and FalseClass types.

the program idea manifests commonly as an OS process

Machines understand languages, that are formal and rigid, with unique and unambiguous instructions that are interpreted in precisely one way. Those formal, abstract languages, and programming languages in general, are hard to understand for humans like ourselves. Primarily, they are tailored towards the requirements of the machine. The user is therefore forced to adapt to the complexity of the formal language.

Instead of forcing humans to understand the complex inner workings of machines, we should construct machines in a way, so they better understand us humans!

Method and theory are like thelanguage of the country you live in: it is nothing to bragabout that you can speak it, but it is a disgrace, as well asan inconvenience, if you cannot.

we never 00:32:28 we never say all that we mean and we never mean all that we say you wouldn't be speaking human language if you did so when politicians tell you I say what I mean and I mean what they're lying already because language doesn't work 00:32:41 that way we we leave a lot of things unspecified and we let the cultural context fill it in so if I say she sat down Who am I talking about you only know who she is if you saw her 00:32:53 come in or I've referred to her earlier and you can follow it so actually why do divorces happen in modern societies because language doesn't really work 00:33:06 well I mean that's a you can tell that language of all because it doesn't work very well there just doesn't communicate as well as we would like it my wife and I have conversations sometimes and we 00:33:19 realized that we didn't know what the other one was talking about we clearly didn't know what the other was talking about we have different cultural assumptions and although if I do figure out what she's talking about then I pretended that's what I was talking 00:33:31 about

ow many sounds do you need to have a language well 00:30:33 think about a computer what can you say on a computer anything right I mean you can type anything that's why people get addicted to Facebook and everything but how many letters does a computer have it 00:30:44 has two zero and one you have a binary digit language and those I would like to call the sounds of the computer zero and one that's how it interprets everything or that's how it presents information 00:30:58 that is interpreted by the program that was created by a person with language you don't really need more than two sounds

let's just 00:29:52 talk a little bit about their vocal apparatus what kinds of sounds could they have made very often when linguists are talking about the evolution of speech they talk about sounds were they capable of making sounds Homo erectus 00:30:05 would have been roughly a talking gorilla they had the vocal apparatus that is much more similar to a gorilla they couldn't have made all the sounds we made the sounds they made would have sound more muffled does that mean they 00:30:19 couldn't have language no it doesn't mean that at all there are a lot of people today that have speech impediments that can't make the same range of sounds we make but they certainly have language

language is much more 00:12:18 complicated than is often presented it's to me the primary unit of language is not the sentence or the word it's the conversation I think that we didn't have language in the archaeological record 00:12:31 until we had a conversation symbols were the beginning but symbols would never have arisen outside of conversations they could have only come about in conversations so we have meaning and 00:12:44 lexical meaning phonetics the history of languages grammar psychology culture on top of all of that so language is quite a complicated thing which is one reason I'm not worried about robots learning 00:12:57 language anytime soon they would have to at least be able to learn culture at the same time

the greatest technology ever invented was language

note that this will show the historical content within the current template – so not necessarily exactly the same as the original page

Rust is integrated with Firefox. This integration is cleverly dubbed ‘oxidation’ and has a whole wiki page dedicated to the project. 

The LISP part, though, is not going well. Porting clever 1970s Stanford AI Lab macros written on the original SAIL machine to modern Common LISP is hard. Anybody with a knowledge of MACLISP want to help?

not designed to be secure

I find C++ quite ugly

XBL is a proprietary technology developed by Mozilla

The person who has acquired knowledge of a language has internalized asystem of rules that relate sound and meaning in a particular way.

languageis a “habit structure” or a network of associative connections,

This is not an attempt to arrest the evolution of the language, but to correct mistakes.

The use ofhyphens in compound words is becoming less frequent exceptwhen essential for clarity of meaning. The customary prac-tice is to write such words as coordinate with the dieresisrather than the hyphen.

and free of globals

There are no static variables and no initialization

"Why have a locked wiki when you can instead just post static Web pages?"

Later in life and irrespective to the character of the relationship held, the good enough approachinforms how communication between people can be practiced. One of the widest known formulasfor that is called Nonviolent Communication, subtitled as the ‘language of life’ [ 39]. The subtitle seemsparticularly appropriate to our case, as it describes a method of communication that does not servesocial programming and allow humans to author and own their speech. A nonviolent communicatordoes not reinforce the boundary cuts and refrains from installing the personware-shaping doublebinds.

actual programming language

I'm approaching the open source portion of this project strictly as "open source" not "open contribution" similar to the SQLite approach

The trouble with redefining "REST" to mean "not REST" is that the first step in learning known techniques to solve a problem is learning the terminology that people use to explain the techniques. If you think you know the terminology, but you have the wrong definition in your mind, you will not be able to understand the explanations, and you will not be able to figure out why you can't understand them, until you finally figure out that the definition you learned was wrong.

1. Focus on items that occur with high frequency in the language as awhole (see Table 3.1 for examples). Such items will occur often inmany different texts.2. Focus on strategies that can be used with most texts (see Table 3.1for examples).

Let's call this style of API pseduoREST or JSON-RPC.

what happens um when we're thinking about our inner states one of the things that we need to recognize is that our introspection when 00:22:54 we we become aware of our beliefs our desires and our hopes and our fears and so forth is all done through language and on the model of language when i decide that i believe that john dunn 00:23:07 gave a great talk this morning when i believe that hal roth is a great scholar of zen and when i believe that alan wallace gave us a beautifully inspirational talk about the role of practice and contemplation in the 00:23:19 understanding of the self and i introspect that way i'm using those sentences alan gave that great talk john gave us a great talk about pramana and so forth as models for my inner states and i'm not 00:23:32 doing that because i looked inside and saw little english sentences in my brain i'm looking i'm doing that by using language as a kind of introspective model that's a matter of self-interpretation 00:23:44 it's easy to forget that because it feels so immediate so language gives us the concepts that we use to think about the world but it is also the model for the concepts of our propositional attitudes like belief 00:23:58 desire knowledge and so forth and as a model we have to recognize that the model the map isn't the reality to go back to what john uh reminded us of he reminded us of earlier introspection in 00:24:11 terms of language gives us an interpretation it doesn't give us an independent reality that is being interpreted and when we think about the madhyamaka 00:24:23 of nagarjuna and chandrakiri we remember that to be empty is to be empty of any intrinsic nature and if we follow chandra charity as i suggested earlier that means that it is to exist only 00:24:37 dependent on conceptual imputation and what i am suggesting now is that all of our inner cognitive states that we introspect we encounter only through a conceptual imputation only through 00:24:50 interpretation only through language and that is they exist conventionally not intrinsically even though they might appear to us to exist just as we see 00:25:02 them and to do so intrinsically

Interleaving is a learning technique that involves mixing together different topics or forms of practice, in order to facilitate learning. For example, if a student uses interleaving while preparing for an exam, they can mix up different types of questions, rather than study only one type of question at a time.Interleaving, which is sometimes referred to as mixed practice or varied practice, is contrasted with blocked practice (sometimes referred to as specific practice), which involves focusing on only a single topic or form of practice at a time.

the more effort they had to put into the study strategy, the less they felt they were learning.

As my colleague Robin Paige likes to say, we are also social beings in a social world. So if we shift things just a bit to think instead about the environments we design and cultivate to help maximize learning, then psychology and sociology are vital for understanding these elements as well.

vulnerable

New insights on infant word learning

"The idea is that over long periods of time, traces of memory for visual objects are being built up slowly in the neocortex," Clerkin said. "When a word is spoken at a specific moment and the memory trace is also reactivated close in time to the name, this mechanism allows the infants to make a connection rapidly." The researchers said their work also has significant implications for machine learning researchers who are designing and building artificial intelligence to recognize object categories. That work, which focuses on how names teach categories, requires massive amounts of training for machine learning systems to even approach human object recognition. The implication of the infant pathway in this study suggests a new approach to machine learning, in which training is structured more like the natural environment, and object categories are learned first without labels, after which they are linked to labels.

The dominant idea is one of attention, by which a representation at a position is computed as a weighted combination of representations from other positions. A common self-supervision objective in a transformer model is to mask out occasional words in a text. The model works out what word used to be there. It does this by calculating from each word position (including mask positions) vectors that represent a query, key, and value at that position. The query at a position is compared with the value at every position to calculate how much attention to pay to each position; based on this, a weighted average of the values at all positions is calculated. This operation is repeated many times at each level of the transformer neural net, and the resulting value is further manipulated through a fully connected neural net layer and through use of normalization layers and residual connections to produce a new vector for each word. This whole process is repeated many times, giving extra layers of depth to the transformer neural net. At the end, the representation above a mask position should capture the word that was there in the original text: for instance, committee as illustrated in Figure 1.

we see by learning to see. The brain evolved the mechanisms for finding patterns, finding relationships in information, 00:04:38 and associating those relationships with a behavioral meaning, a significance, by interacting with the world. We're very aware of this in the form of more cognitive attributes, like language. I'm going to give you some letter strings, and I want you to read them out for me, if you can. Audience: "Can you read this?" "You are not reading this." "What are you reading?" Beau Lotto: "What are you reading?" Half the letters are missing, right? 00:05:04 There's no a priori reason why an "H" has to go between that "W" and "A." But you put one there. Why? Because in the statistics of your past experience, it would have been useful to do so. So you do so again. And yet you don't put a letter after that first "T." Why? Because it wouldn't have been useful in the past. So you don't do it again.

Jean-Claude Gardin (3 April 1925 - 8 April 2013) was a French archaeologist who is recognized as being one of the founders of archaeological computing.

This trick of using a one-hot vector to pull out a particular row of a matrix is at the core of how transformers work.

“So I’m supposed to ask the Lakota Language Consortium if I can use my own Lakota language,” Taken Alive asked in one of many TikTok posts that would come to define his social media presence. 

“No matter how it was collected, where it was collected, when it was collected, our language belongs to us. Our stories belong to us. Our songs belong to us,” Taken Alive, who teaches Lakota to elementary school students, told the tribal council in April. 

The fundamental mistake of Node.js was diverging from the browser

A recent book that advocates for this idea is Range: Why Generalists Triumph in a Specialized world by David Epstein. Consider reading Cal Newport’s So Good They Can’t Ignore You along side it: So Good They Can’t Ignore You focuses on building up “career capital,” which is important for everyone but especially people with a lot of different interests.1 People interested in interdisciplinary work (including students graduating from liberal arts or other general programs) might seem “behind” at first, but with time to develop career capital these graduates can outpace their more specialist peers.

ART. 2. - La Société n'admet aucune communication concernant, soit l'origine du langage~ soit la création d'une langue universelle.

an acknowledgement of network effects: LP is unlikely to ever catch on enough to be the majority, so there needs to be a way for a random programmer using their preferred IDE/editor to edit a "literate" program

memory usage and (lack of) parallelism are concerns

local a (e.g. aPoint) param p (e.g. pPoint) member instance m (e.g. mPoint) static s (e.g. sPoint)

This can get much worse than the above example; the number of \’s required is exponential in the nesting depth. Rc fixes this by making the backquote a unary operator whose argument is a command, like this: size=‘{wc -l ‘{ls -t|sed 1q}}

Giants that prefer the hyphenated spelling—Merriam-Webster, The Chicago Manual of Style, and The New Yorker, have a good reason for doing so. E-mail is a compound noun, made out of two words—“electronic” and “mail.” The e in e-mail is an abbreviation for “electronic,” and it’s used in a lot of other words as well—e-commerce, e-learning, and e-business, for example. There are also other compound nouns formed from an abbreviation and a noun, like the H-bomb, which is short for hydrogen bomb. The general rule of hyphenation in compound words that combine a single letter (or a number) and a word is to hyphenate them. So, based on tradition, e-mail is the correct way to do it.