203 Matching Annotations
  1. Last 7 days
    1. we propose the Transformer, a model architecture eschewing recurrence and insteadrelying entirely on an attention mechanism to draw global dependencies between input and output.The Transformer allows for significantly more parallelization a

      Using the attention mechanism to determine global dependencies between input and output instead of using recurrent links to past states. This is the essence of their new idea.

  2. Nov 2022
    1. But we are drinking sea water, making us thirstier for more.

      Nice analogy. Drinking see water when you’re thirsty makes you even more thirsty. Checking social media in your break makes you even more exhausted.

    1. I do think, however, that we need to consider the pattern Citton and Crary have, each in their own way, identified. I would describe the pattern this way: We inhabit a techno-social environment manufactured to fracture our attention. The interests served by this environment in turn pathologize the resultant inattention. These same interests devise and enforce new techniques to discipline the inattentive subject.
    2. a problem whose centrality was directly related to the emergence of a social, urban, psychic, and industrial field increasingly saturated with sensory input. Inattention, especially within the context of new forms of large-scale industrialized production, began to be treated as a danger and a serious problem, even though it was the very modernized arrangements of labour that produced inattention.
    3. Bradbury’s book is about an imposed and willful alienation from the world effected by media technologies. One scene in particular speaks directly to the question of attention. In it, the protagonist is trying desperately to commit to memory a passage of a book he must shortly surrender to the authorities. But his efforts are undone by an incessant advertising jingle blaring over the loudspeaker on the train he is riding. I think of this scene whenever I’m pumping gas and the screen on the pump starts up with a series of 20 second video ads.
    4. The first is a 1961 short story written by the American author, Kurt Vonnegut. “Harrison Bergeron” is set in a dystopian future where equality is enforced by totalitarian means. The physically gifted, for example, were burdened with weights so as to neutralize their natural advantages. The advantages of the intellectually gifted were neutralized by other means.
    5. is the study of how a society uses its scarce resources. And information is not scarce—especially on the Net, where it is not only abundant, but overflowing. We are drowning in information, yet constantly increasing our generation of it. So a key question arises: Is there something else that flows through cyberspace, something that is scarce and desirable? There is. No one would put anything on the Internet without the hope of obtaining some. It's called attention. And the economy of attention—not information—is the natural economy of cyberspace.
    1. Mind you, this is not a net without corporations and capitalism; they can use the protocols, too, and I’m glad Google gives us usable email and spam protection. But it need no longer be a net corrupted by the business model of mass media imported online: the attention economy. And it need no longer be a net under sole corporate control — and thus, potentially, the influence of malign actors, whether Musk or his pals Putin or Trump.If we gain this promising future, if we return to the net’s founding principles, keep one thing clearly in mind: It won’t be so easy to blame the bad shit on the corporations and nasty nerd boys anymore. The net will be ours along with the responsibility to build and enforce the expectations and standards we wish for. The net is us, or it can be at last.

      Jeff clearly hearking back to the golden era of blogging wrt which values it (should) promote(s). Is the 'attention economy' only the 'corrupted biz model of mass media imported online'? The algo-induced raging sure is. Otherwise not sure: [[3 kwantitatieve veranderingen 20100420210721]] was in full effect way before it, and all three contribute to [[Aandacht is het schaarst 20201013163120]]. Maybe it's the flipside that's key? n:: Being economic with attention, as the core of what an attention economy really is like. Towards the end Jarvis slips in the responsibilities stemming from the triplet of obligations [[Obligation to explain 20120327173752]] [[Wie deelt bestaat 20130131133926]] and [[Obligation to re-use 20191223194129]]

  3. Oct 2022
    1. The mass of Paxson's paper work may appear more clearly nowthan the zest with which he labored, but the essence of his methodwas in the spirit rather than in the product.

      Ahrens and others following him have argued that there is a sort of lightness imbued both in one's thinking processes and life by making and accumulating notes. The cognitive load is lessened by offloading one's thoughts onto pieces of paper that can be revised, compared, and juxtaposed as a means of building some written or creative endeavor, even if it's slowly over time.

      Frederic L. Paxson's mode of life made this seem to be the case for him. There is evidence that he was easier able to manage his daily life by his note taking system. He accumulated no work on his desk and carried none home and was able to more easily give his attention to others.

      Is this a result of breaking things down into tiny, bite sized chunks that were difficult to actually interrupt?

      Was it the system or his particular temperament? Are there other examples of this easier mode of life for note takers? Is there a pattern? What portions can be attributed to the system and one's ability to stick to it versus their particular temperaments?

      Other than small examples in my own life, this may be one of the first examples I've seen of this mode of work. Definitely worth looking at others.

  4. Sep 2022
    1. “Limited individual attention and online virality of low-quality information,” By Xiaoyan Qiu et al., in Nature Human Behaviour, Vol. 1, June 2017

      The upshot of this paper seems to be "information overload alone can explain why fake news can become viral."

    2. One of the first consequences of the so-called attention economy is the loss of high-quality information.

      In the attention economy, social media is the equivalent of fast food. Just like going out for fine dining or even healthier gourmet cooking at home, we need to make the time and effort to consume higher quality information sources. Books, journal articles, and longer forms of content with more editorial and review which take time and effort to produce are better choices.

    3. Unable to process all this material, we let our cognitive biases decide what we should pay attention to.

      In a society consumed with information overload, it is easier for our brains to allow our well evolved cognitive biases to decide not only what to pay attention to, but what to believe.

    1. The challenge for the field of sociable media is not simply to invent ever newer ways of communicating,but also to understand the social implications of ubiquitous and omnipresent communication media.

      This part of the article is extremly pertinent to todays world. With social media being a core aspect of many individuals personalities, socializing and attention in the real world has been affected. Apps like Tiktok with short videos have made many individuals attention span extremely short. Often affecting their time spent on work and other activities. Seeing the long term affects on human attention due to social media apps will be interesting to watch.

    1. To see how this plays out, we can continue looking at matrix shapes. Tracing the matrix shape through the branches and weaves of the multihead attention blocks requires three more numbers. d_k: dimensions in the embedding space used for keys and queries. 64 in the paper. d_v: dimensions in the embedding space used for values. 64 in the paper. h: the number of heads. 8 in the paper.
  5. Aug 2022
    1. And the good news about it is that you can actually train your attention, and it’s not that difficult. In fact, almost every contemplative meditation discipline has to do with just sitting down and paying attention to your breath and noticing how your attention changes. There is a saying that comes from the neuroscientists that neurons that fire together are wired together. When you begin paying attention to your attention, you are developing a capability that enables you to have more control over what’s occupying your mind space.

      attention as mindfulness, and as a muscle to train.

    2. You know, it’s not really that difficult, but it’s not being taught at all.

      Reminds me of my 2008/2010 projects in primary schools on this. I find myself explaining marketing ploys to our 6yo in response to material she sees in print, on billboards, and online. Perhaps I should be doing that more consistently

    3. training on how to understand how you’re deploying your attention.

      There's little training on reflecting and shaping how you wield your attention. Are there resources to be found, wrt workflows / choices / being mindfull of one's attention? Beyond the 'indistractable' material of Nir?

      The exclusionary aspect of attention makes it a scarce resource [[Aandacht is het schaarst 20201013163120]] implying the need to wield it with intent [[Stuur aandacht met intentie 20220213080032]] or it becomes distraction again. It's a moral choice [[Aandacht is een morele keuze 20201217074345]] even. Making such training/understanding important.

    4. And when we now live in an era where you can stand on a street corner in any city of the world, waiting for the light to change, and notice that everyone else — everyone else — standing around you is looking at their phone. There’s a lot of money in capturing people’s attention, and there are a lot of apps that are designed to capture and maintain our attention

      This is, like some of his Stanford in-class attention experiments, a bit geared towards switching on/offline it seems. There's much to be said also about wielding attention within the digital space (see Pegrum/Palalas digital disarray above), and attention as it plays out in the interweaving of the digital and physical (like having information resources available within a conversation).

    5. But attention is really the foundation of thought and communication.

      Aandacht als fundament onder zowel denken als communiceren.

      Pegrum/Palalas 2021 talk about attention literacy as needed to counteract 'digital disarray'. They also call it a macro-literacy, encompassing a long list of 'digital literacies' which are more skills than literacy in the Rheingoldia sense. Bit of term inflation? Does put attention at the top of the heap of digital 'literacies' though. They also do incorporate relationships to others and the informational environment within scope of it a la Rheingold.

  6. Jul 2022
    1. I was particularly interested in Chris Aldrich’s observation that knowledge workers tend to talk in spatial terms about their work, especially if distracted. Following interruptions by colleagues or phone calls at work, people may frequently ask themselves “where was I?” more frequently than “what was I doing?” This colloquialism isn’t surprising as our memories for visual items and location are much stronger than actions. Knowledge workers will look around at their environments for contextual clues for what they were doing and find them in piles of paper on their desks, tabs in their computer browser, or even documents (physical or virtual) on their desktops.
  7. Jun 2022
    1. Contrary to popular belief, students don’t have short attention spans. They can focus for hours on a single project. But it has to feel relevant and meaningful to them, and they need to have the time and the space to accomplish it. It’s not easy in a world of school bells and curriculum maps. However, it’s something we should strive for. We should draw students into the deeper, slower work of creativity — because when that happens, learning feels like magic. - “Myth and Mystery of Shrinking Attention Span” - Dr K. R. Subramanian

      This should be motivation enough for instructors to take the time - assuming their bureaucratic overloads allow it - to find ways to make education relevant. This is something, however, that must be baked into people at a young age. And that's the real problem.

    1. The ability to intentionally and strategically allocateour attention is a competitive advantage in a distracted world. Wehave to jealously guard it like a valuable treasure.

      It would seem that the word treasure here is being used to modify one's attention. Historically in books about "knowledge work" or commonplacing, the word was used with respect to one's storehouse of knowledge itself and not one's attention. Some of the effect is the result of the break in historical tradition being passed down from one generation to another. It's also an indication that the shift in value has moved not from what one knows or has, but that the attention itself is more valued now, even in a book about excerpting, thinking, and keeping knowledge!

      Oh how far we have fallen!

      It's also an indication of the extremes of information overload we're facing that the treasure is attention and not the small tidbits of knowledge and understanding we're able to glean from the massive volumes we face on a daily basis.

    1. The dominant idea is one of attention, by which a representation at a position is computed as a weighted combination of representations from other positions. A common self-supervision objective in a transformer model is to mask out occasional words in a text. The model works out what word used to be there. It does this by calculating from each word position (including mask positions) vectors that represent a query, key, and value at that position. The query at a position is compared with the value at every position to calculate how much attention to pay to each position; based on this, a weighted average of the values at all positions is calculated. This operation is repeated many times at each level of the transformer neural net, and the resulting value is further manipulated through a fully connected neural net layer and through use of normalization layers and residual connections to produce a new vector for each word. This whole process is repeated many times, giving extra layers of depth to the transformer neural net. At the end, the representation above a mask position should capture the word that was there in the original text: for instance, committee as illustrated in Figure 1.
  8. Apr 2022
    1. Der Bestätigungs-fehler – wie auch andere kognitive Verzerrungen – fallen umso gravierender aus, je weni-ger Aufmerksamkeit aufgewendet wird.

      Daher ist Rheingolds infotention so wichtig



    1. In studies comparing European American children withMayan children from Guatemala, psychologists Maricela Correa-Chávez andBarbara Rogoff asked children from each culture to wait while an adultperformed a demonstration—folding an origami shape—for another childnearby. The Mayan youth paid far more sustained attention to the demonstration—and therefore learned more—than the American kids, who were oftendistracted or inattentive. Correa-Chávez and Rogoff note that in Mayan homes,children are encouraged to carefully observe older family members so that theycan learn how to carry out the tasks of the household, even at very young ages.

      American children aren't encouraged to as attentive imitators as their foreign counterparts and this can effect their learning processes.

    1. Still, people at work will rarely see me without a notebook around. When I go to meetings, I always prefer taking notes on paper rather than bringing my laptop along with me.Why is that? Because, so far, I haven’t found any digital solution that gives me as much freedom with so little friction for taking notes, sketching, writing down ideas, etc while keeping my attention on what’s being said around me.To me, digital solutions are clearly not on par with paper note taking. Using digital solutions often takes my attention away, which is horrible.

      digital solutions takes my attention away

    1. Programmable attention centers around making high-level decisions that orchestrate how we later spend our attention. It is rife in everyday life, albeit (usually) in a weak way. Examples include:email inboxes with snoozessetting a recurring event on the calendartodos which appear based on locationGetting Things Done “Tickler” Fileshabit-enforcing systems (e.g. Beeminder, StickK)spaced repetition (e.g. Anki, SuperMemo)ML-trained “smart email inboxes”

      We already use programmable attention in everyday life, but it should be more integrated, potentially at an OS level.

      HeyFocus and ad blockers could also be considered under this umbrella.

    1. next wave of human-computer interaction will focus more on “pushing” us the right, relevant information versus the user “pulling” for information

      This is already happening with ads. It'd be nice if the same techniques were used to pre-emptively help us instead.

  9. Mar 2022
    1. Human minds are made of memories, and today those memories have competition. Biological memory capacities are being supplanted, or at least supplemented, by digital ones, as we rely on recording—phone cameras, digital video, speech-to-text—to capture information we’ll need in the future and then rely on those stored recordings to know what happened in the past. Search engines have taken over not only traditional reference materials but also the knowledge base that used to be encoded in our own brains. Google remembers, so we don’t have to. And when we don’t have to, we no longer can. Or can we? Remembering and Forgetting in the Age of Technology offers concise, nontechnical explanations of major principles of memory and attention—concepts that all teachers should know and that can inform how technology is used in their classes. Teachers will come away with a new appreciation of the importance of memory for learning, useful ideas for handling and discussing technology with their students, and an understanding of how memory is changing in our technology-saturated world.

      How much history is covered here?

      Will mnemotechniques be covered here? Spaced repetition? Note taking methods in the commonplace book or zettelkasten traditions?

    1. operationstisch

      operationstheoretisch - was geschieht - Der Begriff Operation hat mir auch ungemein geholfen, Latours handelnde Akteure besser zu greifen, weil er im Gegensatz zum Handlungsbegriff frei von (menschlicher) Intention ist. Und Intention wird für mich noch ein crucial point, weil Intention fuels Attention und Attention = basale digital literacy nach Rheingold

    1. turning attention to the area of inattention

      Don't know if I am too superficially thinking here, but this reminds me of thoughts about mindfulness - Gives me the idea, that coping with new media will be more successful if it is done mindfully

    1. In the Huffington Post, Stone offers valuable advice on “how to switch from managing time to managing attention”:1. Each evening or morning before you start your day, make a short list of your intentions (the result and feeling of something you want) for the day and by each, write the related to do’s for that day. Try to keep your list to 5 intentions. Conscious-ly choose what you will do and what you will not do. Keep a different list of what you will review for inclusion on other days.2. List only what you really expect to do that day. As other things come to mind, write them on a separate list. By putting these items on a separate list, you are creat-ing the space to be in the moment with each of your day’s priorities. Review that list as you plan for the next day and determine how they fit in to your plans. Give yourself some down time, enjoy your successes at the end of the day.3. Give yourself meaningful blocks of uninterrupted time to focus on each intention. Turn OFF technology each day during those blocks and focus on your intentions.4. At home, be clear about what technology you’ll use and where. Computer in the kitchen? Maybe not.100
    2. When I asked her about the specifics of learning to exercise better executive control over attention, Stone reminded me that “intention is the fuel for atten-tion.”
    3. Recent research reported in the journal Cognition offers evidence that brief distractions from a focal task may improve concentration over the longer run: “We propose that deactivating and reactivating your goals allows you to stay focused,” the study’s authors said. “From a practical standpoint, our research suggests that, when faced with long tasks (such as studying before a final exam or doing your taxes), it is best to impose brief breaks on your-self. Brief mental breaks will actually help you stay focused on your task!”28

      Kleine Pausen machen wenn man ein großes, ein longterm Ziel hat. De- und reaktivieren der intendierten Ziele soll den Forschungsergebnissen zufolge helfen, sich auf ebenjene länger und schließlich erfolgreicher zu konzentrieren.

    4. Paying attention to your breath—the core technique of mindfulness meditation methods—is where Stone sug-gests starting to moderate our online reactions. I’ll get back to that later. For now, I’m convinced that Stone is right to think that attention to breathing could be a tool to help moderate our unthinking, ultimately unhealthy reactions to many online stimuli.

      Atemfokussierung als Mittel bzw. Startpunkt eines Mittels zur Aufmerksamkeitskontrolle - auch im Digitalen.

    5. Gaining control of your attention while you are online requires, first of all, intention. When you formulate a goal, you need to intend to achieve it.

      Intentionale Setzung eines Ziels, das erreicht werden will ist Voraussetzung für Aufmerksamkeitskontrolle.

    6. I’m not sure I would have done so before starting this book because we tend to think of attention passively, as if it is out of our control or only controlled outside of ourselves, by the world around us. That is not true. We can track our own attentional pathways and learn from them.
    7. The first thing I learned is good news if you have been thinking that “attention training” sounds like too much work: you can experience imme-diate benefits by beginning in small ways to exercise mindfulness regarding your attention online. In this realm, taking some control, even if it is a baby step, is far better than passively letting your attention be grabbed without reflection. Growing evidence indicates consistent exercise can strengthen self-control of attention.

      Jede Verbesserung, so klein sie auch ist, von Aufmerksamkeit, ist eine tatsächliche Verbesserung im aufmerksamen Umgang mit digitalen Medien.

    1. Thinking of technology as a kind of writing brings the idea of agency back to the foreground. The same methods of thought you (hopefully) learned to read critically and consider whether or not you agree with a piece of writing can be applied to tech.

      I only recently learned to read critically (through web annotations). This analogy is powerful, but how many people are going to understand it?

  10. Feb 2022
    1. By investigating the patterns of who pays attention to whom, our study provides evidence of a racial attention deficit: Even when in their self-interest, Whites pay less attention to Black peers. Specifically, White Americans rate Black peers as less competent than White ones and are less likely to follow their example as a guide to making a better decision.
    1. Out there in the multiverse is a reality where the web is a complete borefest. Information is the only driving factor to visit a “web page” and PWAs have never come to exist. Custom styling, fancy interactive animations and single-page functionality isn’t even something that can be implemented. The web is just a system of HTML/plaintext documents sharing information and data. Users browse the web in quick bursts to satisfy their queries or read something interesting. Then, they return to real life.
    1. Even though results of these studies are currently under intensescrutiny and have to be taken with a grain of salt (Carter andMcCullough 2014; Engber and Cauterucci 2016; Job, Dweck andWalton 2010), it is safe to argue that a reliable and standardisedworking environment is less taxing on our attention, concentration

      and willpower, or, if you like, ego. It is well known that decision-making is one of the most tiring and wearying tasks...

      Having a standardized and reliable working environment or even workflow can be less taxing on our attention, our concentration, and our willpower leaving more energy for making decisions and thinking which can have a greater impact.

      Does the fact that the relative lack of any decision making about what to see or read next seen in doomscrolling underlie some of the easily formed habit of the attention economy? Not having to actively decide what to read next combined with the random rewards of interesting tidbits creating a sense of flow is sapping not our mental energy, but our time. How can we better design against this?

    2. Give Each Task the Right Kind of Attention

      Ahrens talks about the variety of different tasks that underpin writing and the varieties of attention that each can take. He suggests that for increased productivity that one focus on one sort or type of tasks at a time in each part of the process.

      This sort of structural planning in one's work is possibly the most important planning one can do.

    3. his suggests that successful problem solvingmay be a function of flexible strategy application in relation to taskdemands.” (Vartanian 2009, 57)

      Successful problem solving requires having the ability to adaptively and flexibly focus one's attention with respect to the demands of the work. Having a toolbelt of potential methods and combinatorially working through them can be incredibly helpful and we too often forget to explicitly think about doing or how to do that.

      This is particularly important in mathematics where students forget to look over at their toolbox of methods. What are the different means of proof? Some mathematicians will use direct proof during the day and indirect forms of proof at night. Look for examples and counter-examples. Why not look at a problem from disparate areas of mathematical thought? If topology isn't revealing any results, why not look at an algebraic or combinatoric approach?

      How can you put a problem into a different context and leverage that to your benefit?

    4. On closer look, it becomes obvious how different the tasks are thatare usually summarised under “writing” and how different the kindsof attention are that they require.

      What are the constituent parts of writing and how do they differ based on their functions with respect to attention?

      • note taking
      • composing
      • invention
      • creativity
      • thinking
      • editing
      • structuring
      • outlining
      • proofreading
      • etc.

      Where do each of these sit with respect to the zettelkasten? How can one create flow with respect to each of these or with respect to one or two which may necessarily need to be bound together to accomplish them?

    5. The slip-box provides not only a clear structure to work in, but also forces usto shift our attention consciously as we can complete tasks inreasonable time before moving on to the next one.

      Ahrens provides a quick overview of some research on distraction, attention, and multi-tasking to make the point that:

      The simple structure and design of the zettelkasten forces one's focus and attention on small individual tasks that cumulatively build into better thinking and writing.

      (Summary of Section 9.2)

    6. We also know that theaverage length of TV soundbites has steadily declined over the lastseveral decades (Fehrmann, 2011). During the U.S. presidentialelection in 1968, the average soundbite — that is, any footage of acandidate speaking uninterrupted — was still a little more than 40seconds, but that had fallen to less than 10 seconds at the end of the80s (Hallin 1994) and 7.8 seconds in 2000 (Lichter, 2001). The lastelection has certainly not reversed the trend. Whether that meansthat the media adjust to our decreasing attention span or is causingthe trend is not easy to say.[17]

      Ryfe and Kemmelmeier not only show that this development goes much further back into the past and first appeared in newspapers (the quotes of politicians got almost halved between 1892 and 1968), but also posed the question if this can maybe also be seen as a form of increased professionalism of the media as they do not just let politicians talk as they wish (Ryfe and Kemmelmeier 2011). Craig Fehrman also pointed out the irony in the reception of this rather nuanced study – it was itself reduced to a soundbite in the media (Fehrman 2011).

      Soundbites have decreased in length over time.

      What effects are driving this? What are the knock on effects? What effect does this have on the ability for doubletalk to take hold? Is it easier for doubletalk and additional meanings to attach to soundbites when they're shorter? (It would seem so.) At what point to they hit a minimum?

      What is the effect of potential memes which hold additional meaning of driving this soundbite culture?

      Example: "Lock her up" as a soundbite with memetic meaning from the Trump 2016 campaign in reference to Hilary Clinton.

  11. Jan 2022
    1. Seems easy, right? How about the below code, what will it print? new Promise((_, reject) => reject(new Error('woops'))). catch(error => { console.log('caught', err.message); }); It'll print out an unhandled rejection warning. Notice that err is not defined!
    1. il permet de faire le lien entre l’économie de l’attention, qui est au cœur de la problématique,

      Bien formulé ! L'économie de l'attention n'est pas le probleme mais elle est au coeur de celui-ci. Concernée directement.

    1. Fernandez-Castaneda, A., Lu, P., Geraghty, A. C., Song, E., Lee, M.-H., Wood, J., Yalcin, B., Taylor, K. R., Dutton, S., Acosta-Alvarez, L., Ni, L., Contreras-Esquivel, D., Gehlhausen, J. R., Klein, J., Lucas, C., Mao, T., Silva, J., Pena-Hernandez, M., Tabachnikova, A., … Monje, M. (2022). Mild respiratory SARS-CoV-2 infection can cause multi-lineage cellular dysregulation and myelin loss in the brain (p. 2022.01.07.475453). https://doi.org/10.1101/2022.01.07.475453

    1. Hari puts his general air of unworldly distraction down to his dyspraxia, but it comes across as donnish.

      Johnann Hari has indicated in an interview that he suffered from dyspraxia.

      I wonder how this may or may not affect his writing about being distracted with respect to his book Stolen Focus. Cross reference: https://www.theguardian.com/science/2022/jan/02/attention-span-focus-screens-apps-smartphones-social-media

  12. Dec 2021
    1. I was suddenly deluged with ads for “the world’s thinnest tablet,” which promised not only to replace pen and paper but to help you “Get Your Brain Back.” The company’s Lovecraftian promotional ad, which has racked up nearly three million views, begins with a hissing demon-child clinging to her iPad and proceeds through an animated hellscape complete with attention-sucking brain tubes and notifications circling like sharks. The narrator quavers an ominous warning: “We have to modify technology, or else it will modify us.”

      Given the diversions of modern digital life, perhaps the best way to do one's writing is to do it at the moment of reading the actual references. Often while reading, one isn't as apt to have their attention diverted by the vagaries of life, instead they are focused on the thing at hand. It is while one has this focused attention that they should let their note taking practice while reading take over.

      Even if you are distracted, you can at least maintain focus on a single line of text and your thoughts related to it and write them down in either a summary sentence or with a few related ideas which are sparked by the initial idea.

      (This note is such an example.)

      Then one can start and complete a small idea at a time and then letting them build over time and space, then recollect them to create a piece which then doesn't need to be written and painfully created, but which may only need an outline structure and some final polish and editing.

    1. The transformer model introduces the idea of instead of adding another complex mechanism (attention) to an already complex Seq2Seq model; we can simplify the solution by forgetting about everything else and just focusing on attention.
    1. a lot of people start with learning and then they build things and then they close the circle but there's one key piece missing here and some people hate the word but you 00:29:54 learn to love it eventually it's called marketing and marketing means a lot of things to a lot of people but what it means to me is getting the word out because someone else will if you don't and 00:30:05 you are awesome you just have to realize that maybe not everyone knows right away so you should really talk about it more maybe at conferences see what i did there 00:30:17 um maybe on twitter maybe you can just tell your friends and maybe you can ask people to contribute and to support you like what's wrong with that somehow it's frowned upon in the community that if you do 00:30:30 marketing you're not doing it for real but i think that's not true um i think that if smart people and patient and um passionate people as well 00:30:44 if they did marketing then the world would be a better place because i'm pretty sure the evil guys do marketing so do your homework

      Marketing is very critical but it has negative connotations in the open source community because it is associated with mainstream business , after all, marketing is derived from the word "market".

      Perhaps it is better to think in psychological terms. If we have a great idea, the internet is a way to reach billions of eyeballs. Everyone is, in a sense, forced to compete in an attention economy. Instead of marketing, we can also use the words "attracting attention", because that is really what we are trying to do, be an attention attractor.

      The Indieverse, being developed by knowledge architect Gyuri Lajos, offers an alternative to marketing. Marketing is an attention attractor that relies on a "push" strategy. We are making content and pushing it out to different parts of the world we think may resonate with us to attract attention.

      Instead, the Indieverse, with its built in read and write provenance can act like a "pull" attention attractor. People can discover you through the built in discoverability aspects of the indieverse. Unlike the private sector, which uses this pull method to try to match you to stuff they want to sell you, Indieverse inegrates tools that exposes relevant content to you. If that content has demonstrably improved your life, which can be tracked through your public sharing, you can sponsor or reward that content. Microsponsorship can even be built in.

  13. Nov 2021
    1. We report the first neural recording during ecstatic meditations called jhanas and test whether a brain reward system plays a rolein the joy reported. Jhanas are Altered States of Consciousness (ASC) that imply major brain changes based on subjective reports:(1) external awareness dims, (2) internal verbalizations fade, (3) the sense of personal boundaries is altered, (4) attention is highlyfocused on the object of meditation, and (5) joy increases to high levels. The fMRI and EEG results from an experienced meditatorshow changes in brain activity in 11 regions shown to be associated with the subjective reports, and these changes occur promptlyafter jhana is entered. In particular, the extreme joy is associated not only with activation of cortical processes but also with activationof the nucleus accumbens (NAc) in the dopamine/opioid reward system. We test three mechanisms by which the subject mightstimulate his own reward system by external means and reject all three. Taken together, these results demonstrate an apparentlynovel method of self-stimulating a brain reward system using only internal mental processes in a highly trained subject.

      I can find no other research on this particular matter. It would be helpful to have other studies to validate or invalidate this one. This method of reward requires a highly-trained participant and involves no external means.

    1. The selective-second-order-with-skips model is a useful way to think about what transformers do, at least in the decoder side. It captures, to a first approximation, what generative language models like OpenAI's GPT-3 are doing.
    1. The Query word can be interpreted as the word for which we are calculating Attention. The Key and Value word is the word to which we are paying attention ie. how relevant is that word to the Query word.


    1. Other work on interpreting transformer internals has focused mostly on what the attention is looking at. The logit lens focuses on what GPT "believes" after each step of processing, rather than how it updates that belief inside the step.
    1. The attention layer (W in the diagram) computes three vectors based on the input, termed key, query, and value.

      Could you be more specific?

    2. Attention is a means of selectively weighting different elements in input data, so that they will have an adjusted impact on the hidden states of downstream layers.
    1. Once it becomes clear that attention and praise can be garnered from organizing an attack on someone’s reputation, plenty of people discover that they have an interest in doing so.

      This is a whole new sort of "attention economy".

      This genre of problem is also one of the most common defenses given by the accused as sort of "boogeyman" meant to silence accusers. How could we better balance the ills against each of the sides in these cases to mitigate the broader harms in both directions?

  14. Oct 2021
    1. https://slate.com/culture/2011/08/cathy-n-davidson-s-now-you-see-it-do-the-young-really-rule-in-the-internet-era.html

      A very prescient article by Annie Murphy Paul from 2011. It doesn't review Davidson's book, so much as to take to task some of the underlying optimistic views of the magic of technology. If only we were able to better adapt and evolve to create the sort of changes in humanity to take advantage of the potential benefits that were assumed. Instead, much of the tech sector adapted instead to hijack our slowly evolving attention to benefit themselves.

      I wish we as a culture had had more of this sober sort of outlook about technology at the time.

      I'm now even more intrigued by Paul's new book: The Extended Mind: The Power of Thinking Outside the Brain, which is already in my reading queue.

      <small><cite class='h-cite via'> <span class='p-author h-card'>Annie Murphy Paul </span> in "@ChrisAldrich @amandalicastro @CathyNDavidson Chris, you may be interested in this review of "Now You See It" that I wrote . . . https://t.co/TnnbQ3NHWf" / Twitter (<time class='dt-published'>10/17/2021 10:25:52</time>)</cite></small>

    2. The digital age has brought all of us new and exciting tools that will surely continue to alter the way we learn and work. But focusing one’s attention, gathering and synthesizing evidence, and constructing a coherent argument are skills as necessary as they were before—in fact, more necessary than ever, given the swamp of baseless assertion and outright falsehood that is much of the Web. Some day not too far in the future, the digital natives may find themselves turning down the music, shutting off the flickering screen, silencing the buzzing phone and sitting down to do just one thing at a time.

      Very prescient for 2011!

    3. the work of researchers like Clifford Nass of Stanford University. “Human cognition is ill-suited both for attending to multiple input streams and for simultaneously performing multiple tasks,” Nass has written.
    1. All of these corrections havebeen almost exclusively used with pure water under ambient condition

      these correction methods are more compatible with pure water so far.



    1. According to addiction expert Dr Anna Lembke, smartphones are making us dopamine junkies. So how do we beat our digital dependency?

      Attention to Intention

      Resonance with the topic for the next World Weavers group conversation on Saturday, October 23: Shifting from an attention economy to an intention economy.

    1. On Saturday, October 9, after our World Weavers conversation on the topic Matter is Derivative of Consciousness, I was exploring Value Village, a thrift store in Chilliwack, with my wife, Jayne. I came across a book that fits with the theme for our World Weavers conversation on October 23: Shifting from an attention economy to an intention economy.

      Sacred Economics

      By Charles Eisenstein

      Sacred money, then, will be a medium of giving, a means to imbue the global economy with the spirit of the gift that governed tribal and village cultures, and still does today wherever people do things for each other outside the money economy.

      Sacred Economics describes this future and also maps out a practical way to get there. Long ago I grew tired of reading books that criticized some aspect of our society without offering a positive alternative. Then I grew tired of books that offered a positive alternative that seemed impossible to reach: “We must reduce carbon emissions by 90 percent.” Then I grew tired of books that offered a plausible means of reaching it but did not describe what I personally, could do to create it. Sacred Economics operates on all four levels: it offers a fundamental analysis of what has gone wrong with money; it describes a more beautiful world based on a different kind of money and economy; it explains the collective actions necessary to create that world and the means by which these actions come about; and it explores the personal dimensions of the world-transformation, the change in identity and being that I call “living in the gift.”

      (Page XIX)

  15. Sep 2021
    1. Side note: When I flagged yours as a dupe during review, the review system slapped me in the face and seriously accused me of not paying attention, a ridiculous claim by itself since locating a (potential) dupe requires quite a lot of attention.
  16. Aug 2021
    1. So for each word, we create a Query vector, a Key vector, and a Value vector. These vectors are created by multiplying the embedding by three matrices that we trained during the training process.
    1. I'm going to try provide an English text example. The following is based solely on my intuitive understanding of the paper 'Attention is all you need'.

      This is also good

    2. For the word q that your eyes see in the given sentence, what is the most related word k in the sentence to understand what q is about?
    3. So basically: q = the vector representing a word K and V = your memory, thus all the words that have been generated before. Note that K and V can be the same (but don't have to). So what you do with attention is that you take your current query (word in most cases) and look in your memory for similar keys. To come up with a distribution of relevant words, the softmax function is then used.
    1. First, what were the economies of attention thatguided his commonplacing techniques? Second, what type of impact did his note-taking skillshave upon the way that he arranged information in texts?

      The two questions addressed in this article.

  17. Jul 2021
    1. Setting Up Scope and Topic

      You need to establish boundaries with respect to what you want to learn, otherwise you'll keep going towards whatever catches your attention in the moment.

  18. Jun 2021
    1. The result is to scatter our attention and diffuse our concentration.

      Computer and phone notifications can be insidious. I've personally turned most of them off.

      I also find that reading and annotating with Hypothes.is has helped me to have more focus while reading---even despite the short turnoffs to cogitate a bit, write a bit, and then return.

  19. May 2021
  20. Apr 2021
    1. It feels like it was thrown together in a weekend using parts from "Think To Die" since even the successful act of feeding your chickens has the same blood-splatter-on-camera-lens that you would get from scoring in Think To Die where your goal is to kill all of your people as opposed to this where you are feeding animals, so what's with the blood splatter? It just shows a lack of attention to detail.
  21. Mar 2021
    1. The idea is interesting that if you introduce a slight distraction or speak softly, people will not only have to try harder to hear you but that the "effort moves us into higher gear, activating more vigorous and more analytical brain machinery." (53)

      It's a frequent story in Hollywood that Michael Ovitz used the tactic of speaking softly to get people to listen to him more closely.

      Ought to dig in to see if anyone has done research on this effect.

      Dan doesn't seem to indicate it, but I'm sort of curious what his parenthetical numbers in the text represent or link to?

  22. Feb 2021
    1. we also wrap them in Failure to solve the second problem: spotting potential exceptions is hard
    2. Almost everything in python can fail with different types of exceptions: division, function calls, int, str, generators, iterables in for loops, attribute access, key access, even raise something() itself may fail. I am not even covering IO operations here. And checked exceptions won’t be supported in the nearest future.
    3. You still need to have a solid experience to spot these potential problems in a perfectly readable and typed code.
    4. print will never be actually executed. Because 1 / 0 is an impossible operation and ZeroDivisionError will be raised.
  23. Jan 2021
    1. Chess thinking provides a rich metacognitive context that leads me to believe that we should tease apart three notions that are related but often conflated – attention, flow and concentration. Attention is fundamentally grounded in perception (how we attend), flow is fundamentally grounded in experience (how we feel), and concentration is grounded in praxis (how we purposively coalesce).
    1. Our human tendency is to focus on threats and problems. For the sake of our emotional wellness, it makes sense to modify that automatic tendency. You can’t control the stressors that come your way, but you can influence the focus of your own attention. You can focus on the things that give you back a feeling of control.

      We tend to focus attention towards stressors. Instead we should try to focus on things that give us more control of the situation. That way we can face our stressors with more resources.

  24. Dec 2020
    1. What you pay attention to is going to be your life.

      .. and also, maybe more important, how you pay attention.

      There is a space for creation when deliberately exploiting/exploring the attention mechanism.

  25. Oct 2020
    1. YouTube doesn’t give an exact recipe for virality. But in the race to one billion hours, a formula emerged: Outrage equals attention.

      Talk radio has had this formula for years and they've almost had to use it to drive any listenership as people left radio for television and other media.

      I can still remember the different "loudness" level of talk between Bill O'Reilly's primetime show on Fox News and the louder level on his radio show.

    1. The attention of the audience is a writer's most precious possession, and the value of audience attention is seldom more clear than in writing for the Web. The time, care, and expense devoted to creating and promoting a hypertext are lost if readers arrive, glance around, and click elsewhere. How can the craft of hypertext invite readers to stay, to explore, and to reflect?

      A very early statement about what was about to become the "attention economy"

    1. Third, content collapse puts all types of information into direct competition. The various producers and providers of content, from journalists to influencers to politicians to propagandists, all need to tailor their content and its presentation to the algorithms that determine what people see. The algorithms don’t make formal or qualitative distinctions; they judge everything by the same criteria. And those criteria tend to promote oversimplification, emotionalism, tendentiousness, tribalism — the qualities that make a piece of information stand out, at least momentarily, from the screen’s blur.

      This is a terrifically painful and harmful thing. How can we redesign a system that doesn't function this way?