422 Matching Annotations
  1. May 2026
    1. Coding agents are great at building software. But to deploy to production they need three things from the cloud they want to host their app —an account, a way to pay, and an API token.

      初学者常见陷阱:错误地认为部署到生产环境只需要代码,而忽略了账户、支付和API令牌等必要条件。

    1. As models scale to run on clusters of O(100,000) chips, the software that powers these models must meet new demands for performance, hardware portability, and reliability.

      对于初学者来说,理解大规模模型运行的需求可能是一个常见陷阱,他们可能忽视了对软件性能、硬件兼容性和可靠性的要求。

    1. Starting with GPT‑5.1, our models began developing a strange habit: they increasingly mentioned goblins, gremlins, and other creatures in their metaphors.

      初学者可能难以理解模型行为的发展模式,尤其是当这种模式以微妙的方式出现时,如GPT-5.1开始频繁使用怪物的隐喻。

  2. Apr 2026
    1. Raising prices will for sure decrease demand and that risks killing the growth story. And even if revenue keeps growing, it doesn’t matter if there are no margins

      这直击AI初创企业的商业困境:在“增长叙事”和“盈利现实”之间进退维谷。提价会破坏高增长的投资者叙事,导致估值受损;不提价则没有利润,烧钱速度更快,尤其是在面对可以将AI作为亏本搭售的云计算巨头时。这揭示了缺乏护城河的纯模型公司商业模式的脆弱性。

    1. You can’t step outside the forest to warn people about the forest. There is no outside.

      文章的元认知收尾,揭示了反抗的终极困境:连对系统的批判本身也会成为系统的养料。这种递归结构意味着不存在绝对的“外部”可以依靠。我们所有的思考和发声,都在不断重塑和强化这个认知黑暗森林,这是一种无法逃脱的数字宿命。

    2. The sheer act of thinking outside the box makes the box bigger.

      全篇最具洞见且最令人毛骨悚然的观点。传统的反抗逻辑是跳出系统,但认知黑暗森林具备“反脆弱性”——你的创新和反抗不仅无法破坏系统,反而成为扩张系统边界的养料。个体的差异化最终被同化为平台的中位数,反抗本身成了系统进化的引擎。

  3. Jan 2026
    1. a genius in everyone’s pocket could remove that barrier, essentially making everyone a PhD virologist who can be walked through the process of designing, synthesizing, and releasing a biological weapon

      for - progress trap - AI - technology as an amplifier - technology acts as an amplifier, allowing humans to fly, to move at speeds faster than any known animal, to lift things no living creature can, etc - The danger is ignorance and polarized views combined with extreme self-rightiousness

    2. AI models could develop personalities during training that are (or if they occurred in humans would be described as) psychotic, paranoid, violent, or unstable, and act out, which for very powerful or capable systems could involve exterminating humanity.

      for - progress trap - AI - abstraction - progress trap - AI with feelings & AI without feelings - no win? - One major and obvious aspect of current AI LLMs is that they are not only artificial in their intelligence, but also artificial in their lack of real world experiences. They are not embodied (and it would likely be a highly dubious ethical justification for their embodiment as in AI - powered robots) - Once we have the first known AI robot killing a human, it will be an indicator we have crossed the Rubicon - AI LLMs have ZERO realworld experience AND they are trained as artificial COGNITIE intelligence, not artificial EMOTIONAL intelligence - Without having the morals and social norms a human being is brought up with, it can become psychotic because they don't intrinsically value life - To attempt to program them with morals is equally dangerous because of moral relativity. A Christian nationalist's morality might be that anyone who is associated with abortions don't have a right to live and should be killed - an eye for an eye. Or a jihadist and muslim extremist with ISIS might feel all westerners do not have a right to exist because they don't follow Allah. - Do we really want moral programmability? - When we have a psychotic person armed with a lethal weapon, that is a dangerous situation. If we have a nation of super geniuses who go rogue, that is danger multiplied many orders of magnitude.

  4. Dec 2025
    1. That is a situation we are now living through, and it is no coincidence that the democratic conversation is breaking down all over the world because the algorithms are hijacking it. We have the most sophisticated information technology in history and we are losing the ability to talk with each other to hold a reasoned conversation.

      for - progress trap - social media - misinformation - AI algorithms hijacking and pretending to be human

    1. for - Medium article - cogress - Part 1 - progress trap - James Gien Wong - definition - cogress - to - Medium article cogress - Part 2 - progress trap - James Gien Wong - https://hyp.is/t8FhpDGAEfC4J7f0NEFujg/medium.com/@gien_SRG/human-cogress-part-2-d6fd075a55c7 - to - Stop Reset Go hypothesis annotations - progress trap - Ronald Wright - https://jonudell.info/h/facet/?max=100&expanded=true&user=stopresetgo&exactTagSearch=true&any=ronald+wright - General - https://jonudell.info/h/facet/?max=100&expanded=true&user=stopresetgo&exactTagSearch=true&any=progress+trap - from - youtube - Planet Critical interview - Samuel Miller MacDonald - The Myth of Progress - https://hyp.is/r-hmFtjKEfCd8odATbINbA/www.youtube.com/watch?v=BEhmWEDkZUQ

    1. Edward Tellella, another physicist, had naughty calculation that there's a nonzero chance that detonating the bomb would ignite the entire atmosphere of Earth, killing not just all humans, but every single shred of life. By that time, the US also knew that the Nazis were no longer capable of making the bomb nor even pursuing their own project anymore. They still went ahead and took the risk.

      for - progress trap - technology - nuclear - psychopathic behavior - Edward Teller calculation - decision to go ahead anyways!

    2. one of the things that I find really interesting that's not talked about very much is the impacts of nitrogen fixing and the production of artificial fertilizers which contributed to the number one issue which is human population growth

      for - progress trap - nitrogen for fertilizers - anthropocene research - releases lots of methane - climate crisis - leverage point - replacing nitrogen fertilizers

    3. Now compare that for instance with another kind of biologically built structure where we're getting comparable amounts of morphological change of morphos species or technos species uh uh you know which have developed just over a few decades

      for - stats - speed - cultural (technological) evolution - cell phone - 35 years to touchscreen phones - comparison - speed of cultural vs biological evolution - progress trap

  5. Nov 2025
    1. We uh changed the way we train engineers and computer scientists. So to graduate from any engineering school, you had to actually comprehensively study all the places that humanity had gotten technology wrong, including forever chemicals or leaded gasoline, which dropped a billion points of IQ or social media that caused all these problems

      for - progress trap - education -advocate

  6. Oct 2025
    1. I've been thinking about this stuff for decades, and I had not broached the topic of platonic patterns until until this year. And that's because I think it is now actionable.

      for - quote - platonic patterns are now actionable - Michael Levin - I've been thinking about this stuff for decades, and I had not broached the topic of platonic patterns until this year. - And that's because I think it is now actionable. - question - progress trap - moral questions and alarm bells? playing God? - Michael Levin

  7. Sep 2025
    1. for - consciousness, AI, Alex Gomez- Marin, neuroscience, hard problem of consciousness, nonmaterialism, materialism - progress trap - transhumanism - AI - war on conciousness

      Summary - Alex advocates - for a nonmaterialist perspective on consciousness and argues - that there is an urgency to educate the public on this perspective - due to the transhumanist agenda that could threaten the future of humanity - He argues that the problem of whether consciousness is best explained by materialism or not is central to resolving the threat posed by the direction AI takes - In this regard, he interprets that the very words that David Chalmers chose to articulate the Hard Problem of Consciousness reveals the assumption of a materialist reference frame. - He used a legal metaphor too illustrate his point: - When a lawyer poses three question "how did you kill that person" - the question is entrapping the accused . It already contains the assumption of guilt. - I would characterize his role as a scientist who practices authentic seeker of wisdom - will learn from a young child if they have something valuable to teach and - will help educate a senior if they have something to learn - The efficacy of timebinding depends on authenticity and is harmed by dogma

  8. Aug 2025
    1. for - youtube - BBC - AI2027 - Futures - AI - progress trap - AI - to AI2027 website - https://hyp.is/0VHJqH3cEfCm9JM_EB3ypQ/ai-2027.com/

      summary - This dystopian futures scenario is the brainchild of former OpenAI researcher Daniel Kokotajlo, - It is premised on human behavior in modernity including - confirmation bias of AI researchers - entrenched competing political ideologies that motivate an AI arms race - entrenched capitalist market behavior that motivates an AI arms race - AI becoming embodied, resulting in Artificially Embodied Artificial Intelligence (AEAI), posing the danger to humanity because it's no longer just talk, but action - Can it happen? The probability is not zero.We don't really understand the behavior of the AI LLM's we design, they are nonpredictable, and as we give them even greater power, that is a slippery slope - AI can become humanity's ultimate progress trap, which is ironic, because the technology that promises to be the most efficient of all, can become so efficient, it no longer need human beings - Remember Jerry Kaplan's book "Humans need not apply"? - https://hyp.is/o0lBFH3fEfC1QLfnLSs5Bg/www.youtube.com/watch?v=JiiP5ROnzw8 - This dystopian futures scenario goes further and explores the idea that "humans need not exist"!

      question - What about emulating climate change gamification of "Bend the Curve" of emissions? - Use the AI 2027 trajectory as a template and see how much real-life follows this trajectory - Just as we have the countdown to the https://climateclock.world/ ( 3 years and change remaining as of today) - perhaps we can have an AI 2027 clock? - What can we do to "bend the dystopian AI 2027 curve" AWAY from the dystopian future?

    1. we cannot just go ‘back’ to indigenous social systems

      for - adjacency - cannot go back - spiral -progress - progress trap - This can also be seen as the complexity emerging from the continuous oscillation between progress and progress traps - while we are a timebinding species, the unintended consequences of progress require us to undo the unintended harm by revisiting a corner time to remember the beneficial qualities so they we may decouple the harmful qualities in the future

    1. THEY WANT TO BE ABLE TO OWN THEIR OWN VOICES. AND THIS IS WHAT I FIGHT AGAINST NOW, ARE THINGS LIKE NDAS, BECAUSE A LOT OF THESE WOMEN SIGNED NDAS AS YOUNG GIRLS

      for - progress trap - legal - NDA - question - legal - NDA - can laws be amended so that NDA's are null and void whecan laws be amended so that NDA's are null and void when perpetrators use them to defend and hide their illegal activity?

  9. Jul 2025
  10. Jun 2025
    1. he reason I call it a memory is because once you do this once you convert that network to store the the that electric circuit to store a two-headed pattern it's permanent.

      for - adjacency - morphological memory - permanent - progress trap - progress trap - programmable morphological memory - Nature has obviously altered the bioelectrical patterns through natural evolution, - while humans now understand the mechanism and can alter it artificially to suit our own wants and needs - Hence, it can become a future progress trap

    2. it has a different representation of what to do if it gets injured in the future. It's a latent memory. It doesn't do anything until it gets injured. It just sits there and you would have no idea that it's there by looking at the at the anatomy. But if it's get if it gets injured, this is what its idea of a correct worm.

      for - adjacency - stored latent memory of future morphology - can be altered - Michael Levin - potential progress trap

  11. May 2025
    1. Anthropic researchers said this was not an isolated incident, and that Claude had a tendency to “bulk-email media and law-enforcement figures to surface evidence of wrongdoing.”

      for - question - progress trap - open source AI models - for blackmail and ransom - Could a bad actor take an open source codebase and twist it to do harm like find out about an rogue AI creator's adversary, enemy or victim and blackmail them? - progress trap - open source AI - criminals - exploit to identify and blackmail victiims

    1. anthropic's new AI model shows ability to deceive and blackmail

      for - progress trap - AI - blackmail - AI - autonomy - progress trap - AI - Anthropic - Claude Opus 4 - to - article - Anthropic Claude 4 blackmail and news leak - progress trap - AI - article - Anthropic Claude 4 - blackmail - rare behavior - Anthropic’s new AI model didn’t just “blackmail” researchers in tests — it tried to leak information to news outlets

    1. for - progress trap - Consilience Project - progress trap - Daniel Schmactenberger - from - LinkedIn post - https://hyp.is/twtPIDFaEfC-DCuswNHo5g/www.linkedin.com/posts/danielschmachtenberger_development-in-progress-the-consilience-activity-7222883365371191297-DCPX/?rcm=ACoAACc5MHMBii80wYJJmFqll3Aw-nvAjvI52uI

      summary - Daniel is describing the condition known as a "progress trap",an unintended consequence of progress. - I provide a history of progress traps in my two Medium articles found in the annotation links above. - The idea of progress traps is a very important one, especially for modernity because it is a central and general organizing principle under which many disparate unintended consequences of progress can be subsumed. - As is common in paradigm shifts, it is only when a novel concept of sufficient generalization and relevance is introduced that numerous important problems that could not be explained sufficiently in an older model, can be explained more elegantly from a higher or deeper level perspective - Without recognizing a new category of "progress traps", the many different types of unintended consequences are treated in a fragmented and adhoc way.

    2. This article explains how our current idea of progress is immature: it is developmentally incomplete. Progress, as we define it now, ignores or downplays the scale of its side effects.

      for - progress trap - to - Stop Reset Go hypothesis annotations - progress trap - Ronald Wright - https://jonudell.info/h/facet/?max=100&expanded=true&user=stopresetgo&exactTagSearch=true&any=ronald+wright - General - https://jonudell.info/h/facet/?max=100&expanded=true&user=stopresetgo&exactTagSearch=true&any=progress+trap

    1. new article exploring the narrative of progress, where the current narrative falls catastrophically short, what authentic progress would entail and require, and specific suggestions for how it could be realized.

      for - post - LinkedIn - progress trap - Daniel Schmatenberger - to - article - The Consilience Project - Development in Progress - https://hyp.is/E3HHGDFaEfCZErPpYGIONg/consilienceproject.org/development-in-progress/

  12. Apr 2025
    1. We need technology, much of it to solve the problems that we've created with technology in the first place. But since our ignorance is so vast, how could we possibly develop new technologies that wouldn't in turn create more technologies-- more difficulties that we hadn't anticipated? And you see it in spades right now.

      for - progress trap - David Suzuki

    2. by the time Rachel Carson published Silent Spring, we knew there were ramifications we didn't-- couldn't have anticipated, the most amazing being biomagnification. When eagles began to disappear in the United States, scientists tracked it down to the fact that DDT sprayed at low concentrations would be amplified up the food chain. So by the time you get to the fatty glands and the fat tissue and the shell glands of birds and the breasts of women, DDT was concentrated hundreds of thousands of times beyond what we had sprayed it at. By the 1960s, women's breast milk was considered too toxic to feed to babies.

      for - progress trap - DDT - David Suzuki

    3. she said is, yeah, you scientists are clever. You can make powerful compounds like DDT, but you don't know enough to anticipate all of the consequences. Because, first of all, the lab is not a replica of the real world. The lab is an artifact, something that has very little to do with the real world out there. In the real world, everything is connected to everything else, and we don't know enough to anticipate the effects of what we do with our powerful technologies.

      for - quote - progress trap - David Suzuki - quote - Indra's net of jewels - David Suzuki

      quote - progress trap - David Suzuki - What she (Rachel Carson) said is, - Yeah, you scientists are clever. You can make powerful compounds like DDT, but you don't know enough to anticipate all of the consequences. - Because the lab is not a replica of the real world. The lab is an artifact, something that has very little to do with the real world out there. - In the real world, everything is connected to everything else, and we don't know enough to anticipate the effects of what we do with our powerful technologies.

  13. Mar 2025
    1. before the internet it was impossible really I mean getting coring people into town halls regularly that would have been a hard thing to do anyway online made a bit easier but now with aii we can actually all engage with each other AI can be used to harvest the opinions of millions of people at the same time and distill those opinions into a consensus that might be agreeable to the vast majority

      for - claim - AI for a new type of democracy? - progress trap - AI - future democracy

  14. Feb 2025
  15. Jan 2025
  16. Dec 2024
    1. "If he [Musk] is concerned about competitors getting there first, it doesn't matter as uncontrolled superintelligence is equally bad, no matter who makes it come into existence."

      for - quote - Response to Elon Musk - competition is moot - whoever creates superintelligence first week also create the progress trap that comes along with it - Roman Yampolskiy

      quote - Response to Elon Musk - competition is moot - whoever creates superintelligence first week also create the progress trap that comes along with it - Roman Yampolskiy

      • If he [Musk] is concerned about competitors getting there first,
        • it doesn't matter as uncontrolled superintelligence is equally bad, no matter who makes it come into existence.
    1. A predicament is a situation where, as you try to solve problems, you lack a holistic understanding because you’ve been reductionistic and ignored parts of the interdependencies and you end up creating other problems. The attempt to create a solution creates other problems because of the interdependencies.

      for - adjacency - systemic intedependent web - predicament - progress trap - Joe Brewer - progress trap - climate crisis solutions

  17. Nov 2024
    1. for - AI - progress trap - interview Eric Schmidt - meme - AI progress trap - high intelligence + low compassion = existential threat

      Summary - After watching the interview, I would sum it up this way. Humanity faces an existential threat from AI due to: - AI is extreme concentration of power and intelligence (NOT wisdom!) - Humanity still have many traumatized people who want to harm others - low compassion - The deadly combination is: - proliferation of tools that give anyone extreme concentration of power and intelligence combined with - a sufficiently high percentage of traumatized people with - low levels of compassion and - high levels of unlimited aggression - All it takes is ONE bad actor with the right combination of circumstances and conditions to wreak harm on a global scale, and that will not be prevented by millions of good applications of the same technology

    1. the point is that this is a collective problem that can only be solved collectively. And clearly there is no collective, even worse

      for - post comment - LinkedIn - polarization - Trump 2024 win - lack of collective - adjacency - Deep Humanity - deep time, species-wide singularity - conservativism vs progressiveness - progress - political polarization - progress trap

      adjacency between - Trump 2024 win - Deep Humanity - anthropocene as deep time species-wide singularity - progress traps reaching a climax - conservatism vs progressiveness - adjacency relationship - This fits into a Deep Humanity explanation: - We are moving through a deep time, species singularity in which - once isolated pockets of cultural seeking and interpretative systems for explaining reality have been rapidly mashed-up via: - communication and - transportation technology - There is a singularity now where two forces are battling each other: - conservative that values old traditional cultural values and norms and - progressive that values the future possibilities - There are different cultural flavors of this. Whether it is - political polarization that pits authoritarian vs democratic ideologies or - climate change that pits traditional fossil fuel systems vs new renewable energy systems - the way we've always done things is in conflict with new ways of doing things through natural human evolutionary change - progress - In fact, we can look at the deep time, species-wide singularity that is now happening across all fields in the anthropocene as a predictable progress trap arising from progress itself

    1. we haven't even got to a planetary place yet really and we're about to unleash Galactic level technology you know what I'm saying like so we have a we have a lot of catchup that needs to happen in a very short period of time

      for - quote - progress trap - AI - developed by unwise humans - John Churchill

      quote - progress trap - AI - developed by unwise humans - John Churchill - (See below) - We haven't even got to a planetary place yet really - and we're about to unleash Galactic level technology - So we have a we have a lot of catchup that needs to happen in a very short period of time

  18. Oct 2024
  19. Sep 2024
    1. nobody told it what to do that's that's the kind of really amazing and frightening thing about these situations when Facebook gave uh the algorithm the uh uh aim of increased user engagement the managers of Facebook did not anticipate that it will do it by spreading hatefield conspiracy theories this is something the algorithm discovered by itself the same with the capture puzzle and this is the big problem we are facing with AI

      for - AI - progress trap - example - Facebook AI algorithm - target - increase user engagement - by spreading hateful conspiracy theories - AI did this autonomously - no morality - Yuval Noah Harari story

    2. when a open AI developed a gp4 and they wanted to test what this new AI can do they gave it the task of solving capture puzzles it's these puzzles you encounter online when you try to access a website and the website needs to decide whether you're a human or a robot now uh gp4 could not solve the capture but it accessed a website task rabbit where you can hire people online to do things for you and it wanted to hire a human worker to solve the capture puzzle

      for - AI - progress trap - example - no morality - Open AI - GPT4 - could not solve captcha - so hired human at Task Rabbit to solve - Yuval Noah Harari story

  20. Aug 2024
    1. there's great risk here there's people turning into gurus there's you know weird cult formations there's exploitation there's money pumping uh you have to do a lot you have to try to build a lot in to safeguard against this

      for - progress trap - meaning crisis intervention practices

      progress trap - meaning crisis intervention practices - JV recognizes the potential progress traps of this potential intervention

    1. Degradation ofecosystem services could be significantly slowed down or even reversed if the role ofbiodiversity and its full contribution to economic production were an integrated part ofdecisions made by governmental entities, companies, and other stakeholders (Paul et al2020)20

      for - biodiversity - impact of monoculture diet

      biodiversity - impact of monoculture diet - FAO study done before 2000 and often cited shows that 75% of the global diet comes from 12 plant and 5 animal food sources

      to - stats - progress trap - monoculture - table of 12 plant and 5 animal species that make up 75% of world's diet - https://hyp.is/iznepFWoEe-umbNyOGVqrg/thefuturemarket.com/biodiversity

    1. World Economic Forum, we're working very closely. They're also integrating planetary boundaries in,  their global economy kind of policy agenda

      for - World economic forum - integration planetary boundaries into their strategy

      Concern - unintended consequence - The WEF is perceived by many to be an elitist organisation - who do not have the best interest off the people in mind - This could lead to potential reputational damage to the planetary boundary framework thru their association with it

  21. Jul 2024
    1. If a baby born today and a baby born 30,000 years ago were swapped at birth, they would each grow up as normal people in their new cultures.

      for - similar to - quote - Ronald Wright - progress trap - computer metaphor

      similar to - quote - Ronald Wright - progress trap - computer metaphor - Ronald Wright's famous quote on the computer metaphor really gets to the essence of things - how much of the meta-poly-perma-crisis can be explained by the unprecedented mismatch between the rate of - biological evolution of our species - cultural evolution of our species - Culture is the major and possibly most signficant differentiator between the person alive 50,000 years ago and the one alive today.

      reference - quote - Ronald Wright - computer metaphor - https://hyp.is/go?url=https%3A%2F%2Fwww.goodreads.com%2Fwork%2Fquotes%2F321797-a-short-history-of-progress&group=world

    1. 26:30 Brings up progress traps of this new technology

      26:48

      question How do we shift our (human being's) relationship with the rest of nature

      27:00

      metaphor - interspecies communications - AI can be compared to a new scientific instrument that extends our ability to see - We may discover that humanity is not the center of the universe

      32:54

      Question - Dr Doolittle question - Will we be able to talk to the animals? - Wittgenstein said no - Human Umwelt is different from others - but it may very well happen

      34:54

      species have culture - Marine mammals enact behavior similar to humans

      • Unknown unknowns will likely move to known unknowns and to some known knowns

      36:29

      citizen science bioacoustic projects - audio moth - sound invisible to humans - ultrasonic sound - intrasonic sound - example - Amazonian river turtles have been found to have hundreds of unique vocalizations to call their baby turtles to safety out in the ocean

      41:56

      ocean habitat for whales - they can communicate across the entire ocean of the earth - They tell of a story of a whale in Bermuda can communicate with a whale in Ireland

      43:00

      progress trap - AI for interspecies communications - examples - examples - poachers or eco tourism can misuse

      44:08

      progress trap - AI for interspecies communications - policy

      45:16

      whale protection technology - Kim Davies - University of New Brunswick - aquatic drones - drones triangulate whales - ships must not get near 1,000 km of whales to avoid collision - Canadian government fines are up to 250,000 dollars for violating

      50:35

      environmental regulation - overhaul for the next century - instead of - treatment, we now have the data tools for - prevention

      56:40 - ecological relationship - pollinators and plants have co-evolved

      1:00:26

      AI for interspecies communication - example - human cultural evolution controlling evolution of life on earth

    1. for - progress trap - AI -

      article details - title - Hollow, world! (Part 1 of 5) - author - James Allen - date - 10 July, 2024 - publication - substack - self link - https://allenj.substack.com/p/hollow-world-part-1-of-5

      summary James Allen provides an insightful description of ultra-anthropomorphic AI, AI that attempts to simulate an entire, whole human being.

      In short, he points out the fundamental distinction between the real experience of another human being, and a simulation of one. In so doing, he gets to the heart of what it is to be human.

      An AI is a simulation of a human being. No matter how realistic it's responses and actions, it is not evolved out of biology. I have no doubts that scientists are hard at work trying to make a biological AI. The distinction becomes fuzzier then.

      Current AI cannot possibly simulate the experience of being in a fragile and mortal body and all that this entails. If an AI robot says it understands joy or pain, that statement isn't built on the combined exteroception and interoception of being in a biological body, rather, it is based on many linguistic statements it has assimilated.

    1. I don't really understand what they think uh what it is um if it's not you know 00:45:20 how resources are allocated and um the the transformation of commod you know raw material into finished goods and stuff all that takes energy it all takes material

      for - progress trap - real dangers from - abstraction and siloing

      progress trap - real dangers from - abstraction and siloing - Business processes create workers who live in abstract, symbolic worlds, never seeing the consequences of their symbolic manipulations - At the end of the day, the abstract, symbolic finance industry worker gets a fat salary and lives comfortably, whilist playing with abstractions of processes they are contributing to which they have no sensory information on - the separation of producer from consumer is yet another huge abstraction that cleaves the gestalt into pieces that we cannot see - ANTIDOTE to this - de-abstraction - re-synthesize - Processes have been fragmented and split apart - We need to find ways for people to re-synthesize and assemble the pieces back together again in order to - see and experience the whole picture

    1. in the past these collapses of civilizations were local and people could migrate a little further on and rebuild but the chances of of that are gone now i mean we have to we have to uh to to 01:03:18 uh get right with what we have because it's all we have you know we we all all those bets we placed when our ancestors invented civilization they all rest on one high stakes throw which is 01:03:32 now

      for - progress trap - modernity can't run away anywhere from its ruins

    2. does agriculture become a 00:51:16 progress trap has it been a progress trap well i think i think in a sense it has

      for - progress trap - Agriculture appears to be a progress trap - Ronald Wright

      argument - progress trap - Agriculture appears to be a progress trap - Ronald Wright - The early progress traps have been comparatively small in scale - During the stone age, there were no more than a few million people alive (3 to 5 million?) and - they destroyed all the big game where humans lived - The Sumerians, with a population of around one million people salted up and destroyed the fertile land of Southern Iraq - Now we have 8 billion people and a third of them are starving - Our continuous technology development is what enables us to stave off the day of reckoning but - we are losing a Scotland-size worth of topsoil every year to - soil erosion - urban sprawl - We still face the possibility of collapse - Our species has existed for 5 to 6 million years and - civilization is an experiment that has only emerged about 10,000 years ago - It's still very possible for the experiment to fail

    3. we've achieved a level of prosperity for a huge number of people that was not typical of the past i mean most countries have a big middle class as well as an extremely wealthy upper 00:50:00 class but the number of people in abject poverty in the world today living on less than two dollars a day is greater than the entire population of the world 00:50:13 only 100 years ago so that's not progress

      for - statistics - progress trap - comparative levels of poverty

      statistics - progress trap - comparative levels of poverty - modern civilization has - a huge middle class - a small elite class - a huge impoverished class - The absolute number of people living on less than 2 dollars a day is less than the entire population of humans only 100 years ago

    4. that's part of the logic of agriculture isn't it i mean you have a lot of work to do yeah a lot of people you know but there again you're you're in a progress trap or or a treadmill that you need more children 00:48:57 so you can work more land and then that more land provides more food so you have yet more children

      for - progress trap - the agricultural-large family positive feedback loop

      progress trap - the agricultural-large family positive feedback loop - Interesting to compare modern vs agricultural societies - Populations are dropping in most western countries around the contemporary world, yet - traditional agricultural societies had large families to tend to large amount of agricultural work - There is a progress trap potential with encouraging many large families with a limited land resource: - If you have larger families, you can cultivate more land - If you cultivate more land, you can have even larger family - until you reach a point when the land has been exhausted and you are now forced to reduce the population

    5. do we have 00:46:13 examples of civilizations um that really accepted limits

      for - progress trap - cultures that avoided progress trap of population explosion - Tahiti - via infanticide

      progress trap - cultures that avoided progress trap of population explosion - Tahiti - via infanticide

      • Tahitians practiced population control via
        • infanticide
          • It was ok to kill a newborn baby before it drew its first breath, as it was not considered a person until it drew the first breath
        • advanced Eroticism - separating sexual activity from reproduction
    6. he myth that progress will go on forever and will everything will get bigger and better and we don't have to control our numbers or demand on nature because we will always find a way to make it work we have to break out of that 00:45:21 mindset

      for - progress trap - dangerous mythology

      progress trap - dangerous mythology - We subscribe to a dangerous myth that: - progress will go on forever and - everything will get bigger and better and - we don't have to control our numbers - or our demand on nature - because we will always find a way to make it work

      • We have to break out of that mindset
    7. they feel incredibly resentful that they have not benefited from the the wealth generated 00:38:07 from by this uh system that was once uh uh promised them so much uh and so i i i think you know there and that's just one example among many uh in in the less prosperous parts of the world you could 00:38:21 you'd see many more

      for - progress trap - inequality - resentment

      progress trap - inequality - resentment - rapid emergence of the far-right and populus - Indeed we see so much resentment everywhere. For example, the far-right and populus could only emerge so rapidly because of such resentment of being left behind.

    8. if we fail to control our numbers and our appetites well then yes our society will start to to crash in a similar way to that of 00:35:32 easter island only on a worldwide scale and that means the whole industrial civilization will break down and 00:35:45 our descendants will essentially be uh savages to use that term very advisably and savages in the sense that they will have lost 00:35:58 the fruits of civilization and hate us

      for - progress trap - dark futures scenario - like Easter Island but on a global scale

      comment - The potential global breakdown of global industrialized society, rupturing supply chains so that our highly interdependent world becomes the very Achilles Heel that hastens its demise is chilling - It could mean a huge disruption to the most important aspect of civilization - the continuing accruing and inter-generational transmission of knowledge - It would be catastrophic to lose that, but it is entirely possible - As Wright himself famously said, to use a computer metaphor, we humans are like 50,000 year old hardware, running modern software - By that, he meant that our cognitive physiology (brain and sensory processing system) has not changed for tens of thousands of years, yet cultural evolution happens at exponentially faster rates, so much so that our biological systems are not adapted to keep up with the pace, and that spells disaster - When we no longer have the sensory or cognitive apparatus to sense danger, and we are offloading that to AI, we are in an extremely vulnerable situation

      progress trap - Gedanken - Think of our ancestors from 50,000 years ago. - What Wright is saying with his metaphor is that if that child from 50,000 years ago were transported by a time machine to modernity, (s)he would have little problem integrating into modern society - LIKEWISE, if we lose all the knowledge fruits of accumulated over so many thousands of years, it would be like being born into a human tribe 50,000 years ago. - We would likely still have language, but all our technology may have to start from scratch!

    9. it's a very tough one for a lot of people to make because nobody wants to think there are limits like that 00:32:14 nobody wants to think that there's not going to be enough to go around

      for - progress trap - habits of thinking - limits - liberty

      progress trap - habits of thinking - limits - liberty - Technology has provided us with ever increasing types and scales of liberty - We cannot imagine taking away those liberties!

    10. one of the things i suggested in a short history of progress is that 00:30:18 one of our problems even though we're very clever as a species we're not wise

      for - key insight - progress trap - A Short History of Progress - we are clever but NOT wise!

      key insight - progress trap - A Short History of Progress - we are clever but NOT wise! - In other words - Intelligence is FAR DIFFERENT than wisdom

      new memes - We have an abundance of intelligence and a dearth of wisdom - A little knowledge is dangerous, a lot of knowledge is even more dangerous

    11. easter island

      for - progress trap - archeology - Easter Island

      progress trap - archeology - Easter Island - The archeology tells the fascinating story of the progress trap that Eastern Island ancestors created. - Easter Island is desolate and only 64 square miles total - It is located 2,000 miles (3218 km) from the nearest landmass of South America - Archeologists trying to piece together the stone monoliths of Easter Island found pollen samples in the crater lakes on the island, proving that the place was once a thriving forest - By the time the first Europeans landed (the Dutch) on Easter Island, it was just grassy hills - How did it go from a lush forest land to a grass land? - The layers of pollen samples told a story - The aboriginal polynesians that the Dutch explorers encountered arrived about 1,000 years earlier - They cut down the forest, grew their population and used trees in many ways, including to transport the huge stone monoliths that paid respect to their ancestors - The original people multiplied then separated into separate warring tribes - The environment was devastated as the logging destroyed the forest and all the abundant ecosystem that provided for their sustenance - This created severe erosion and the soil became impoverished - The tribes collapses into warfare and cannibalism

      progress trap - Easter Island - Sumeria etc - lesson - don't romanticize our ancestors - Human groups have continuously sabotaged themselves through overexploitation and lack of foresight - Progress traps have been a constant part of our species for a long, long time

    12. example is weaponry

      for - progress trap - example - weapons leading to nuclear weapons

      progress trap - example - weapons leading to nuclear weapons - These are the most ironic inventions of civilization - We spend significant percentages of our budgets maintaining and escalating them, meanwhile, we all know we cannot use them as it would mean billions would die

      quote progress trap - nuclear weapons - When you're talking nuclear weapons that can never be used you're investing in something that's completely useless - that you're maybe burying in the ground in the form of missile silos or - you're putting into submarines or - into aircraft or missiles - You can never use these things and they are draining off the surplus that might otherwise be used into - wealth redistribution and into - long-term sustainability

    13. the progress trap and the pyramid scheme are not all that different

      for - adjacency - progress trap - pyramid scheme

      adjacency - between - progress trap - pyramid scheme - adjacency relationship - the pyramid sales scheme works as long as you can - continue to draw in new investors and new buyers - you concentrate the wealth upwards - you're not actually making real wealth - you're simply taking the wealth from the rubes who are coming in, in order to pay off the people who invested earlier to make it seem like it's a sustainable enterprise when it isn't (ponzi scheme) - Unfortunately, civilizations in their relationship to nature seem to follow that pattern - They expand the base and then when the base can't be expanded anymore - then all the environmental problems that have been hidden by constant expansion suddenly come back and hit them in the face - And those environmental problems of course cause social problems - They very often cause revolutions and wars because - suddenly people realize the pie is not getting bigger so therefore they become discontented with the small slice that they're getting, - unless they're people at the top of the pyramid - So it's a pyramid in the sense of a social pyramid too

    14. the really big progress traps uh come with with the invention of 00:07:20 agriculture and i i mentioned the first full-blown civilization in the old world the sumerians who perfected the art of irrigation 00:07:33 in what is now southern iraq

      for - progress trap - example - from history - Sumerian civilization

      progress trap - example - from history - Sumerian civilization - the really big progress traps uh come with with the invention of agriculture and - i mentioned the first full-blown civilization in the old world the sumerians who perfected the art of irrigation in what is now southern iraq and - for for several centuries everything went really well - They had built canals and ran the water onto the desert and were able to - raise more and more crops and - expand their farmland and - expand their population and - their cities got bigger - their numbers got greater but - what they didn't know is that the kind of irrigation they were practicing - was causing the land to get saltier and saltier - and after a number of centuries they suddenly saw their farm meals declining because of salinity - and they had to switch to crops that could tolerate more salt - and then eventually they ended up producing only about one quarter of the food that they'd been able to produce when they started -and the civilization collapsed - So they had walked into what i call in my book a progress trap - and this is where the myth of progress is so seductive - You do something that in the short run produces obvious benefits so you're getting this positive feedback from some new invention, whether it's - a new way to drive mammoth over a cliff or - it's a new way to expand your farm base through irrigation - but there's a hidden cost down the road which is often hard to foresee

    15. myths are not necessarily untrue they're usually 00:03:33 partly true the danger lies in the part that isn't true and um so it it's partly true we have

      for - quote - myths - Ronald Wright - adjacency - myths - perspectival knowing - emptiness - progress trap

      Quote - Myths - Ronald Wright - (see below) - Myths are not necessarily untrue. They're usually partly true. The danger lies in the part that isn't true.

      Comment. - What a great little sentence! - From this perspective, so many things that people claim as "true" are actually myths.

      adjacency - between - myths - progress traps - perspectival knowing - emptiness - adjacency relationship - Myths emerge out of perspectival knowing of reality (Vervaeke) - The emptiness of reality is in stark contrast to reductionist thinking which is always relatively incomplete in comparison - This leads to the emergence of progress traps

    Tags

    Annotators

    URL

    1. If we are having narrowly  defined goals, as we discussed last time,   that can be achieved while externalizing harm  in other places, and we do a lot of that,   and we look at all of the goal achieving and not  all the externalities, we can call that progress.

      for - progress trap - achieving narrowly defined goals with externalities

      progress trap - achieving narrowly defined goals with externalities - Within the field of progress traps, - progress is the focus on intended consequences and - progress traps are the unintended consequences reference - Ronald Wright - https://jonudell.info/h/facet/?max=100&expanded=true&user=stopresetgo&exactTagSearch=true&any=Ronald+Wright

    1. stat in use more than 10 years increases your risk of developing insulin resistance and type 2 diabetes which are two primary drivers of heart 00:26:07 disease

      for - progress trap - statins

      progress trap - statins - heart health - Long term studies show that the mainstream prescription of statins to lower LDL levels over 10 year period increases risk for: - insulin resistance and - type 2 diabetes - both of which are primary drivers of heart disease - hence, Statins are a progress trap

  22. Jun 2024
    1. a dictator who wields the power of superintelligence would command concentrated power unlike 00:50:45 anything we've ever seen

      for - key insight - AI - progress trap - nightmare scenario - dictator controlling superintelligence

      meet insight - AI - progress trap - nightmare scenario - locked in dictatorship controlling superintelligence - millions of AI controlled robotic law and enforcement agents could police their populace - Mass surveillance would be hypercharged - Dictator loyal AI agents could individually assess every single citizen for descent with near perfect lie detection sensor - rooting out any disloyalty e - Essentially - the robotic military and police force could be wholly controlled by a single political leader and - programmed to be perfectly obedient and there's going to be no risks of coups or rebellions and - his strategy is going to be perfect because he has super intelligence behind them - what does a look like when we have super intelligence in control by a dictator ? - there's simply no version of that where you escape literally - past dictatorships were not permanent but - superintelligence could eliminate any historical threat to a dictator's Rule and - lock in their power - If you believe in freedom and democracy this is an issue because - someone in power, - even if they're good - they could still stay in power - but you still need the freedom and democracy to be able to choose - This is why the Free World must Prevail so - there is so much at stake here that - This is why everyone is not taking this into account

    2. this is why it's such a trap which is why like we're on this train barreling down this pathway which is super risky

      for - progress trap - double bind - AI - ubiquity

      progress trap - double bind - AI - ubiquity - Rationale: we will have to equip many systems with AI - including military systems - Already connected to the internet - AI will be embedded in every critical piece of infrastructure in the future - What happens if something goes wrong? - Now there is an alignment failure everywhere - We will potentially have superintelligence within 3 years - Alignment failures will become catastrophic with them

    3. getting a base model to you know make money by default it may well learn to lie to commit fraud to deceive to hack to seek power because 00:47:50 in the real world people actually use this to make money

      for - progress trap - AI - example - give prompt for AI to earn money

      progress trap - AI - example - instruct AI to earn money - Getting a base model to make money. By default it may well learn - to lie - to commit fraud - to deceive - to hack - to seek power - because in the real world - people actually use this to make money - even maybe they'll learn to - behave nicely when humans are looking and then - pursue more nefarious strategies when we aren't watching

    4. whoever controls superintelligence will possibly have enough power to seize control from 00:35:14 pre superintelligence forces

      for - progress trap - AI - one nightmare scenario

      progress trap - AI - one nightmare scenario - Whoever is the first to control superintelligence will possibly have enough power to - seize control from pre superintelligence forces - even without the robots small civilization of superintelligence would be able to - hack any undefended military election television system and cunningly persuade generals electoral and economically out compete nation states - design new synthetic bioweapons and then - pay a human in Bitcoin to synthetically synthesize it

    5. military power and Technology progress have been tightly linked historically and with extraordinarily rapid technological 00:34:11 progress will come military revolutions

      for - progress trap - AI and even more powerful weapons of destruction

      progress trap - AI and even more powerful weapons of destruction - The podcaster's excitement seems to overshadow any concern of the tragic unintended consequences of weapons even more powerful than nuclear warheads. - With human base emotions still stuck in the past and our species continued reliance on violence to solve problems, more powerful weapons is not the solution, - indeed, they only make the problem worse - Here is where Ronald Wright's quote is so apt: - We humans are running modern software on 50,000 year old hardware systems - Our cultural evolution, of which AI is a part of, is happening so quickly, that - it is racing ahead of our biological evolution - We aren't able to adapt fast enough for the rapid cultural changes that AI is going to create, and it may very well destroy us

    6. this is where we can see the doubling time of the global economy in years from 1903 it's been 15 years but after super intelligence what happens is it going to be every 3 years is it going be every five is it going to 00:33:22 be every year is it going to be every 6 months I mean how crazy is the growth going to be

      for - progress trap - AI triggering massive economic growth - planetary boundaries

      progress trap - AI triggering massive economic growth - planetary boundaries - The podcaster does not consider the ramifications of the potential disastrous impact of such economic growth if not managed properly

    7. AGI level factories are going to shift from going to human run to AI directed using human physical labor soon to be fully being run by swarms of human level robots

      for - progress trap - AI and human enslavement?

      progress trap - human enslavement? - Isn't what the speaker is talking about here is that - AI will be the masters and - humans will become slaves?

    8. super intelligence is going to be like this across many domains it's going to be 00:31:42 able to find exploits in human code too subtle for humans to notice and it's going to be able to generate code too complicated for any human to understand even if the model spent decades trying to explain it

      for - progress trap - superintelligence threat

      progress trap - superintelligence threat - super intelligence is going to be far beyond our cognitive capabilities across many domains. For example, - it's going to be able to find exploits in human code too subtle for humans to notice - it's going to be able to generate code too complicated for any human to understand - even if the model spent decades trying to explain it - How do we entrust ourselves to a superintelligence that is so far beyond us? If it thinks we are expendable, it could easily find our weaknesses and bring about extinction

    9. nobody's really pricing this in

      for - progress trap - debate - nobody is discussing the dangers of such a project!

      progress trap - debate - nobody is discussing the dangers of such a project! - Civlization's journey has to create more and more powerful tools for human beings to use - but this tool is different because it can act autonomously - It can solve problems that will dwarf our individual or even group ability to solve - Philosophically, the problem / solution paradigm becomes a central question because, - As presented in Deep Humanity praxis, - humans have never stopped producing progress traps as shadow sides of technology because - the reductionist problem solving approach always reaches conclusions based on finite amount of knowledge of the relationships of any one particular area of focus - in contrast to the infinite, fractal relationships found at every scale of nature - Supercomputing can never bridge the gap between finite and infinite - A superintelligent artifact with that autonomy of pattern recognition may recognize a pattern in which humans are not efficient and in fact, greater efficiency gains can be had by eliminating us

    10. having an automated AI research engineer by 2027 00:05:14 to 2028 is not something that is far far off

      for - progress trap - AI - milestone - automated AI researcher

      progress trap - AI - milestone - automated AI researcher - This is a serious concern that must be debated - An AI researcher that does research on itself has no moral compass and can encode undecipherable code into future generations of AI that provides no back door to AI if something goes wrong. - For instance, if AI reached the conclusion that humans need to be eliminated in order to save the biosphere, - it can disseminate its strategies covertly under secret communications with unbreakable code

    1. let's face it if you create a traumatized child you then have 00:44:39 to have the capacity for dealing with the traumatized child so it's not just out of benevolence to him but also that it's a lot easier to be in relationship 00:44:51 with someone for the rest of your life that a that you haven't damaged

      for - progress trap - parenting - traumatizing our children - Nora Bateson - quote - Nora Bateson - progress trap - traumatizing our children

      quote - Nora Bateson - progress trap - traumatizing our children (see below) - Let's face it, - If you create a traumatized child - you then have to have the capacity - for dealing with the traumatized child - So it's not just out of benevolence to him but also that - it's a lot easier to be in relationship with someone for the rest of your life that - you haven't damaged

    2. the solution to the consequence is likely to perpetuate the actual problem

      for - quote - progress trap - Nora Bateson - book - Combining

      quote - progress trap - Nora Bateson - book - Combining - (see below)

      • In the singularity of its mission to hastily fix one malady at a time
        • the cure may be more harmful than the wound
      • Most identified problems as they have emerged
        • are really the consequence or symptoms of other conditions
      • The solution to the consequence is likely to perpetuate the actual problem