Correspondence concerning this article should be addressed to Iza Ray Korsmit (iza.korsmit@mail.mcgill.ca).

The experimental protocol was certified for ethical compliance by the McGill University Research Ethics Board II.

IRK was supported by funding from the Prins Bernhard Cultuurfonds (The Netherlands). This project was also funded by a Canadian Social Sciences and Humanities Research Council Insight Grant (435-2021-0224), a Social Sciences and Humanities Research Council Partnership Grant (895-2018-1023), and a Canada Research Chair (950-231872) to SMc.

Part of this research was presented at the Society for Music Perception and Cognition Conference, Portland, Oregon (2022). The authors would like to thank Bennett K. Smith for programming the experimental interface and assisting with the experiment execution on Prolific, and Philippe Macnab-Seguin for creating the chromatic scales for the second experiment.

Iza Ray Korsmit, Marcel Montrey, Alix Yok Tin Wong-Min, & Stephen McAdams McGill University, Montreal, Canada

Grimaud and Eerola (2022) compared instrument ensembles of strings, woodwinds, and brass in a study where participants either rated the emotions they perceived or manipulated musical parameters to produce a certain emotion. They found that strings were associated with increased anger and fear, woodwinds with decreased anger and fear, and brass with decreased fear, in the cases of both emotion perception and production. For the other emotions (joy, sadness, calmness, power, surprise), however, results were less consistent between perception and production, indicating that the emotion-instrument association may also depend on context of the task.

following the constructionist idea that an individual's personality and background will influence the affect they perceive or feel, we considered several sources of individual differences as moderating factors in the effects of instrument family, pitch height, and affect locus.

This research follows a constructionist approach to musical affect (Cespedes-Guevara & Eerola, 2018). That is, although we are interested in the "bottom-up" influence of certain musical features on musical affect, we believe these cannot be adequately evaluated without considering the "top-down" effects of context and individual differences that are present when affects are constructed. The perception or induction of affect does not merely arise in response to a stimulus but is also formed in relation to the individual and the context.

the context of a task (like perception, production, or induction) may change the effect of musical features.

Furthermore, as the method of reporting on perceived and induced affect may influence the construction of an affect (e.g., facilitating categorical perception), we also compare two different affect representations.

This research follows a constructionist approach to musical affect (Cespedes-Guevara & Eerola, 2018). That is, although we are interested in the \'bottom-up\' influence of certain musical features on musical affect, we believe these cannot be adequately evaluated without considering the \'top-down\' effects of context and individual differences that are present when affects are constructed. The perception or induction of affect does not merely arise in response to a stimulus but is also formed in relation to the individual and the context.

Cognitive surrenderA paper that came out this year asked: if you’re working with AI a lot, and you’re using it as a machine to answer all of your questions, what happens with System 1 and System 2?

2. Validator Another basic role for AI is validating your understanding. To do this, you ask it to review your notes for errors or gaps, do basic fact checking, or critique your reasoning. Again, you can do this via the chat interface, but I also experimented with passing my notes in Obsidian using the Copilot plugin and in Emacs using gptel. Example: After reading The Epic of Gilgamesh, I wrote a note in Obsidian summarizing its plot. When I asked ChatGPT to critique my summary, it pointed out that I’d given the central character a redemption arc that isn’t present in the text. I’m so accustomed to the standard hero’s journey, that I projected it onto the book — and an LLM helped me correct this ‘hallucination.’ Suggested prompt: Here are my notes on [WORK]. What important ideas did I miss or underemphasize? Don’t rewrite my notes — just flag the gaps.

For the record, my posts aren’t written or conceived with an LLM, although I know an increasing number of people who use one to write a first draft and then edit. I’m not a fan. The whole point of the web — its beauty — is that it’s unrelentingly human and diverse.

Given the high prevalence of such sounds in everyday life, having misophonia can have large negative effects on one's functioning in personal, academic, and work environments.

Although there are many idiosyncrasies in what may trigger a person with misophonia, the most common triggers are created by other humans, such as the sound of someone chewing, clearing their throat, tapping their foot, or typing on a keyboard.

Misophonia is a psychological disorder that is characterized by severe aversive responses to specific environmental sounds (i.e., triggers).

This indicates that misophonia is not a purely auditory processing disorder but is also influenced by a top-down process of source identification.

an fMRI study found that people with misophonia show increased response in the anterior insular cortex (AIC) in response to misophonic sounds, compared to control participants and other unpleasant or neutral sounds (Kumar et al., 2017).

Both the subjective judgment of aversiveness and the physiological measure of skin conductance response (SCR) increase when people with misophonia are presented with triggers (Edelstein et al., 2013).

The disorder is not yet recognized by the Diagnostic and Statistical Manual − 5th version (DSM-5; American Psychiatric Association, 2013), but there has been an increasing amount of research on the characterization and treatment of misophonia (Vitoratou et al., 2021; see also Brout et al., 2018, for a review).

Composers and music researchers had previously analyzed and annotated 65 movements from the Classical, Romantic, and early Modern repertoire in terms of the Taxonomy of Orchestral Grouping Effects (McAdams et al., 2022).

In a study by McAdams and Goodchild (2018), orchestral simulations created with OrchSim were compared perceptually to commercial recordings and were shown to be of high quality.

These results confirm with orchestral excerpts the findings of studies on isolated tones with dyads or triads of instruments in which the presence of impulsive instruments reduces the perception of blend (Lembke et al., 2019; Reuter, 1996; Tardieu & McAdams, 2012).

This provides additional support for its significant contribution to blend in Fischer et al. (2021).

Lembke et al. (2017) demonstrated that combinations of sustained and impulsive instruments blend less well.

structuring by affecting sequential grouping through the segregation of auditory streams played by different instruments and segmental grouping through timbral contrasts (McAdams et al., 2022).

Several other spectral and spectrotemporal descriptors were found to play a role in blend perception in orchestral works by Fischer et al. (2021). These include spectral flatness and spectral crest (different measures of the degree to which the spectrum is denser or has more emergence of spectral components), and spectral variation (the degree of variation of the spectral shape over time).

Fischer et al. (2021) studied the blends of multi-instrument streams in the context of orchestral stream segregation in predominantly Romantic orchestral excerpts. They found that within-family instrument combinations blended better than between-family combinations. They demonstrated the role played by overlap in timbre correlates of spectral flatness (a measure of the tonalness/noisiness or density of the spectrum), spectral skewness (related to the shape of the spectral envelope), and spectral variation (evolution of the spectral envelope over time), as well as cues derived from the scores such as onset synchrony and the consonance of concurrent pitch relations.

McAdams et al. (2022) distinguish two subcategories of these two types of blend: stable and transforming.

Lembke, S.-A., Parker, K., Narmour, E., & McAdams, S. (2019). Acoustical correlates of perceptual blend in timbre dyads and triads. Musicae Scientiae, 23(2), 250–274.

Several other spectral and spectrotemporal descriptors were found to play a role in blend perception in orchestral works by Fischer et al. (2021).

Lembke and McAdams (2015) found that the degree of spectral overlap between constituent sounds enhanced blend perception.

Tardieu and McAdams (2012) extended this work with combinations of unison sustained and impulsive instruments (including pitched percussion and string pizzicati).

McAdams et al. (2022) introduce other notions related to blend as well.

This in turn creates an effect of perceptual unity (McAdams, 1989).

Four important concurrent grouping cues predict the perceptual fusion of sound events (McAdams, 1984): onset synchrony, harmonicity, coherent frequency behavior, and common spatial origin.

To this, McAdams et al. (2022) have added segmental grouping (chunking into perceptual units).

When the sudden drop to a pianissimo occurred towards the ending of the piece, the perceived arousal responses of CHM and WM dropped slightly but rose again immediately to end on a high arousal. These two groups of listeners appear to have anticipated a return to a loud and majestic close and therefore kept their arousal responses higher than those of the NM.

CHM, who are more experienced with the instruments and compositional techniques used in Chinese orchestral music, might have had an idea of which features figure more prominently in the communication of particular intentions, and therefore would have more information available for their judgments.

The perception of affective intentions in music is influenced by the degree of familiarity listeners have with a musical tradition, the content implicated in the music, and the complex sonic environment created by the composer's creation and the musicians' interpretation.

The version that participants heard was a premier of the work by the Taipei Chinese orchestra.

The communication of emotions or affect takes place when listeners perceive emotional meaning that is expressed by performers in music (Juslin, 2013a, 2013b).

An understanding of a tonal schema with its associations to happiness and sadness has been consistently found to influence listeners who have grown up in a culture of Western music.

Its mirmode function estimates the modal strength of the music in terms of ''majorness'' and ''minorness.''

The perception of consonance also plays an important role in the music listening process—combinations of tones that are consonant are perceived as more positively valenced than dissonant ones (Harrison & Pearce, 2020).

Musical structures such as pitch relations are perceptually salient and provide important information for listeners (e.g., Gabrielsson & Lindstrom, 2010; Krumhansl, 1998; Krumhansl & Kessler, 1982).

Iqa' (plural iqa'at) is used to describe a rhythmic cycle. Iqa'at are made up of two different basic building blocks, the dum and tak, onomatopoeias derived from the sound produced on membranophones such as the darabuka.

H5. Being more culturally bound, musical cues that are learned, such as modal structures, metrical relations, and so on, will exert a greater influence on listeners' perceived valence ratings than on their arousal ratings.

This is a simple PDF file. Fun fun fun.

using SemanticCommit, we recorded all instances of edits, check for conflicts, make change, local, and global resolution actions using telemetry.

A sub-task was considered a failure if the participant was unable to complete it within the time limit.

Finally, we conducted an informal interview about their experience.

After both tasks were completed, participants filled out a final survey to compare the two conditions.

We chose GPT-4o for performance and latency reasons, as it performed optimally against our evals.

We also ran evaluations of model latency and classification performance under varying false positive rates for the following LLMs by OpenAI: GPT-4o, GPT-4o-mini, and o3-mini.

For each task, participants were tasked with integrating three new pieces of information into the memory, one at a time ("sub-tasks").

We ensured each list was 30 items long as our pilot studies suggested this was long enough that manual detection starts to become unwieldy (users need to scroll up and down the document), but short enough that participants could become familiar in a short period.

We adapted two intent specifications from our evals: Mars Game Design Document and Financial Advice AI Agent Memory, as these tasks mapped to the two paradigmatic types covered in Sections 2 and 2.1 (design documents, and AI memory of the user).

We recruited 12 participants (7 female, 5 male) through the mailing lists of two research universities and one multinational technology company.

We chose OpenAI's ChatGPT Canvas as a baseline for five reasons: (i) it is a popular, commercially available tool, hence it is likely familiar to users; (ii) it provides a document editing view, where users can select text and ask GPT to rewrite it, or chat with an AI to make global edits; (iii) it employs a similar class of model (GPT-4o); (iv) it supports similar editing features as SemanticCommit like inline text selection, conflict highlighting, and a diff view, while adding free-form editing; and (v) similar interfaces like Anthropic Artifacts tended to rewrite the specification entirely, and did not offer Canvas's "diff" view to allow for a fair comparison.

With participant consent, we recorded audio and screen-casts, and participants were encouraged to think aloud.

Four coauthors created the evals, and two coauthors manually double-checked all conflicts, a process that took several days.

We ran one pilot study with five users of our card-based interface, and a second with four users of a revised interface.

Our explorations went through substantial iterations and prompt prototyping over a period of eight months, evolving in response to two pilot studies and progressing from a card-based interface to a list of texts.

We iterated on prompts using ChainForge [5] by setting up an evaluation pipeline against our datasets, which allowed us to observe the effects of prompt changes and model choices.

To measure statistical significance, we used Mann–Whitney–Wilcoxon tests and report the p-values.

For qualitative analysis, the first author performed open coding on participant responses and audio transcripts to identify themes, which were used to interpret the qualitative results.

In the post-task surveys, we collected self-reported NASA Task Load Index (TLX) scores, Likert-scale ratings for ease of use, and responses on how well the AI helped participants identify, understand, and resolve semantic conflicts.

Each condition had a time limit of 15 minutes, after which the participant completed a post-task survey.

Before each task, participants received a tutorial on the assigned tool and were given five minutes to explore it using a test document.

Both the order of task assignment and tool assignment were counterbalanced and randomly assigned.

We conducted a controlled within-subjects study with mixed methods, comparing SemanticCommit with a baseline interface.

We run end-to-end on our four eval datasets using GPT-4o and GPT-4o-mini and report the mean ± stddev for accuracy, precision, recall, and F1 scores for the three approaches in Figure 5.

We compare our end-to-end system against two simpler methods: (i) DropAllDocs, which adds all documents to the context for conflict classification; and (ii) InkSync [56] which generates a JSON list of string-replace operations.

In order to minimize relevance assessment issues, we apply a PageRank-based relevance ranking over the KG, akin to HippoRAG [36].

We implement the back-end using a knowledge-graph (KG) RAG architecture [36] consisting of two phases: pre-processing and inference.

Through a within-subjects study with 12 participants comparing SemanticCommit to a chat-with-document baseline (OpenAI Canvas), we find differences in workflow: half of our participants adopted a workflow of impact analysis when using SemanticCommit, where they would first flag conflicts without AI revisions then resolve conflicts locally, despite having access to a global revision feature.

We implement the back-end using a knowledge-graph (KG) RAG architecture [36] consisting of two phases: pre-processing and inference.

In order to minimize relevance assessment issues, we apply a PageRank-based relevance ranking over the KG, akin to HippoRAG [36].

We compare our end-to-end system against two simpler methods: (i) DropAllDocs, which adds all documents to the context for conflict classification; and (ii) InkSync [56] which generates a JSON list of string-replace operations.

We run end-to-end on our four eval datasets using GPT-4o and GPT-4o-mini and report the mean ± stddev for accuracy, precision, recall, and F1 scores for the three approaches in Figure 5.

We conducted a controlled within-subjects study with mixed methods, comparing SemanticCommit with a baseline interface.

Both the order of task assignment and tool assignment were counterbalanced and randomly assigned.

Before each task, participants received a tutorial on the assigned tool and were given five minutes to explore it using a test document.

Each condition had a time limit of 15 minutes, after which the participant completed a post-task survey.

In the post-task surveys, we collected self-reported NASA Task Load Index (TLX) scores, Likert-scale ratings for ease of use, and responses on how well the AI helped participants identify, understand, and resolve semantic conflicts.

For qualitative analysis, the first author performed open coding on participant responses and audio transcripts to identify themes, which were used to interpret the qualitative results.

To measure statistical significance, we used Mann–Whitney–Wilcoxon tests and report the p-values.

We iterated on prompts using ChainForge [5] by setting up an evaluation pipeline against our datasets, which allowed us to observe the effects of prompt changes and model choices.

Our explorations went through substantial iterations and prompt prototyping over a period of eight months, evolving in response to two pilot studies and progressing from a card-based interface to a list of texts.

We ran one pilot study with five users of our card-based interface, and a second with four users of a revised interface.

Four coauthors created the evals, and two coauthors manually double-checked all conflicts, a process that took several days.

With participant consent, we recorded audio and screen-casts, and participants were encouraged to think aloud.

These semantic conflicts require dedicated support to detect, visualize, and resolve. Semantic conflict resolution interfaces must go beyond visualizing what changes were made, to what changes could be made, where they should be made, and what the effects might be. This resembles feedforward: affordances that help the user foresee the impact of an action [67, 93].

Today with LLMs, we are less limited by this constraint, and solutions to the problem of human-machine communication might be better found in cybernetics theory [9] than static formalism.

This reflects the principle of feedforward [67, 93] in communication theory—"a needed prescription or plan for a feedback, to which the actual feedback may or may not confirm" [79]—where a communicator provides "the context of what one was planning to talk about" [64, p. 179-80] in order to "pre-test the impact of [its output]" on the listener [34, p. 65].

One illustration is using menus, instructions, graphical user interfaces (GUI), or virtual reality to access any particular computer's features.

An appealing alternative to conventional text-based interfaces through graphical user interfaces is the direct use of hands as an input device to provide natural human-computer interaction.

A more thorough description of the current tools and techniques for interacting with computers as well as recent developments in the subject is provided in the next section.

The evolving multi-modal and Graphical user interfaces (GUI) enable humans to interact with embodied character agents in a way that is not possible with other interface paradigms.

The widely used graphical user interfaces (GUI) of today are found in desktop applications, internet browsers, mobile computers, and computer kiosks.

and the narrowing of the medullary cavity of tubular bones (C) (diaphyseal stenosis).

In this work, we introduce a new paradigm for exploring a large corpus of small documents by identifying roles at the phrasal and sentence levels, then slice on, reify, group, and/or align the text itself on those roles, with sentences left intact.

AbstractExplorer instantiates new minimally lossy SMT-informed techniques for skimming, reading, and reasoning about a corpus of similarly structured short documents: phrase-level role classification that drives sentence ordering, highlighting, and spatial alignment.

AbstractExplorer has a unique combination of LLM-powered (1) faceted comparative close reading with (2) role highlighting enhanced by (3) structure-based ordering and (4) alignment. An ablation study (N=24) validated that these features work best together. A summative study (N=16) describes how these features support users in familiarizing themselves with a corpus of paper abstracts from a single large conference with over 1000 papers.

We contribute: • Novel SMT theory-informed text analysis and rendering techniques for enabling cross-document skimming and comparative close reading at scale • AbstractExplorer, which instantiates these techniques for familiarizing oneself with a corpus of ∼1000 CHI paper abstracts. • Three studies informing and evalutaing the benefits, challenges, and interactions between these techniques.

However, in a narrow set of cases, we believe AI can undermine, rather than defend, democratic values. Some uses are also simply outside the bounds of what today’s technology can safely and reliably do. Two such use cases have never been included in our contracts with the Department of War, and we believe they should not be included now:

The real annoying thing about Opus 4.6/Codex 5.3 is that it’s impossible to publicly say “Opus 4.5 (and the models that came after it) are an order of magnitude better than coding LLMs released just months before it” without sounding like an AI hype booster clickbaiting, but it’s the counterintuitive truth to my personal frustration

A generative AI like ChatGPTData Analyst can take on the role of the evaluation soft-ware. It is expected that this manner of use will make thestudents' work easier, as less emphasis needs to be placedon the programming itself. Instead, teachers can incorpo-rate exercises that encourage students to code more effi-ciently and accurately with the assistance of AI. Thisshifts the focus from finding the right command or func-tion to examining and understanding the data moreclosely. As a consequence, students are better enabled tointerpret the results of statistical evaluation software cor-rectly, thus fulfilling goal 8 of the GAISE report.

statistical knowledge is still required in order toformulate the correct prompts and to ensure that the AIdoes not leave out any step of the analysis.

ChatGPT Data Analyst clearly produced a false resulthere, precisely because the application assumptions for theANOVA were not checked.

The results show that generative AI canfacilitate data analysis for individuals with minimal knowledge of statistics,mainly by generating appropriate code, but only partly by following standardprocedures.

PWA have language deficits that require bespoke AAC supports. These supports may beenhanced by LLMs in software systems that use spoken user input to provide relevantsuggestions that have grammatical and speech production support.

Perseverations that are input into the system are essentially mag-nified by the system’s suggested sentences,

Participant 2 stated the age of her daughters (“Name1 is 18, Name2 is21”), Aphasia-GPT transformed it as “Name1 is 18 and 21”, which is an impossible, butrelated, hallucination

Aphasia-GPT is a real-time, AI-enabled web app designed to expand the words providedby a user into complete sentences as suggestions for a user to select.

nearly half of the workers who received sloppy AI-generated material from coworkers saw those coworkers as less creative, capable and reliable, while smaller amounts saw them as less trustworthy and intelligent.

The cost of the time that it takes fix "workslop" could add up too, with a $186 monthly cost per employee on average, according to a survey of desk workers by BetterUp in partnership with the Stanford Social Media Lab. Forty percent of the workers surveyed said they received "workslop" in the last month and that it took an average of two hours to resolve each incident.

“Younger workers aren’t necessarily more careless, but they’re often using AI more frequently and earlier in their workflows," Dennison said. "There is also a training gap. Organizations often assume younger employees intuitively understand AI, yet provide little guidance on verification, risk, or appropriate use cases. As a result, AI may be treated as an answer engine rather than a support tool."

About 70% of managers say their direct reports have made AI-related mistakes in the past year

58% said direct reports submitted work that contained factual inaccuracies generated by AI tools, while fewer reported that AI failed to account for critical contextual factors. Other issues cited include low-quality content, poor recommendations and inappropriate messaging.

59% of managers saying that they had to invest additional time to correct or redo work created by AI. Similarly, 53% said their direct reports had to take on extra work, while 45% said they had to bring in co-workers to help fix the mistake.

they damaged client relationships.

While 18% of managers said they did not suffer any financial losses from the mistakes, and 20% said those losses were less than $1,000, a significant number reported bigger losses. Twelve percent said those losses were more than $25,000, while 11% said between $10,000 and $24,999. Another 27% placed the value of those losses above $1,000 but below $10,000.

34% said Gen Z followed by 26% who blamed millennials, with 18% citing Gen X and just 9% saying it was baby boomers.

“AI is reliable when used as an assistant, not a decision-maker," Dennison said. "Without human judgment and clear processes, speed becomes a risk, and efficiency gains can turn into costly mistakes,”

“Employees treat AI outputs as finished work rather than as a starting point. Current AI tools are very good at generating fluent content, but they don’t understand context, business nuance, risk, or consequences. That gap shows up in factual errors, missing constraints, poor judgment calls, and tone misalignment.”

AI fatigue is real and nobody talks about it

The scenarios Wooldridge imagines include a deadly software update for self-driving cars, an AI-powered hack that grounds global airlines, or a Barings bank-style collapse of a major company, triggered by AI doing something stupid. “These are very, very plausible scenarios,” he said. “There are all sorts of ways AI could very publicly go wrong.”

“It’s the classic technology scenario,” he said. “You’ve got a technology that’s very, very promising, but not as rigorously tested as you would like it to be, and the commercial pressure behind it is unbearable.”

The race to get artificial intelligence to market has raised the risk of a Hindenburg-style disaster that shatters global confidence in the technology, a leading researcher has warned.Michael Wooldridge, a professor of AI at Oxford University, said the danger arose from the immense commercial pressures that technology firms were under to release new AI tools, with companies desperate to win customers before the products’ capabilities and potential flaws are fully understood.

OpenClaw, like many other open-source tools, allows users to connect to different AI models via an application programming interface, or API. Within days of OpenClaw’s release, the team revealed that Kimi’s K2.5 had surpassed Claude Opus and became the most used AI model—by token count, meaning it was handling more total text processed across user prompts and model responses.

Low-cost Chinese AI models forge ahead, even in the US, raising the risks of a US AI bubble Nvidia’s latest earnings report reassured some. But Chinese AI models are fast gaining a following around the world, underlining concerns over an ‘AI bubble’ centered on high-investment, high-cost US models.

One of the largest PC suppliers, Dell, was reported to be planning a price hike that could raise hardware costs by hundreds of dollars. Interestingly, for consumers opting for higher memory configurations, this would now require a significant price increase. Here were the price increases that were reported across a variety of products: $130–$230 increase for Dell Pro and Pro Max notebooks and desktops configured with 32 GB of memory $520–$765 increase for systems configured with 128 GB of memory $55–$135 increase for configurations with a 1 TB SSD $66 increase for AI laptops equipped with an NVIDIA RTX PRO 500 Blackwell GPU (6 GB) $530 increase for AI laptops equipped with an NVIDIA RTX PRO 500 Blackwell GPU (24 GB) Similarly, companies like ASUS and Acer were also reported to be bumping up PC pricing to cope with memory shortages, and according to Acer's Chairman, Jason Chen, the BoM (Bill of Materials) for several products within Acer's portfolio has risen dramatically, leaving no choice but to increase prices to ensure consistent supply. Small-scale manufacturers like Framework are also looking to increase the cost of upgrading RAM on existing configurations, indicating a widespread "price hike" wave approaching gamers.

the humans involved may have simply lost the plot and may not understand what the program is supposed to do, how their intentions were implemented, or how to possibly change it.

AI infrastructure developers cannot wait five years. In many cases, they cannot wait six months, because waiting six months costs billions of dollars of lost opportunities.

we might move again. The point is that we can. We can because we own our prompts, our skills, our databases, our memory architecture, they all live in our bar. None of it lives inside OpenAI or Anthropic. When we moved, we rewired the model layer and everything else stayed put. That’s the whole trick, really. If you control the pieces that make your agents smart, switching the engine underneath is just plumbing.

What if I actually did have dirt on me that an AI could leverage? What could it make me do? How many people have open social media accounts, reused usernames, and no idea that AI could connect those dots to find out things no one knows?

AI Doesn’t Reduce Work—It Intensifies It

I’m going to cure my girlfriend’s brain tumor.

Owning a $5M data center

I miss thinking hard.

AI voor woningbouw

Sovereign Minds Leadership and Work in the AI Era Structure, Agency, and Judgment Under Acceleration

The pipeline runs from women as the original “computers” in the 1940s, through the masculinization of computing that pushed women into typing pools and administrative support, through the automation of those roles, to AI assistants today automating what remains: scheduling, reminding, organizing, emotional management.

“Obedient and obliging machines that pretend to be women are entering our homes, cars and offices,” warned UNESCO’s Director for Gender Equality, Saniye Gülser Corat, in the agency’s landmark 2019 report.

Standard Retrieval-Augmented Generation (RAG) over documents is a good first step, but it fails when faced with complex, cross-domain enterprise questions. It finds text that looks similar, which isn’t the same as finding facts that are related.

steer AI models, and to shaping the character and constitution of these models

I do see a path to a slight moderation in AI development that is compatible with a realist view of geopolitics

By the nature of unknown unknowns it is impossible to make an exhaustive list,

AI companies, semiconductor companies, and perhaps downstream application companies generating ~$3T in revenue per year,

we cannot ignore the potential for abuse of these technologies by democratic governments themselves.

brainwashing

a true panopticon on a scale that we don’t see today

Putting these two concerns together leads to the alarming possibility of a global totalitarian dictatorship. Obviously, it should be one of our highest priorities to prevent this outcome.

authoritarian governments might use powerful AI to surveil or repress their citizens in ways that would be extremely difficult to reform or overthrow.

has a small number of specific hard-line prohibitions, and one of them relates to helping with the production of biological (or chemical, or nuclear, or radiological) weapons. But all models can be jailbroken

a genius in everyone’s pocket could remove that barrier, essentially making everyone a PhD virologist who can be walked through the process of designing, synthesizing, and releasing a biological weapon

I believe the only solution is legislation—laws that directly affect the behavior of AI companies,

not all AI companies do this, and the worst ones can still be a danger to everyone even if the best ones have excellent practices.

a clockwork watch may be ticking normally, such that it’s very hard to tell that it is likely to break down next month, but opening up the watch and looking inside can reveal mechanical weaknesses that allow you to figure it out.

similar to how neuroscientists study animal brains by correlating measurement and intervention to external stimuli and behavior

AI models are grown rather than built

almost always follows this constitution

Constitutional AI,

The concern is that there is some risk (far from a certainty, but some risk) that AI becomes a much more powerful version of such a person, due to getting something wrong about its very complex training process.

AI models could develop personalities during training that are (or if they occurred in humans would be described as) psychotic, paranoid, violent, or unstable, and act out, which for very powerful or capable systems could involve exterminating humanity.

For example, AI models are trained on vast amounts of literature that include many science-fiction stories involving AIs rebelling against humanity.

we know that AI models are unpredictable and develop a wide range of undesired or strange behaviors, for a wide variety of reasons. Some fraction of those behaviors will have a coherent, focused, and persistent quality

We now know that it’s a process where many things can go wrong.

What should you be worried about? I would worry about the following things: