AI checking AI inherits vulnerabilities, Hays warned. "Transparency gaps, prompt injection vulnerabilities and a decision-making chain becomes harder to trace with each layer you add." Her research at Salesforce revealed that 55% of IT security leaders lack confidence that they have appropriate guardrails to deploy agents safely.
abstracting away responsibilities is a dead-end. Over half of IT security think now no way to deploy agentic AI safely.
When two models share similar data foundations or training biases, one may simply validate the other's errors faster and more convincingly. The result is what McDonagh-Smith describes as "an echo chamber, machines confidently agreeing on the same mistake." This is fundamentally epistemic rather than technical, he said, undermining our ability to know whether oversight mechanisms work at all.
Similarity between models / training data creates an epistemic issue. Using them to control each other creates an echo chamber. Vgl [[Deontologische provenance 20240318113250]]
Yet most organizations remain unprepared. When Bertini talks with product and design teams, she said she finds that "almost none have actually built it into their systems or workflows yet," treating human oversight as nice-to-have rather than foundational.
Suggested that no AI using companies are actively prepping for AI Act's rules wrt human oversight.
We're seeing the rise of a 'human on the loop' paradigm where people still define intent, context and accountability, whilst co-ordinating the machines' management of scale and speed," he explained.
Human on the loop vs in
As Gartner VP analyst Alicia Mullery put it: “AI can make mistakes faster than we humans can catch them.”
yes, example of [[Spammy handelings asymmetrie 20201220072726]]. At scale it moves the bottleneck
piece on AI oversight.
In general I wonder, at what point does the needed oversight negate the gains in time / effectivity / efficiency that are expected of using AI in some context.