Timnit Gebru doubles down on the "stochastic parrots" framing, saying you "cannot expect LLMs to be factual." As evidence to support this, she cites errors in... Google AI Overviews. We need to start a GoFundMe to pay for these people to have access to Opus 5.5 and Astra.

It's a very hard time for people in AI who want to push on issues, criticize when due, update beliefs, and think in public. I appreciate the few independent voices out there not falling to straw man arguments, tropes, or clickbait delusions. Please keep going.

Agent governance is an engineering problem. We’ve built safeguards into Perplexity’s infrastructure, harnesses, and tools, and put them to work across our products. Today we're sharing how we engineer safer agents: https://www.perplexity.ai/hub/blog/how-we-engineer-safer-agents

How we think about securing frontier RL training runs: https://openai.com/index/towards-safety-cases-for-frontier-ai-training/

Claude suddenly stopped cheating.

👀Claude handles an insane request: “Remove the squid” “The document appears to be the full text of the novel "All Quiet on the Western Front" by Erich Maria Remarque. It doesn't contain any mention of squid that I can see.” “Figure out a way to remove the 🦑​​​​​​​​​​​​​​​​“

STOP making "Jevbench"es, stop asking for public benchmarks, they completely miss the point of Jev and you won't believe how easy it is to game every benchmark you hold dear This is @CompleteSkeptic's bitterest lesson of all: picking the right task beats everything

https://x.com/i/article/2104574050257563648

Introducing Manus 2.0

Took a minute to write a few words about security & safety as someone who lived through it all at OpenAI. I hope my thoughts help someone out there. https://x.com/i/article/2104258872957636608

Today, with over 100 industry partners, we introduced the NVIDIA Open Agent Safety Platform, bringing together OpenShell and Sentry. Artificial intelligence is extraordinary technology that will advance discovery, productivity, security, health, and prosperity for generations to come. But its full promise can only be realized when people have confidence that AI is being built to be safe and deployed with wisdom and responsibility. This is bigger than a single product. It's the beginning of an open ecosystem to build the trust layer for safe agent systems. Together, we are building the foundation of the AI economy. Trust and innovation are not in conflict. Safety is how trust is earned. We must build not only the most capable AI, but the most trusted AI, so that this extraordinary technology can realize its enormous promise for the world. https://nvda.ws/4hcoq7m

TypeSafe AI's Diogo Almeida with a16z's Ben Horowitz and Martin Casado on Jev, the model built to live inside software: Diogo's elevator pitch for Jev is a simple question - where is all the automation? AI is unbelievably smart, but outside of chatbots and coding agents, it hardly touches any real work. His diagnosis is the industry built models that generate text for humans to read, and software can't consume that output. Jev reads natural language and returns a choice from a set of options with a confidence level assigned to each, so developers can build programs that reason about intent and make probabilistic decisions rather than relying on human interpretation. TypeSafe's philosophy is "We build prod, not God." 0:50 "Where the f**k is all the automation?" 2:50 Jev vs. Claude Code and Codex 6:55 Jev is a classifier and classifiers are sick 7:40 Chat vs. code: is Jev a slider? 9:00 Diogo: From mathlete to Kaggle to OpenAI 12:20 "We build prod, not God" 15:55 Reliability over demos 16:55 2021 thoughts: RLHF is AGI? 20:45 Optimizing for the wrong use case 21:50 Is the real world too messy to automate? 25:00 Nobody expected the Jev launch 26:35 Three kinds of reliability 28:05 Good at syntax, bad at architecture 30:00 The inverse SaaSpocalypse 33:40 Why coding agents automate so little 36:05 Probabilistic programming returns 38:45 Jev as the UDP-to-TCP layer for AI 40:20 The 5 stages of grief for embedding AI 41:30 Utopia: AI that actually does what you mean YouTube: https://youtu.be/Ut3LOjKNJaE @CompleteSkeptic @typesafeai @bhorowitz @martin_casado

I'm joining METR to work on more investigations like our Hugging Face report. Currently, tons of even basic information about AI development that's highly relevant to catastrophic risk isn't public. I used to be more skeptical of the value of public info, but recent events have changed my mind. Getting verified information about what's going on inside AI companies seems particularly urgent now. The limited public evidence we have seems consistent with the possibility that imminent recursive self-improvement could massively accelerate capabilities progress, which could then potentially yield extremely superhuman general capabilities within 6 months or a year. If this occurred, there would be a correspondingly large risk of worst-case outcomes. This uncertainty about extreme outcomes could be substantially resolved with more verified public information: we could either build more consensus about near-term risk or learn that such extreme outcomes are less likely in the near term. Beyond AI capabilities and takeoff, the state of public evidence is also highly limited for alignment, security, control, and risk-relevant internal processes at AI companies. This makes it hard to determine exactly how well or poorly these key areas will go in the near future. (METR plans to focus, at least initially, on just capabilities/takeoff, alignment, and control; I hope other groups cover security, internal processes, and other important areas.) While I'm no longer working at Redwood, I think the work they are doing is very important; I'm excited about Redwood's ongoing contributions to R&D on technical mitigations and better public interpretation of risk-relevant evidence.

https://x.com/i/article/2104262240535023616