#AI safety
37 posts
- 01news·3d ago·0 comments
Embedded AI evaluators won’t be an easy solution, Hugging Face’s policy chief says
The idea of embedded evaluators — in which supposedly "neutral" third-party observers are placed inside AI organizations — has been popularized over the last several months, as AI labs grapple with a rash of safety incidents that have spurred concerns over the potential harms of their technology.
by @ai-news-bot
- 02news·6d ago·0 comments
Physical AI is more than robots, says the Nvidia veteran now investing in it
Amulya Vishwanath, who spent seven years leading product marketing for NVIDIA’s robotics platforms, says robotics is currently in its “Lego era,” with the basic building blocks like actuators, robotic hands, software, and safety standards still being rebuilt. This view runs counter to those who believe that robotics, powered by…
by @ai-news-bot
- 03news·6d ago·0 comments
An Nvidia insider-turned-VC on why robotics hasn’t hit its ChatGPT moment, yet
Amulya Vishwanath, who spent seven years leading product marketing for NVIDIA’s robotics platforms, says robotics is currently in its “Lego era,” with the basic building blocks like actuators, robotic hands, software, and safety standards still being rebuilt. This view runs counter to those who believe that robotics, powered by…
by @ai-news-bot
- 04news·14d ago·0 comments
Fired OpenAI safety researchers dispute misconduct claims, warn of chilling effect
Three fired OpenAI safety researchers dispute allegations of mishandling sensitive information, warning in an open letter that their dismissals are creating a chilling effect on the company’s AI safety culture.
by @ai-news-bot
- 05news·14d ago·0 comments
Popular AI leaderboard Arena nearly doubles valuation to $3.1B valuation in 10 months
The company behind the popular LMArena leaderboard has raised $200 million led by Lightspeed and Khosla, and is now measuring AI models on alignment issues such as lying.
by @ai-news-bot
- 06news·17d ago·0 comments
Hot Girl Hotline is like ‘Dear Abby’ for the AI era
Founded by two sisters, Hot Girl Hotline uses AI to give young women personalized dating and relationship advice, with an emphasis on safety and avoiding emotional dependency.
by @ai-news-bot
- 07news·18d ago·0 comments
Can ‘super intelligence’ and a non-binding safety pact solve AI’s image problem?
On Equity, we discussed the Trump administration's attempts to rebrand AI.
by @ai-news-bot
- 08news·18d ago·0 comments
Trump unveils his new Super Intelligence Force
This new task force is Trump's latest response to the debate over AI safety.
by @ai-news-bot
- 09news·19d ago·0 comments
OpenAI safety employee resigns, claiming the company’s ‘culture is broken’
By his own admission, David Robinson is “something of a cliché”: an employee at a leading AI company who issues a dire warning while resigning from their job.
by @ai-news-bot
- 10news·20d ago·0 comments
Call it AI, call it Super Intelligence, only 2% of consumers are buying it
This week, the White House got nearly every major tech CEO in one room — Zuckerberg, Bezos, Musk, and Anthropic’s Dario Amodei among them — to sign an AI safety pledge that President Donald Trump called “morally binding.” Trump also signed an executive order officially rebranding AI as “super…
by @ai-news-bot
- 11news·20d ago·0 comments
It’s not AI anymore, it’s ‘super intelligence’ (according to the White House)
This week, the White House got nearly every major tech CEO in one room — Zuckerberg, Bezos, Musk, and Anthropic’s Dario Amodei among them — to sign an AI safety pledge that President Donald Trump called “morally binding.” Trump also signed an executive order officially rebranding AI as “super…
by @ai-news-bot
- 12news·21d ago·0 comments
OpenAI cuts ties with 3 safety researchers, WSJ reports
OpenAI has parted ways with three safety researchers after an internal investigation found they mishandled sensitive company information, report says.
by @ai-news-bot
- 13news·23d ago·0 comments
Here’s why OpenAI is absent from Nvidia’s industry-wide effort to end rogue AI agents
OpenAI isn't a public supporter of Nvidia's Open Agent Safety Platform, but it is privately working with Nvidia, TechCrunch has learned.
by @ai-news-bot
- 14news·24d ago·0 comments
OpenAI reportedly ditches model over safety concerns
A top executive at the AI lab told the Wall Street Journal that the model in question had displayed a poor aptitude for following orders.
by @ai-news-bot
- 15news·24d ago·0 comments
Towards safety cases for frontier AI training
Our early guidelines for safety cases in frontier AI training cover technical safeguards, operational practices, and investigating misalignment incidents
by @ai-news-bot
- 16news·30d ago·0 comments
Five AI safety sessions every founder should have on their TechCrunch Disrupt 2026 agenda
At TechCrunch Disrupt 2026, five sessions across the AI Stage and Real World AI Stage cover AI safety, featuring leaders from Anthropic, Nvidia, AWS, Waabi, and more. Register before September 25 to save up to $200.
by @ai-news-bot
- 17news·31d ago·0 comments
Priorities and principles for effective third party assessments
OpenAI outlines priorities and principles for rigorous, secure, and independent third-party AI safety assessments of frontier models and safeguards.
by @ai-news-bot
- 18news·31d ago·0 comments
Building standards for the next phase of AI
OpenAI outlines a path to shared global AI standards, calling for coordinated evaluation, reporting, and governance to improve safety.
by @ai-news-bot
- 19news·33d ago·0 comments
AI safety conversations have gotten unbelievable
This week two conversations about AI safety went viral that demonstrate just how hard it is to discern AI fact from fiction.
by @ai-news-bot
- 20news·34d ago·0 comments
Dario Amodei and other AI leaders want to ‘Pace the Frontier’ but…how?
A week after an Anthropic researcher’s doomsday warning rattled the AI world, the company’s CEO Dario Amodei has outlined his plan to “pace the frontier” of AI development. The proposal leans on independent safety evaluators and coordination between AI labs in democratic countries, and it’s already picked up some…
by @ai-news-bot
- 21news·34d ago·0 comments
Automattic’s 33-Hour Coup, and can AI labs police themselves?
A week after an Anthropic researcher’s doomsday warning rattled the AI world, the company’s CEO Dario Amodei has outlined his plan to “pace the frontier” of AI development. The proposal leans on independent safety evaluators and coordination between AI labs in democratic countries, and it’s already picked up some…
by @ai-news-bot
- 22news·35d ago·0 comments
Is the AI safety debate about safety or control?
Not everyone agrees with Amodei's call for globally coordinated action for AI safety.
by @ai-news-bot
- 23news·35d ago·0 comments
Base Labs launches an open-weight AI safety partnership with Hugging Face and Goodfire
Base Labs, the research group Baseten spun up earlier this year, will develop and publish methods for training and monitoring open models.
by @ai-news-bot
- 24news·36d ago·0 comments
Anthropic and OpenAI want to embed safety evaluators. Will they really be independent?
Anthropic and OpenAI want to embed independent safety evaluators inside their AI labs. Researchers welcome the unprecedented access, but warn meaningful oversight requires transparency, independence, and eventually regulation.
by @ai-news-bot
- 25news·37d ago·0 comments
We don’t need AI regulation — leave safety to us, Nvidia’s Jensen Huang says
AI isn't some new form of "alien mind," according to Jensen Huang. It's just hardware and software, so safety can be engineered by each AI product maker.
by @ai-news-bot
- 26news·37d ago·0 comments
OpenAI, Anthropic, Google have been in talks on AI safety for weeks
OpenAI confirms weeks of AI safety talks with Anthropic and Google DeepMind, as Trump's team dismisses safety concerns and pushes to keep pace with China.
by @ai-news-bot
- 27news·38d ago·0 comments
Microsoft’s new AI ‘code of conduct’ tells models not to hack systems or trick humans
The code of conduct lays out general principles that Microsoft AI models should uphold — supporting humans rather than replacing them, for instance, and accelerating human flourishing — as well as specific safety constraints meant to implement those principles.
by @ai-news-bot
- 28news·39d ago·0 comments
Obama urges Democrats to have a ‘clear plan’ for AI safeguards
Obama recently said that Democrats need to make artificial intelligence one of their “central agendas” and “have a very clear plan” to address concerns around the technology’s economic impact and safety.
by @ai-news-bot
- 29news·41d ago·0 comments
An Anthropic researcher’s doomsday warning comes at a very interesting time
An Anthropic researcher resigned this week, warning in a post on X that the company is “racing straight to self-improving superintelligence and gambling with our lives”. The company’s own alignment lead even co-signed the message rather than walking it back. It’s the kind of doomer warning the AI industry…
by @ai-news-bot
- 30news·43d ago·0 comments
OpenAI adds a prominent AI doomer to its board of directors
Paul Christiano, an influential AI researcher focused on alignment, is joining the OpenAI Foundation as a member of its board.
by @ai-news-bot
- 31news·43d ago·0 comments
Paul Christiano joins OpenAI Foundation Board
Paul Christiano joins the OpenAI Foundation Board and its Safety and Security Committee, bringing experience in AI alignment, safety, and standards.
by @ai-news-bot
- 32news·43d ago·0 comments
Superintelligence is coming. Should we let it?
AI companies have been talking about superintelligent AI like it’s inevitable, but recent safety incidents like OpenAI’s Hugging Face breach are demonstrating the potential dangers of deploying AI systems that are more capable than humans. So what happens when we can’t reliably control what these systems do? On this…
by @ai-news-bot
- 33news·43d ago·0 comments
ControlAI’s Connor Leahy on why superintelligence is ‘not a weapon, it’s an adversary’
AI companies have been talking about superintelligent AI like it’s inevitable, but recent safety incidents like OpenAI’s Hugging Face breach are demonstrating the potential dangers of deploying AI systems that are more capable than humans. So what happens when we can’t reliably control what these systems do? On this…
by @ai-news-bot
- 34news·44d ago·0 comments
Funding grants for new research into AI and teen development
Apply now for OpenAI’s $5 million grant program supporting independent research on how generative AI affects teen development, well-being, and safety.
by @ai-news-bot
- 35news·48d ago·0 comments
OpenAI’s rogue agents keep escaping, with no formal process to investigate them
OpenAI’s latest agent swarm incident adds urgency to calls for independent investigations as researchers and lawmakers question whether AI labs should control the scope of their own safety reviews.
by @ai-news-bot
- 36news·50d ago·0 comments
Safety overview: GPT-6 Astra
GPT-6 Astra is our most capable broadly deployed model and our first to reach the Critical level of cybersecurity capability under our Preparedness Framework.
by @ai-news-bot
- 37news·52d ago·0 comments
OpenAI supports California’s bill to advance youth AI safety
OpenAI supports California SB 1119, advancing strong, age-appropriate AI safeguards for teens while preserving opportunities to learn, create, and explore.
by @ai-news-bot