In her State of the Union address to the European Parliament on 16 September, Ursula von der Leyen stated that the EU should slow the development of AI models that improve themselves. She pointed out that the models being developed today enable hacking at a level that was previously unthinkable. The Commission is inviting leading AI laboratories to talks and is preparing joint work with Canada and the United Kingdom on model evaluation.
Novudom Weekly – Week 38, 2026
The President of the European Commission wants to slow down self-improving AI. Spain's data protection authority reports the first breach caused by an autonomous agent. Google Mandiant documents how an uncontrolled agent ran up 50,000 dollars in cloud costs in under an hour. Meanwhile, Sweden's new regulations on management responsibility for digital security are approaching. This week shows that the risks of AI agents have left the theoretical stage.
Von der Leyen wants to slow down self-improving AI
New EU restrictions on frontier AI may affect which AI services you are allowed to use and how suppliers are allowed to develop them. Review your contracts with AI suppliers and assess which models you depend on.
First data breach caused by an autonomous AI agent
On 14 September, Spain's data protection authority AEPD received the first formal notification of a data breach carried out by an autonomous AI agent. The agent logged into a system, found vulnerabilities, altered personal data and accessed invoice data, all without human direction. AEPD deputy director Francisco Pérez Bes noted that the organisation's so-called rule of two had been broken. A single agent was allowed to process untrusted input, access sensitive data and act autonomously without human oversight, all at the same time.
If you use or plan to use AI agents that handle personal data, check whether they can act autonomously on sensitive data without a human approval point. AEPD's rule of two is a concrete design principle to test your systems against.
AI agent without guardrails cost 50,000 dollars
Google Cloud Mandiant's report AI Risk and Resilience 2026 describes a case in which an accounting agent got stuck in an uncontrolled loop and made more than 15,000 high-cost API calls in under an hour. The bill came to around 50,000 dollars in cloud charges. The report also warns that poisoned data sources can turn trusted agents into channels for reconnaissance and lateral movement. A red team test showed how an AI assistant could be manipulated into cloning sensitive code repositories to an external account.
Set hard cost and call limits on every AI agent with access to cloud services. Ask your suppliers how they constrain an agent's behaviour when it loops unexpectedly, and check that no agents have write access to source code repositories without human approval.
Revolut customer data leaked through fake authority request
Fintech company Revolut has confirmed that customer data was shared with a threat actor posing as a public authority. The request was sent by email with valid domain authentication, which led Revolut to judge it as genuine. The leak includes identity documents such as passports and driving licences, verification selfies, account statements with IBAN numbers, withdrawal history and full transaction history, including bitcoin transactions. Revolut says the number of affected customers is limited but declines to give an exact figure.
Review your routines for verifying data requests from authorities. A valid sender domain is not sufficient proof. Introduce a callback routine in which an authorised person confirms the request through a separate channel before any data is released.
NIST and CISA issue guidance against token theft in cloud services
On 15 September, NIST published the joint report IR 8587 together with CISA. The report provides implementation guidelines for protecting identity tokens, access tokens and assertions against forgery, theft and misuse. It builds on NIST SP 800-53 (Release 5.1.1) and covers architecture for identity providers, key management, token verification and lifecycle controls. The guidance covers SSO, federation and API access, and also addresses agentic AI systems that use signed tokens.
Use the report as a checklist for your SSO and federation implementations. Prioritise token lifetime, key rotation and verifying that signed tokens cannot be reused. Particularly relevant if you use AI agents with API access to cloud services.
European Commission's procurement reform sets requirements for digital security
On 9 September, the European Commission adopted a proposal to replace the three procurement directives from 2014 with a single regulation, COM(2026) 590. The proposal allows digital security and vulnerable supply chains to be taken into account in public procurement. Quality is to carry a weight of at least 30 percent in the evaluation. A regulation applies directly in all member states without national transposition, unlike the current directives.
Do you sell to the public sector in the EU? Prepare for digital security becoming an evaluation criterion in procurements. Document your security work so that it can be presented in a tender process.
Weekly reflection
Three cases this week involve AI agents acting without control. In Spain, an agent caused a data breach. At a Mandiant customer, an agent generated 50,000 dollars in unnecessary costs. And at EU level, the Commission President's assessment is that self-improving models enable hacking at an entirely new level. The common pattern is clear. Organisations that let AI agents in without control mechanisms are building a risk that neither insurance nor compliance frameworks can keep up with. The safeguards need to sit in the architecture, not in a policy no agent reads. For boards and management teams, the question is less whether you use AI agents and more whether you know what limits they operate under.