James Hughes, VP of Solutions Engineering at Rubrik
If something goes wrong in the financial sector, market disruption, loss of customer confidence, and systemic risk can follow quickly. As the Bank of England recently warned, advanced AI models could effectively “crack” cyber systems and expose vulnerabilities at scale, raising the prospect of faster, more sophisticated attacks on financial infrastructure. That risk is no longer theoretical.
Anthropic is continuing to roll out Claude Mythos Preview, an AI system capable of identifying thousands of previously unknown vulnerabilities and generating working exploits at unprecedented speed, to a select number of companies. What once required time, expertise and resources can now be done almost instantly. For financial services, where systems are deeply interconnected and time-sensitive, that shift fundamentally changes the risk equation.
A prime target for manipulation
As AI adoption accelerates, institutions must ensure innovation does not come at the cost of security. AI introduces clear benefits, from faster fraud detection to more informed decision-making, but also new risks. More than 75% of financial firms now use AI, and as exposure grows, financial firms must ensure resilience is built in from the outset, not retrofitted after the fact.
AI is now embedded in critical financial processes, from credit approvals to trading and fraud detection. These systems operate at speed and scale, often with limited human oversight, making them highly attractive targets for cyber attackers. In fact, research finds that only 23% of European businesses have full visibility over their AI systems.
Unlike traditional cyber incidents, attacks on AI systems can be subtle and persistent. Threat actors can manipulate training data or inputs to influence outcomes over time, meaning a compromised model may appear to function normally while quietly producing distorted results. This could mean misclassifying fraudulent transactions, introducing bias into lending decisions, or influencing trading activity in ways that are difficult to detect.
In a world where attackers can now discover and weaponise vulnerabilities at machine speed, the challenge is no longer just detecting an attack. It is recognising that by the time you detect it, the damage may already be underway.
The risks of moving too quickly
Despite these challenges, financial institutions are under pressure to adopt AI quickly. The Treasury Select Committee recently pointed out that embracing new technology allows enterprises to gain an edge. But in the race to deploy AI, are adequate safeguards being left behind?
In financial services, the stakes are uniquely high. These systems do not operate in isolation, but directly impact savings, investments and access to credit. A failure is not contained to a single application; it can ripple across markets and economies.
This pressure is also taking a toll internally. Recent research shows that 79% of CISOs in the financial sector say cyber threats and heightened resilience demands are impacting their mental health. It is a clear signal that the pace of change, combined with the weight of responsibility, is creating strain at the very point where strong oversight is most needed.
At the same time, many organisations lack full visibility into the data used to train their models, making it difficult to ensure integrity. If that data is compromised, the model’s outputs will be too. And while AI systems are designed to perform under normal conditions, they are rarely stress-tested against adversarial scenarios – precisely the conditions they are now most likely to face.
There is growing concern that safeguarding is not keeping pace with adoption. In a threat landscape accelerated by AI, that gap is widening.
Why resilience must come first
Financial institutions must operate on the assumption that AI systems will fail – whether through error, manipulation or exploitation at scale – and plan accordingly. The downstream impact is immediate and tangible: flawed credit decisions, missed fraud, market disruption, regulatory breaches and direct financial loss. Crucially, detection alone is no longer sufficient. When attackers can move faster than response cycles, resilience is defined by how quickly an organisation can recover.
Without clear, forensic-level visibility into how AI systems make decisions and how data flows through them, firms cannot diagnose issues quickly or prevent them from recurring. That lack of visibility extends disruption and amplifies both financial and reputational damage. Just as critical is the ability to act fast. If an AI-driven process is compromised, organisations need to roll systems back to a known, clean state within minutes, not hours or days. Where the time between vulnerability discovery and exploitation is collapsing, the ability to rapidly return to a trusted state becomes the defining factor in limiting impact.
A call for control, not caution
AI will continue to shape the future of financial services. Its benefits are transformative, and its adoption is inevitable. But without the right controls, it also introduces new and complex cyber risks at a speed the industry has not previously faced.
The answer is not to slow innovation, but to match it with resilience. Financial institutions must ensure that AI systems are secure, observable and recoverable from the outset, not after a system is compromised. The real risk is that AI could be manipulated, at scale and at speed, without being detected until it is too late.
