Manesh Tailor, EMEA Field CTO, New Relic
In the fast-paced world of financial services, downtime is more than an inconvenience – it puts revenue, reputation, and customer trust at risk. From trading platforms to payment systems, a single outage can create a domino effect that reverberates across the market. The highly public and frequent outages affecting major financial institutions serve as a powerful and costly reminder of this reality.
The Staggering Price of Downtime
Recent research highlights the immense financial damage caused by IT outages. For organisations in the EMEA region, a single high-business-impact outage carries an annual median cost of $102 million (£79.9 million). For firms specifically in the UK & Ireland, that figure is $38 million (£28.3 million). Breaking that down further, the median cost per hour of a high-impact outage in EMEA is $2 million (£1.49 million), which equates to more than $33,000 (£24,000) for every single minute of downtime.
The data shows that high-business-impact outages occur far more frequently than many would suspect—as often as weekly for 37 percent of surveyed organisations.
The Flawed “Firefighting” Approach
The traditional approach of managing these outages is fundamentally broken. Organisations have often adopted a reactive “firefighting” culture where problems are addressed only after they have already caused a negative impact. This strategy is unsustainable and dangerous. It drains millions in resources and diverts valuable engineering talent from innovation to crisis management.
Our data shows that, on average, more than a quarter (26 percent) of engineering teams’ time is spent addressing disruptions. Worse yet, a significant number of customers (41 percent) still learn about software and system issues through outdated means, such as manual checks, complaints from internal stakeholders, or, worst of all, from their own customers.
Embracing a Proactive, Data-Driven Mindset
The solution lies in shifting from a reactive model to a proactive one. This requires a new engineering mindset that prioritises prevention over reaction. Observability is a key component of this shift. It’s not just about monitoring systems but about gaining deep, real-time insights into the health of your entire digital ecosystem. The research confirms this: 63 percent of respondents said their mean time to detection (MTTD) and 64 percent said their mean time to resolution (MTTR) improved measurably since adopting observability solutions.
This new mindset isn’t just about implementing a tool; it’s about embedding a culture of engineering excellence from the very beginning of the software development lifecycle (SDLC). It’s about building resilience, planning for failure, and making observability a core part of the process, even before the first line of code is written.
However, this cultural shift can only succeed with buy-in from key stakeholders. A lack of a cohesive IT strategy often results from decentralised decision-making and a chaotic mix of tools across different departments. While the average number of tools per EMEA organization is four, our data shows that 10 percent of EMEA organisations have consolidated to a single observability tool, up from just two percent in 2022. Furthermore, a remarkable 44 per cent plan to consolidate their tools within the next year. The benefits of this consolidation are clear: boosted productivity, enhanced security, and superior data for making informed decisions.
Rebalancing Speed with Stability
In the relentless pursuit of speed, particularly within Agile methodologies, a critical element has sometimes been overlooked: stability. While Agile is a powerful framework for rapid development, a singular focus on velocity can inadvertently lead to a neglect of foundational engineering practices like thorough architectural planning, formalised testing, and comprehensive documentation.
To build a more robust and sustainable approach, it’s worth revisiting principles from methodologies like Six Sigma. Originating from manufacturing, Six Sigma provides a structured, data-driven approach to eliminating defects and improving processes. Its five core principles—Define, Measure, Analyse, Improve, and Control (DMAIC)—are directly applicable to software engineering and can be powered by modern observability tools.
By using observability to power this methodology, engineers can proactively identify and prevent issues before they impact customers.
Building a resilient digital infrastructure isn’t about slowing down innovation. It’s about building a solid, reliable foundation that can handle the complexity of modern systems. By re-embedding these engineering practices, financial institutions can protect themselves against the inevitable, securing both their digital infrastructure and their reputation for the long term.
