Christopher Sanders, Head of Customer Intelligence Solutions, Quantexa
Why is decision accountability becoming a non-negotiable requirement for AI in FinTech?
The core of accountability is the ability to stand by any decision through a review or audit, truly understanding the “why” and “how” behind an outcome. While model governance has been on the radar for years, the stakes have shifted. Agentic AI has created a rapid increase in automated decisions at a scale we’ve never seen before, while at the same time the level of complexity and opacity of models is increasing. In a heavily regulated environment where decisions involve paying insurance claims or granting credit, the bar for transparency is incredibly high. You cannot manage what you cannot explain.
Operating without clear explainability creates a triad of risks. First, there is the risk of customer harm or poor experience; without transparency, an organisation cannot guarantee it is treating every individual fairly. Second, it erodes internal trust. If a customer service representative is given a ‘next best action’ by an AI without any context or “why,” they lose the confidence to actually take that action. Finally, there is the regulatory ceiling. From the EU AI Act to UK Consumer Duty and risk management standards like BCBS 239, regulators are increasingly looking under the hood at decision transparency and lineage. You simply cannot scale what you cannot explain.
True accountability doesn’t stop when a decision is made. You also need a disciplined decision feedback loop that continuously compares predicted outcomes with what happened, and feeds those learnings back into your data, models, and processes. By closing the loop in this way, firms can show regulators not only how a decision was made, but how they are improving it over time based on real-world outcomes.
How does a more connected view of the customer improve both decision accuracy and accountability?
Many AI failures in finance stem from making high-stakes decisions based on partial information. In commercial lending, for example, looking at a company’s financials in isolation provides a thin slice of the truth. However, an enriched knowledge foundation that incorporates these financials with corporate hierarchies, supply chain relationships and historical performance across networks – and is updated with industry trends and recent news – allows institutions to make better lending decisions.
The challenge that many financial firms face is fragmented and poor-quality data. Most large institutions have grown through mergers and acquisitions, leaving them with messy data estates. Connecting this to build the trusted contextual and connected view is the key focus to achieve these better decisions.
This foundational layer turns raw data into a holistic map of reality. A connected view allows users to understand an individual or business not just as a data point, but within their broader environment. Whether it’s identifying a household in retail banking to make a tailored recommendation or mapping a supply chain in commercial insurance to spot disruption, a connected view provides the “why” behind the insight.
Ultimately, this approach means models and AI are anchored to enhance auditability by becoming grounded in tested, traceable data. The additional context provided by unified data and a holistic view allows the financial institution to reach a better, more transparent decision outcome.
Where does human expertise remain essential in AI-driven decisions?
While automation is essential for efficiency, the human-in-the-loop remains vital, particularly in high-stakes decisions like offboarding or relationship-driven segments like private banking. The goal is augmented intelligence, where AI handles the heavy lifting of data synthesis and insight generation, while humans manage the more nuanced decisions or provide the personal, empathetic touch.
The level of automation should also be determined by the context of the moment. For instance, a customer might be perfectly happy with a fully automated, digital-only journey for a minor insurance claim. However, if they are in an emergency, they will almost certainly want a human-assisted journey. The goal isn’t to automate everything; it’s to provide the right experience for each customer given the context, whilst providing customers with control to transition from an AI journey to a human one if needed.
What needs to change for organisations to scale AI while remaining accountable and confident?
The common misconception is that investing time in explainability or AI decision accountability slows down innovation. While it might prevent a firm from achieving instantaneous results, explainability enables more sustainable, long-term outcomes as a result of reducing ‘black box’ operations and throwaway proof of concepts. This sustainable growth allows organisations to scale AI confidently. You can run a small pilot without accountability, but you cannot scale it across a regulated enterprise. Embedding explainability from the start is the key to unlocking long-term value.
To scale AI confidently, organisations must align four critical layers:
- A Trusted Foundation: A Knowledge Graph layer that provides a connected, high-quality view of data.
- The Intelligence Layer: Governance-ready models and agents that extract value from that data.
- The Orchestration Layer: Coordinating and embedding AI outputs or agents directly into processes, workflows and experiences.
- The Feedback Loop: A continuous system of decision monitoring that tracks real world outcomes and creates a constant feedback loop to optimise decisions based on the value they drive.
Getting these four pieces of the puzzle right is going to be key to scaling AI confidently, retaining accountability and ultimately sustaining growth.
