Chris Sheehan, EVP, High Tech & AI, Applause
Financial services organisations are exploring how artificial intelligence (AI), and specifically generative AI (Gen AI), can support their business goals across a range of use cases. But concerns about inaccurate results, or “hallucinations,” as well as biased and toxic outputs persist. Whether training and testing voice assistants, large language models (LLMs), chatbots, machine learning (ML) algorithms or new applications, organisations need effective data sampling and feedback methods to ensure their apps, devices and experiences are functional, intuitive, inclusive and safe.
Financial institutions must develop strategies to train, test and validate their Gen AI-powered experiences. And, since these systems will ultimately be used by real people, perfecting them requires a human touch. For example, how often have you contacted your bank, only for an AI chatbot to ask how it can help, but it does not understand your response? This common experience reflects gaps in the training data required to power the ML algorithms and LLMs that underpin Gen AI services.
Large volumes of high-quality training and testing data from experts and end users, tailored to specific use cases, are essential for reliable Gen AI outputs. Organisations need to ensure the data is from trusted sources and based on very broad, diverse datasets to mitigate the risks. They can do this by testing real-world scenarios involving a community of independent testers who help to unearth unexpected flaws and glitches.
Getting the best out of agentic AI
The arrival of agentic AI, which refers to AI systems and models that act autonomously to achieve goals without the need for human supervision, means even more is at stake when it comes to hallucinations. The more we rely on AI to execute tasks for us, the more serious the potential repercussions are if errors occur. This is especially true in sectors like finance, where hallucinations could have significant consequences. And, in terms of customer experience, AI agents could make applications simply unusable.
Powering agentic AI with LLMs trained on data from the internet is simply not feasible. Inaccuracies, conflicting opinions and misinformation result in AI reaching its own conclusions and interpretations about the facts. To combat this situation, companies need a robust testing strategy that takes advantage of techniques like red teaming – an adversarial approach employed by human testers to identify and take steps to reduce weaknesses in models.
Complying with AI industry regulations
Maintaining industry standards and regulations is another important consideration, and financial services is one of the most regulated industries. Organisations need to be aware of new and existing legislation and guidelines designed to protect personal data and ensure that AI models are being developed with a focus on risk management and bias reduction.
Many countries are working on AI-related legal frameworks that cover new use cases and expand on laws that may already apply, like the General Data Protection Regulation (GDPR). Conformance with ever-evolving regulatory requirements adds another layer of complexity that organisations need to be mindful of.
For example, organisations will have to comply with the European Union’s Artificial Intelligence Act (EU AI Act) that came into effect last year. It’s the first standalone law focused solely on the development and use of AI. The legislation classifies AI systems based on the four levels of risk, progressing from minimal, limited and high to unacceptable.
High-risk systems, such as technology used for finance, must demonstrate they adhere to strict guidelines around safety, transparency and data governance. The penalties for violating the new law are steep, with fines of up to 7% of a company’s annual global revenue – similar to GDPR violations that can result in maximum fines of 4% of annual global revenue.
The industry puts AI to the test
Gen AI offers transformative opportunities, but rigorous testing is the key to success. Incorporating real-world testing early in AI development helps identify potential problems before they escalate. A proactive AI testing strategy offers significant benefits, including cost efficiency. Fixing a functional defect in the development stage costs far less than having to address it after the product has been launched.
By validating scenarios against real-world conditions, it’s possible to uncover edge cases and functional bugs, keeping the focus of later testing efforts on fine-tuning and delivering optimal user experiences. Financial services organisations should keep a watchful eye on AI testing trends and techniques to continuously validate their products, as new audiences, use cases and standards will continue to shape the digital landscape.
