The treasure chest of data at the end of the real time payments rainbow

Steve Wilcockson, Data Science expert at KX


The digital payments landscape is in constant flux, as it’s shaped by a multitude of payment companies racing to meet growing payment demands. The growth is such that analysts struggle to pin down the total size of the global digital payments landscape. All anyone can agree on is that the number is massive. A Deloitte & Capital IQ survey suggests the value of transactions should reach 11.3Trillion by 2026 while, according to PwC, the number of cashless payments will double by 2030.

The introduction of new FinTechs, regulations and technologies have caused the payments landscape to evolve, leaving some of its incumbent payment players behind. Suddenly, keeping customers is the name of the game and it’s a difficult game to play. Payments companies are being kept on their toes to have a competitive edge in a market that expects fast, safe, simple, and affordable services. Whilst, regulators expect fair payment transactions to ensure ethical market practices and shareholders push for growth.

How can payment firms try to balance these sometimes-opposing priorities? The straightforward answer is data. Every transaction provides a significant amount of in-flight data on when, where, how often and how much, which – if analyzed in real-time and coupled with historical data – opens a treasure chest of data for payment companies to use. The data is there, but the challenge for payment firms today is in accessing, analyzing and making decisions on it fast enough for it to be useful.

Steve Wilcockson

Becoming real-time ready

Payments organizations are able to detect fraud by processing and analyzing the movement of transactions using decision management platforms. If potential fraud is detected, instant actions can be taken to cancel the transaction or alert the customer to the possibility of fraud and advise on appropriate decisions. This sounds simple in principle, but certain challenges need to be overcome before payment firms can truly extract value from real-time data analytics.

For example, speed is crucial when every microsecond counts, transaction analysis needs to happen as close to real-time as possible. This can be difficult if firms are stuck with clunky data architecture and slow databases and pipelines. Payment organizations need to make sure they also have scalable infrastructure that can handle processing the sheer volume of transactions, tens of thousands every single second, every day, 365 days a year. If they don’t, they risk missing out on insights that are most valuable at the moment they are analyzed. Finally, the expense of analyzing and processing terabytes of data, by monitoring transactions and complex machine learning – real-time or historical – has to be taken into account in the company’s bottom line.

Once these challenges are thought through, payment firms will be able to more effectively tackle fraud.  As the needed insights are being analyzed and fed into the right decision-making platforms at the right time to flag any potentially fraudulent activity.

The opportunities

Most payment companies can ’see’ the astronomical amount of data that flows through their platforms on a daily basis. But, analyzing and transforming this data into value is not so easily accomplished.

As companies grow, both organically and through M&As, a certain level of technical debt inevitably comes with it. Legacy systems can outgrow their usefulness and are no longer fit for purpose and others become siloed over time. The lack of interconnectedness and interoperability mean systems are unable to handle efficient processing in real time. It is essential for payment organizations to re-evaluate their operations and identify these issues and potential risks. The more a company is set up to utilize the insights extracted from data in real-time the more value-add services it can provide to its customers and remain competitive.

To leverage the most out of the data available, payment companies need the right technologies and protocols to make their analytics applications more scalable, connected and easy to deploy. A new approach to databases is needed, with advanced data analytics that boasts efficient Python integration and SQL querying of data. Allowing firms to deploy effective decision making in real time by detecting anomalies and processing the insights gathered to reach the coveted treasure trove of data.

Payments organizations have been using databases and data warehouses for years. However, outdated legacy systems and slow data infrastructures are holding some payment firms back whilst they try to stay ahead in an increasingly competitive market. Now is the time for these companies to relook at their systems and turn challenges, such as cost, scalability and speed, into opportunities. To deliver continuous streams of insights, both in real-time and combined with historical data, to reach the treasure chest of data at the end of the real time payments rainbow.

Explore more