Fujitsu Laboratories of Europe has developed an innovative artificial intelligence-based technology that overcomes traditional data reconciliation challenges, applying cutting edge Dynamic Data Loading (DDL) to automate complex knowledge processing. Fujitsu’s technology significantly simplifies the data integration and reconciliation process, dynamically loading data during the runtime process using AI and microservice-based crawlers to create a fully scalable and intelligent platform.  

Fujitsu has focused on solving the problems associated with analyzing huge volumes of data reliably, accurately and rapidly in the financial domain. Fujitsu’s reconciliation technology is unique in the combination of advanced features used for record linkage. The main innovation lies in the entity type and domain recognition methods that use the system’s knowledge base to recommend the candidates. Domain recognition provides considerable context to the user and to the system itself to facilitate the reconciliation process. Another innovationstems from the use of AI to automate tasks, with the system learning from user decisions and feedback, resulting in a progressively more customized user experience, matched by a high level of automation for simple, repetitive tasks. The technology is based on Fujitsu’s cutting edge entity reconciliation and microservice orchestration techniques, transforming high volume data reconciliation practice for any application. 

Dr Adel Rouz, CEO of Fujitsu Laboratories of Europe, explains: ‘‘We have extensive co-creation experience, particularly resulting from deep learning and AI projects in the financial services sector, and have applied this to develop an innovative new approach to complexdynamic data loading and integration challenges. Our AI-based approach breaks new ground in terms of the scalable, accurate and intelligent analysis of huge volumes of data. We have combined three essential ingredients : a powerful data reconciliation mechanism with advanced and flexibile analysis technologies, supported by an interactive virtual assistant. Our initial application involves a platform for regulatory authorities, with future applications involving any industry handling massive data volumes, such as the finance sector, healthcare, retail and manufacturing. It is an exciting development, which we are confident has wide-ranging future potential - for example being applied to identify cross-border relationships, by integrating open data sets from around the world.’’ 

Fujitsu’s new approach applies a novel methodology for record linkage, using the system’s knowledge base for enhanced entity type and domain recognition. For example, it uses linkage types such as company and company, and company and person. In most cases, the incoming dataset lacks data property descriptions or any reference to standard vocabularies/ontologies. Therefore, the person in charge of the data integration has to guess relevant meta-information (e.g. the meaning of property names) or to ask the data provider. With Fujitsu’s technology, the dataset is contextualized, with minor input from the user. Additionally, the system’s capability to learn from user actions is an important reconciliation feature. In Fujitsu’s reconciliation technology, learning from user decisions and feedback results in a more customized user experience and a high level of automation for simple and repetitive tasks.

The figure above details the reconciliation workflow components of DDL technology, together with an example of reconciled company datafrom the finance sector. The workflow is divided into four main blocks:

•           Data Property Reconciliation module
•           Entity Type and Domain Reconciliation module
•           Entity Disambiguation module
•           Knowledge Base Storage

During a one-month trial period, data reconcilation times and data loading times were significantly reduced, typically taking one week compared to one month previously. Importantly, the process of reconciling a financial dataset, such as the one used in the example, produces a knowledge graph that can be used for future data reconciliation tasks.

Fujitsu Laboratories of Europe is a Center of Excellence for Fujitsu’s advanced research into machine learning and deep learning, as part of the digital solutions and services being developed under the Fujitsu’s Human Centric AI approach Zinrai. Fujitsu Laboratories of Europe’s activities include extensive collaboration and co-creation with Fujitsu customers and research organizations across Europe, including San Carlos Clinical Hospital in Madrid (with the HIKARI AI intelligent healthcare solution), the University of Seville (data analytics for tourism applications), and the 5G Innovation Centre in the UK.