Big Data doesn’t need to be big, or daunting

Big Data doesn’t need to be big, or daunting

Dr Nicolai Baldin

CEO & Founder at Synthesized

Views 760

Big Data doesn’t need to be big, or daunting

28.02.2020 12:00 pm

Collecting and collating customer information in order to create new, or enhanced products and services is a strategy as old as business itself, but the process involved has always been a huge time and resource drain for companies.

Yet in the digital era we now live in, companies are creating and handling more data than ever before. They are awash with it, to the point where many firms even struggle to identify what data is mission-critical. Digital transformation projects have put data at the centre of every organisation, with businesses estimated to have spent nearly $1.2 trillion globally to digitise their operations to fit the ever-expanding digital economy. 

With such transformations underway across huge swathes of businesses, they do still face two, quite fundamental issues around “big data”.

Firstly, many companies simply overlook the hidden costs behind data such as the time and resources required.  For example, for those involved in data science, it is estimated that 80% of their work is solely focused on the acquisition and preparation of data. This leaves them with just 20% capacity to focus on actual data analysis, which can provide crucial insights to propel a business forward.

Secondly, a significant issue is also how to actually access and share data to make it truly useful for commercial purposes, especially at a time when data is now seen as a competitive advantage for companies. For example, it is estimated that up to 60% of all data within an enterprise is unused.

Issues around how data can be safely shared are now paramount. In fact, businesses have never faced more complex regulatory requirements as they do now in regards to how data should be shared. The General Data Protection Regulation (GDPR), introduced in 2018 into the European Union to impose stiffer data protection regulations and require businesses to obtain consent in certain situations in order to process data.

Making this regulatory environment even more complex is that individual countries also have their own data protection laws. Germany has the Federal Data Protection Act (FDPA) which sets even stricter requirements for processing personal data than GDPR. Meanwhile, France has revised its Data Protection Act to impose stricter rules concerning biometric information collection. In Italy, the country is implementing more rigorous criminal sanctions around the unlawful processing of personal data.

The list goes on and on of countries seeking to deal with the issue of how data is collected and handled, with an eye, overall, on ultimately protecting personal, consumer data as much as possible.

With such hidden costs and regulatory requirements placed on businesses, big data would appear to be quite big, and quite daunting.

Yet technology is now disrupting (for the better) these central issues. Thanks to artificial intelligence (AI) and machine learning (ML) technologies huge advances are being made in how businesses can analyse data in a much more cost-effective and efficient manner, while also ensuring regulatory compliance.

These technologies have helped to create synthesized data, which is able to mimic original data, without any of the data protection headaches. How is such accurate data created? An ML engine generates synthetic data sets by learning and reinforcing the structure of information found in original data. It is able to move beyond surface statistics to capture and reproduce the complex multi-dimensional patterns underlying realistic data sets. Unlike anonymisation and encryption techniques, this approach enables any company to share better quality information in a format useful for machine learning and analytics, without, crucially, disclosing any information about individual data points.

You might ask how close to original data is synthetic data? Research from MIT found that synthesized data can give the same results as real, original data. Synthesized data also completely takes away the risk of non-compliance with data regulations such as GDPR. Finally, to top it off, the ML technology is so powerful it can generate this data in minutes and break the siloes associated with data completely, thus breaking those hidden costs associated with big data.

It can be hard to know where to start in trying to tackle the challenge provided by big data. Yet synthesized data is a real, viable approach which takes away many of the risks associated with collection and handling of huge amounts of data. The benefits of this approach are now too hard to ignore - knowledge and insights can be unlocked without jeopardising personal data.

Latest blogs

Otabek Nuritdinov Safenetpay

A strong fintech needs more than just access to funding

  Investors, both private and institutional, are excited about investing in fintechs that are in the payments services business. What are the issues that really should matter to you, as a client? In 2019, institutional investors Read more »

Martijn Bos Holland FinTech

Get your head up in the clouds, it’s good for business

How Digital Transformation is reshaping competition in financial services The message is clear and it’s coming at us from all sides: digitalize now. No business unit seems to be immune to the onslaught of cloud-based, AI-driven, real-time, Read more »

Sonny Aulakh Pure Storage

How to support remote working without compromising productivity

As the need to work remotely continues to impact the daily lives of people and businesses around the globe, it places unexpected demand on IT departments. How do you transition supporting 30% of your workforce to work remotely to 100% in a matter of Read more »

Martijn Bos Holland FinTech

Making it through the rain: Finance in times of turmoil

You’d need to be living on a remote island, without electricity or internet to not be aware of what the world is going through right now – a medical crisis that has spread across the world and disrupted supply chains, goods and services production, Read more »

James Devoy Sysnet Global Solutions

PCI DSS and Remote Assessments

COVID-19 is obviously changing many aspects of daily life. Some will be short term measures to see us all through these times, although I wonder how many will become more permanent fixtures in our lives. The PCI SSC has provided guidance to allow Read more »

Related Blogs

Grant Eggleton P2 Energy Solutions

Big Data in the Upstream Sector

It’s not the data itself, but the ability to put it towards some useful task that matters. In the oilfield, a simple yet powerful way of making use of data and predictive analytics is for automatic surveillance by exception. By learning from the Read more »

Danny Molhoek, Lexmark

How Big Data and IoT Can Drive Behaviour Change

Today, you can control the temperature in your house from your office. Your car can update its maps from the garage. You can even boil the kettle for your morning cup of tea before leaving your bed. All this is possible due to the convergence of the Read more »

Rahul Singh HCL Technologies

Putting Your Money on Big Data and Analytics for Guaranteed Wins

Banks and financial services have always had a voracious appetite for data. They have a finger on the pulse of company balance sheets, interest rates, money supply, exchange rates, inflation rates, stock market movements, bond yields, customer Read more »

Ivy Schmerken FlexTrade

Quant Funds Get Sentimental About Big Data

Big Data Fed by Social Media Social media is creating a torrent of data every minute.  But how much of this firehose is relevant to making better investment decisions? As hedge funds and asset managers eye an explosion in unstructured data from Read more »

Ivy Schmerken FlexTrade

Managing Big Data After Brexit

After the UK’s June 23 referendum to leave the European Union, traders and investors awoke to a global sell off in stocks, a surge in U.S. equity trading volumes, and volatility in foreign exchange rates as the British pound sterling fell to Read more »

Free Newsletter Sign-up
+44 (0) 208 819 32 53 +44 (0) 173 261 71 47
Download Our Mobile App
Financial It Youtube channel