Big Data is transforming how many companies operate. By collecting an enormous amount of data from a variety of sources and subjecting it to in-depth analysis, the door has been opened to unprecedented insight into areas such as customer behavior, predictive marketing and real-time product insights. Those harnessing Big Data in this way are realizing benefits such as increased revenue, greater workforce productivity, lower customer churn, and improved product and supply chain efficiency.

As we enter ‘The Age of IoT,’ the number of data points requiring analysis jumps from the millions to the billions. No longer is there time to upload a large database into an analytics engine, spend a few days crunching numbers and gradually draw conclusions. By then it will be much too late: customers will have moved on, insight and business advantage will be lost and opportunities squandered.

By collecting data from sensors, components and every conceivable type of equipment and system, tens of billions of data points become available for analysis. IT research firm International Data Corp (IDC) estimates that by 2020 the digital universe will reach 44 zettabytes, or 44 trillion gigabytes. That’s almost a tenfold increase over current totals.

Big Data and IoT in the Transportation Revolution

Big Data and IoT will revolutionize the transportation industry in the coming years. As we enter an age of automated, self-driving cars an incredible amount of information will be generated by all manner of IoT devices and analyzing and interpreting actionable insights from this massive amount of data will be the challenge of successful companies. 

The only way to accomplish this is through the introduction of predictive analytics and machine learning to constantly observe the endless numbers of data points and identify evolving patterns. In essence, the machine learns how to correlate related activities and score those associations based upon intelligent algorithms. When combined with data analytics in the context of IoT, machine learning automation uncovers trends and relationships in data sets far too complex or large for a human to deal with. What’s more, its accuracy increases with use courtesy of adaptive algorithms.

The goal of companies in the coming transportation revolution will be to effectively deploy these machine learning algorithms. Giving them the capability to react in real-time to subtle shifts in customer behavior, data anomalies across devices, and shifting market dynamics and quickly provide meaningful insights for the sales, marketing and data teams.    

Revolutionizing Urban Parking with Predictive Analytics: Case in Point with Parking-Meter Giant, Parkeon

Take the case of transportation giant Parkeon, who created Path to Park, the first mobile app for urban parking assistance that predicts city areas where drivers are more likely to find available parking. The application uses real-time IoT data from parking meters that is processed with the machine learning and predictive analytics capabilities within Dataiku’s Data Science Studio (DDS).  

The resulting application processes a massive volume (Gb/s) of IoT data from the millions of transactions coming from parking meters. The data is ultimately combined with geographical information from OpenStreetMap and used to predict the behavior of cars and traffic by building a real time model of each city that can analyze and predict ‘parking pressures’ on city streets.

Detecting Data Anomalies: How Data Science can Improve the Accuracy of Real-Time Data in IoT devices  

Another example is Coyote, the European leader in real-time road information, who has developed an experimental machine learning data science platform that analyses anomalies in their IoT devices to improve the accuracy of road information for their 4.8 million users across Europe.

Coyote’s IoT devices and apps rely heavily on the accuracy of incoming data.  In particular, driving speed limits within their embedded maps have to updated constantly with real-time data from other Coyote IoT devices.

The company used Machine Learning within Dataiku DSS to develop an algorithm that leverages vast amounts of IoT-derived data.  The solution segments roads into sections and analyses patterns in each section. This enabled Coyote to build a predictive model that estimated the speed limit of the road section. The Machine Learning process facilitated the detection of speed limit anomalies and, consequently, enabled Coyote to estimate the global quality & reliability of the displayed speed limit.

The R&D project has increased speed limit reliability by 9% on a sample test, potentially resulting in more efficient and accurate real-time road information and travel time calculations for the 4.8 million Coyote users across Europe. In addition, thanks to the new platform’s focus on teamwork & cooperation, which enabled employees with differing skill-sets to work together, Data Mining & Visualization are now widespread within Coyote and there is a growing awareness of Smart Data issues.

IoT Data Science with Dataiku DSS

These solutions were made possible by the technology behind Dataiku (www.dataiku.com), maker of the collaborative data science software platform Dataiku Data Science Studio (DSS).  Dataiku DSS makes it possible for organizations to reap the benefits of data science thanks to a collaborative interface for both expert and beginner analysts and data scientists. Dataiku offers a complete and accessible advanced analytics software platform that leverages all the tools and technologies required to develop a data research and development department that can build data-driven solutions to improve business.

Dataiku DSS can be used to quickly build predictive services and data products that transform raw data into business impacting products including:

• Churn Analytics

• Fraud Detection

• Logistic Optimization

• Data Management

• Demand Forecasting

• Spatial Analytics

• Lifetime Value Optimization

• Predictive Maintenance

• and much more