A Vision of Rapid, High-Quality Data Analysis for All Businesses
By Ryohei Fukimaki, dotData CEO and Founder
It is becoming increasingly important for enterprises to leverage data to be competitive. Yet, there are three challenges related to embracing data utilization that all businesses share: it takes time, advanced skills and expertise are required, and the potential outcomes are not fully maximized. Together, these challenges make it difficult for enterprises to fully leverage their data for business growth.
Data analytics is not simply prediction by machine learning. Rather it is a process involving many different steps, including data preparation, feature engineering, machine learning, visualization, and model operationalization. Until now, completing this process for just a single project would have taken months. Moreover, a wide variety of highly-skilled personnel are needed for each step – such as domain experts, data scientists, data engineers, and BI engineers – and processes and outcomes have tended to be highly dependent on the experience and intuition of each individual. For feature engineering in particular, it has long been thought that this step can only be done by experts, as it requires deep domain knowledge. Furthermore, the results derived from machine learning have tended to be “black-box”, so often these results could not be fully leveraged in businesses. For enterprises to benefit from the full utilization of their data, it is necessary to resolve these challenges and streamline data analysis and application.
dotData’s approach to data science solves these problems through AI and automation. The development of the dotData Platform stemmed from my experience in leading more than 100 data analysis projects at NEC, across a variety of industries. What I found is that, no matter the industry, a common thought process could be applied to how to build the data analytics process. From that experience, I was able to invent automated feature engineering, which previously had been the most time-consuming and manual step, requiring high levels of skill and domain knowledge. The automation of feature engineering is core to dotData in that we can use AI to design hypotheses for features, and automate analytical processes that are applicable to various industries, businesses, or data.
Because dotData can automatically execute data analysis processes from data preparation through feature engineering and machine learning, to model operationalization, it solves the data analytics challenges related to time and skill sets that have existed until now. For example, a data analytics use case for a customer of a financial business, which previously required two or three months of work by data scientists, can now be done in less than a day, with equal or better accuracy. As it becomes possible to complete projects significantly faster, there will be an exponential increase in the number of experiments and the discoveries of new use cases. In addition, our approach provides full transparency and interpretability where the basis for the derived results is apparent and, therefore, can easily be implemented in business operations with high confidence and accountability. As data analytics becomes more efficient, enterprises can operationalize it as part of their everyday processes and accelerate their data-driven initiatives. We have made it possible for all businesses to utilize AI and machine learning, and have in fact already achieved major results across a number of industries.
As data science automation is adopted, processes that once relied on peoples’ experience and intuition will instead be executed efficiently using data. As a result, enterprises of all types will be able to analyze data more efficiently, create better products and services, and be more productive, ultimately benefitting society as a whole.