AI Automation for BI Professionals

What if you could tell that an essential robotic process in your manufacturing line was about to break down? What if your finance department could provide you with a list of customers most likely to default on their payments? What if your marketing department could rank the planned campaigns in their budget based on the likelihood of success? The answer to these, and countless other questions, are at the heart of predictive analytics. As the world of Business Intelligence (BI) continues to evolve, describing “what happened” through dashboards and reports is no longer sufficient. To provide genuine value, modern BI professionals must frequently deliver dashboards and reports that can help line-of-business users make better, smarter decisions faster. Of course, the challenge is that most BI systems do not have predictive analytics capabilities “out of the box,” and purpose-built predictive systems are often too limiting or designed for use-cases that are too inflexible for the majority of organizations.

For most BI organizations, the big challenge in delivering useful, user-friendly predictive dashboards is developing the underlying technology that powers predictive systems: Artificial Intelligence and Machine Learning. The automation of AI development was made possible in recent years with the advent of AutoML platforms. These new platforms provided a new impetus for modern-day BI developers to expand their toolset to offer higher value to the organization. The problem, however, is that most AutoML platforms cater to data scientists – not BI users – so how do you move forward? What does a BI team look for in an AutoML platform to ensure a faster time-to-value and higher likelihood of success? To make the right choice, BI organizations must look for five key features in their AI/ML Automation toolset:

1 – Start With The State of Your Data

When it comes to data, most BI teams have a great deal of confidence in the readiness, cleanliness, and state of their data. After all, to build effective and compelling dashboards and reports, data consolidation, unification, and cleansing are paramount. The reality is, however, that for what may be an “acceptable” level of impurity in your BI reports might cause problems in the ML models that will be at the heart of your predictive dashboards. While data “prep” is quickly becoming a core feature in the world of AutoML, it’s essential to understand the level of manual effort required to connect to and prep the data for AI/ML purposes. A real modern AutoML platform – or AutoML 2.0 – should provide a high degree of automated data prep and cleansing, doing the heaviest lifting for the BI team before the data is fed to any models.

2 – Feature Tables – The Heart of your AI Model

Before even embarking on a predictive analytics project, BI teams must understand and embrace the concept of the “feature table.” In fact, unlike BI and dashboarding, where relational data is king, the ML algorithms that are at the heart of your future predictive models rely purely on “flat” tables. These consolidated “mega tables” contain all the rows of data needed to build models and the “right” mix of columns of data sourced from various systems of record to provide richness to your model. The building of this “feature table,” as it’s known, creates the biggest headaches. In a typical ML model building scenario, multiple iterations of “trial and error” must be performed before the right mix of columns from disparate source systems provide the best outcomes. The complexity of “testing use cases” is why building feature tables can often take months of arduous labor that involves large multi-discipline teams. AutoML platforms are notorious for claiming that they perform “automated” feature engineering – but the devil is in the details. AutoML 2.0 platforms must provide a fully automated means of attaching to your pre-built data warehouse and automatically building feature tables through robust AI processes. With the right AutoML 2.0 platform, your BI team can take the feature engineering process from months to just a few hours.

3 – ML Optimization – The Right Model for The Right Use-case

With your Feature Table built, your BI team is now ready to test multiple ML algorithms to understand which one will perform best for any given use-case. Once again, the key is automation. Building and optimizing your ML models by hand requires deep expertise in data science, domain knowledge and a fair amount of time. The selection and tuning of ML algorithms are where most AutoML platforms will perform quite nicely. AI automation enables the system to test, validate hundreds of algorithms and select the one or an ensemble of algorithms that will optimize the outcome based on business requirements and constraints. AutoML 2.0, automates the data and feature engineering, is streamlining FE automation and ML automation as a single pipeline and one-stop-shop. With AutoML 2.0, the full-cycle from raw data through data and feature engineering through ML model development takes days, not months, and a team can deliver 10x more projects.

4 – Deployment Options – Real-Time or Not?

With an ML model developed, how do you move into your production environment? Once again, having the right platform is a critical part of the process. With most AI/ML models, there will be two forms of integration – either directly into your BI dashboarding environment – or (for specialized applications) – in the form of containerized deployments. Unless your AI model addresses particular use-cases (like factory automation), the best course of action will likely be to integrate your AI/ML model from your platform into your BI platform of choice. In any deployment scenario, being able to quickly and dynamically update and redeploy AI/ML models as your business needs change must be a critical component of your strategy. As the events of the past nine months have shown, external conditions can suddenly make models obsolete or – at the very least- in bad need of update. It’s critical to be able to make changes as quickly as possible with little, if any, downtime.

Automation – The Heart of the Matter

In the world of predictive analytics, moving from the desire of predictive dashboards to actual deployed, usable dashboards that provide tangible value can be an arduous, costly, and painful journey. Accelerating the timeline to develop and implement the underlying AI/ML models requires investing in AI Automation tools that can go beyond merely automating the selection and optimization of your ML algorithms. AutoML tools must also provide means of taking the hardest parts of the process out of your hands to help you focus on building business value for the organization, instead of building feature tables and prepping data.
Automation enables more people, such as BI developers, analysts, or data engineers to incorporate predictive analytics in an agile manner at scale.