What makes data science operationalization so hard?

There are four critical aspects of data science operationalization that make it challenging to implement. First, is the quality of code. Because data scientists use tools like Python and R to develop models, the code is often not of “production quality.” Moving the code to production means that a fair amount of rework has to take place to re-code the models using SQL code that is native to the production database.

Integration viability

The second problem is the integration challenge. Integrating data and scoring pipelines with the multitude of systems that are often associated with data science projects requires a lot of integration work that is time-consuming and highly technical.

Model Monitoring & Maintenance

Even when models are appropriately integrated, they must be maintained. Accuracy of metrics and model prediction accuracy must be continuously monitored, and models need to be adjusted over time as data changes. This process involves retraining models regularly, which is time-consuming and expensive.

Scalability

Data science models often rely on a tiny subset of the full available data set. In a churn model, for example, the models might be developed on less than 40% of the available data, but in production, the models need to scale to process 100% of available customer data to predict churn. Another aspect of scalability is the ability of the server to scale up and down depending on the level of power required. Many customers underestimate the computer power required and have problems when ML models break or fail.

Portability

In most organizations, the data science team uses software tools and configurations that are often markedly different from production environments. That means that taking models developed by data scientists and operationalizing them entails porting the code to platforms and systems not initially taken into account during model development.