fbpx

Automated Feature Selection Methods to Assess Client Risk

By Hari Naryanan

In industries like Financial Services, Housing, and Insurance, the automated scoring of client risk can provide critical benefits. By quantifying the potential hazards for an organization, risk profiling helps organizations build sustainable long-term growth and minimizes losses in economic downturns. However, traditional client risk scoring methodologies are often not sufficient but can be augmented with machine learning-based models. Client Risk Profile Examples Popular use cases across different domains include: Which of my customers might default on loan payments? (Finance)Can a supplier deliver goods on time? (Manufacturing)Will my new tenant pay the rent on time? (Housing)What are the risks associated with a specific property or driver? (Insurance) Challenges with Traditional Risk Profiling Methods Traditionally, businesses solved these use-cases with rule-based approaches. A banker might have used a credit score and household income to determine loan eligibility. Simple rule-based logic is easy to implement but does not provide enough scalability and robustness…

An Improved Approach to Time-Series Forecasting

By dotData

New automated feature engineering tools remove the need to choose between accuracy and interpretability. Revenue and demand forecasting are among the most common use cases in data science, with abundant available data and clear business value across multiple industries. However, little agreement remains about the ‘best’ approach for building such forecasting models. New automated feature engineering tools are making that debate less relevant. Drawbacks of Traditional Forecasting Solutions Algorithms such as ARIMA (Autoregressive Integrated Moving Average), SARIMA (Seasonal Autoregressive Integrated Moving Average), or XGBoost, remain popular due to their flexibility and interpretability. In a search for superior predictive performance, practitioners have been learning and incorporating an ever-expanding catalog of AI-based neural network architectures into their forecasting. Despite often outperforming classical approaches in forecasting accuracy, neural networks are not a one-size-fits-all solution. Neural networks are mostly uninterpretable, leaving the business with limited insights into what’s driving the forecast, limiting business impact…

Reflections from ODSC East 2021

By Sachin Andhare

This was the second year in a row that the premier data science conference went virtual due to the Covid-19 pandemic. Overall the experience was much better this year with a breadth of research topics as well as industry coverage from machine learning for Time Series Data, Transformers in natural language processing (NLP) to Deep Neural Networks for visual quality inspection in manufacturing.

Is The Pace Of AI Development Slowing Down?

By Sachin Andhare

Got Data Science Platform, Visualization, and MLOps Tools, yet struggling with scaling AI? Feature Engineering holds the key to faster development! Data science, analytics, and BI leaders in disparate industries such as financial services, retail, and manufacturing have been spending heavily on AI tools, upgrading data infrastructure, augmenting BI with ML. Your organization may already have an AI Center of Excellence (CoE) to support LoB’s where the teams are building predictive applications that can predict churn, detect fraud, and forecast inventory.  Yet, for the vast majority of enterprise customers, the AI development has been slow, AI initiatives have not scaled according to expectations. What can you do to scale AI development, accelerate adoption and propel innovation?  You need to step back, analyze the data science process and focus on three core buckets in the  development workflow - Data Preparation, Feature Engineering, and Machine Learning. More specifically, answer three critical questions: Who…

The 10 Commandments of AI & ML (P1)

By Sachin Andhare

  Building an enterprise AI application is problematic. Scaling ML and getting user adoption is even more challenging. Here are the rules to follow. The big day is finally here! After months of squabbles, discussions, and fistfights, you have managed to get the budget allocated for evaluating machine learning. You have identified the right AutoML tool and are eager to get started on your first proof of concept. Where do you begin? This project’s success will determine the course of the digital transformation journey, production roll-out at other business units, and perhaps earn you a fancy new title! If you want to do ML the right way, you need to follow a set of commandments. In this two-part blog series, we’ll cover the ten rules. Here are the top 5 : Start with critical objectives for the project, and think about alignment with business goals.  Identify top use cases, key…

Demystifying Feature Engineering for Machine Learning

By Sachin Andhare

What is Feature Engineering FE is the process of applying domain knowledge to extract analytical representations from raw data, making it ready for machine learning. It involves the application of business knowledge, mathematics, and statistics to transform data into a format that can be directly consumed by machine learning models. It starts from many tables spread across disparate databases that are then joined, aggregated, and combined into a single flat table using statistical transformations and/or relational operations. Let’s say you are addressing a complex business problem such as predicting customer churn or forecasting product demand using applied machine learning. Assuming a team is in place and the business case identified, where do you start? The first step is to collect the relevant data to train the machine learning (ML) algorithms. This is usually followed by the selection of the appropriate algorithm or ensemble of algorithms. Choosing the right algorithm depends…

What IS Feature Engineering?

By Walter Paliska

What Is Feature Engineering?(And Why Do We Need To Automate it?) The past few years have seen the rapid rise in the adoption of Artificial Intelligence (AI) and Machine Learning (ML) for a multitude of commercial use-cases. Beyond the “cute” factor of AI that can pick a cat out of a photo array, AI and Machine learning are being deployed to model and predict lending risk, to understand and manage customer churn, provide product recommendations, help with programmatic advertising and much more. The challenge for the business community is that the underlying practice that is at the heart of AI and Machine Learning - data science - is rooted in a complex world of statistical analysis, data manipulation, programming and more. Most businesses don’t have enough data scientists - a fact illustrated by research in 2018 by LinkedIn that showed that there would be a shortfall of over 150,000 people…

AutoML and Beyond – Part 2

By dotData

Watch Part 2 (the Conclusion) of "AutoML and Beyond." With AutoML trending in data science, our CEO spoke at #Ai4Finance on data preparation, aggregating tables, feature engineering, the #AutoML process, and AutoML’s missing gaps.  Video: AutoML and Beyond - Part 2 Share On [social_warfare ] Related Articles

AutoML and Beyond – Part 1

By dotData

With AutoML trending in data science, our CEO spoke at #Ai4Finance on data preparation, aggregating tables, feature engineering, the #AutoML process, and AutoML’s missing gaps.  We’ll post the Conclusion / Part 2 next Thursday.  Video: Part 1 – AutoML and Beyond. Share On [social_warfare ] Related Articles

Ai4 Finance: dotData CEO Promotes Automated Feature Engineering

By dotData

We are proud to share a few photos from the #Ai4Finance2019 New York Conference.  With such a great turnout, our founder and CEO Ryohei Fujimaki displayed many of the new #datascience #innovations where dotData is paving the way for #AutomatedFeatureEngineering. Related Articles