Ryohei Fujimaki, Ph.D., founder and CEO of dotData explains Feature Engineering and why it needs to be automated https://bit.ly/2UMwZKs #datascience #AutoML
Machine learning can help enterprises prevent fraud, find anomalies and predict customer churn. The most critical step in AI/ML is to select the right features to train AI/ML models. Features are a crucial part of the data science workflow. Feature engineering is the process of using domain knowledge and statistics to transform raw data into a format that machine learning models can use. When making predictions about customer churn, we analyze historical behavior and create hypotheses, test them, and then make predictions about customer churn. ML algorithms extract the business hypothesis from historical data, such as logistic regression, decision tree, and support vector machine.
We may write many SQL-like queries to perform temporal aggregation on two tables to extract temporal user behavior patterns. Historical patterns can be the basis of a Machine Learning Model. But a model can only make accurate predictions if the hypotheses are correct. Feature engineering is an iterative process of generating features for a machine learning model, which requires domain knowledge and technical knowledge. It is not possible to automate. Feature engineering automation significantly lowers skill barriers in data science, allowing faster project execution without full domain knowledge.
The AutoML approach to machine learning automation addresses preparing raw data for machine learning and AI applications. AutoML must also resolve several challenges for modern organizations that leverage AI and machine learning. Data and feature engineering is automated with AutoML 2.0, streamlining FE automation and ML automation. Feature engineering is a vital part of machine learning and BI. AutoML 2.0 automates feature engineering and ML, helping enterprises to automate feature engineering and ML workflows. Read more about this at Datanami
