How to Optimize Hyperparameter Tuning for Machine Learning Models

Ryohei Fujimaki, Ph.D., founder and CEO of dotData, discusses why enterprises must readjust hyperparameters as part of any ongoing maintenance.  Full article on TechTarget | Search Enterprise AI –  https://bit.ly/2JgevNc

Optimizing the hyperparameters of an AI model comes at a cost, and experts recommend that tuning these parameters during the design process instead of retrofitting them is a way to ensure better performance and accuracy. For example, hyperparameters may need to be tuned while training a new model for a customer in California. Enterprises should make hyperparameter tuning part of their design process. If the process relies on manual effort, the AI model quality is likely to decrease over time. Zillow uses hyperparameter optimization in its development process to tune machine learning algorithms for particular use cases, improving performance.

Machine learning algorithms require hyperparameter tuning for noisy data but not for clean data. It’s essential to tune hyperparameters as part of your routine. Linda is a fan of hyperparameter tuning and believes it is important to make it part of the training process. The recommendation is to test the best possible combination of hyperparameters on a subset of the training data to identify the final configuration. Data science platforms can use AutoML to evaluate new model parameters early in the training process. Simpler models learn more from training data, while complex models overfit the training data.

Machine learning scientists should use prudence to select suitable models, depending on the application. Hyperparameters in model training affect the time it takes for the model to converge. The data scientist needs to specify many hyperparameters before training a neural network on the data. A hyperparameter tuning process consists of deciding on a method for tuning hyperparameters and a tradeoff between time, compute, and tooling costs. When will hyperparameter tuning be done in the development cycle? Tuning is done in tandem with iterating your feature set and model. Hyperparameters for a model must be tuned for business goals and not for model accuracy. Read more at TechTarget