Statistical Machine Learning Week 12 Hyperparameter Optimization KerasTuner Hyperband Example

>> YOUR LINK HERE: ___ http://youtube.com/watch?v=pC5EsO92R4M

#neuralnetworks #deeplearning #tensorflow • What are hyperparameters? • Variables that control the training process and structure of a machine learning model. • Set before training begins and remain constant during training. • Have a significant impact on the model's performance. • Types of hyperparameters: • Model hyperparameters: Affect the architecture of the model. • Examples: Number of hidden layers in a neural network, number of neurons per layer. • Algorithm hyperparameters: Influence the learning process itself. • Examples: Learning rate in gradient descent, regularization strength. • • Common techniques for hyperparameter optimization: • Grid Search: You define a grid of possible values for each hyperparameter, and the algorithm evaluates every single combination. • Simple to implement and guarantees finding the best combination within the specified grid. However, it is computationally expensive. • Random Search: Instead of exhaustively trying every combination, random search samples a predefined number of configurations from the hyperparameter space. • Often more efficient than Grid Search, especially when some hyperparameters are more important than others. However, it does not guarantee finding the absolute best combination. • Bayesian Optimization: This technique uses a probabilistic model (often a Gaussian Process) to learn the relationship between hyperparameters and model performance. • Can be more complex to implement. The performance depends on the choice of the probabilistic model and acquisition function. • Hyperband: This is an early stopping technique designed for hyperparameter optimization. It focuses on quickly identifying and stopping poorly performing configurations, allowing more resources to be allocated to promising ones. • • • Although hyperparameter optimization is rapidly advancing, comprehensive coverage of this topic in textbooks remains limited. However, a wealth of research articles provides valuable insights into this crucial aspect of machine learning. • For a thorough overview of the techniques mentioned earlier, the following paper offers a particularly comprehensive and up-to-date analysis. • Bischl, Bernd, et al. Hyperparameter optimization: Foundations, algorithms, best practices, and open challenges. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 13.2 (2023): e1484. • Link: https://wires.onlinelibrary.wiley.com... (Open Access) • • Keras, and specifically the Keras Tuner library, provides tools for implementing hyperparameter optimization techniques. We will explore some of these techniques in this lecture. • Install and import the Keras Tuner: • ```pip install -q -U keras-tuner``` • • ```import keras_tuner as kt```

#############################

New on site