I tried to hypertune the dropout layer, no of layers, activation function for each layer and number of units in each layer. I used hyperband to tune the hyperparameters.
Everytime I got a different optimal results for each of these hyper parameters. I also noticied that the number of trials each time was 30. Is there a way to increase the number of trials so that all possible hyper parameter combination is evaluated? The objective is to get the same set of hyper parameters after tuning the hyper parameters. The hyper parameter space that I tried are as follows:
#Activation Function
hp_activation = hp.Choice(âactivationâ, values=[âseluâ,âreluâ,âsigmoidâ,âtanhâ])
#Dropout
hp_dropout = hp.Float(âdropoutâ, min_value=0.1, max_value=0.9, step=0.1)
#Learning Rates
hp_lr = hp.Choice(âlrâ, values=[1e5, 1e4, 1e3])
#Number of Layers
hp.Int(name=âdense_layer_numâ, min_value=2, max_value=20)
#Number of Units in each layer
hp.Int(âunits_â+str(i), min_value=16, max_value=512, step=16)
Hi @neerajkumar
Welcome to our community!
Just for my understanding, which is the range of the different optimal results for each of these hyper parameters?
About your question " Is there a way to increase the number of trials so that all possible hyper parameter combination is evaluated?" maybe you can remove the âstop_earlyâ callback.
Anyway I donât expect improvements because early stopping is used precisely to âmonitor the validation loss and stop training if itâs not improving after N epochsâ (in this case 5).
Hope this can help
BR
Hi @fabioantonini,
Below are the values that I got for different runs. I ran the tuner 6 times, last two times without early stopping. Still the number of trials were 30.
Run No 
1 
2 
3 
4 
5 
6 
Activation Function 
selu 
relu 
selu 
tanh 
tanh 
selu 
Dropout Rate 
0.1 
0.3 
0.1 
0.6 
0.4 
0.4 
No of Layers 
12 
2 
5 
14 
18 
6 
No of Units in 1 Layer 
468 
128 
416 
336 
496 
336 
No of Units in 2 Layer 
48 
144 
416 
208 
256 
144 
No of Units in 3 Layer 
512 

480 
304 
480 
480 
No of Units in 4 Layer 
128 

320 
160 
272 
304 
No of Units in 5 Layer 
464 

496 
416 
112 
16 
No of Units in 6 Layer 
224 


112 
272 
416 
No of Units in 7 Layer 
192 


48 
240 

No of Units in 8 Layer 
368 


272 
384 

No of Units in 9 Layer 
416 


416 
32 

No of Units in 10 Layer 
224 


64 
384 

No of Units in 11 Layer 
288 


224 
384 

No of Units in 12 Layer 
384 


400 
432 

No of Units in 13 Layer 



240 
320 

No of Units in 14 Layer 



304 
128 

No of Units in 15 Layer 




448 

No of Units in 16 Layer 




208 

No of Units in 17 Layer 




144 

No of Units in 18 Layer 




112 

Hi @fabioantonini ,
Upon some research, I found that max_trials argument is allowed in BayesianOptimization Tuner, RandomSearch Tuner and as part of Oracle argument in Sklearn tuner. But for Hyperband Tuner, which I am using, I did not find any way of specifying max_trials.
A few sites specified that the max trails in hyperband tuner depends on the max_epochs and hyperband_iterations, but I have not been able to increase the number of trials by changing either of two.
1 Like