This note is based on Coursera course by Andrew ng.
(It is just study note for me. It could be copied or awkward sometimes for sentence anything, because i am not native. But, i want to learn Deep Learning on English. So, everything will be bettter and better :))
INTRO
What are hyperparameters?
MAIN
Parameters our model are W and b, and there are other things we need to know such as the learning rate alpha, the number of iteration, the number of hidden layers, the number of hidden units, and choice of activation function. These parameters control the ultimate parameters W and b. So, we call all of these things above hyper parameters.
When we are training a deep net for our own application, we find that there may be a lot of possible settings for the hyper parameters that we need to just try out. So, applied deep learning today is very impirical process where we often might have an idea for the best value of hyper parameters.
For example, alpha = 0.01 is that we want to try. Then we try it out and see how that works, and the based on that outcome we might want to chagne the learning rate to 0.05.
CONCLUSION
One rule of thumb is just try a few value for the hyper parameters and double check if there is a better value for the hyper parameters and as we do so we slowly gain intuition as well about the hyper parameters that work best for our problems. It seems like unstisfying part of deep learning that we just have to try on all the values, but this is one area where deep learning research is still advancing.