Which Activation Function Should I Use?
3,070
by Super User, 7 years ago
All neural networks use activation functions, but the reasons behind using them are never clear! Let's discuss what activation functions are, when they should be used, and what the difference between them is.
Sample code from this video:
https://github.com/llSourcell/Which-Activation-Function-Should-I-Use
Please subscribe! And like. And comment. That's what keeps me going.
More Learning resources:
http://www.kdnuggets.com/2016/08/role-activation-function-neural-network.html
http://cs231n.github.io/neural-networks-1/
https://www.quora.com/What-is-the-role-of-the-activation-function-in-a-neural-network
https://stats.stackexchange.com/questions/115258/comprehensive-list-of-activation-functions-in-neural-networks-with-pros-cons
https://en.wikibooks.org/wiki/Artificial_Neural_Networks/Activation_Functions
https://stackoverflow.com/questions/9782071/why-must-a-nonlinear-activation-function-be-used-in-a-backpropagation-neural-net
https://papers.nips.cc/paper/874-how-to-choose-an-activation-function.pdf
http://neuralnetworksanddeeplearning.com/chap4.html
https://medium.com/towards-data-science/activation-functions-in-neural-networks-58115cda9c96
https://medium.com/autonomous-agents/mathematical-foundation-for-activation-functions-in-artificial-neural-networks-a51c9dd7c089
Join us in the Wizards Slack channel:
http://wizards.herokuapp.com/
And please support me on Patreon:
https://www.patreon.com/user?u=3191693
Sample code from this video:
https://github.com/llSourcell/Which-Activation-Function-Should-I-Use
Please subscribe! And like. And comment. That's what keeps me going.
More Learning resources:
http://www.kdnuggets.com/2016/08/role-activation-function-neural-network.html
http://cs231n.github.io/neural-networks-1/
https://www.quora.com/What-is-the-role-of-the-activation-function-in-a-neural-network
https://stats.stackexchange.com/questions/115258/comprehensive-list-of-activation-functions-in-neural-networks-with-pros-cons
https://en.wikibooks.org/wiki/Artificial_Neural_Networks/Activation_Functions
https://stackoverflow.com/questions/9782071/why-must-a-nonlinear-activation-function-be-used-in-a-backpropagation-neural-net
https://papers.nips.cc/paper/874-how-to-choose-an-activation-function.pdf
http://neuralnetworksanddeeplearning.com/chap4.html
https://medium.com/towards-data-science/activation-functions-in-neural-networks-58115cda9c96
https://medium.com/autonomous-agents/mathematical-foundation-for-activation-functions-in-artificial-neural-networks-a51c9dd7c089
Join us in the Wizards Slack channel:
http://wizards.herokuapp.com/
And please support me on Patreon:
https://www.patreon.com/user?u=3191693
-
Super User uploaded a new media, Which Activation Function Should I Use?
7 years ago
