How ReLU really does introduce non-linearity to a neural network
Quora user Nathan, make a really nice and useful comment on Quora, (https://www.quora.com/Why-is-ReLU-non-linear/answer/Nathan-Yan-2?comment_id=39524791&comment_type=2), about how ReLU really does introduce non-linearity to a neural network. Respect. He made a short little program that simulates a random ReLU neural network, with one hidden layer. The network takes a scalar input, and produces a scalar output. To generate […]Read More