Layers in a neural network



1-How do we decide the number of layers to be used in a neural network model for a classification task ?

2- Is there an alternative to the sigmoid function that can be used in a neural network layer?

3- Can you please give me an intuitive explanation of what layers in a neural network do?



Hi @NSS,

  1. Deciding the number of layers of a neural network is a hyperparameter (like deciding the k value for k Nearest Neighbor). It is problem dependent and also your requirements. It is generally observed that increasing the number of layers increases accuracy, but there are some caveats (like overfitting and high resource utilization)

  2. Yes there are. There’s tanh, rectified linear unit, etc. (And research is still going on for more!). Here’s a link to some commonly used activation functions.

  3. This video gives an in-depth understanding of how neural network works. In a network architecture, each layer (which contains numerous neurons) works on the previous layer, extracting more global and invariant features. This can be explained in a face detection problem as that the start layers work as an edge detector, extracting lines and boundaries, whereas as you go deep, you get mouth and eye detector, and as you go deeper, you get whole face detectors


Thank you so much for taking out the time and answering. The sources you mentioned were very helpful.