Considerations for including a feature / input in a neural network?



I have a big Neural network, which I have trained. You can consider this as version n of the current model. Now I get an additional variable / feature (which can be a independent variable or also have correlation with existing features).

What are the considerations to include / exclude this feature in the neural network? Is there any standard way to decide this?



Here are a few tricks I have learnt over time:

  • If the new feature has been derived from already existing features, you can almost leave it out for sure. Neural network would likely do the derivation themselves.
  • If the new input / feature has a high correlation with existing features (> 0.7), you can leave it out as there will be very little value in adding this feature. On the other hand, if the feature has low correlation (< 0.3), you are likely to gain from including it in the network. It is kinda trial and error for the range in between.
  • You can also check the correlation between the output and the inputs - a high score means the input will surely have information related to the output. On the other hand, if it is low, it may or may not be useful. If the variable has non-linear relationship, the correlation could be low, but neural networks would be able to learn the non linear relationship.

Hope this helps.