How to explain an output of Neural Nets to a business user?



Dear all,

How do we explain a output of neural nets to a business user. I find this to be a serious limitation when using neural network for Risk Analytics.

Balaji S.R



You are right, one of the biggest problems with neural networks is that black box nature of it’s output. It doesn’t provide you equations to relate to the inputs and understand the relationships.

One of the papers, which I read recently mentioned following technique in some other context. However, I think this can help you with what you are trying to do:

  1. Train the neural net and get the predictions out of neural nets.
  2. Next, use a simpler technique (e.g. Regression) on the prediction from neural network to understand the relationships. This might sound counter-intuitive, but the paper mentioned that these simpler techniques, can achieve similar results as that of neural networks.
  3. You can then use the output from these techniques to explain the model to the business users.

As I said, the technique was mentioned in a different context and I can’t remember the author / link to the article, but this technique can help you decipher some of the black box computations in neural nets.

Hope this helps.




Thank you for the explanation.

Balaji S.R


@Kunal, Are you saying using the final predictors used in NN, run a regression and use rank predictor based on standardised beta/test statistic/p values etc?



No, What I am saying is that you use NN to get the predictions and then use regression on the predictions from Neural nets. This would give you a far simple model with most of the learning from NN in most of the simple scenarios.


Ok. For everyone knowledge i am stating in detail what we discussed. Please let me know if this is not what you meant.

Looking at confusion matrix of NN model separate out predicted=1 & predicted=0 .
Then run 2 different logistic model for predicted =1 & predicted=0.



@Kunal can you confirm this is what you meant?


Yes…you are right in interpreting it.


Is there any typical neural nets metrics to validate the NN output like other predictive modelling techniques?