Hello,
While generally ensemble methods are used on the dataset for the same type of prediction algorithm,I was wondering if one could use a combination of predictions from different models-like from randomforests,knn and svm for forming ensembles and then applying it on train data.
Can someone please help me understand how this might be done?
Yes, we can combine the output of different models and this is the one of the type of ensemble modeling. Here we simply take a simple or weighted average of the predictions from different model. For business problems with binary target variables like predicting likelihood of fraud this sometimes give better result in terms of predicting the event rate. The major limitation of this approach is that the overall accuracy of this combined model lies between the accuracies of the individual models. So this method is definitely not advisable for modeling continuous variables as overall accuracy matters most there.
Regards,
Imran
