What is the standard way for doing validation for a Machine Learning model?



Can you really believe in your model after training? Not until you have validated it. So how would one approach this problem? Also suggest some resources for this validation pipeline


I used the R "caret" package for Cross Validation. There are loads of articles on how caret package can be utilised for this purpose.


Could you give a link to the resources? (Possibly resources in python would be preferable)


You can refer these links

But these are for R. Some Python expert can share any additional information on this.


The link http://machinelearningmastery.com/how-to-estimate-model-accuracy-in-r-using-the-caret-package/
has 5 different validation method. While developing any model is there any logic in choosing a particular validation technique?


I typically start with Data Split or Bootstrap and build a couple of features and models.
During the later stages, I use K-fold Cross Validation most of the times.

Repeated K-fold and Leave-one out take very long to complete depending on how much data you have.
If time is not a constraint , then one should also try these so as to prevent over-fitting.