No improvement using boosting


I have a dataset which consists of continuous predictors. Problem is binary classification and I have used RF model with 75% accuracy on test set.
I have tried boosting alog gbm and xgboost but no improvement in accuracy.
Can anyone guide on what could be the issue.
Thanks in advance.




I would suggest you to focus on Feature engineering (create new features or feature transformation for better explanation of target variable). You need to focus on “Hypothesis generation” and “Data Exploration” stages more to have better understanding about data set, this will help you to create new feature or feature transformation. Post this you can focus on Ensemble methods (combine diverse models) to have better results.

You can focus on these articles to have better understanding about ensemble methods:

Hope this helps!



Where are you on the train set ? 95% or 78% ? If 78% as mentioned features 95% then try to increase the size of you training set.
Hope this help


Thanks Imran for the insights and useful links.