What is the main advantage of feature evaluation based approaches to dimensionality reduction, over the popular statistical ones (e.g. PCA)?
From what I think you mean by feature evaluation based approach, I am assuming you are referring to feature ranking, correlation filters and evaluation of features based on entropy and gain.
Based on the above assumption,
The main advantages of feature evaluation based approaches would be you have a much better understanding of the feature and how it is affecting the dependent variable/ outcome. It is better from a business perspective.Also feature evaluation based approaches provide a lot of scope for feature engineering and transforms, because as you know the model building process is iterative.
That said, however dimensionality reduction approaches have their own uses. These algorithms are essential when you have to deal with large number of features or anonymized features. In applications like image processing it is almost impossible to work with feature evaluation based approaches.
So to summarize, feature evaluation based approaches are better to use in a business setting where the model will be essential in bringing about a change in a real life process.