is it possible to combine logistic regression, SVM, decision tree for classification

# Combine Classification Agorithms

**tillutony**#2

Hi Erigits,

on High level answer inline.

**Multiple classifiers - Definition.**

• Multiple classifier – a set of classifiers whose individual

predictions are combined in some way to classify new

examples.

• Various names: ensemble methods, committee, classifier

fusion, combination, aggregation,…

• Integration should improve predictive accuracy.

**Why could we integrate classifiers?**

• Typical research → create and evaluate a single learning

algorithm; compare performance of some algorithms.

• Empirical observations or applications → a given algorithm

may outperform all others for a specific subset of problems

•There is no one algorithm achieving the best accuracy for all

situations!

• A complex problem can be decomposed into multiple subproblems

that are easier to be solved.

• Growing research interest in combining a set of learning

algorithms / classifiers into one system

„Multiple learning systems try to exploit the local

different behavior of the base learners to enhance

the accuracy of the overall learning system”

**Multiple classifiers** – why do they work?

• How to create such systems and when they may perform

better than their components used independently?

• Combining identical classifiers is useless!

A necessary condition for the approach to be useful is

that member classifiers should have a substantial level of

disagreement, i.e., they make error independently with

respect to one another

**Why do ensembles work?**

• The Statistical Problem arises when the hypothesis space is too large

for the amount of available data. Hence, there are many hypotheses with

the same accuracy on the data and the learning algorithm chooses only

one of them! There is a risk that the accuracy of the chosen hypothesis is

low on unseen data!

• The Computational Problem arises when the learning algorithm cannot

guarantee finding the best hypothesis.

• The Representational Problem arises when the hypothesis space does

not contain any good approximation of the target classes

Hope this helps

Regards,

Tony

**jalFaizy**#3

Hi @erigits, in addition to @tillutony I would recommend you read this blog on ensemble learning.