Course Content

DECISION TREE CLASSIFIER

Decision Trees (DTs) are a non-parametric supervised learning approach for classification and regression. The objective is to build a model that predicts the value of a target variable using basic decision rules learned from data attributes. A tree can be thought of as a piecewise constant approximation. In the example below, decision trees learn from data to approximate a sine curve using a series of if-then-else decision rules. The deeper the tree, the more complicated the decision rules and the more accurate the model.

Decision trees could be used for classification and also regression. DecisionTreeClassifier is a class capable of performing multi-class classification on a dataset. As with other classifiers, DecisionTreeClassifier takes as input two arrays: an array X, sparse or dense, of shape (n_samples, n_features) holding the training samples, and an array Y of integer values, shape (n_samples,), holding the class labels for the training samples:

The code below, is the example how DecisionTreeClassifier is capable of both binary (where the labels are [-1, 1]) classification and multiclass (where the labels are [0, …, K-1]) classification, by using the Iris dataset.

  • from sklearn import tree
    X = [[0, 0], [1, 1]]
    Y = [0, 1]
    clf = tree.DecisionTreeClassifier()
    clf = clf.fit(X, Y)

    clf.predict([[2., 2.]])
    clf.predict_proba([[2., 2.]])

    from sklearn.datasets import load_iris
    from sklearn import tree
    iris = load_iris()
    X, y = iris.data, iris.target
    clf = tree.DecisionTreeClassifier()
    clf = clf.fit(X, y)
    tree.plot_tree(clf)

    import graphviz
    dot_data = tree.export_graphviz(clf, out_file=None)
    graph = graphviz.Source(dot_data)
    graph.render("iris")
    dot_data = tree.export_graphviz(clf, out_file=None,
                          feature_names=iris.feature_names,
                          class_names=iris.target_names,
                          filled=True, rounded=True,
                          special_characters=True)
    graph = graphviz.Source(dot_data)
    graph

    from sklearn.datasets import load_iris
    from sklearn.tree import DecisionTreeClassifier
    from sklearn.tree import export_text
    iris = load_iris()
    decision_tree = DecisionTreeClassifier(random_state=0, max_depth=2)
    decision_tree = decision_tree.fit(iris.data, iris.target)
    r = export_text(decision_tree, feature_names=iris['feature_names'])
    print(r)

THE MAIN CODE OF DECISION TREE

  • from sklearn.tree import DecisionTreeClassifier

Full Code of Decision Tree Classifier Exercise

See the full code below to get the final result like the figure above: