Decision Tree

A decision tree is a model that makes predictions by asking a sequence of simple questions about the input—like a flowchart you might use to troubleshoot a problem. Each question narrows down the possibilities until the model reaches an answer.

How a decision tree works

A decision tree splits the data into smaller and smaller groups using rules such as “is income > 60k?” or “is the word ‘free’ present?”. Each internal point is a node, each question is a split, and the final endpoints are leaves. For classification, a leaf predicts a class (often the majority class of training examples that reached it). For regression, a leaf predicts a number (commonly the average target value in that leaf).

Choosing good splits (the learning part)

Training a tree means picking splits that make the child groups “purer” with respect to the target. Common criteria include Gini impurity and entropy (information gain) for classification, and mean squared error reduction for regression. Trees keep splitting until a stopping rule kicks in, such as maximum depth or a minimum number of samples per leaf. Without these controls, trees can overfit by memorizing noise.

Why it matters in practice

Interpretability: You can inspect the rules, which helps in credit scoring or medical triage.
Handles mixed data: Works with numeric and categorical features (often after encoding).
Strong baseline and building block: Single trees are useful, and they power ensembles like random forests and gradient boosting.

from sklearn.tree import DecisionTreeClassifier
model = DecisionTreeClassifier(max_depth=4, random_state=0)

A Decision Tree is a supervised learning model that predicts an output by applying a sequence of feature-based tests, represented as a tree of split rules leading to leaf predictions (a class label for classification or a numeric value for regression). It matters because it provides an interpretable, nonparametric baseline that handles nonlinear interactions and mixed feature types, and it serves as a core building block for ensembles like Random Forests and Gradient Boosted Trees.

A Decision Tree is like a flowchart you might use to make a choice: “Is the email from someone you know?” If yes, go one way; if no, go another. Keep asking simple questions until you reach a final answer, like “spam” or “not spam.”

In supervised learning, a Decision Tree learns these question-and-branch rules from examples where the right answer is already known (called “labeled” data). It can be used to sort things into categories (like fraud vs. not fraud) or to predict a number (like a house price). The big appeal is that its decisions are easy to follow and explain.