Decision Tree Evaluation

Learning (how the splits would be chosen from data, e.g. CART or ID3/C4.5)

1. Start with all training rows at the root.
2. For every feature and every candidate threshold, split the rows in two and measure the impurity of the children:
   • Entropy: H = −Σ p_c · log2 p_c over classes c; information gain = H(parent) − weighted average H(children).
   • Gini impurity: G = 1 − Σ p_c², the chance of mislabeling a random row with a random label from the node. CART picks the split with the lowest weighted Gini.
3. Keep the best split, recurse on each child.
4. Stop when a node is pure, too small, or too deep; the leaf predicts the majority class. Pruning later removes splits that do not help on validation data.

Example: 14 days with 9 "yes" and 5 "no" have entropy −(9/14)log2(9/14) − (5/14)log2(5/14) ≈ 0.940 bits. A split whose children are purer lowers the weighted entropy; the largest drop wins.

Walkthrough

Case 0 evaluates four days (one per leaf):

day 1 {humidity 60, wind 10, temp 25}
  humidity 60 <= 75  -> left;  wind 10 <= 20  -> left   leaf "yes"
day 2 {humidity 85, wind 5, temp 30}
  humidity 85 > 75   -> right; temp 30 > 21   -> right  leaf "no"
day 3 {humidity 70, wind 30, temp 18}
  humidity 70 <= 75  -> left;  wind 30 > 20   -> right  leaf "no"
day 4 {humidity 90, wind 12, temp 18}
  humidity 90 > 75   -> right; temp 18 <= 21  -> left   leaf "yes"
answers = ["yes", "no", "no", "yes"]

Complexity

When to use it

• Strengths: human-readable rules, no feature scaling needed, handles mixed numeric and categorical data and non-linear interactions, very fast prediction.
• Weaknesses: a single deep tree overfits and is unstable (small data changes can change the whole tree); axis-aligned splits approximate diagonal boundaries poorly.
• vs. random forests / gradient boosting: many trees averaged or added together are far more accurate and stable, at the cost of interpretability.
• vs. logistic regression: linear models extrapolate smoothly and need less data; trees capture interactions without feature engineering.
• vs. k-nearest neighbours: trees summarize the data into rules and predict in O(depth) instead of scanning the training set.

Common pitfalls

• Inconsistent tie handling at the threshold: decide once whether value == threshold goes left (<=, as here) and use the same rule when learning and predicting.
• Missing features: day[feature] is undefined, and undefined <= 75 is false, so the walk silently goes right. Validate inputs or learn a default direction.
• Growing the tree until every leaf is pure: perfect training accuracy, poor generalization. Limit depth or prune.
• Information gain favours features with many distinct values (like an ID); C4.5's gain ratio corrects for that.
• Comparing categorical features with <=: encode categories properly or use multiway splits.

What to watch in the visualizer

• tree is drawn as a binary tree: internal nodes are labelled with their feature, leaves with the answer.
• In predict, node steps down one level per comparison; value shows the feature value being tested.
• answers fills with one label per day.

Example

Input: days = [
  { humidity: 60, wind: 10, temp: 25 },
  { humidity: 85, wind: 5, temp: 30 },
  { humidity: 70, wind: 30, temp: 18 },
  { humidity: 90, wind: 12, temp: 18 }
]
Output: ["yes", "no", "no", "yes"]

Constraints

• Every day has numeric humidity (%), wind (km/h) and temp (°C).
• 0 <= days.length <= 100