Which metric is used to quantify how mixed a group is, often used to measure impurity in decision trees?

Get ready for the GARP Risk and AI Exam with flashcards and multiple choice questions. Each question comes with hints and explanations. Prepare for success!

Multiple Choice

Which metric is used to quantify how mixed a group is, often used to measure impurity in decision trees?

Entropy measures how mixed the class labels are at a node. In decision trees, impurity refers to that mixing. If every sample in the node belongs to one class, entropy is zero, meaning no uncertainty. If the classes are evenly distributed, entropy is at its maximum, indicating high impurity. The formula -sum p_i log2 p_i uses p_i as the proportion of each class in the node, which makes entropy a natural gauge of uncertainty. Splits aim to reduce this uncertainty, or entropy, leading to higher information gain and purer child nodes. Accuracy, precision, and recall, by contrast, describe overall predictive performance or class-specific performance, not how mixed a node’s labels are. That’s why entropy is the best choice for quantifying impurity in decision trees.

Entropy measures how mixed the class labels are at a node. In decision trees, impurity refers to that mixing. If every sample in the node belongs to one class, entropy is zero, meaning no uncertainty. If the classes are evenly distributed, entropy is at its maximum, indicating high impurity. The formula -sum p_i log2 p_i uses p_i as the proportion of each class in the node, which makes entropy a natural gauge of uncertainty. Splits aim to reduce this uncertainty, or entropy, leading to higher information gain and purer child nodes. Accuracy, precision, and recall, by contrast, describe overall predictive performance or class-specific performance, not how mixed a node’s labels are. That’s why entropy is the best choice for quantifying impurity in decision trees.

More practice questions