Decision trees are one of the oldest machine learning techniques and still one of the most useful for business applications. Unlike neural networks, which are essentially black boxes, decision trees are interpretable, you can read exactly why a prediction was made.
The Basic Structure
A decision tree is a series of yes/no questions applied to your data. At each node, the algorithm asks a question about a variable. Depending on the answer, the data goes left or right. At the end of the tree, the leaf nodes, you get a prediction or classification.
When to Use Them
Decision trees work well when you need to explain your model's logic to a non-technical stakeholder. When the data has non-linear relationships but you do not have enough data to train a neural network. When interpretability is a compliance requirement. For business rules that need to be auditable.
The Limitation
Individual decision trees overfit easily, they learn the training data too well and perform poorly on new data. This is why Random Forests (many trees averaged together) are almost always preferred in practice. A single decision tree is useful for exploration and explanation. For production predictions, use an ensemble.