Rule-Based Learning

Rule-based learning covers the machine learning methods that induce rules of the form «if these conditions hold, then this conclusion» from examples. The result is a model a person can read rule by rule, which sets it apart from neural networks and other opaque models.

It should not be confused with rule-based systems, where the rules are not learned but written by hand. What separates the two is where the rules come from, not what they look like.

Hand-Written Rules Versus Learned Rules

Classic expert systems ran on rules drafted by human specialists and an inference engine that chained them together. MYCIN was developed at Stanford during the 1970s to work on the diagnosis and treatment of infectious diseases. Rule-Based Expert Systems (1984), edited by Buchanan and Shortliffe, documents how its knowledge base grew through a cycle: the expert proposed or corrected rules, the knowledge engineer added them, and both checked the changes against cases. MYCIN did not learn: it is the canonical example of a rule-based system.

Rule-based learning reverses the process: an algorithm examines labeled data and generates the rules itself, what the field calls rule induction. Foundations of Rule Learning by Fürnkranz, Gamberger, and Lavrač (2012) is presented as both a textbook and a comprehensive reference on research in inductive rule learning.

The Classic Algorithms

Rivest formalized decision lists in 1987 — ordered sequences of rules — and proved they can be learned efficiently from examples. CN2 (Clark and Niblett, 1989) induced comprehensible rules while tolerating noisy data. In 1993, Holte showed with 1R that on most datasets studied, the best rule based on a single attribute was about as accurate as the rules induced by most of the systems evaluated. Cohen introduced RIPPERk in 1995 as a modified IREP algorithm, reporting nearly linear scaling with the number of examples and efficient processing of noisy datasets containing hundreds of thousands of cases.

Quinlan's C4.5 system (1993) starts from classified cases, looks for patterns, and can express them as decision trees or sets of if-then rules. The book's own contents devote separate chapters to building and pruning trees, turning trees into rules, and simplifying rule sets, so reducing it to “trees, not rules” erases an explicit part of the system.

What It Is Used for Today

Rule induction survives wherever understanding the model is mandatory. Writing in Nature Machine Intelligence in 2019, Rudin argued for designing inherently interpretable models for high-stakes decisions instead of explaining black boxes after the fact. As a separate example from that family, RiskSLIM by Ustun and Rudin learns risk-scoring systems with small integer coefficients and interpretability constraints. Agrawal and Srikant introduced Apriori in 1994 to discover association rules among items in large sales-transaction databases; the paper frames the problem in basket data and names applications such as cross-marketing, store layout, and segmentation by buying patterns. And the survey by Marra, Dumančić, Manhaeve, and De Raedt in Artificial Intelligence (2024) defines neuro-symbolic AI as the integration of symbolic reasoning and neural networks and compares systems across seven dimensions, including whether they learn parameters or structures. There is no reliable census of how widely rule induction is used in industry; what is documented is its role in these niches.

Limits

Rules handle unstructured data poorly — images, audio, free text — territory where deep learning dominates. Rule sets can overfit noise, which is why RIPPER prunes aggressively, and very long rule lists lose the readability that justifies them. Hand-written rule systems face a different bottleneck: extracting and maintaining the knowledge of human experts.