Three tests for industrial AI before you buy it

On July 25, 2026, discussion of factory AI often puts three very different tasks under one label: predicting failures, detecting defects, and reducing energy use. The promise sounds similar —greater efficiency—, but each task uses different data, changes a different decision, and needs its own metric. When they are blended, a successful vision test can be marketed as though it proved predictive maintenance.

The 60-second decision

Before buying a system, complete this sentence: “we want the system to recommend or perform ___ in order to improve ___, measured as ___.” If the answer is “use AI to optimize the factory,” there is not yet a testable problem.

NIST’s 2026 roadmap for smart manufacturing, published July 3, identifies these applications alongside less visible challenges: industrial data quality, integration of sensors and heterogeneous systems, and reliable, explainable operation. The issue is not simply getting a larger model. A plant needs to see what the system detects, what action it triggers, and what changes afterwards.

Maintenance: prediction is not repair

Predictive maintenance begins with condition signals: vibration, temperature, pressure, cycles, alarms, and maintenance records. NIST’s asset condition management framework describes the aim as assessing current condition, diagnosing it, and estimating future health so maintenance can be planned.

The measure is not model accuracy alone. It can be unplanned downtime, maintenance cost per asset, or avoided failures without creating unnecessary replacements. A correct alert that arrives too late does not prevent a breakdown; an overly sensitive one can fill a schedule with false inspections. The operator should see the signal, the threshold, and the decision: inspect, reduce load, or continue.

Quality: finding a defect is not enough

Inspection uses images, measurements, or process data to flag anomalies in a part or batch. The NIST MEP guide separates defect detection from failure prediction and resource management. Useful measures here include customer escape rate, false rejects, scrap, and inspection time.

The critical question is what happens after the result. A system can flag a weld, but someone must decide whether to repair, reject, or sample further. Keeping images and reviewed decisions exposes whether performance degrades when lighting, material, supplier, or camera changes. Without that loop, a laboratory accuracy score says little about a real line.

Energy: optimize one variable without moving the problem

For energy, a system may forecast demand, suggest schedules, adjust setpoints, or spot anomalous consumption. A useful measure is often energy per good unit produced, not total kilowatt-hours. Stopping a line cuts consumption, but it does not make production more efficient. Quality, safety, delivery requirements, and demand peaks also matter.

NIST lists data availability, initial cost, skills, privacy, and legacy-system integration as barriers. They are implementation conditions, not administrative details. If an energy meter cannot be linked to product, shift, and machine state, a model may optimize a number that does not represent operating cost.

A pilot you can audit

Start with one asset, one part family, or one production cell. Record a baseline for a defined period. Specify who receives the recommendation, which actions are allowed, and when the system is reversed. Compare results with an equivalent operation and retain both successes and errors.

Do not promise a savings percentage before measuring it. The documented evidence says these methods can support maintenance, inspection, and resource management; it does not show that every plant will get the same effect. The durable skill is demanding the full mechanism: input data, decision, plant metric, and human review. When those four elements are clear, industrial AI stops being a label and becomes a hypothesis a factory can test.