Evidence quality · Condition monitoring · Product evaluation
What evidence supports a predictive-maintenance claim?
Preserve what was known before failure, measure whether warnings allow useful action, and evaluate the maintenance intervention separately from the prediction.
Preserve the sequence of evidence
Define the claim being evaluated: detecting a present fault, predicting a future condition or improving maintenance outcomes. For each evaluated event, keep the observation time, information available then, prediction issued and later observed outcome. These records make the sequence independently inspectable.
NASA researchers’ work on prognostic evaluation measures predictions against later outcomes while accounting for uncertainty. Their framework includes when predictions become sufficiently accurate, rather than judging only the last estimate before failure. An impressive retrospective plot leaves that timing question unresolved.
Freeze the evaluation procedure before reviewing the test outcomes. Reconstruct only the inputs that would have been available at each prediction time, including their original delays and missing readings. Label retrospective experiments clearly and reserve new operating periods for subsequent verification.
Measure useful warning under relevant conditions
The NASA paper defines prognostic horizon around a specified accuracy requirement and the time remaining before the observed end of life. Use the underlying question: does the warning arrive early enough for the response the operation could actually perform? State the uncertainty around that estimate.
Review missed events, warnings without the predicted event and the distribution of warning times. Separate equipment types and operating conditions that could change performance. Agree the observation window and event definition before calculating rates, and keep insufficiently observed assets visible.
Ask the responsible reliability engineers to judge the required evidence and operating limits. A useful planning prediction may still be unsuitable for a safety-critical decision. Preserve required maintenance and safety controls throughout any evaluation, with qualified safety review where the proposed use demands it.
Evaluate the intervention as a separate claim
A 2025 review by NIST researchers examines condition monitoring together with maintenance deployment, performance measures and evaluation methods. Its framework makes the operational intervention part of the assessment. Prediction performance alone leaves the effect of the resulting maintenance policy unanswered.
Record whether staff received the warning, what they decided, what work occurred and what happened afterward. An intervention can change the future outcome being measured. Have the evaluation owner explain how such cases enter the analysis instead of labeling every uneventful intervention a prevented failure.
For an improvement claim, specify the comparison, observation period and changes in equipment use or maintenance practice. Report prediction quality, intervention burden and operational outcomes separately. A credible conclusion makes clear which benefit was observed and which remains an assumption requiring further evidence.