Do not clean the export

The most common reason a first assessment underperforms is not the modeling. It is that the SCADA export arrived pre-cleaned. Someone reasonable, wanting to be helpful, removed the curtailed periods, filtered the outliers, and interpolated across the gaps - and in doing so removed most of the information that separates a genuine fault from a period of low wind resource.

Take each of those in turn. Curtailed periods removed means a model cannot learn which conditions trigger a constraint, so the constraint reappears later as an unexplained group deficit. Outliers removed means the physically impossible readings that identify a failing sensor are gone, and the sensor is then treated as trustworthy. Gaps filled means a manufactured value is presented as a measurement, and the model has no way to tell the difference, which is the worst of the three because it is undetectable downstream.

The general principle is that anomalies are the signal in condition monitoring. A conventional data pipeline treats an outlier as noise to be suppressed because its consumer is a report; a diagnostic pipeline treats it as the observation of interest. Applying the reporting convention to diagnostic data removes exactly the rows the diagnosis needed, and the resulting dataset looks cleaner by every summary statistic anyone would compute on it.

There is a legitimate counter-position. Operators clean exports because their own reporting requires it, and because raw historian output frequently contains artifacts - duplicated timestamps, daylight-saving discontinuities, unit inconsistencies - that break naive consumers. That is all true. The distinction that matters is between structural repair, which fixes how data is represented, and semantic filtering, which decides which observations count. The first is welcome. The second belongs to the analysis, not to the export.

The practical version of this is that the validation stage is a deliverable rather than a formality. Before any modeling runs, the transferred data is checked for channel coverage, timestamp integrity, frozen sensors, and physically impossible values, and the resulting report regularly identifies instrumentation faults that predate the engagement by years. Several operators have found the validation output more immediately useful than the first set of findings, which says something about how rarely anybody looks at raw operational data.

None of which means raw data is sufficient. A machine with three months of history, or one whose channel set cannot support an expectation, should be reported as out of scope rather than modeled at low confidence and listed alongside the rest. Refusing to produce a finding is a capability, and it is the one most easily traded away for a more complete-looking report.

The request, then, is unusually simple: send the record as the historian wrote it, tell us where automatic interpolation is enabled, and include the alarm and status log for the same period. Everything else - alignment, unit resolution, channel mapping - is our work, and it is the first stage of how a finding gets made.