Predictive maintenance from sensors and logs

Predictive Maintenance from Sensors and Logs is a plant operations process in which machines continuously produce sensor readings and log entries that are monitored for signs of emerging faults. At present, shift supervisors and maintenance technicians manually review data from the MES system, machine logs and sensor feeds, diagnose whether an anomaly represents a real fault or a false alarm, decide on priority, and raise a service ticket by hand. The process runs several times a week in bursts, totalling around 500 cases per year at roughly 45 minutes of manual effort each, and the detection almost always happens reactively, after a machine has already slowed or stopped.

The desired end state is a system that ingests sensor and log data continuously, scores each anomaly against historical patterns, and automatically opens and routes a service ticket for routine high-confidence cases without any human touch. Cases where the model is uncertain, where the fault pattern is non-standard, or where sensor data is noisy are flagged for review by a shift supervisor or technician, who confirms or dismisses the alert. Their decisions feed back into the model to sharpen future classifications. The compliance context is significant: product safety and traceability requirements mean that every automated decision must be logged in enough detail to support a recall investigation, so human oversight on uncertain cases is a regulatory and liability consideration, not just an operational preference.

The client team consists of a plant manager, two shift supervisors, and a maintenance technician. The automation engineer, who has a PLC background and is actively learning Python, will build and maintain the pipeline. The primary system is a legacy MES with a readable database, supplemented by machine logs, sensor feeds, PLC systems, email, and spreadsheets. The readable MES database and existing Python capability are the two most important assets for the build.

The recommended path is a Hybrid AI workflow with human-in-the-loop gates, and the reasoning comes directly from the shape of this specific process rather than from any general preference for AI solutions. Around 75% of cases follow recognisable patterns that a well-trained anomaly scoring model can classify with high confidence, which makes those cases strong candidates for full automation. The remaining 25% involve non-standard fault signatures, noisy sensor data, or insufficient historical precedent, and on those cases the combination of product-safety compliance requirements and real recall risk means autonomous action is not acceptable. The hybrid design treats the human review queue not as a fallback for when the system fails but as a permanent, intentional feature of the architecture. That distinction matters for how you build and communicate the system internally.

The client's preferred tech stack, Python and the readable MES database, is genuinely well matched to this approach, and it is worth being explicit about why. A Python-based ingestion and scoring pipeline can connect directly to the MES database without any additional middleware layer, which removes a significant source of integration complexity and ongoing maintenance overhead. The automation engineer's PLC background is an underrated asset here: understanding how machines produce data and what normal operating envelopes look like is often the hardest knowledge to acquire when building an anomaly model, and this team already has it. The recommended build is therefore feasible with the existing team, with targeted upskilling rather than external specialist hiring.

Comparing the Hybrid path to its nearest alternatives makes the case clearer. A fully automated AI Workflow scored six out of ten, and it is viable only if confidence thresholds are set so conservatively that uncertain cases always route to human review anyway, which effectively recreates a hybrid under a different name. There is no meaningful benefit to calling it fully automated if the compliance requirements force you to retain human gates regardless. An AI Agent scored five, and the honest assessment is that full autonomy on a safety-critical process is premature before the team has accumulated a substantial period of supervised operation and before the automation engineer has grown significantly beyond current Python capability. Agents are powerful but they require robust observability and error-handling discipline that takes time to develop. Traditional Code also scored five. Rule-based threshold alerting on the MES database is something the automation engineer could implement today, and for some machines it may be exactly the right starting point. The limitation is that static rules will generate the false alarms the client is already frustrated by, because real fault signatures in a mixed machine fleet are rarely clean enough for simple thresholds to distinguish signal from noise. A statistical or lightweight ML scoring layer adds real value over pure rules, and the data the team already holds is sufficient to build one.

There are honest tradeoffs to name. The hybrid path is more complex to build and maintain than a purely rule-based system. The model requires retraining as machines age and fault taxonomies evolve, and that ongoing maintenance commitment needs to be owned by someone, ideally the automation engineer with a defined quarterly review cadence involving the plant manager. The audit trail design, covering sensor values, model scores, timestamps, and reviewer decisions, must be treated as a first-class requirement from the start, not something retrofitted after go-live. And the entire system's reliability depends on the quality and completeness of the MES history and sensor coverage, which is why the data audit in the first implementation milestone is the most important single step in the project. If the data foundation is weak, the model will produce too many false positives and the team will stop trusting it quickly. Confirming data quality before writing a single line of model code is not caution for its own sake; it is the primary risk mitigation for the whole programme.

The top contender is the Hybrid path, and the case for it over a Fully Automated AI Workflow comes down to one concrete constraint: product safety compliance. A fully automated AI Workflow could handle the straightforward majority of detections well, but the moment it auto-closes a low-confidence case that turns out to be a real fault, the business is exposed, because there is no human sign-off in the log and no clear accountability if that fault reaches a customer. Setting the confidence threshold high enough to avoid that outcome effectively recreates a human review queue anyway, so you end up with a hybrid in practice without the intentional design that makes it reliable and auditable.

The AI Agent path scores lower not because the technology is wrong for this domain, but because full autonomy is premature here. An agent that can orchestrate data collection, scoring, ticket creation and escalation end-to-end is a reasonable long-term direction, but it requires a level of model maturity and team capability that does not exist yet. The automation engineer is still building Python skills, the fault taxonomy is only partially explicit, and the machine fleet is mixed in terms of sensor coverage. Running an agent over that foundation without a supervised period first is how you get a system nobody trusts within two months of launch.

Traditional Code, meaning static threshold rules written in Python against the MES database, is worth addressing because it is the path the team might instinctively reach for. Rule-based alerting is understandable, cheap to build, and the automation engineer could have something working quickly. The problem is that static rules struggle with the variety of fault patterns the team themselves described, and they are the primary source of the false alarms the plant manager is already frustrated with. Adding a statistical or lightweight ML scoring layer over the same data infrastructure delivers meaningfully better precision for a modest increase in build complexity, and that difference compounds over 500 runs a year.

The build starts with a data audit before a single line of model code is written. The automation engineer, working with the maintenance technician, maps every data source: the MES database schema, the machine log formats, and which machines in the fleet actually have usable sensor feeds. The output is a clear picture of which historical fault records are complete enough to train on and which machines will need manual observation to fill the gaps. This milestone typically takes two to three weeks and it is the most important one, because a model trained on incomplete history will produce the false alarm rates that kill trust in the system on day one. Tools here are straightforward: Python with pandas and SQLAlchemy to query the MES database directly, and whatever file-based access the machine logs expose.

Once the data foundation is confirmed, the automation engineer builds a scheduled ingestion pipeline in Python that reads from the MES database and available sensor feeds, normalises the time-series readings into a consistent format, and writes them to a local store, something as simple as a PostgreSQL instance or even structured flat files at this stage. The pipeline runs on a cron schedule or a lightweight orchestrator like Prefect or Apache Airflow, depending on how much operational overhead the team is comfortable managing. The key design requirement is idempotency: the pipeline must handle gaps, retries and duplicate readings without corrupting the history it is building, because that history becomes the training data for the model.

The anomaly scoring layer sits on top of the ingestion pipeline and this is where the intelligence lives. For a first version, a statistical approach using isolation forest or a simple LSTM-based sequence model trained in scikit-learn or PyTorch is appropriate. The maintenance technician acts as ground truth during validation, reviewing a sample of past cases and labelling them as true faults or noise. Confidence thresholds are set conservatively: anything below a defined score goes to a human review queue rather than auto-generating a ticket. High-confidence detections create service tickets automatically via the existing system, whether that is a shared email inbox, a tool like Jira Service Management, or a direct write back to the MES if it supports it. Every automated decision, the sensor values that triggered it, the model score, and the timestamp, is written to an audit log in a format that can be queried during a recall investigation.

The supervised pilot phase is where the system earns the team's trust. For four to six weeks the pipeline runs in parallel with the existing manual process, and shift supervisors review every automated decision alongside their normal checks. False positives and missed detections are flagged, labelled, and fed back into the model as new training data. This phase is also when confidence thresholds get tuned to the actual noise characteristics of this specific machine fleet, something no amount of upfront modelling can substitute for. At the end of the pilot, the plant manager reviews the false alarm rate and detection rate against the targets set at the start, and makes the call to retire the manual process. Ongoing governance is a quarterly review where the team checks for model drift, adds any new fault types encountered since the last review, and confirms the audit log is complete and queryable.