What Is a Process Historian? Industrial Data Historians Explained
A process historian explained — how time-series historians store plant data, how they differ from SQL databases, compression, and how PLC/SCADA tags get logged.
A process historian is a purpose-built time-series database that continuously records the value of every instrumentation tag in a plant — temperature, flow, pressure, motor current, valve position — at rates from milliseconds to minutes, and stores that data for months or years. Unlike a general-purpose relational database, a historian is designed specifically for the write-once, read-many, time-ordered nature of industrial process data: it compresses raw samples by 80–95 % without losing engineering accuracy, retrieves millions of data points in milliseconds, and serves that data to operators, engineers, and analytics systems simultaneously.
Historians sit between the real-time control layer (PLCs and SCADA) and the business/analytics layer (MES, ERP, digital twins). They are the institutional memory of a process plant.
What a Process Historian Is
At its core, a process historian is a server application that:
- Collects tag values from PLCs, DCS controllers, SCADA servers, and field instruments via OPC, proprietary drivers, or direct database connectors.
- Compresses those values using lossless or near-lossless algorithms before writing to disk.
- Stores the compressed data in a proprietary or open time-series format optimised for sequential reads.
- Serves data back to clients — trend displays, reports, analytics engines, REST APIs — with automatic decompression and interpolation.
Every data point carries three fields: tag name, timestamp, and value (sometimes with a quality flag). That triple is the fundamental atom of historian storage. A plant with 10,000 tags sampled every second generates roughly 864 million data points per day before compression — which is why a historian exists and a spreadsheet does not.
Anatomy of a Historian
| Component | Role |
|---|---|
| Collector / Interface | Reads tag values from the source (PLC, SCADA, DCS) via OPC DA/UA or a vendor driver |
| Compression engine | Reduces stored samples using deadband or swinging-door compression before write |
| Time-series store | The on-disk data store, optimised for chronological sequential access |
| Retrieval engine | Handles client queries: raw, interpolated, summary (avg, min, max, etc.) |
| Client API layer | Exposes data via OPC HDA, REST/GraphQL, ODBC, or a native SDK |
Why Not Just Use a SQL Database?
This is the most common question from engineers first encountering historians. SQL databases are excellent tools — but they are fundamentally unsuited to continuous industrial time-series data for three reasons.
1. Write Throughput and Scale
A medium-sized plant might have 50,000 tags sampled at 1 Hz. That is 50,000 INSERT operations per second, every second, indefinitely. A conventional SQL engine uses row-based storage with indexes maintained on every write. At that scale the index maintenance overhead alone saturates a server. Historians use append-only columnar or compressed binary storage — writes are fast because data is simply appended chronologically with no index update per row.
2. Compression at Acquisition Time
Process values do not change uniformly. A tank level may sit at 45.3 % for 30 minutes before a pump starts. A SQL database stores every second-by-second reading. A historian applies deadband or swinging-door compression (see below) and stores only the readings that represent a meaningful change. Compression ratios of 10:1 to 20:1 on slowly changing analogue values are routine, rising to 100:1 on steady-state values.
3. Time-Oriented Retrieval
Historians answer queries like "give me the average reactor temperature for every 15-minute interval over the past six months" in sub-second time by design. Relational databases must scan and aggregate row data; a historian's retrieval engine is purpose-built for time-range scans, interpolation between stored samples, and statistical summaries.
When should you use a SQL database instead? For event logs, alarm logs, batch records, or any data where you need JOIN operations across relational entities. Most industrial architectures use both: the historian for continuous analogue and digital tag history, and a relational database for discrete records and metadata.
How a Process Historian Works
Step 1 — Collectors and Interfaces
A collector (sometimes called an interface) is a software agent that polls or subscribes to a data source and forwards values to the historian server. Common source protocols include:
- OPC DA — legacy poll-based Windows interface; still common in older plants. See our OPC UA tutorial for the modern successor.
- OPC UA — the preferred modern standard; supports subscriptions so the source pushes changes rather than the collector polling on a fixed interval.
- OPC HDA — historical data access; used to back-fill the historian from another historian or from a DCS's internal buffer.
- Vendor drivers — proprietary interfaces for DCS systems such as Emerson DeltaV, Honeywell Experion, or Yokogawa CENTUM.
- MQTT/Sparkplug B — increasingly used for IIoT and PLC integration, particularly when data moves through an MQTT broker.
Each collector is configured with a scan class — the rate at which it reads source values. Typical scan classes are 500 ms, 1 s, 5 s, 30 s, and 60 s. Faster scan classes increase load on both the source device and the historian; choosing an appropriate scan rate is an engineering judgment based on the speed of the process.
Step 2 — Tags and the Tag Database
Every signal recorded in a historian is associated with a tag (also called a point or PI point in some vendors' terminology). The tag defines:
- Tag name — typically mirrors the PLC/SCADA tag name (e.g.,
REACTOR_01.TEMP_IN) - Engineering units — °C, bar, m³/h
- Scan class — the collection rate
- Compression parameters — deadband and exception deviation values
- Data type — float, integer, string, Boolean
Well-structured tag naming is critical. A plant that starts with ad-hoc tag names ends up with a historian that nobody can query without intimate plant knowledge. The SCADA best practices guide covers tag naming conventions that translate directly to historian configuration.
Step 3 — Compression
Compression is where historians earn their keep. Two algorithms dominate industrial practice.
Deadband (Exception Reporting)
Before a new value is stored, the historian checks whether it has changed by more than a configured exception deviation (deadband) from the last stored value. If the change is less than the deadband, the value is discarded. If it exceeds the deadband, it is passed to the next stage.
Example: A temperature sensor with a deadband of 0.5 °C. If the tag reads 82.1 °C and the previous stored value was 82.3 °C, the difference is 0.2 °C — below the deadband — so the reading is not stored. Only when the temperature moves to 82.8 °C or higher (or 81.8 °C or lower) will a new sample be stored.
Swinging-Door Compression
What is swinging-door compression? Swinging-door compression is a lossless linear interpolation algorithm that stores only the inflection points of a continuously changing signal, discarding intermediate samples that lie within a straight-line error tolerance of the stored trajectory. Imagine holding a door hinged at the last stored point: as new values arrive, you swing the door to follow them. When a new value falls outside the angular tolerance of the door, the previous value is stored and the door re-hinges. The result is that a linearly ramping signal (such as a temperature rise during start-up) is stored with only its start and end points — every intermediate sample is recoverable by interpolation.
The combination of deadband and swinging-door gives historians their compression power: steady-state values are discarded by deadband; ramp segments are compressed to two points by swinging-door.
Step 4 — Retrieval and Interpolation
When a client requests data, the historian retrieval engine:
- Locates the relevant time range in the compressed store.
- Decompresses and, if the client requested interpolated data (evenly spaced timestamps), reconstructs intermediate values by linear interpolation between stored samples.
- Optionally aggregates: average, minimum, maximum, standard deviation, time-weighted average over user-defined intervals.
Time-weighted average (TWA) is a critical concept in process industries. Because samples are not evenly spaced after compression, a simple arithmetic average over-weights transient spikes and under-weights long steady-state periods. TWA weights each sample by the duration it was held before the next change, giving a mathematically accurate average of the process condition over the time window.
Historian vs SCADA vs Relational DB vs Data Lake
| System | Primary Role | Data Model | Query Strength | Retention |
|---|---|---|---|---|
| SCADA / HMI | Real-time monitoring and control | Live tag values | Current value, short trend | Minutes to hours |
| Process Historian | Continuous time-series archive | Tag + timestamp + value | Time-range trends, TWA, statistics | Months to years |
| Relational DB | Structured records, transactions | Rows and tables | JOINs, relational queries | Application-defined |
| Data Lake | Raw data landing zone for analytics | Files / objects (Parquet, CSV) | Batch analytics, ML feature stores | Years; cold storage |
| MES / ERP | Production/business records | Business objects | Production orders, KPIs | Operational + compliance |
The historian occupies a unique layer: it is not trying to replace SCADA (which operates in real time at the control layer) and it is not a data lake (which ingests many data types and serves batch analytics). For the specific problem of "what was the value of tag X at time T?" over an extended historical window, no other system matches a historian's combination of compression efficiency and retrieval speed.
For context on where SCADA fits in the broader system hierarchy, see SCADA vs MES vs ERP.
Common Industrial Historians
Several historians dominate the market. Each has a different architecture, licensing model, and integration footprint.
AVEVA PI System (formerly OSIsoft PI)
PI System is the most widely deployed historian in process industries. Its architecture separates the PI Data Archive (the time-series store) from the PI Asset Framework (PI AF), a hierarchical asset model that maps raw tag data onto equipment and process unit context. PI AF is what allows engineers to query "all reactor inlet temperatures" by asset class rather than by knowing each individual tag name.
PI System uses a proprietary compressed binary format and exposes data via PI Web API (REST), OPC HDA, OLEDB, and .NET SDK. It is the de-facto standard in oil & gas, chemicals, and utilities, but its per-tag licensing model makes large-scale IIoT deployments expensive.
AVEVA Historian (formerly Wonderware Historian)
Built on Microsoft SQL Server with a proprietary compression layer on top, AVEVA Historian is deeply integrated with the AVEVA System Platform / InTouch SCADA stack. Its SQL foundation makes it more approachable for engineers comfortable with relational queries; its compression and retrieval performance is competitive for mid-size plants.
Canary Historian
Canary is a .NET-native historian with a reputation for straightforward configuration and competitive pricing. It uses its own compressed binary store and exposes an OPC HDA interface alongside REST and ODBC. Popular in food and beverage, water/wastewater, and smaller manufacturing sites.
Ignition with Tag Historian Module
Inductive Automation's Ignition platform includes a Tag Historian module that stores to any JDBC-compatible database (MySQL, Microsoft SQL Server, PostgreSQL). Ignition does not implement compression at the acquisition layer in the same way dedicated historians do — it relies on the underlying database's compression features. This approach trades historian-specific write efficiency for SQL queryability and straightforward integration with Ignition SCADA projects.
InfluxDB (Open Source / Cloud)
InfluxDB is an open-source time-series database increasingly used as the historian layer in modern IIoT architectures, particularly where data arrives via MQTT and flows through an edge computing gateway. InfluxDB's line protocol, Flux query language, and Grafana integration make it attractive for greenfield sites. It lacks the deep OPC HDA integration and asset framework of PI System, but for cloud-native architectures where the data pipeline is MQTT → InfluxDB → Grafana, it is a practical and cost-effective choice.
Use Cases for Process Historians
Process Trend Analysis and Root-Cause Investigation
The most fundamental historian use case: when a process event occurs — a batch failure, an upset, an equipment trip — engineers retrieve the historian trend for all relevant tags in the 30 minutes before the event. Because the historian stores every tag continuously, engineers can examine what the process was doing before anyone suspected a problem. This is impossible in a SCADA system, which typically keeps only a short rolling buffer of raw trend data.
OEE Calculation and Production Reporting
Overall Equipment Effectiveness (OEE) requires availability, performance, and quality data over production periods. Historians provide the continuous time-series data (motor running states, throughput rates, quality rejects) that OEE calculations aggregate. Time-weighted averages and sum aggregations over shift intervals are standard historian queries used to populate OEE dashboards.
Predictive and Condition-Based Maintenance
Vibration, temperature, current draw, and other condition indicators recorded continuously over months or years form the baseline for predictive maintenance models. Historians are the data source for both rule-based alerting (is this bearing temperature trending up over the past week?) and ML-based anomaly detection that requires months of historical context as training data.
Energy Management
Energy management programs require accurate, continuous measurement of power consumption at the equipment and cell level. Historians record power meter data alongside process data, enabling energy intensity calculations (energy per unit of production) and identification of parasitic loads during idle periods.
Digital Twin Calibration and Validation
Digital twins require continuous comparison between the simulated process model and the real plant. Historians provide the real-plant time-series data feed that the digital twin continuously reconciles against. Without a historian, a digital twin has no ground truth.
The Controls View: Getting PLC and SCADA Tags into the Historian
This is where automation engineers spend most of their historian configuration time.
OPC vs Direct Driver
The cleanest integration path for modern PLCs is OPC UA: configure an OPC UA server on the PLC (Siemens S7-1500, Allen-Bradley CompactLogix, Mitsubishi iQ-R all support this natively or via gateway), then configure the historian's OPC UA collector to subscribe to the relevant node IDs. The PLC pushes value changes to the collector on deadband, reducing polling load on the controller.
For legacy PLCs that do not support OPC UA, historians offer proprietary drivers (e.g., AVEVA's SuiteLink, Ignition's Allen-Bradley or Siemens drivers) that poll the PLC directly over its native protocol. The OPC UA tutorial walks through setting up an OPC UA server on common platforms.
Where a SCADA server is already collecting all tag data, it is often simpler to configure the historian collector to read from the SCADA server via OPC DA or OPC UA rather than adding a second connection to each PLC.
Choosing Scan Rates
Scan rate selection is an engineering trade-off:
- Too fast: unnecessary load on the PLC communications processor; larger pre-compression data volume; higher historian CPU usage.
- Too slow: missed transient events; inability to reconstruct fast dynamics; poor trend resolution for troubleshooting.
General guidance by signal type:
| Signal Type | Typical Scan Rate |
|---|---|
| Safety-critical analog (pressure relief, temperature interlock) | 500 ms – 1 s |
| Process control loop PV and SP | 1 s – 5 s |
| Utility (cooling water flow, compressed air pressure) | 5 s – 30 s |
| Motor run/stop status (digital) | 1 s |
| Energy meters | 5 s – 60 s |
| Environmental / ambient conditions | 60 s |
Choosing Deadbands
Deadband should be set to the resolution of meaningful change for the process, not to the sensor's noise floor. Setting deadband too tight (e.g., 0.01 % for a 0–100 % level tag) defeats compression. Setting it too wide misses real process movement.
A practical starting point: set the deadband to 0.2–0.5 % of the engineering range for most analogue process variables, then review after a week of operation. Tags with very slow process dynamics can use wider deadbands (1–2 %); tags used for high-resolution troubleshooting should use narrow deadbands (0.1 %).
Feeding Analytics and Digital Twins
Historians increasingly serve as the data source for analytics pipelines. The typical modern architecture:
PLC/DCS → OPC UA → Historian → REST/OPC HDA → Analytics Platform (Python, Power BI, etc.)
For ML workloads, it is common to extract large blocks of historian data (months of tag history) into a data lake or feature store in Parquet format, then train models offline. The historian continues to serve real-time inference by streaming recent tag values to the model serving layer via REST or MQTT.
Frequently Asked Questions
What is a process historian? A process historian is a purpose-built time-series database that continuously records the value of every instrumentation tag in an industrial plant — temperature, flow, pressure, motor current, valve position — at rates from milliseconds to minutes. It stores this data for months or years using compression algorithms that reduce storage by 80–95 % while preserving engineering accuracy. The historian forms the bridge between the real-time control layer (PLCs and SCADA) and the analytics, reporting, and maintenance systems that need access to plant history.
How is a historian different from a database? A general-purpose relational (SQL) database stores data in rows and tables optimised for JOINs and relational queries. A historian stores data in compressed, time-ordered binary format optimised for the specific query pattern of industrial process data: "give me the value of tag X from time A to time B, interpolated every 5 seconds, with a time-weighted average per hour." At the write throughput required for continuous multi-tag acquisition (thousands of writes per second), relational databases suffer index maintenance overhead that historians avoid by using append-only compressed stores. The two are complementary: historians for continuous analogue/digital tag history; SQL databases for event logs, batch records, and relational metadata.
What is swinging-door compression? Swinging-door compression is a lossless linear interpolation algorithm that stores only the inflection points of a continuously varying signal, discarding intermediate samples that fall within a defined linear error tolerance. The name comes from the visual analogy: imagine a door hinged at the last stored sample. As new values arrive, the door swings to follow them. When a new value falls outside the angular tolerance of the door, the previous value is committed to storage and the hinge resets. A linearly ramping signal is stored with only its start and end points; intermediate values are reconstructed by interpolation on retrieval. Combined with deadband filtering, swinging-door compression routinely achieves 10:1 to 20:1 compression on active analogue signals.
How does data get into a historian? Data enters a historian through collectors (also called interfaces or agents) — software components that connect to data sources and forward tag values to the historian server. The most common collection paths are OPC UA subscriptions (the source PLC or SCADA pushes value changes to the collector), OPC DA polling (the collector polls the source on a fixed scan interval), and vendor-proprietary drivers for DCS systems. Each tag is configured with a scan rate (how often the collector reads the source) and compression parameters (deadband and swinging-door deviation). The collector applies deadband filtering before forwarding to the historian, which then applies swinging-door compression before writing to disk.
Summary
A process historian is the purpose-built archive layer for continuous industrial time-series data. Its core value is not simply storage — it is the combination of high-throughput collection from OPC and vendor interfaces, intelligent compression that makes multi-year retention economically practical, and fast retrieval that turns months of tag history into actionable trends, OEE metrics, and predictive maintenance features in sub-second query time.
For controls engineers, the practical work is in the configuration details: selecting scan rates appropriate to process dynamics, setting deadbands that preserve meaningful changes while achieving compression, and structuring tag names that make historian data queryable without plant-floor tribal knowledge. Get those three decisions right and the historian becomes one of the most valuable diagnostic and analytics assets on the plant network.
