We are at a crossroad as advances in information systems challenge our fundamentals, and so they should. But what are we at risk of losing as we progress? In this article, process historian is used synonymously with enterprise historian. As a rule, vintage and modern historians both remain in active use across a wide range of applications, and modern historians carry real added benefit as developers build new capability onto newer platforms. The question of old or new isn't really either/or, the honest answer is both, and that distinction matters.
Beyond visualisation, user controls, and alarms, SCADA can act as a front-end processor, a data concentrator for historians. SCADA systems gather data from multiple sources and condense it so operators can manage assets effectively in real time. The better the SCADA and its configuration, the better the outcome for operators and the business. SCADA produces rich data sets that have historically been hard to access, which is exactly the gap the data historian fills, adding context and high-performance access for end users including corporate.
Process data is sampled readings from devices and software systems, mostly instruments monitoring elements like pressure, temperature, and speed, and it can produce very high sample volumes. When systems are architected, the data management plan, how the payload is handled by design, is too often neglected because systems start small and work fine as a proof of concept. Even a small system using a process historian will quickly overwhelm an alternative relational database with that same flood of data.
Process historians store this data highly compressed, making them a strong repository for the life of the system. Alternative technologies would need a much larger commitment to disk or cloud storage, though cloud storage is an easy choice if the historian is already cloud-hosted. Several topologies exist, including hybrid ones, but the most common pattern is a process data repository alongside a separate business data repository, an important distinction given how different the two types of data really are.
Process historians offer several access methods for client applications or third-party interfaces, an EAM, an ERP, or any business system that can consume the data, via SQL, REST APIs, .NET APIs, and similar. The volume of process data dwarfs typical business data (think comparing a bank statement's data volume to the moon), and that difference has to shape how interfaces, reports, and client tools are built. New historian technologies are significantly faster than the pioneering generation, which is worth remembering before comparing pioneer-era historian performance against newer alternative technologies and drawing the wrong conclusion.
Process historians can be expensive to purchase outright, and subscription models, while softening the optics, still carry total lifecycle costs worth scrutinising. That high ongoing cost pushes early adopters to look elsewhere, particularly where the historian isn't critical to operations. There are creative alternatives for capturing process data directly from PLCs or IoT devices into generic databases, and the established historian vendors have decades of reliability and mature, out-of-the-box tooling behind them, tooling that needs careful consideration before walking away from it. Modern alternatives are still maturing in how they visualise and, more importantly, manage large volumes of time-series data.
A common workaround is to cube or precondition process data before storing it, converting raw data into something lighter based on what seemed valid at setup time. Anyone who has worked with process data for long knows there is far more to discover across an asset's entire lifecycle than is obvious at setup, that's precisely why they're called lessons learned. Deprecating raw data reduces its value, sometimes the smallest undetectable change is exactly what AI needs to protect against unnecessary asset failure, and that's only possible if the raw data was preserved rather than displaced.
Yes, technically, you could also use a spreadsheet, but most process historians already offer client tools including spreadsheet plug-ins for exactly this reason. Whatever the solution, the client tools and the captured data shouldn't inhibit a data-driven business from finding new insight. Key factors worth weighing include the type of data (real time or time series), payload cost related to cloud data volume, open-source policy, acquisition methods, cloud data security and recovery mandates, and the longevity of the application itself. This list isn't exhaustive, but it shows the depth of consideration an architecture decision like this deserves.
Perhaps the greatest risk with alternative technologies is product continuity. Some organisations that adopted alternative IT solutions in the last few years are already discovering those applications have reached end of life, virtually unheard of for industrial automation vendors. Given engineering cost dwarfs software cost in this industry, a forced migration to a new platform magnifies the impact on the business considerably.
The hybrid model offers the best of both worlds, and arguably it should never have been framed as an either/or debate in the first place. Big data's rise simply replicated elements of industrial systems into the corporate space, dashboards moved to the boardroom while the underlying data source stayed the same. Modern visualisation tools sell well, but process historian tooling exists to be robust, hardened, and conservative, by design, it isn't meant to change at the pace consumer applications do, and changing it that fast would be irresponsible.
The hybrid model also manages the quieter cost: payload. In countries including Australia, the cost of repeatedly moving high volumes of data to and from the cloud is genuinely limiting. A well-considered data management plan keeps that cost under control even as the user base of data consumers grows, which is part of why Microsoft, Amazon, and others have built or acquired dedicated time-series technologies, an acknowledgement that scale needs purpose-built technology, not generic databases.
Next generation historians are likely to matter more as infrastructure components than as feature-rich client applications. Increasing security-by-design requirements won't simplify historian technology, if anything the opposite, though keeping feature-rich client applications out of the more secure network layers offers some consolation. As organisations build more of their own analytical tools and in-house skills, the appeal of feature-rich historians will fade in favour of open access to contextualised data, and a historian that can't provide that context won't last. In some respects, historians are likely to return to their original purpose, just with far greater performance, security, and robustness than before.