Data Migration Is Translation, Not Copy

Historian and SCADA platforms compete on their native value proposition. Each one stores values, quality, time and context in the way that best serves its own strengths, which means they are different by design, not by accident.

That makes the common assumption of a one-to-one migration analytically flawed. Moving data into a system built on different rules is never a simple copy. It is a translation, and every translation has choices in it. Left to default behaviour, those choices are made silently, and the result is data that looks complete but is less trustworthy than it was before the move.

Data integrity declines unless something extra is bundled into the migration. The rest of this page sets out what that extra is.

Twelve things separate a migration that improves your data from one that quietly degrades it. The first six protect the integrity of the data itself. The second six keep the project moving so it finishes sooner.

Integrity carried across

Quality keeps its meaning

Every historian has its own way of saying a value is good, uncertain or bad, and the definitions rarely line up. We translate source quality and status into the target platform's native quality mechanism on the same point, so a failed instrument never arrives looking like a healthy one.

Values arrive as precise as they left

Data types and timestamp resolution differ between platforms. A large totaliser or energy register squeezed into a smaller number type will overflow, and timestamps rounded on the way through blur fast events. We map types and time resolution faithfully, so the numbers in the new system are the numbers that were recorded.

Context travels with the data

A historian is more than a list of tags. Years of asset structure, equipment relationships and engineering metadata give the data its meaning, and that structure is what dashboards and reports depend on. We rebuild it in the target alongside the history, and references to decommissioned equipment are identified and retired instead of being carried into the new model.

Names land in the new standard

Long-lived systems accumulate naming conventions from every SCADA platform and integrator that has touched them. We convert legacy names to the new naming standard by rule, so the migrated data arrives already aligned, without anyone hand-editing hundreds of thousands of spreadsheet rows.

History reflects what actually happened

Over decades, field changes such as RTU replacements and rewiring can invert how a signal is recorded, with the correction applied only in screen logic. Copied as stored, a running pump reads as stopped. We apply known corrections per point, device or site, so the migrated record matches operational reality.

Clean, ordered time series

Re-runs create duplicates, legacy systems hold colliding timestamps, and writing data in the wrong order fragments the target's archives. We resolve duplicates and collisions before transfer and write in an order the target handles efficiently, so the new archive starts clean and stays performant.

Delivered without stalling

Throughput sized to the dataset

At utility scale, history runs to hundreds of billions of records. Moved one point at a time, that is a multi-year job. We migrate in parallel with coordinated workers and remove redundant records before transfer, bringing the job down to weeks.

Nothing is lost silently

Networks drop and servers time out over a long migration. Some tools respond by cutting the point in progress short and still reporting success. Our migrations break history into bounded chunks, retry interruptions automatically and track the state of every chunk, so a failure is caught and recovered, never hidden.

Migrate while the new system runs live

Most organisations want live data flowing into the new platform before the old one is retired. We backfill history underneath live data without interrupting it, with no temporary servers and no archive merges.

The source is never touched

Legacy historians are often licensed to the hardware they run on, and production servers are no place for migration software. We read the source remotely, so the existing historian, its host and its licence stay exactly as they are.

Recent data first

Not all history is needed on day one. We move the most recent operational years first, so the new platform can be commissioned and put to work sooner, then the longer compliance history follows in the background.

Evidence at the end

Every point is read back from the target and compared against the source, with counts, values and quality reconciled and exceptions reported. Acceptance testing rests on evidence for every point, not on a handful of sample trends.

The first six of those outcomes are what the something extra looks like in practice. Quality is translated, context is rebuilt, names are standardised, history is corrected and duplicates are removed, all as part of the move.

A migration is the one time every record in the system passes through a single process. That makes it the cheapest moment the data will ever have to be fixed. Done well, the new platform starts with better data than the old one ever held.

Every migration follows the same disciplined pattern, design it, prove it, then scale it.

1. Discover. We profile how both platforms actually store values, quality, time and context, including the quirks that only show up in real data.
2. Agree the translation rules. Every mapping decision is documented and signed off with your team before any data moves.
3. Prove on a pilot. The rules are tested on a representative subset and read back, so problems surface while the exposure is small.
4. Migrate in priority order. Recent operational history first, then the longer compliance record, with every chunk tracked.
5. Verify and hand over. Point-by-point reconciliation and exception reporting give you the evidence for acceptance testing.

• Time. A migration measured in weeks instead of years shortens the period of running two historians side by side.
• Capex. Backfilling under live data avoids standing up temporary servers to stage the history.
• Opex. Context rebuilt during the migration means nobody has to rebuild asset models and naming by hand afterwards, and fewer post-migration data defects means less ongoing rework.

Our approach is not tied to any one platform. We adapt it to the source and target systems for each job, and the historians we see most often in Australian critical infrastructure include the following.