One number, in every place it appears.
Schema design, delta sync, migration, and the reconciliation job that catches the drift before an auditor does. Unglamorous work that decides whether anybody trusts the software.
- Delta sync
- changed rows, not full reloads
- Reversible
- every migration, both directions
- Drift alerts
- before the quarterly audit
Who this is for
You probably need this if…
01
Two reports, two answers
The same question asked in two systems returns different numbers, and the meeting stops while somebody works out which one is right.
02
A nightly job that keeps failing
It reloads everything, it takes four hours, and when it breaks at two in the morning nobody finds out until somebody opens a report.
03
History is trapped in the old system
You cannot switch it off because five years of records live in it and nobody has ever successfully moved them out.
What this covers
The work, in detail.
7 capabilities
01
Schema & modelling
Data structures designed for the questions you will be asking in two years, not only for the screen that is needed next month.
02
Delta sync
Changed records only, with tombstones for deletions and a conflict policy written down before anybody actually hits a conflict.
03
Reconciliation
Scheduled comparison across systems, producing a report that names the rows which disagree rather than a count that does not help.
04
Migration & backfill
Moving history off the old system in a rehearsed run with a rollback path, and verified row counts on both sides afterwards.
05
Reporting & dashboards
Operational views built on a read model, so that a heavy report cannot slow down the software people are trying to use.
06
Event pipelines
Ordered, durable event streams and the job queue that processes them without losing anything when a worker restarts.
07
Retention & archival
What gets kept, for how long, and what happens at the end of it. Written down, because at some point somebody is going to ask.
How we approach it
Positions we actually hold.
Opinions cost something to have. These are the ones we would argue for on your project, including where they make the work slower.
01
Deletes need a tombstone
A record that vanishes from the source must be distinguishable from one that was never sent. Sync bugs are almost always deletion bugs.
02
Rehearse the migration
Full dry run on a copy, verified counts, timed. The real run should be a repeat of something that already worked, not a first attempt in front of an audience.
03
Every migration goes both ways
A down-migration is written and tested at the same time as the up. A rollback should not be the first time anybody thinks about reversing it.
04
Reconciliation is a feature
Anything syncing two sources gets a scheduled job that checks them and reports drift. It costs a day to build and it saves an audit.
Technology
What we work with.
Defaults, not requirements. If you already run something else and have a team who knows it, we work in yours.
- Databases
- PostgreSQL, SQLite, Redis
- Access
- Prisma, SQL, row-level security
- Pipelines
- Job queues, change data capture, scheduled reconciliation
- Reporting
- Read models, materialised views, CSV & API export
Where this shows up in delivery
- Larder and InvtoryX both run delta sync between local stock and a central record, conflict resolution included.
- Sill carries audit logging and multi-tenancy at the data layer rather than in application code.
- We have moved production history off legacy systems in rehearsed, verified, reversible runs.
Bought as part of
Ecommerce & POS
Fixed-price project
AI Integration
Fixed-price project
Systems Integration
Fixed-price project
Data & Analytics
Fixed-price project
This practice is never sold on its own. It is quoted inside one of the engagements above, as part of a single number.
Proof
Where we have actually done this.
Projects we designed, shipped and wrote up. Each one names the decision that was genuinely hard to get right.
Questions
What people ask about data & sync.
- How long does a data migration take?
- The rehearsal takes longer than the migration. Budget about a week of dry runs for anything with real history behind it, after which the live run is usually a matter of hours.
- Can we keep the old system running in parallel?
- Yes, and for anything financial we recommend it for at least one full cycle. Reconciliation runs across both until you are satisfied that they agree.
- Who decides which system wins a conflict?
- You do, in writing, in week one. After that it is code. An undefined conflict policy is exactly how the same number ends up different in two places.
- Do we need a data warehouse?
- Usually not at your size. A read model in the same database answers most operational reporting without giving you a second system to maintain and pay for.
Related reading
Written on this, by us.
Data & Integration · 7 min read
The Data Migration Is the Project
Replacing a system is mostly moving its data, and that part is a business decision rather than a technical one. Why migrations overrun, and how to scope one honestly.
Data & Integration · 7 min read
The Same Number in Two Places
Your POS says one thing and your accounting package says another. Reconciling them is somebody's entire Monday. Here is why systems drift apart, and the boring job that keeps them together.
Data & Integration · 6 min read
Every Metric Means Three Things, and That Is Why Your Dashboards Disagree
Two reports, two answers, both technically correct. The problem is almost never the data. It is that nobody wrote down what the words mean.
Usually combined with
All practice areas →Tell us what you’re trying to build.
Describe the problem in your own words. We’ll come back within one business day with a scope, a number and a date.
Phone
+1 (407) 796-2376Reply time
One business day, from an engineer
Based in
Orlando, Florida · serving the United States
What happens next
- A reply within one business day, from an engineer
- A thirty-minute call, with no qualifying call before it
- A written scope and a fixed number, if it fits