◆
No lossy ingestion
Raw data kept as it arrived. Aggregates become a view rather than a destructive choice.
Every aggregation at ingestion is a question made permanently unanswerable. Those decisions were made under storage economics that no longer apply, and most organisations have never revisited them.
Used for warehousing, analytics, event capture and the reporting built on them.
What this looks like in practice
Raw data kept as it arrived. Aggregates become a view rather than a destructive choice.
A new question is a query. It does not require a pipeline to be commissioned first.
How long you keep something is a business choice, not a consequence of what storage costs.
Which workload spent what, so the expensive query is identifiable rather than mysterious.
The pipeline decisions that constrain most analytics teams were storage-cost decisions in disguise, taken years ago by people who were right at the time.
Somebody asks what customers did in the eight weeks before they churned. If the pipeline summarised to weekly totals, the answer does not exist and never will. Keeping raw data is not a storage preference; it is the difference between an analytics function and a reporting function.
Less transformation at ingestion means less to maintain and less to be wrong.
Numbers traceable to the events behind them rather than to another summary.
Giving partners access without handing over an extract that immediately goes stale.
Old data still queryable rather than restored from a backup when someone asks.
We stopped aggregating on ingest and within a quarter the analytics team was answering questions the business had given up asking.
For most teams, yes, and more affordable than the pipelines built to avoid it. The aggregation decisions in your estate were usually taken under economics that have since changed considerably.
No. Teams typically keep existing pipelines and begin retaining raw data alongside them, which means the change is additive rather than a migration.
As long as it is useful to you. The point is to make that a business decision rather than something storage cost decides on your behalf.
Yes, attributable per workload, which is usually the first thing a platform team wants and rarely what they have.
Start on something that will still be right at a hundred times your size, and pay now what a startup should pay.
Move in stages with the old arrangement still running, and cut over only once the replacement has carried real traffic.
Get capacity when the experiment is ready, keep what produced a result, and put models in front of users without a separate platform.
Ship the application your users touch, on an address that stays put, with every version one click behind you.
Build the tools your own teams need, at a cost that makes small improvements worth doing.
Give your engineers a platform that removes decisions rather than adding them, without a platform team to run it.
Begin with one workload. Keep what you have running. Decide whether to continue once you have seen it carry something real.
No migration required to begin. Nothing to tear down if you stop.