Customer Story – Peter Chaplin, Co-founder, Datazag
Most threat intelligence arrives after the attack has started. Datazag shows up while it’s still being set up.
That’s not just a marketing line. It’s an engineering choice that shapes everything about how the company works, what infrastructure it needs, and why predictable costs matter more than almost anything else when you’re measuring the whole internet, around the clock, every day.
What Datazag actually does
Datazag tracks the infrastructure behind the internet’s domains: who hosts them, who runs their email, how they’re secured, and how all of that changes over time. Over 411 million domains are in the system. More than 360 million of those currently resolve – the rest are parked, expired, or never properly set up, so Datazag tracks the full set and focuses resolution on the domains that answer.
That data feeds managed security providers, cyber insurers, email service providers and enterprise security teams. Some use it as alert feeds inside their existing security tools. Some use it as risk reports across a company’s domain portfolio. Others pull it directly into Snowflake, Databricks or DuckDB and join it to their own data.
There’s also the Datazag Observatory – free, publicly available research on how the internet’s domains, email and hosting actually work.
Where the idea came from
It started with email. Peter’s background includes running email sending infrastructure, where you learn quickly that a domain’s DNS tells you a great deal about who’s behind it and what they’re about to do.
Most threat intelligence is reactive. It tells you about infrastructure that was already used in an attack. Datazag wanted to catch the setup stage instead: the domain being registered, the certificate being issued, the mail records being configured. Doing that properly meant measuring the whole internet themselves rather than relying on someone else’s feed.
What 3 million lookups an hour actually looks like
Not every domain needs the same attention. New and higher-risk domains get checked daily for their first 30 days – even longer if they change. The full corpus refreshes on a rolling two-month cycle. Most domains don’t change much month to month, so Datazag stores the changes rather than every observation. That keeps the history compact and makes changes easy to find.
The workload runs on a pool of worker servers fed by queues. Work is split by urgency: newly seen domains and high-priority checks go to the front, routine refreshes fill the remaining capacity, and failed lookups go to a separate retry queue so they don’t hold up everything else. A smaller set of servers handles the data lake, with each one responsible for a distinct stage of processing.
Keeping collection and processing separate is deliberate. A heavy analysis job should never slow down live detection.
The 10-second window
Every publicly trusted website certificate is recorded in public Certificate Transparency logs. Datazag reads those logs directly as certificates appear. When a new domain looks like it’s impersonating a brand or a platform – a fake login page, for instance – it gets resolved immediately and checked continuously until it goes live.
The target is to raise an alert within about 10 seconds of the domain resolving.
That matters because phishing campaigns are short. A domain might be used for a few hours. If a customer’s security tools can block it before the first malicious email lands, the attack fails. Hearing about it a day later is simply too late.
The infrastructure problem with consumption-based pricing
Datazag started on a major public cloud using managed services – quick to get running, but a poor fit as the operation scaled to hundreds of millions of domains.
Monitoring internet infrastructure isn’t like running a conventional application. The workload doesn’t stop when customers aren’t using the product. DNS records need refreshing, new infrastructure needs investigating, and large datasets need processing around the clock. On consumption-based pricing, every improvement in coverage or refresh frequency became a new cost calculation.
“We wanted an architecture where we could continuously expand what we observe without costs increasing in direct proportion to every additional query. Predictable compute capacity gives us the freedom to collect more signals, refresh data more frequently and experiment with new methods. Ultimately, that means better intelligence for our customers.”
The result is an infrastructure strategy that uses dedicated compute for continuous collection and processing, alongside cloud services for storage, distribution and customer access.
Why Contabo
For Datazag, the decision came down to predictable economics, performance, and the ability to scale a distributed collection operation handling around 3 million DNS lookups an hour.
Since adopting Contabo, Datazag has expanded from monitoring a few hundred thousand domains to tracking hundreds of millions. The infrastructure has supported that growth without the economics getting in the way, and the company is now planning to replicate its deployment across a second Contabo data centre to increase capacity and strengthen resilience.
The benefit isn’t simply lower infrastructure expenditure. It’s being able to direct more resources towards what matters to customers: broader coverage, fresher intelligence, and faster detection.
What changed
When Datazag started, it was processing 100,000 to 200,000 domains. Today it maps the whole internet with over 90% coverage across domains and their DNS records. Costs and the ability to update data continuously are significantly lower than alternative vendors – and while some competitors run on weekly or monthly batch cycles, Datazag updates daily.
That’s not a minor operational difference. In threat intelligence, the gap between daily and weekly is the gap between catching an attack and reading about it afterwards.
What’s next
Datazag’s datasets are launching on the Snowflake and Databricks marketplaces, so organisations can join the data to their own without it leaving their environment. Alert feeds already deliver into Microsoft Sentinel, with more integrations in progress.
The research side is expanding too. Just this month Datazag published “Who Really Runs Business Email?” – a look at how the world’s largest companies run their email compared with the rest of the internet.
The internet keeps growing. Datazag keeps watching.
If you’re running data infrastructure at scale and want costs that don’t rise every time you do more, take a look at our server plans – and explore what Datazag is building at datazag.com and observatory.datazag.com.
Want to be featured? We’re always looking for customers doing interesting things on Contabo infrastructure. Get in touch.
