Industry
NicheDB 0.2.0: the domain layer, counted daily
chovy DEV Community
1 views
Twenty seven commits had merged into NicheDB since the first release and none of them had reached anybody. The CLI publishes from a GitHub release and from nothing else, so a merge to main is a midpoint rather than a finish. 0.2.0 cuts them loose.
The headline is a new collection: domains.
The layer everything else runs on
NicheDB tracks packages, filings, launches, weather, markets, crime, public money, housing, jobs and news. It had no view of the layer all of that lives on, which is the domain name system's own market. It does now, built from ntlddata.com's free JSON API. No key, CC-BY, and aggregate only, because ICANN's zone file terms permit analysis but not redistribution.
Eight feeds, each with a page, RSS, JSON Feed, an API endpoint and MCP tools:
How many domains exist across the new gTLDs and which way that moved, day by day, with the pre-2012 generics counted separately
The whole register of 1,114 TLDs with its registry operator, inferred backend, root signing and delegation date
The biggest daily gainers and the TLDs losing the most names
Sunrise and claims windows with the dates registries filed with ICANN, strings contracted but not yet delegated, and registries being transitioned to an emergency operator
TLDs changing operator, registrars moving platform, accreditations granted and terminated
What the collection refuses to conflate
Two clocks. Domain counts come from zone files and are a day old. Anything attributed to a registrar comes from ICANN's monthly reports and runs about a quarter in arrears, because zone files carry no registrar information at all. Every figure of the second kind is carried with the month it describes and is never merged with the daily count.
Two populations. New gTLDs and the pre-2012 generics are never added together. A source names one and labels its rows with it.
Three traps, found by probing rather than by reading
The overview endpoint never answers without an explicit scope, and never answers for scope=all. Both hang until the client gives up, while scope=new returns in under a second. So the totals source refuses scope=all rather than scheduling a guaranteed timeout on every run.
The page size parameter is floored at ten. Asking for three rows returns ten while the response's own metadata says three, so a "top five movers" source that trusted the parameter would publish ten.
Detected timestamps carry no zone, and a bare Date parse reads them as local time, which files a midnight row under the previous day for every reader west of UTC.
One more came from the live data rather than the docs. The first day of the daily series carries zero adds, zero drops and zero net, because it had no day before it to difference against. Across a thousand registries that cannot be a real day, so it publishes as movement not yet known instead of "+0 on the day", which would read as a flat day rather than a missing one.
Everything else in 0.2.0
Weather as it is issued rather than forecast. Automotive with a VIN history report scored against a measured complaint baseline. Markets with the world's exchanges as a register and the corporate actions that move them. Crime by city, state and force area. Public money across three governments. Housing and jobs as governments measure them. AI incidents. News, where every story files under the desk that published it.
Plus the fixes worth naming: dedup now folds a source against its own second feed, not only against the other sources, so a publisher exposing one article through two of its feeds stops arriving twice. The dedupe keys reach Postgres as a real array rather than a comma joined string, which had been aborting whole ingestion runs.
Reading it
The collection is at https://nichedb.dev/c/domains and every feed on it is public. The CLI is npm i -g @profullstack/nichedb@latest.
Data on the domains collection is from nTLDData (ntlddata.com), CC-BY-4.0.
Read original: https://dev.to/chovy/nichedb-020-the-domain-layer-counted-daily-28id
← Previous
The Evolution of WSL Support in JetBrains IDEs
Next →
PowerShell ile Kanıt Dosyası Üretirken Yaşadığım 580 Baytlık Hata
Related
Carmakers Have a New Idea to Boost EV Range: Add a Gas Engine
Industry
0
Hacker News
Worried Anthropic researchers warn that AI ‘could kill all humans’
Industry
0
The Verge (Tech)
The firewall that was not there
Industry
2
Dev.to (EN Zone)
“Tweet” and the bird logo apparently enter the public domain
Industry
2
Hacker News
Comments0
No comments yet — be the first