I built a domain expiry monitor, and to test it against something other than my own domains I pointed it at the "our work" pages of 30 US web design agencies — the client sites they link to publicly — and checked every domain against its registry. The results were mildly alarming, but the interesting part was everything that went wrong on the way there. The results first 65 client domains across 14 agencies: 1 had already expired, 60 days earlier 3 were inside 45 days, the nearest with 3 days left the rest ran from 46 days to several years out, clustering at 3–6 months I'm not naming the agencies or the domains. Telling someone privately that their client's domain is about to lapse is useful; publishing it is exposing somebody's client. Why RDAP and not WHOIS WHOIS returns free text in roughly as many formats as there are registrars. RDAP (RFC 7482/9083) returns JSON, over HTTPS, with a documented shape. The single nicest property: a 404 is an answer. In RDAP it means "this domain is not registered", which is precisely the event worth alerting on. No string matching on "No match for domain". const res = await fetch(`${base}domain/${name}`, { headers: { Accept: 'application/rdap+json' }, }) if (res.status === 404) return { registered: false } Three things that will bite you 1. The registrar's name is hidden in a jCard There's no registrar field. The registrar is an entity with the registrar role, and its human-readable name lives inside a vCard-in-JSON structure: const entity = (body.entities || []).find((e) => (e.roles || []).includes('registrar')) const card = entity?.vcardArray?.[1] || [] const fn = card.find((f) => Array.isArray(f) && f[0] === 'fn') const name = fn?.[3] ?? null // "GoDaddy.com, LLC" vcardArray[1] is an array of ["fn", {}, "text", "GoDaddy.com, LLC"] triples. That awkwardness is why so many tools show you IANA registrar ID 146 instead of a name — and "where do I renew this" is the actual operational question for anyone holding domains across several registrars. 2. IANA's bootstrap does not cover every TLD The bootstrap file maps TLDs to RDAP servers, and .de, .co and .gg simply aren't in it. There's no authoritative endpoint to ask. The right response is to say so. "The registry publishes no expiry date" is a fact about the registry, not about the domain, and it's more useful to a user than a number you inferred from somewhere else. Any tool that confidently shows an expiry date for a .de is guessing. 3. Public suffixes cartwright.co.uk naively reduces to co.uk. Query that and you get the registry's own record, then cheerfully report the registry's expiry date as your user's domain expiry. The full Public Suffix List is thousands of entries; even a small hardcoded set of co.uk, com.au, co.nz and friends prevents the embarrassing version of this bug. The bug that nearly discredited the whole thing My first extractor took every href on the page. For one agency it confidently reported licdn.com, hotjar.com, addtoany.com and gmpg.org as their clients. All four came from the <head>: <link rel="preconnect"> hints and the XFN profile link WordPress writes into every page. I started adding them to a blocklist, then stopped — every new analytics vendor adds another entry, and the blocklist can never be finished. The structural rule can: // Only anchors. A person clicks anchors, and only anchors point at clients. const re = /<a\b[^>]*?\bhref\s*=\s*("([^"]*)"|'([^']*)'|([^\s">]+))/gi One line, and the entire category of failure disappears. A related one: an agency's second domain isn't the agency's client. One firm's site was at atendesigngroup.com and they also owned aten.io; another at openmediafoundation.org owned open.media. Matching only the exact domain you crawled isn't enough — comparing the leading label catches both. About a third of sites hard-block automated requests This one changed the architecture. I wrote the fetcher with fetch, and got 403 on every single site, including with a complete set of genuine browser headers. Not a Cloudflare JS challenge you can wait out — a flat 403, identical in headless Chrome and in a real headed one. It's blocking by IP reputation. I switched to driving a real browser, which fixed it for about two thirds of sites. That turned out to matter for a second reason I hadn't considered: most agency portfolios render their client grid client-side, in Webflow or React. Even a successful fetch would have returned markup with no client links in it, and I'd have spent a day blaming my extractor. The unglamorous finding Only 14 of 30 agency sites linked three or more client domains at all. The rest show client work as screenshots with no anchor. That's a lot of backlinks being left on the floor. If you want to look at what I built with this, it's at dropperch.com — free tier monitors domains. But the RDAP notes above are the genuinely reusable part, and they're yours whether or not you ever look at it.