A gateway already knows how to authenticate the sender. It knows far less about where the links inside the body actually go. This page is about wiring a batch lookup of DNS-verified, currently-resolving phishing hostnames into the delivery path, re-checking at click time, and going back through mail you already delivered when the list changes overnight.
SPF, DKIM and DMARC are excellent at the question they were designed to answer. That question is not the one a credential-harvesting link asks.
An attacker registers a domain, publishes a clean SPF record, signs outbound mail with DKIM, and sets a DMARC policy that aligns perfectly. The message authenticates because it genuinely was sent by the domain it claims to come from. Nothing in that record set says the domain was created yesterday to imitate a payroll portal.
The lookup is deliberately narrow. It takes a hostname and reports whether that exact host has been observed and confirmed as live phishing infrastructure, with a category, a DNS status and the timestamp of the last verification. It makes no judgement about tone, sender reputation or business context.
The practical effect of that split shows up in any quarantine review. Pull a week of messages a mail team released after a user complained they were held wrongly, then pull a week of messages that reached a mailbox and later turned into an account-compromise ticket. The two piles barely overlap. The released mail is usually marketing, transactional notifications and correspondence from small senders with imperfect DNS hygiene, held on reputation or authentication grounds. The compromise pile is usually mail that authenticated cleanly and scored low, carrying one link to a host nobody had ever seen before. Tightening authentication policy makes the first pile bigger without touching the second.
A host-level known-bad check attacks the second pile directly. When a message is held because login-verify-payroll.example appears on a list of hostnames confirmed to be resolving and confirmed to be phishing, the release conversation with the recipient takes ten seconds. When a message is held because a composite score crossed 5.4, that conversation takes ten minutes and often ends in a policy exception that quietly weakens the gateway for everybody.
The same lookup is asked three times in a mail path, at three different moments, and each moment answers a question the others cannot. All three are covered in turn below; they differ only in call pattern.
Attachment sandboxing, display-name analysis, impersonation rules and the human process behind a payment-change request all remain necessary, and a message with no link at all is invisible to a URL check by construction. What the lookup provides is a floor under the link surface of inbound mail — the part of the message the user actually interacts with, and the part where a wrong click costs a password rather than an afternoon.
The work is mostly parsing. The lookup itself is a single request that can carry up to a hundred hostnames.
Pull URLs from the HTML part, the plain-text alternative, link attributes, and any markup carried inside an attachment.
Follow shorteners and redirectors to the destination, keeping every intermediate hostname as its own candidate.
Collapse to distinct hostnames, drop anything already cached, and post the remainder in blocks of a hundred.
Quarantine, strip, rewrite or tag, and write the matched host into a message header for later review.
Links live in a href attributes, but they also live in image-map regions, in background and source attributes a preview pane will fetch, in the plain-text alternative some clients render instead of the HTML, and inside attachments — a PDF with an embedded action URL, an HTML file posing as an invoice, a calendar invitation carrying a join link. Attachment-borne links matter more than their volume suggests, because a user who has already opened a file is past the point where they were being careful. Extract from everything you can parse, and treat the plain-text part as a first-class source rather than a fallback.
It puts a shortener, or a marketing-platform click tracker, or an open redirect on a legitimate site, and the destination sits three hops away. A gateway that checks only the host written in the message checks the shortener and learns nothing. Unwrap the chain and check every hostname you touch on the way, because intermediate hops are frequently attacker-controlled staging domains that already appear on the verified list even when the final landing page is a fresh registration nobody has seen yet.
The lookup is keyed on hostname, so forty links to the same newsletter domain in one message are one lookup rather than forty. Across a whole mail stream the same senders, tracking domains and content-delivery hosts recur all day, so tens of thousands of messages typically resolve to a few thousand distinct hosts before any caching is applied. Batch those in blocks of a hundred against POST /api/v1/batch, which charges one credit per domain in the request, and the per-message cost stops being a line item worth arguing over.
A verified phishing host in the visible body is a strong enough signal to hold the message outright in most environments. A verified host reached only after three redirects, inside a message that otherwise authenticates, might be better handled by stripping that link and delivering the rest, so the recipient still gets the invoice they were waiting for. Whichever you choose, write the matched hostname and the stage that caught it into a header. That header is what makes the quarantine queue reviewable and the retro-hunt described further down actually possible.
URL rewriting exists because hours pass between accepting a message and a user clicking it, and the infrastructure behind the link moves inside that window.
Rewriting is straightforward mechanically. The gateway replaces each link in the body with a URL pointing at its own click service, carrying an opaque token that maps back to the original destination. When the recipient clicks, the request lands on the click service first, which resolves the token, re-evaluates the destination, and either issues a redirect or renders a block page. Every mail-security product offering time-of-click protection is doing some version of this, and the entire value of it sits in what happens during that re-evaluation.
A message is accepted at 09:14. At that moment the link points at a host absent from every verified list, because it was registered forty minutes earlier and has served nothing yet. The recipient is in meetings and opens the message at 16:40. Somewhere in between, the host started serving a cloned single-sign-on page, was observed, was confirmed to be resolving, and entered the daily rebuild. A check performed at delivery says clean. A check performed at 16:40 against the same-day database says phishing — and only the second answer helps the person about to type a password.
This is why the click-time lookup should hit the live API rather than a snapshot the click service loaded at startup. Use GET /api/v1/check?domain=host&apikey=key for the single destination, honour the cache you already keep for the day, and accept the one credit for hosts you have not seen. The response carries last_checked and dns_status alongside the verdict, which lets the click service distinguish a host confirmed live an hour ago from one whose verification is older — useful when you want a hard block for the first and an interstitial warning for the second.
Pretending otherwise makes for a bad deployment, and mail teams who have run a rewrite service for a few years will recognise every item on this list.
The unwrapping work from stage two pays a second dividend here. If the click service re-follows the chain rather than trusting the destination it recorded at delivery, it catches the case where a benign-looking intermediate was repointed after the message was accepted. Open redirects on legitimate domains are used for exactly this: the host in the message never changes and never becomes suspicious, but the parameter it forwards to is swapped once the campaign is ready.
The API key is passed as a query parameter or in the POST body and must be used only from server-side code — the gateway, the milter, the click service. It never belongs in a browser extension, a mail-client add-in, or any JavaScript that reaches a recipient. If a rewrite interstitial needs a verdict, have the click service fetch it and render the result, rather than letting the page call the API itself.
Three mechanisms often described as overlapping in fact answer three different questions, and only one of them is about the destination.
| Question the gateway needs answered | SPF / DKIM / DMARC | Content & reputation scoring | DNS-verified host lookup |
|---|---|---|---|
| Did this domain really send the message? | Exactly what it is for | Inferred, not proven | Out of scope |
| Is the destination of the link hostile? | Never evaluated | Only by reputation proxy | The single question asked |
| Does it work on a domain registered this week? | The attacker publishes valid records | Age is a weak signal alone | Once observed and DNS-verified |
| Does it survive a compromised internal mailbox? | Internal mail authenticates perfectly | Trusted sender, low score | The link is checked regardless of sender |
| Can the verdict be explained in one line? | Alignment failed | A composite score, rarely legible | This exact host is confirmed phishing |
| Does it flag domains that have already gone dark? | Not applicable | Stale entries persist in feeds | Non-resolving entries are dropped |
| Does it catch a message with no link at all? | Only if the sender is spoofed | Its strongest ground | Nothing to look up |
Read the last row before the others. A payment-redirection request written in plain prose, with no URL and no attachment, is invisible to a host lookup and always will be. That is the case for keeping impersonation rules, display-name analysis and a finance process that verifies bank-detail changes out of band. The table is not an argument that one column replaces another; it is an argument that the middle column has been asked to cover the second row for years and has never been good at it.
Feeds that accumulate indicators without ever retiring them generate quarantine hits on hosts that stopped resolving months ago — the campaign is over, the registration has lapsed, the link in the message goes nowhere — and every one of those still costs an analyst a review. A list that verifies DNS on each rebuild and drops whatever no longer resolves keeps the queue populated with things that are actually live, which is the difference between a queue a team works and a queue a team learns to ignore.
Credits are consumed per hostname, not per message — which is why deduplication and caching are the whole cost story.
Work an example. A gateway accepts 40,000 inbound messages on a normal weekday, and each message averages five distinct hostnames after parsing — 200,000 host references. Deduplicating within the message drops that sharply; deduplicating across the whole stream drops it much further, because the same newsletter platforms, tracking domains and content hosts appear in message after message. In practice a stream of that size resolves to a few thousand genuinely distinct hostnames a day. Call it 6,000. That is 60 batch requests and 6,000 credits, or roughly 180,000 credits a month.
Against the published packages that sizes easily. Professional is Professional at $249/month for 100,000 lookups; Business is Business at $499/month for 250,000 lookups; Enterprise is Enterprise at $999/month for 750,000 lookups. A well-cached gateway at the volume above sits comfortably inside Business for a year, and a multi-tenant provider running the same pipeline across many customer domains — where the cache hit rate is higher still, because tenants share most of the internet — will find Enterprise or larger the right shape. lookups reset monthly and no subscription is attached to the API itself, so an unusually noisy month does not reset anything.
Both are worth designing around before the first invoice rather than after it, and both are ordinary engineering rather than anything to do with the list.
Once one mailbox is taken, the attacker is operating inside every trust relationship the gateway was built on.
The sequence is familiar to anyone who has run an incident. A single user authenticates on a cloned sign-in page. The attacker takes the session, adds a mail rule filing replies into an obscure folder, reads enough of the mailbox to learn who this person emails and how they write, and then sends. The mail they send is internal. It originates from the real account, on the real platform, and it authenticates because it genuinely is that user. Sender reputation is perfect. Display-name analysis has nothing to flag. Every control watching the perimeter is watching the wrong door.
The attacker still needs somewhere to send the next recipient — a document-share lure, a supposedly shared file, a payment portal — and that somewhere is infrastructure they control. If the internal mail path runs the same extraction and the same lookup as the inbound path, a hostname already on the verified list is caught on an internal message exactly as it would have been on an external one. It is the one control in the stack whose accuracy does not depend on the sender being untrusted.
Internal mail often bypasses the gateway entirely for latency reasons. On a hosted platform, fixing that means a rule routing internal traffic through the scanning connector, or an API-based scanner that reads messages after they land and can soft-delete from mailboxes. On a self-hosted setup it means the milter runs on the internal submission path as well as the inbound one. The extraction and batching logic is identical either way, so the incremental engineering is small even where the mail-flow change is politically awkward.
A held internal message is far more likely to be genuine business mail than a held external one, so the release path has to be fast and the reviewer needs to see why it was held without opening the message. This is where the header written at stage four earns its place: the queue shows the matched hostname, the stage that caught it and the verification timestamp. Make release a two-part action — deliver the message, and record the hostname as reviewed — otherwise the same host generates the same review every time it appears in a thread.
When the sender is one of your own accounts, the right response is not merely to hold the message; it is to check that mailbox for hostile rules, revoke its sessions, and look at what else it sent in the preceding hours. Teams who already run this playbook will recognise the overlap with incident response and with the alerting they push into SIEM integration — the mail verdict is the trigger, not the conclusion.
Most gateways can enforce all of this. What is usually missing is a written statement of what the gateway is supposed to do, so the settings drift.
POST /api/v1/batch in blocks of no more than 100. One credit is consumed per domain submitted.The most valuable thing about a list rebuilt every twenty-four hours is the diff, and the diff is only useful if you kept a record of what you let through.
POST /api/v1/batch
Up to 100 hostnames per request, one lookup each. The workhorse call for the inbound and internal scanning paths.
GET /api/v1/check?domain=
A single hostname answered against the current database. Used by the click service when a rewritten link is followed.
Daily CSV feed + changelog
The full database as CSV on a separate subscription, plus the daily added and removed lists that drive the backwards search.
Retro-hunting works because of a simple asymmetry: a host unknown at 09:14 is frequently known by the following morning's rebuild. If you journal delivered mail, or if you simply retained the evidence headers described in the rule outline, you can answer the question that matters after the fact — did anybody receive a message pointing at a host we now know is phishing? The job is unglamorous. Pull the day's added list from the changelog, match it against the hostnames recorded for the last thirty days of delivered mail, and produce a list of recipients. It runs in minutes and it finds things nothing else in the stack will.
Either way the incident starts from a specific hostname and a specific recipient list rather than a vague alert, which is exactly what makes it actionable.
Matching thirty days of journaled hostnames one lookup at a time would be enormous and pointless; loading the day's added list locally and doing a set intersection costs nothing per record. The feed is a separate subscription from API credits precisely because the two workloads differ — one is per-decision and latency-sensitive, the other is bulk and offline. That same feed also suits DNS resolvers, firewall block lists and the offline analysis described on the daily feed page.
Here is the shape of the two calls a gateway makes: the batch request during delivery, and the single check from the click service.
# Delivery path: every distinct host extracted from one batch of messages
curl -s -X POST https://phishingdetectionapi.com/api/v1/batch \
-H "Content-Type: application/json" \
-d '{"apikey":"YOUR_KEY",
"domains":["mail.tracking-partner.example",
"sso-account-verify.example",
"cdn.newsletter-platform.example"]}'
# Click service: one host, answered against today's database
curl -s "https://phishingdetectionapi.com/api/v1/check\
?domain=sso-account-verify.example&apikey=YOUR_KEY"
# Response shape returned for each domain
{
"domain": "sso-account-verify.example",
"is_phishing": true,
"category": "phishing",
"dns_status": "active",
"last_checked": "2026-07-27T04:18:00Z",
"confidence": "high",
"database_size": 390000
}Clause 8 says so deliberately. Mail that stops moving because a lookup timed out is an outage your users feel within minutes, and a gateway that holds mail on infrastructure failure will be switched off by an administrator at two in the morning. Deliver, mark the header, and let the re-scan queue pick the message up. That behaviour also means the check can be introduced into a live mail path without a maintenance window, which matters when the alternative is never introducing it at all.
If the gateway feeds a broader detection programme, the verdicts are worth exporting rather than discarding. A hostname appearing repeatedly across many recipients is a campaign, and knowing which department it reached first is genuinely useful to the people running security awareness programmes. Teams already correlating indicators across controls will want the same verdicts alongside their threat intelligence pipeline, and organisations operating their own resolvers usually pair mail scanning with DNS filtering so a host blocked in mail is also unreachable from the browser. Where the same organisation also ships an agent to laptops, the verdicts belong in endpoint protection as well, since that is the layer that still applies when the device is off the corporate network.
The objections below come up in roughly this order, and every one of them is reasonable.
The lookup itself is typically under 50 milliseconds, and a batch call covering up to a hundred hostnames costs about the same as a single one — it is one round trip regardless of how many domains it carries. For a message whose hosts are already in the shared cache, the added time is a local memory read.
Because the two answer differently shaped questions. A reputation service scores a host on a spectrum, blending age, hosting, category and observed behaviour, tuned to a threshold you generally cannot inspect. A verified known-bad list answers one binary question about one hostname and tells you when it last confirmed the answer.
It will not be on the list, and no honest description of this control claims otherwise. A hostname registered forty minutes ago that has not yet been observed serving anything cannot be in any verified database, because verification requires observation. A delivery-time lookup against a brand-new host returns clean, and returning clean is the correct behaviour.
Yes, though the integration point moves. Instead of a milter or content filter in the SMTP path, you either route mail through a connector that performs the scan before delivery, or you run an API-based scanner that reads messages as they land and acts on them afterwards. The second pattern delivers first and remediates within seconds, which is weaker than blocking but far easier to deploy.
Share one cache across tenants for the lookup itself, and account per tenant separately. Tenants overwhelmingly see the same internet — the same newsletter platforms, the same cloud hosts, the same trackers — so a shared cache gives a hit rate no single-tenant deployment can match. What each tenant then costs you is the count of hostnames genuinely new when their mail arrived.
The message is delivered, the header records that the scan was skipped, and the message enters the re-scan queue. This is a deliberate design choice rather than a limitation: any control placed inline in a mail path has to degrade toward delivery, because held mail is an outage and users escalate outages fast.
The list contains hostnames confirmed as phishing infrastructure and verified as currently resolving, so a large marketing platform's sending or tracking domains are not the sort of thing that lands on it. The more realistic edge case is a legitimate host that was compromised and used to serve a phishing page — in which case the block is correct while the page is live, and the entry leaves the database once the host stops resolving that way.
The value of this control comes from being narrow. Being clear about the edges is what keeps it trusted.
Wire-transfer fraud, vendor bank-detail changes, gift-card requests and most of what gets filed under business email compromise are conversations, not URLs. They are defeated by out-of-band verification and a finance process that does not treat email as an instruction channel. If a supplier emails new bank details and somebody pays them, no URL database was ever going to be part of that story.
When the bait is a document on a mainstream file-sharing service, or a page on a general-purpose hosting subdomain, the hostname is shared with an enormous amount of legitimate traffic and blocking it is not an option. This is a real and growing gap, and the honest answer is that the unit of decision — the hostname — is the wrong unit for that case. Page-level analysis and attachment sandboxing are the controls that address it.
Everything above about click-time re-checking and the daily changelog is an attempt to shrink that window, not to pretend it does not exist. A campaign that registers, sends and burns its infrastructure inside a couple of hours can complete entirely within the gap, and the retro-hunt will find it the next morning — useful for remediation, useless for prevention.
It removes the large population of hostnames already caught and still live, and that population is not small: the database holds more than 390,000 DNS-verified active phishing domains at any moment, rebuilt every twenty-four hours so dead infrastructure is dropped rather than accumulated. Every one of those hosts is a link that will not resolve into a login form in one of your users' browsers. That is a floor, not a ceiling, and a mail team that describes it that way internally will get more out of it than one that oversells it.
Batch every extracted hostname during delivery, re-check at click time, and let the daily changelog tell you what you delivered before anyone knew better.
