Security

DNS Security Explained: DNSSEC, DNS over HTTPS, DNS over TLS, and Why Your Queries Are Probably Leaking Right Now

A principal cloud architect breaks down DNSSEC, DNS over HTTPS, DNS over TLS, and the real-world attacks that exploit unprotected DNS infrastructure.

Layered diagram showing DNS query path with DNSSEC chain-of-trust validation and encrypted DoH/DoT transport overlaid on a network topology

Twenty years in this field and DNS security still doesn’t get the respect it deserves. I’ve watched organizations spend millions on next-gen firewalls, endpoint detection, and SIEM platforms, then leave their DNS completely unprotected. They’ll audit every API endpoint and harden every SSH configuration, but their resolvers are sitting open, logging nothing, trusting everything, and broadcasting query metadata to anyone in a position to intercept it.

DNS is the phone book of the internet. Everyone knows that. What fewer people internalize is that for most of its history, DNS operated entirely in the clear, with no authentication and no encryption. Every domain your servers resolve, every internal service name that leaks out through a misconfigured forwarder, every query your developers make from a coffee shop: all of it visible to anyone between you and your resolver.

NIST published a major update to their DNS security guidance in early 2026 (SP 800-81r3), the first significant revision in over a decade. That document exists because the threat landscape around DNS has fundamentally changed. Before we talk about solutions, let me show you exactly what you’re up against.

The DNS Attack Surface You Are Probably Ignoring

DNS attacks fall into a few categories. Cache poisoning has been around since the 1990s, but the Kaminsky attack in 2008 turned it from a theoretical concern into a practical emergency. The trick is that DNS resolvers cache responses to reduce latency. If an attacker can inject a forged response before the legitimate one arrives, the resolver caches the bad entry and serves it to everyone who asks. Kaminsky’s insight was that you don’t need to spoof the exact query: you can flood a resolver with responses for random subdomains while simultaneously forging the glue records that direct the resolver to your malicious nameserver. The attack worked at scale because the birthday paradox makes it far easier to guess a 16-bit transaction ID than people expected.

DNS hijacking is different. Instead of poisoning a cache, an attacker redirects queries at the infrastructure level: compromising the registrar account, modifying NS records on the authoritative server, or intercepting traffic at an ISP or on a local network. The Sea Turtle campaign, publicly attributed to state-level actors, hijacked DNS for government and military targets across multiple continents by compromising registrar credentials. The victims had no idea anything was wrong because, from their perspective, their DNS configuration hadn’t changed.

DNS amplification sits in a third category. UDP-based DNS responses can be 70 times larger than the query that triggered them, especially when requesting ANY records or DNSSEC-signed responses. An attacker spoofs the source IP of a small query, sends it to thousands of open resolvers, and directs all those amplified responses at the target. This is pure volumetric DDoS. I’ve seen organizations absorb hundreds of gigabits per second through this vector.

There is also DNS exfiltration, which is the sneaky one. Data moves out by encoding it in subdomain labels: c29tZXNlY3JldGRhdGE.attacker.com, where the long string is base64-encoded payload. Because most firewalls inspect packets for known protocols but pass DNS on port 53 without scrutiny, exfiltration tunnels hide in plain sight. Malware like DNSCat2 has used this technique for years. If you don’t have DNS logging with anomaly detection, you will not catch it.

DNS attack surface diagram: cache poisoning, DNS hijacking, amplification, and exfiltration attack vectors

DNSSEC: Proving Your Answers Are Real

DNSSEC adds cryptographic signatures to DNS data. When a zone is DNSSEC-signed, every record set gets a corresponding RRSIG record. Resolvers that support DNSSEC validation follow a chain of trust from the root zone down through TLDs to the authoritative nameserver for the domain in question. If any signature fails to validate, the resolver returns SERVFAIL rather than serving a potentially forged response.

The chain of trust works like this. The root zone is signed with a key pair maintained by IANA. Each TLD (.com, .net, .io) publishes a Delegation Signer (DS) record that contains a hash of the child zone’s Key Signing Key (KSK). The child zone publishes its own DNSKEY records and signs its records with a Zone Signing Key (ZSK). A validating resolver walks this chain: it trusts the root’s public key (distributed out-of-band as a trust anchor), verifies the DS records at each delegation point, and ultimately validates signatures on the records it actually cares about.

This sounds good in theory. In practice, DNSSEC has a few sharp edges that took me years to fully appreciate.

First, DNSSEC proves authenticity, not confidentiality. Your responses are signed, but they’re still transmitted in plaintext over UDP. An observer can’t forge a response, but they can absolutely read it. DNSSEC and encrypted DNS (DoH/DoT) solve orthogonal problems.

Second, operational complexity is real. Key rollover is a process that’s easy to get wrong. Many organizations have had outages because their KSK rollover didn’t propagate correctly before the old key expired. The 2017 ICANN root KSK rollover was delayed by a year precisely because too many resolvers still had the old key hard-coded and would have broken if the rollover had proceeded on schedule.

Third, DNSSEC introduces NSEC and NSEC3 records for authenticated denial of existence, which means you can prove a domain doesn’t exist. NSEC (the original) allows zone enumeration: an attacker can walk the entire zone by following NSEC chains. NSEC3 uses hashing to prevent this, but even hashed names are vulnerable to offline dictionary attacks if your zone uses predictable naming conventions.

Fourth, DNSSEC validation rates remain stubbornly low. As of 2026, somewhere between 30-40% of global resolvers actually validate DNSSEC signatures. If you sign your zone but the resolver your users are hitting doesn’t validate, you get exactly zero protection from the cryptographic signatures you just spent a day implementing.

Despite these limitations, you should sign your zones. For critical infrastructure (payment systems, authentication endpoints, internal PKI), DNSSEC eliminates an entire class of attack at the cost of operational overhead that is manageable with the right tooling. BIND, PowerDNS, and all major cloud providers support DNSSEC with varying degrees of automation.

For deeper background on how DNS resolution works before you add the security layers, the DNS resolution process article covers the recursive lookup chain in detail. And if you’re unclear on DNS record types like DS, DNSKEY, and RRSIG records that DNSSEC introduces, read that first.

DNS over TLS and DNS over HTTPS: Encrypting the Query

DNSSEC prevents response forgery. DoT and DoH prevent query interception. These are separate problems requiring separate solutions, which is why you often deploy both.

DNS over TLS wraps DNS queries in a TLS connection on port 853. The resolver and the client establish a TLS session, then speak ordinary DNS wire format inside that encrypted tunnel. From a network perspective, DoT is identifiable because port 853 is exclusively used for this purpose. An enterprise can allow port 853 to specific trusted resolvers and block everything else, giving IT visibility into which resolver clients are actually using.

DNS over HTTPS wraps DNS queries in HTTPS on port 443. From a network observer’s perspective, DoH traffic looks identical to ordinary web traffic. This is intentional: the design goal was to prevent ISPs and network operators from observing or blocking DNS queries. DoH has a JSON API mode and a DNS wire-format mode (the application/dns-message content type). Major browsers started shipping with DoH enabled by default in 2019 and 2020, which immediately created a conflict with enterprise network security teams.

Here’s the tension that I’ve had to explain to CISOs more times than I can count: DoH is excellent for protecting individual privacy on untrusted networks. It is a serious headache for enterprise DNS visibility. When browsers make their own encrypted DNS queries directly to Cloudflare or Google, bypassing your corporate resolver, your DNS logging, your protective DNS filtering, and your data exfiltration detection all have a blind spot the size of every HTTPS connection. The Chromium-based browsers handle this reasonably: they have a “Secure DNS” mode that falls back to plaintext if the detected resolver doesn’t support DoH, and they respect enterprise policy settings. Firefox’s approach was more aggressive, enabling DoH regardless of network configuration in some cases.

For enterprise deployments, the practical answer is to run your own DoH or DoT resolver internally and push clients to use it via policy. Pi-hole, Unbound, and Knot Resolver all support DoH and DoT on the recursive resolver side. On the authoritative side, name servers don’t typically need to support DoH: only recursive resolvers, which talk to clients directly, need to implement it.

Comparison of DNS over TLS (port 853) vs DNS over HTTPS (port 443) showing transport differences, enterprise visibility, and use case trade-offs

DNS over QUIC (DoQ) is the newer entrant, standardized in RFC 9250. It uses QUIC as the transport layer instead of TCP or TLS-over-TCP, which means it inherits QUIC’s properties: no head-of-line blocking, faster connection establishment, and native multiplexing. DoQ runs on port 8853 by convention. Adoption is still early in 2026, but if you’re looking at QUIC and HTTP/3 for your application traffic, DoQ is the logical extension of that investment to your DNS infrastructure.

Protective DNS: The Layer You Actually Need

Beyond encrypting DNS queries and validating signatures, there’s a third approach that often delivers the most immediate security value: running your recursive lookups through a resolver that actively inspects and filters them.

Protective DNS services maintain threat intelligence feeds and block queries to known malicious domains at the resolver level. Cloudflare’s 1.1.1.1 for Families, Cisco Umbrella, Infoblox BloxOne, and similar services block malware command-and-control domains, phishing sites, and newly registered domains with suspicious characteristics before a single packet reaches the destination.

The architecture is straightforward: you point your resolvers (or your clients) at the protective DNS service, configure your firewall to block direct port 53 traffic to any other destination, and suddenly every DNS query in your environment is being inspected against a threat feed. This is not a replacement for endpoint detection or network monitoring, but it is a remarkably high-leverage control because it works before any payload is delivered.

I deployed Umbrella across a financial services client a few years back. Within the first week, it blocked several thousand queries to domains that were part of known botnet infrastructure. The endpoints making those queries had passed EDR checks. The outbound firewall hadn’t flagged anything. DNS-layer inspection caught what everything else missed.

The NIST SP 800-81r3 update specifically calls out protective DNS as a baseline control recommendation. For organizations without a mature security operations center, it is one of the highest-value controls you can implement with minimal infrastructure investment.

One thing to watch for: category-based filtering can produce unexpected false positives in cloud environments. If your protective DNS service categorizes certain CDN subdomains as suspicious because they were recently registered, you’ll see connection failures in unexpected places. Always have an exception process and monitor blocked queries actively rather than treating the filter as a set-and-forget control.

The relationship between DNS security and zero trust security architecture is direct: a zero trust model assumes you can’t trust the network, which means you also can’t trust DNS unless you’ve verified it end-to-end. Encrypted DNS to a known-good resolver is part of implementing that assumption consistently.

Response Rate Limiting: Stopping Amplification at the Source

If you’re running authoritative nameservers, you need Response Rate Limiting (RRL). RRL is a feature in BIND, NSD, and Knot that limits how many similar responses a nameserver will send to the same source IP address within a time window. When an attacker spoofs your IP and sends queries to open resolvers, you’re not the target, but your nameservers are being used as amplifiers. RRL breaks this by truncating or dropping responses above the threshold.

The configuration is straightforward in BIND:

rate-limit {
    responses-per-second 5;
    referrals-per-second 5;
    nodata-per-second 5;
    nxdomains-per-second 5;
    slip 2;
    window 15;
};

The slip parameter controls how often a truncated response is sent instead of a dropped one. A truncated response signals the client to retry over TCP, which is harder to spoof. Legitimate clients will retry; attackers spoofing source IPs won’t get the benefit.

Split-Horizon DNS and Internal DNS Security

Enterprise environments typically run split-horizon DNS: internal resolvers that serve private zone data, separate from the authoritative nameservers that serve external queries. This is a security control in itself, since it means internal service names and RFC 1918 addresses never appear in externally visible DNS responses. But split-horizon configurations introduce their own risks.

The most common failure I’ve seen is DNS rebinding. An attacker registers a domain and configures its authoritative nameserver to return a very short TTL. When a browser inside your network resolves that domain, it gets a legitimate external IP. After the TTL expires, the attacker’s nameserver returns your internal RFC 1918 address. The browser, following same-origin policy, now treats the attacker’s domain as if it’s the internal address. The attacker can now make requests from the user’s browser to internal services that have no authentication because they assume they’re not accessible from outside.

Protection: configure your recursive resolvers to refuse to return RFC 1918 addresses for external domains. In BIND, this is done with deny-answer-addresses. In Unbound, use private-address and private-domain options to block private IP ranges from appearing in externally resolved answers.

DNS logging deserves special attention here. If you’re logging DNS queries at all, you want enough detail to reconstruct: which IP queried which domain, at what time, and what response they received. This is the foundation of detecting exfiltration, identifying compromised hosts, and retrospectively investigating incidents. Forward your DNS logs to your SIEM or log aggregation platform. The volume can be high for large environments, so consider sampling or filtering out known-good domains (Alexa top 1000, your own internal zones) before indexing everything.

Internal DNS security architecture showing split-horizon DNS, private zone isolation, recursive resolver hardening, and protective DNS integration

VPN and DNS Leaks: The Configuration Mistake That Neutralizes Your Security

One failure mode I’ve seen trip up engineering teams repeatedly: running a VPN without ensuring DNS queries go through the tunnel. If your VPN client routes only data traffic through the tunnel but continues to send DNS queries to the local ISP resolver, you have a DNS leak. The ISP can see every domain you’re resolving even though they can’t see the traffic that follows.

DNS leak testing is trivial: tools like dnsleaktest.com show you which resolvers your DNS queries are reaching. If you’re using a VPN for security or privacy and the results show your ISP’s resolver, your DNS queries are leaking.

The correct fix is to ensure that your VPN configuration routes DNS traffic through the tunnel and pushes a resolver configuration that sends queries to a resolver inside the tunnel endpoint. WireGuard, which we covered in depth in the WireGuard VPN guide, handles this through its DNS option in the interface configuration. OpenVPN handles it through dhcp-option DNS. The important thing is to test after configuration, not assume.

For the relationship between DNS and TLS specifically: if you understand how TLS works and how SNI exposes the destination hostname even in encrypted HTTPS traffic, you’ll appreciate why DNS-layer encryption is necessary but not sufficient for privacy. Both DNS queries and SNI headers reveal destination intent, and Encrypted Client Hello (ECH) is the complementary standard addressing the SNI exposure.

Practical Implementation Roadmap

After two decades of helping organizations improve their DNS security posture, here’s the sequence I recommend:

Week one: Enable DNS query logging on your recursive resolvers. You can’t secure what you can’t see. Even before you deploy any additional controls, visibility is the foundation.

Week one: Deploy protective DNS. Point your internal resolvers at Cloudflare Gateway, Cisco Umbrella, or an equivalent service configured with your policy requirements. This delivers immediate value against known threats.

Month one: Sign your authoritative zones with DNSSEC for any zones that host critical services (authentication, APIs, payment endpoints). Automate key rollover from the start rather than treating it as a one-time operation.

Month one: Enable DoT or DoH on your recursive resolvers for outbound queries to upstream resolvers. Your internal resolver talking to a DoT-capable upstream (Cloudflare 1.1.1.1:853, Google 8.8.8.8:853) encrypts the leg that’s most exposed to interception.

Month two: Deploy DoT or DoH for client-to-resolver communication, at least for mobile devices and remote workers. Push resolver configuration via MDM so clients use your company’s DoT resolver rather than whatever is configured on the local network.

Month three: Configure RRL on your authoritative nameservers, review your split-horizon configuration for rebinding vulnerabilities, and audit resolver configurations for open recursion (your resolvers should not answer DNS queries from arbitrary internet IPs).

Ongoing: Monitor DNS logs for anomalies: high query rates to newly registered domains, unusually long subdomains (potential exfiltration), queries to domains that have no legitimate business purpose, and repeated NXDOMAIN responses that could indicate C2 beaconing or misconfigured malware.

Where This Fits in Your Security Architecture

DNS security is not a standalone project. It’s a layer in a defense-in-depth model.

DNSSEC addresses response authenticity. DoH and DoT address query confidentiality. Protective DNS addresses threat intelligence filtering. RRL addresses amplification. Split-horizon and rebinding protection address internal exposure. Query logging addresses visibility and forensics.

No single control addresses all of these. The organizations I’ve seen get compromised via DNS were usually missing one specific piece: they had DNSSEC but no logging, or they had logging but no filtering, or they had a VPN but no leak detection.

DNS is part of every attack chain. Phishing uses DNS. Malware uses DNS for C2. Data exfiltration uses DNS. Supply chain attacks use DNS to redirect software update endpoints. Getting DNS security right doesn’t prevent all attacks, but it removes an entire class of vectors that are trivially exploitable when left unaddressed.

BGP security faces a structurally similar problem: an infrastructure protocol designed for a trusted network that now operates in an adversarial environment. The solutions in both cases (RPKI for BGP, DNSSEC for DNS) add cryptographic authentication to protocols that were never designed with it, and both face adoption challenges that make the protections only partially effective at internet scale. But both are worth deploying anyway, because partial protection is better than none, and because the organizations that do deploy them are measurably harder targets than those that don’t.

Start with logging and protective DNS this week. Sign your critical zones this month. Roll out DoT to your resolvers next quarter. The threat is real, the solutions are mature, and the window for treating DNS as a “just works” infrastructure component is long closed.