Stop Editing /etc/resolv.conf: Practical systemd-resolved on Linux

nameserver 127.0.0.53 options edns0 trust-ad search corp.example.com ~~~ Nothing you wrote survived. On a system running `systemd-resolved`, `/etc/resolv.conf` is not configuration. It is output. The real configuration lives in `/etc/systemd/resolved.conf`, in per-link `.network` files, in DHCP leases, and in the resolver's runtime state. Editing the generated file is like editing a log to change a program's behavior. That is the small version of the problem. The larger version is that `systemd-resolved` is not just a caching DNS forwarder. It is a per-link query router, and most "DNS is broken" incidents come from misunderstanding which link and which server the resolver chose. Here is the model, the failure modes, and a debugging sequence that tells you what actually happened. ## The daemon in the middle `systemd-resolved` provides a caching, DNSSEC-validating stub resolver, plus LLMNR and MulticastDNS support. Local programs can reach it through three different interfaces: 1. **The native D-Bus or Varlink API**, which returns rich metadata such as which link answered and whether the data was DNSSEC-authenticated. 2. **glibc's NSS resolver**, through the `nss-resolve` module. This is how ordinary `getaddrinfo()` calls reach the resolver. 3. **A DNS stub listener** on `127.0.0.53` and `127.0.0.54`, port 53. This exists for programs that speak raw DNS and bypass NSS. The two stub addresses are not identical. `127.0.0.53` is the full resolver, including LLMNR and mDNS. `127.0.0.54` is a proxy-only listener: it forwards messages to upstream servers largely unmodified, does not validate DNSSEC, and does not answer LLMNR or mDNS. It exists for compatibility with software that needs a plain forwarding DNS server. Check what is listening before you debug anything else: ~~~bash systemctl status systemd-resolved --no-pager sudo ss -lntup 'sport = :53' ~~~ On a working host you should see `127.0.0.53:53` and `127.0.0.54:53`. If another process owns port 53, read on. That is one of the classic failures. ## Four ways /etc/resolv.conf can exist The `systemd-resolved.service` manual documents four supported modes. The mode is detected automatically, mostly by what `/etc/resolv.conf` points to. - **Symlink to `/run/systemd/resolve/stub-resolv.conf`.** The file contains `nameserver 127.0.0.53` plus the search domains currently in use. This is the recommended mode. It is generated and updated by the resolver. - **Symlink to `/usr/lib/systemd/resolv.conf`.** A static file containing only the stub address. No search domains. Useful in minimal images, less useful on a laptop that moves between networks. - **Symlink to `/run/systemd/resolve/resolv.conf`.** This file lists all known upstream DNS servers. Programs that read it talk directly to those servers and bypass `systemd-resolved` entirely. You lose the cache, DNSSEC validation, and per-link routing. - **A regular file managed by something else.** In this mode `systemd-resolved` is a consumer, not the provider. It reads the file to discover DNS servers, but it does not manage it. That last point is the one people miss. When `/etc/resolv.conf` is a symlink to the stub file, `systemd-resolved` ignores its contents as input. Adding a `nameserver` there does not tell the resolver to use that server. It only changes what other programs read, and the file will be regenerated. Find out which mode you are in: ~~~bash readlink -f /etc/resolv.conf resolvectl status | head -5 ~~~ The status output includes a line like `resolv.conf mode: stub` or `resolv.conf mode: foreign`. Believe that line over whatever you remember doing to the machine six months ago. ## Where the settings actually live There are four layers, and they are not equal: **Global configuration** lives in `/etc/systemd/resolved.conf` and in drop-ins under `/etc/systemd/resolved.conf.d/*.conf`. This is where you set system-wide DNS servers, DNSSEC, DNS-over-TLS, and cache behavior. **Per-link static configuration** lives in `/etc/systemd/network/*.network` files when `systemd-networkd` manages the interface. The relevant settings are in the `[Network]` section: `DNS=`, `Domains=`, `DNSDefaultRoute=`, `DNSSEC=`, `DNSOverTLS=`, and `DNSSECNegativeTrustAnchors=`. In the `[DHCP]` section, `UseDNS=no` tells networkd to ignore DNS servers offered by DHCP. **Per-link dynamic configuration** comes from DHCP, IPv6 router advertisements, NetworkManager, or other services. This is why a VPN can quietly change your DNS behavior without touching a file you edited. **Runtime configuration** comes from `resolvectl dns`, `resolvectl domain`, and friends. It is not written to disk. It disappears on reboot or when the link disappears. It is excellent for testing and terrible as a permanent fix. A practical primary-link configuration looks like this: ~~~ini # /etc/systemd/network/10-lan.network [Match] Name=eth0 [Network] DHCP=yes DNS=192.0.2.53 Domains=~corp.example.com DNSDefaultRoute=yes [DHCP] UseDNS=no ~~~ `UseDNS=no` is doing real work here. Without it, the DHCP-provided DNS servers are added alongside the static one, and the resolver may prefer a server you did not intend. With it, the link uses `192.0.2.53` and nothing else. A VPN link that should only receive corporate queries looks different: ~~~ini # /etc/systemd/network/90-wg0.network [Match] Name=wg0 [Network] DNS=10.20.0.53 Domains=~corp.example.com ~20.10.in-addr.arpa DNSDefaultRoute=no ~~~ Apply networkd changes with: ~~~bash sudo networkctl reload sudo networkctl reconfigure wg0 ~~~ For a quick experiment, the runtime equivalent is: ~~~bash sudo resolvectl dns wg0 10.20.0.53 sudo resolvectl domain wg0 '~corp.example.com' '~20.10.in-addr.arpa' sudo resolvectl default-route wg0 no ~~~ Then `resolvectl revert wg0` puts the link back to its discovered settings. Do not stop at this step and call it a fix; it will not survive the next link reset. ## How a query picks a server This is the part worth learning properly. The rules are documented, deterministic, and simpler than the folklore. For a unicast DNS query, `systemd-resolved` does this: 1. If the name matches a configured routing domain, pick the **best match** — the domain with the most labels. Send the query to all DNS servers on every link that has that best-matching domain. 2. If no domain matches, send the query to all links marked as a default route, plus any globally configured DNS servers. 3. If there are no default-route links and no global servers, use the compiled-in fallback servers. 4. Otherwise, fail. There is no suitable server. Search domains and route-only domains both participate in routing. The tilde is what changes the behavior: - `Domains=corp.example.com` is a **search domain**. It is appended to single-label names, and it also routes matching queries to this link. - `Domains=~corp.example.com` is a **route-only domain**. It routes matching queries to this link but is not used as a search suffix. - `Domains=~.` is the root route-only domain. It claims every name that is not matched by a more specific domain. Use it when you want one link to be the preferred resolver for everything else. `DNSDefaultRoute=` controls the fallback. If you do not set it, the automatic rule is: a link with a route-only domain other than `~.` defaults to `no`; a link without such a domain defaults to `yes`. That automatic rule is responsible for a lot of accidental VPN DNS leaks. A link configured with the search domain `corp.example.com` and no route-only domain becomes a default route, so it receives queries for the public internet too. Add the tilde, or set `DNSDefaultRoute=no` explicitly, and the leak stops. When multiple links carry the same best-matching domain, the query is sent to all of them in parallel and the first successful answer wins. That is why two DNS servers in the same lookup scope must serve equivalent data. The resolver is not doing a vote; it is doing a race. ## The search-domain trap that breaks muscle memory The `systemd-resolved` manual has a compatibility section, and one paragraph deserves to be quoted: > Search domains are not used for suffixing of multi-label names. Any name with at least one dot is always interpreted as a FQDN. And: > This stub resolver does not implement this at all: multi-label names are only resolved as FQDNs. If your search domain is `corp.example.com`, then: ~~~bash getent hosts db # may try db.corp.example.com getent hosts db.prod # queries db.prod. as-is, no suffix ~~~ The second command does not try `db.prod.corp.example.com`. Traditional glibc behavior, with `ndots` and a search list, often would. Here, `ndots` is not implemented. If you rely on `ssh db.prod` resolving through a search suffix, it will fail even though `ssh db` works. Single-label names have a separate rule. By default, A and AAAA queries for a single-label name are not sent to unicast DNS at all. They are tried through LLMNR, and then through search domains if any are configured. `ResolveUnicastSingleLabel=yes` changes that and forwards them upstream, but the manual warns that this is non-standard and can leak internal names to servers you do not control. Prefer a search domain or an FQDN. The `.local` suffix has its own rule: it is reserved for MulticastDNS by RFC 6762. Queries for `.local` names are not routed to unicast DNS unless you explicitly configure that domain as a search or routing domain. Do not put your company's internal namespace under `.local` and then wonder why the resolver refuses to ask a normal DNS server about it. ## DNSSEC and DNS-over-TLS without fairy tales Two defaults matter: - `DNSSEC=` defaults to `allow-downgrade`. Validation is attempted, but if the server does not support DNSSEC properly, the resolver turns validation off. This is pragmatic and deliberately vulnerable to downgrade attacks. - `DNSOverTLS=` defaults to `no`. DNS is sent over UDP. You can see the effective state per link in `resolvectl status`, in a line like `DNSSEC=no/unsupported` or `DNSSEC=allow-downgrade/supported`. The first half is the mode; the second half is whether the current server appears to support it. If you set `DNSSEC=yes`, every lookup must validate. That is the point, but it has consequences: - A private zone that is not signed will fail validation. The resolver returns a failure, not an unsigned answer. - A broken or outdated trust anchor can make the whole internet fail validation. - A server that does not support DNSSEC will produce failures for every name, not a quiet fallback. For a private, unsigned zone, do not disable DNSSEC globally. Add a negative trust anchor for that subtree, either per link: ~~~ini # inside the [Network] section of the corporate link DNSSECNegativeTrustAnchors=corp.example.com ~~~ or globally in a file: ~~~text # /etc/dnssec-trust-anchors.d/corp.negative corp.example.com ~~~ Then restart the resolver. Negative trust anchors tell the validator "do not require authentication below this name." They are a scalpel, not a global off switch. Strict DNS-over-TLS has a similar shape. This configuration is intentionally strict: ~~~ini # /etc/systemd/resolved.conf.d/dot.conf [Resolve] DNS=1.1.1.1#cloudflare-dns.com 9.9.9.9#dns.quad9.net FallbackDNS= DNSOverTLS=yes DNSSEC=yes ~~~ The `#hostname` part is not decoration. It sets the TLS Server Name Indication and gives the resolver a name to validate the certificate against. Without it, the certificate is checked against the server's IP address, which usually fails for public resolvers. `DNSOverTLS=yes` is strict: if a server does not support TLS, queries to it fail. `opportunistic` tries TLS and falls back to plaintext when the server does not support it, which is exactly the downgrade you were trying to avoid. Use `opportunistic` only when you value compatibility over authentication. The global `DNSOverTLS=` also applies to per-link servers unless the link overrides it. If DHCP hands you a resolver that does not speak TLS, strict mode will fail on that link. Either stop accepting DHCP DNS (`UseDNS=no`) or set `DNSOverTLS=no` on that specific link. ## The failures you will actually hit **1. The stub is silently disabled by a port conflict.** If something else already listens on `127.0.0.53:53` or `127.0.0.54:53`, `systemd-resolved` turns its stub listener off. `/etc/resolv.conf` still points at `127.0.0.53`, so every traditional program times out while `resolvectl query` may still work through the native API. Check with `sudo ss -lntup 'sport = :53'` and `journalctl -u systemd-resolved -b`. The fix is to stop the other resolver or disable the stub deliberately with `DNSStubListener=no` and point `/etc/resolv.conf` somewhere that actually answers. **2. NSS is not using the resolver.** If `/etc/nsswitch.conf` does not list `resolve` on the `hosts:` line, glibc reads `/etc/resolv.conf` and uses `nss-dns`. In stub mode that still reaches the stub, but you lose the native API's metadata and you are depending on the generated file being correct. Check the line: ~~~bash grep '^hosts' /etc/nsswitch.conf ~~~ A working line puts `resolve` before `dns`, for example: ~~~text hosts: files myhostname resolve [!UNAVAIL=return] dns ~~~ **3. You edited the symlink target.** If `/etc/resolv.conf` points at `/run/systemd/resolve/stub-resolv.conf`, your edit went into `/run` and will be overwritten. This is not a permissions problem. It is the wrong file. **4. Split DNS was configured as a search domain.** `Domains=corp.example.com` routes corporate queries to the VPN and also makes the VPN a default route under the automatic rule. Public queries can end up at the corporate resolver. Use `~corp.example.com` and, when the link should never be a fallback, `DNSDefaultRoute=no`. **5. Strict DNSSEC met an unsigned private zone.** The resolver returns validation failures for names that used to work. Add a negative trust anchor for the private subtree, or set `DNSSEC=no` on that one link. Do not turn off validation globally because one internal zone is unsigned. **6. Strict DNS-over-TLS met a DHCP server that cannot do TLS.** The link fails, often only after a network change. Either use explicit `DNS=` entries with `#hostname` and disable DHCP DNS on that link, or relax `DNSOverTLS=` for that link. **7. A container got a copy of the host's resolv.conf.** Inside a container, `127.0.0.53` is the container's loopback, not the host's resolver. Container runtimes normally generate a container-appropriate file; copying the host file into an image or bind-mounting it breaks DNS in a way that looks like a network outage. Let the container runtime manage it, or point the container at a reachable resolver. ## A debugging sequence that narrows the path Start with the question "which path is failing?" There are three paths into resolution, and they fail differently. ~~~bash # Native API: shows the chosen link and richer metadata resolvectl query --cache=no --legend=no internal.example.com # NSS: what ordinary applications see getent hosts internal.example.com # Raw DNS to the stub: what a program that ignores NSS sees dig @127.0.0.53 internal.example.com ~~~ The decision table is short: - `resolvectl` works, `getent` fails: fix NSS (`/etc/nsswitch.conf`, `nss-resolve`). - `resolvectl` works, `dig @127.0.0.53` fails: the stub listener is not running or is not reachable. Check the port conflict and the unit logs. - All three fail: the query is not reaching a working upstream server. Check routing, `resolvectl status`, and the selected link. - `resolvectl` fails with `--cache=no` but succeeds normally: you are looking at a cache or stale-data problem, not an upstream problem. A successful native query looks like this (trimmed from a Debian 13 host running systemd 257.13): ~~~text $ resolvectl query --legend=no deb.debian.org deb.debian.org: 2a04:4e42:400::644 -- link: eth0 2a04:4e42:600::644 -- link: eth0 151.101.2.132 -- link: eth0 (debian.map.fastlydns.net) ~~~ The `-- link:` annotation is the useful part. It tells you which interface's DNS configuration produced the answer. If that is the wrong link, the problem is routing, not the upstream server. Then inspect the policy: ~~~bash resolvectl status resolvectl domain ~~~ A representative status block looks like this: ~~~text Global Protocols: +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported resolv.conf mode: stub Link 2 (eth0) Current Scopes: DNS LLMNR/IPv4 LLMNR/IPv6 Protocols: +DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported Current DNS Server: 192.0.2.53 DNS Servers: 192.0.2.53 192.0.2.54 DNS Domain: corp.example.com Default Route: yes ~~~ Read it in this order: `resolv.conf mode`, `Current DNS Server`, `DNS Domain`, `Default Route`. Those four lines explain most routing surprises. For cache and server behavior: ~~~bash sudo resolvectl statistics # hit/miss and validation counters sudo resolvectl show-cache # systemd 254+ sudo resolvectl show-server-state # systemd 255+ sudo resolvectl monitor # systemd 252+, live local query stream sudo resolvectl flush-caches ~~~ `flush-caches` is not a universal fix. It clears the resolver's cache, not your browser's, not a local dnsmasq, and not an application's own cache. The resolver already flushes automatically when the network configuration changes; manual flushing is for debugging and for the rare case where an upstream changed a record early. If the counters do not move when you query, you are not talking to this resolver. Go back to the NSS and stub checks. For the resolver's own view of what went wrong: ~~~bash sudo journalctl -u systemd-resolved -b --no-pager sudo resolvectl log-level debug # reproduce the problem sudo resolvectl log-level info ~~~ Debug logging is verbose and can include query names. Turn it back down when you are done. ## A clean cutover If you are moving a host from a hand-managed `resolv.conf` to `systemd-resolved`, do it in a way you can reverse. ~~~bash # Keep the old file (or symlink) around sudo cp -a /etc/resolv.conf /root/resolv.conf.bak # Start the resolver and point the compatibility file at the stub sudo systemctl enable --now systemd-resolved sudo ln -sf /run/systemd/resolve/stub-resolv.conf /etc/resolv.conf # Verify the mode and a real lookup resolvectl status | head -5 resolvectl query --cache=no example.com getent hosts example.com ~~~ If NetworkManager owns the interfaces, tell it to use the `systemd-resolved` DNS plugin instead of writing its own file: ~~~ini # /etc/NetworkManager/conf.d/90-dns.conf [main] dns=systemd-resolved ~~~ Then reload it: ~~~bash sudo systemctl reload NetworkManager ~~~ If `resolvectl status` reports `resolv.conf mode: foreign`, the symlink did not take effect — something else is rewriting the file. Find that manager before you fight it. Two programs writing `/etc/resolv.conf` is not a configuration; it is a race. ## When not to use it `systemd-resolved` is a stub resolver and router. It is not a full recursive resolver. If you run Unbound on `127.0.0.1` because you want local recursion and your own root hints, do not point clients at both resolvers. Pick one place to own the query path. If Unbound binds `0.0.0.0:53`, it will also collide with the stub listener. If you need a DHCP server, use dnsmasq or another DHCP daemon. If you need a full-featured authoritative server, use Knot, NSD, or BIND. The mistake is treating `systemd-resolved` as a replacement for all of them because it happens to answer DNS queries. ## The short checklist - `/etc/resolv.conf` is generated when it is a symlink into `/run/systemd/resolve/`. Do not edit it. - Configuration belongs in `resolved.conf` drop-ins, `.network` files, or the system that owns the link. - `~domain` is route-only; a bare `domain` is search plus route and can make a link a default route. - Multi-label names are never suffixed with search domains. `ndots` is not implemented. - Single-label names do not go to unicast DNS without a search domain or an explicit override. - `DNSSEC=yes` fails closed; use a negative trust anchor for unsigned private zones. - `DNSOverTLS=yes` needs `#hostname` for certificate validation, and it applies to DHCP-provided servers too. - Diagnose with `resolvectl query --cache=no`, `resolvectl status`, `getent hosts`, and `dig @127.0.0.53` before changing anything. The next time DNS breaks, resist the urge to open `/etc/resolv.conf`. Run `resolvectl status` first. The answer is usually in the link that should not have been a default route, or in the search domain that was never a search domain at all. ## References - [systemd-resolved.service(8)](https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html) — stub listeners, synthetic records, routing rules, `/etc/resolv.conf` modes, glibc compatibility - [resolved.conf(5)](https://www.freedesktop.org/software/systemd/man/latest/resolved.conf.html) — `DNS=`, `Domains=`, `DNSSEC=`, `DNSOverTLS=`, cache settings - [resolvectl(1)](https://www.freedesktop.org/software/systemd/man/latest/resolvectl.html) — `query`, `status`, `statistics`, `monitor`, `show-cache`, per-link commands - [systemd.network(5)](https://www.freedesktop.org/software/systemd/man/latest/systemd.network.html) — `DNS=`, `Domains=`, `DNSDefaultRoute=`, `UseDNS=`, `DNSSECNegativeTrustAnchors=` - [dnssec-trust-anchors.d(5)](https://www.freedesktop.org/software/systemd/man/latest/dnssec-trust-anchors.d.html) — positive and negative trust anchors - [RFC 6762](https://www.rfc-editor.org/rfc/rfc6762) — Multicast DNS and the `.local` namespace - [RFC 4795](https://www.rfc-editor.org/rfc/rfc4795) — LLMNR