.mx TLD DNSSEC Outage: 2023-04-04

Date: April 4, 2023

Overview

This page gives some details on the .mx (Mexico) TLD DNSSEC outage on April 4, 2023.

Timeline / DNSViz

DNSSEC Debugger

Verisign's DNSSEC Debugger doesn't archive results, so here's a screenshot of my web browser's output from April 4, 2023:

April 4, 2023 .mx TLD DNSSEC outage as shown by DNSSEC Debugger

OpenDNS: with and without DNSSEC

DNSSEC can be disabled in queries via the CD (checking disabled) bit. Let's compare DNS queries with and without DNSSEC.

$ dig +dnssec ns mx. @resolver1.opendns.com.

; <<>> dig 9.10.8-P1 <<>> +dnssec ns mx. @resolver1.opendns.com.
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 62588
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 4096
; EDE: 6 (DNSSEC Bogus)
;; QUESTION SECTION:
;mx. IN NS

;; Query time: 29 msec
;; SERVER: 208.67.222.222#53(208.67.222.222)
;; WHEN: Tue Apr 04 02:21:15 UTC 2023
;; MSG SIZE rcvd: 37

You have to disable DNSSEC to make DNS queries work:

$ dig +cd ns mx. @resolver1.opendns.com.

; <<>> dig 9.10.8-P1 <<>> +cd ns mx. @resolver1.opendns.com.
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 53764
;; flags: qr rd ra; QUERY: 1, ANSWER: 6, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;mx. IN NS

;; ANSWER SECTION:
mx. 86400 IN NS c.mx-ns.mx.
mx. 86400 IN NS e.mx-ns.mx.
mx. 86400 IN NS i.mx-ns.mx.
mx. 86400 IN NS m.mx-ns.mx.
mx. 86400 IN NS o.mx-ns.mx.
mx. 86400 IN NS x.mx-ns.mx.

;; Query time: 29 msec
;; SERVER: 208.67.222.222#53(208.67.222.222)
;; WHEN: Tue Apr 04 02:21:15 UTC 2023
;; MSG SIZE rcvd: 133

drill trace

Since DNSSEC contains so much garbage, I put the complete drill trace into its own file, with the relevant portion below (emphasis added):

[T] mx. 86400 IN DS 61885 8 2 bd34dc7ad75d94125810337700c6d0f9ee5dbfd4d52d13c2d88ac5e70deb8f79
;; Domain: mx.
;; Signature ok but no chain to a trusted key or ds record
[S] mx. 86400 IN DNSKEY 256 3 8 ;{id = 10935 (zsk), size = 1024b}
mx. 86400 IN DNSKEY 257 3 8 ;{id = 61885 (ksk), size = 2048b}
[S] Existence denied: mx. A
;;[S] self sig OK; [B] bogus; [T] trusted; [U] unsigned

Logfile examples

These Unbound log entries come from different Unbound instances, running different operating systems in different geographical regions.