dnssek.info DNSSEC Outage: 2021-03-30

Date: March 30, 2021

Overview

This page gives some details on the dnssek.info DNSSEC outage on March 30, 2021. dnssek.info provides what it calls the "DNSSEC EARLY WARNING SYSTEM (DEWS)" for expiring and expired TLD RRSIGs...

Timeline / DNSViz

2021-03-29 21:30:16 UTC — RRSIGs expire
2021-03-30 02:41:41 UTC — first personally observed dnssek.info DNSSEC failure
2021-04-02 17:36:36 UTC — last personally observed dnssek.info DNSSEC failure

DNSSEC Debugger

Unlike DNSViz, Verisign's DNSSEC Debugger doesn't archive results, so here's a screenshot of my web browser's output from March 30, 2021:

March 30, 2021 dnssek.info DNSSEC outage

Google Public DNS

DNSSEC can be disabled in queries via the CD (checking disabled) bit. Let's compare DNS queries with and without DNSSEC.

With Google Public DNS, because of DNSSEC, queries fail:

$ dig +dnssec a www.dnssek.info. @8.8.8.8.

; <<>> DiG 9.9.5-9+deb8u19-Debian <<>> +dnssec a www.dnssek.info. @8.8.8.8.
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 13873
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 512
;; QUESTION SECTION:
;www.dnssek.info. IN A

;; Query time: 106 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Mon Mar 29 22:50:00 EDT 2021
;; MSG SIZE rcvd: 44

You have to disable DNSSEC to make DNS queries work:

$ dig +cd a www.dnssek.info. @8.8.8.8.

; <<>> DiG 9.9.5-9+deb8u19-Debian <<>> +cd a www.dnssek.info. @8.8.8.8.
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 45593
;; flags: qr rd ra cd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;www.dnssek.info. IN A

;; ANSWER SECTION:
www.dnssek.info. 3599 IN A 192.101.186.107

;; Query time: 123 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Mon Mar 29 22:50:00 EDT 2021
;; MSG SIZE rcvd: 60

Logfile examples

[1617072101] unbound[389:0] info: validation failure <dnssek.info. A IN>: signature expired from 172.105.242.234 for key dnssek.info. while building chain of trust
[1617072426] unbound[389:0] info: validation failure <www.dnssek.info. A IN>: signature expired from 45.56.97.49 for key dnssek.info. while building chain of trust
[1617371836] unbound[389:0] info: validation failure <www.dnssek.info. A IN>: signature expired from 172.105.242.234 for key dnssek.info. while building chain of trust
[1617372037] unbound[389:0] info: validation failure <dnssek.info. A IN>: signature expired from 45.56.97.49 for key dnssek.info. while building chain of trust
[1617384789] unbound[389:0] info: validation failure <www.dnssek.info. A IN>: signature expired from 172.105.242.234 for key dnssek.info. while building chain of trust
[1617384996] unbound[389:0] info: validation failure <dnssek.info. A IN>: signature expired from 172.105.242.234 for key dnssek.info. while building chain of trust