www.getdnsapi.net DNSSEC Outage: 2020-03-22
Date: March 22, 2020
Overview
This page gives some details on the www.getdnsapi.net/A DNSSEC outage on March 22, 2020. getdns is a DNSSEC library and strong supporter of DNSSEC.
Timeline / DNSViz
- 2020-03-22 11:52:40 UTC — RRSIGs expire
- 2020-03-22 15:30:45 UTC — expired RRSIGs (archive.is copy)
- 2020-03-22 16:46:10 UTC — last personally observed www.getdnsapi.net/A DNSSEC failure
DNSSEC Debugger
Unlike DNSViz, Verisign's DNSSEC Debugger doesn't archive results, so here's a screenshot of my web browser's output from March 22, 2020:

There's also a copy saved by archive.is.
Google Public DNS: with and without DNSSEC
DNSSEC can be disabled in queries via the CD (checking disabled) bit. Let's compare DNS queries with and without DNSSEC.
With DNSSEC, DNS queries fail:
$ dig +dnssec a www.getdnsapi.net. @8.8.8.8
; <<>> DiG 9.10.3-P4-Debian <<>> +dnssec a www.getdnsapi.net. @8.8.8.8
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 62991
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 512
;; QUESTION SECTION:
;www.getdnsapi.net. IN A
;; Query time: 399 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Sun Mar 22 11:59:51 UTC 2020
;; MSG SIZE rcvd: 46
You have to disable DNSSEC to make DNS queries work:
$ dig +cd a www.getdnsapi.net. @8.8.8.8
; <<>> DiG 9.10.3-P4-Debian <<>> +cd a www.getdnsapi.net. @8.8.8.8
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 15728
;; flags: qr rd ra cd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;www.getdnsapi.net. IN A
;; ANSWER SECTION:
www.getdnsapi.net. 449 IN A 185.49.141.37
;; Query time: 270 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Sun Mar 22 11:59:52 UTC 2020
;; MSG SIZE rcvd: 62
drill trace
Since DNSSEC contains so much garbage, I put the complete drill trace into its own file, with the relevant portion below (emphasis added):
[T] Existence denied: www.getdnsapi.net. DS
;; No ds record for delegation
;; Domain: www.getdnsapi.net.
;; No DNSKEY record found for www.getdnsapi.net.
[B] www.getdnsapi.net. 450 IN A 185.49.141.37
;; Error: DNSSEC signature has expired
;;[S] self sig OK; [B] bogus; [T] trusted
dns.google.com
dns.google.com is related to but separate from Google Public DNS. During this DNSSEC outage, dns.google.com showed the following for www.getdnsapi.net/A:

There's also a copy of this output thanks to archive.is.
Logfile examples
These Unbound log entries are from different servers in different geographical regions:
- [1584878425] unbound[28019:0] info: validation failure <www.getdnsapi.net. A IN>: signature expired from 192.16.197.229
- [1584878951] unbound[28019:0] info: validation failure <www.getdnsapi.net. A IN>: signature expired from 185.49.141.53
- [1584981855] unbound[260:0] info: validation failure <www.getdnsapi.net. A IN>: signature expired from 192.16.197.229
- [1584981970] unbound[28019:0] info: validation failure <www.getdnsapi.net. A IN>: signature expired from 192.16.197.229