nohats.ca DNSSEC Outage: 2020-09-11 to 2020-09-12

Date: September 12, 2020

Overview

This page gives some details on the nohats.ca DNSSEC outage from September 11 to September 12, 2020. This isn't the first DNSSEC outage for nohats.ca, and not even the first one in 2020.

DNSViz / Timeline

Here's a screenshot of DNSViz output during the nohats.ca DNSSEC outage:

September 12, 2020 nohats.ca DNSSEC outage at DNSViz

This data was also saved by archive.is.

DNSSEC Debugger

Verisign's DNSSEC Debugger doesn't archive results, so here's a screenshot of my web browser's output from September 12, 2020:

September 12, 2020 nohats.ca DNSSEC outage

Zonemaster

Please note: Zonemaster requires javascript to display text.

Google DNS: with and without DNSSEC

DNSSEC can be disabled in queries via the CD (checking disabled) bit. Let's compare DNS queries with and without DNSSEC.

With DNSSEC, DNS queries fail:

$ dig +dnssec a nohats.ca. @8.8.8.8

; <<>> dig 9.10.8-P1 <<>> +dnssec a nohats.ca. @8.8.8.8
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 11058
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 512
;; QUESTION SECTION:
;nohats.ca. IN A

;; Query time: 343 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Fri Sep 11 23:01:22 UTC 2020
;; MSG SIZE rcvd: 38

You have to disable DNSSEC to make DNS queries work:

$ dig +cd a nohats.ca. @8.8.8.8

; <<>> dig 9.10.8-P1 <<>> +cd a nohats.ca. @8.8.8.8
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 56449
;; flags: qr rd ra cd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;nohats.ca. IN A

;; ANSWER SECTION:
nohats.ca. 21599 IN A 193.110.157.102

;; Query time: 280 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Fri Sep 11 23:01:23 UTC 2020
;; MSG SIZE rcvd: 54

drill trace

Since DNSSEC contains so much garbage, I put the complete drill trace into its own file, with the relevant portion below (emphasis added):

nohats.ca. 86400 IN DS 35434 8 2 8a3b88f46994974633d77fd25be8b11ddd47b78640fb8dfbb5c1fc65995d8048
;; Domain: nohats.ca.
[B] nohats.ca. 3600 IN DNSKEY 256 3 8 ;{id = 37759 (zsk), size = 2048b}
nohats.ca. 3600 IN DNSKEY 257 3 8 ;{id = 35434 (ksk), size = 2048b}
[B] nohats.ca. 86400 IN A 193.110.157.102
;; Error: No keys with the keytag and algorithm from the RRSIG found
;;[S] self sig OK; [B] bogus; [T] trusted

dns.google.com

dns.google.com is related to but separate from Google Public DNS. During this DNSSEC outage, dns.google.com showed the following for nohats.ca:

September 12, 2020 dns.google.com output for nohats.ca

Logfile examples

These logs come from different servers in different geographical regions: