.au (Australia) DNSSEC Outage: 2022-03-22
Date: March 22, 2022
Overview
This page gives some details on the .au DNSSEC outage of March 22, 2022.
Timeline / DNSViz
- 2022-03-22 04:40:22 UTC — first personally observed .au DNSSEC failure
- 2022-03-22 04:47:14 UTC — Bogus DNSSEC
- 2022-03-22 04:48:50 UTC — Bogus DNSSEC
- 2022-03-22 06:18:29 UTC — Bogus DNSSEC
- 2022-03-22 06:48:41 UTC — last personally observed .au DNSSEC failure
OpenDNS: with and without DNSSEC
DNSSEC can be disabled in queries via the CD (checking disabled) bit. Let's compare DNS queries with and without DNSSEC.
$ dig +dnssec a google.com.au. @resolver1.opendns.com.
; <<>> dig 9.10.8-P1 <<>> +dnssec a google.com.au. @resolver1.opendns.com.
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 9455
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 4096
; EDE: 6 (DNSSEC Bogus)
;; QUESTION SECTION:
;google.com.au. IN A
;; Query time: 8 msec
;; SERVER: 208.67.222.222#53(208.67.222.222)
;; WHEN: Tue Mar 22 04:54:53 UTC 2022
;; MSG SIZE rcvd: 48
You have to disable DNSSEC to make DNS queries work:
$ dig +cd a google.com.au. @resolver1.opendns.com.
; <<>> dig 9.10.8-P1 <<>> +cd a google.com.au. @resolver1.opendns.com.
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 41861
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;google.com.au. IN A
;; ANSWER SECTION:
google.com.au. 300 IN A 142.251.33.99
;; Query time: 7 msec
;; SERVER: 208.67.222.222#53(208.67.222.222)
;; WHEN: Tue Mar 22 04:54:53 UTC 2022
;; MSG SIZE rcvd: 58
AusNOG mailing list
This DNSSEC outage was discussed on the AusNOG mailing list, in the thread [AusNOG] .AU DNSSEC Broken. Page archived here and here.
dns-operations mailing list
It was also discussed on the dns-operations mailing list hosted by DNS-OARC, in the thread [dns-operations] .au DNSSEC issues. Page archived here and here.
drill trace
Since DNSSEC contains so much garbage, I put the complete drill trace into its own file, with the relevant portion below (emphasis added):
[T] au. 86400 IN DS 34882 8 2 8de6a7a1aa8ac4ad4dc732074f43c08be6cb88cfbc245207e8fd70b3014e32b4
;; Domain: au.
[B] au. 43200 IN DNSKEY 256 3 8 ;{id = 17788 (zsk), size = 2048b}
au. 43200 IN DNSKEY 257 3 8 ;{id = 34882 (ksk), size = 4096b}
[B] Error verifying denial of existence for au. type A: No keys with the keytag and algorithm from the RRSIG found
;;[S] self sig OK; [B] bogus; [T] trusted
Logfile examples
- [1647924022] unbound[95966:0] info: validation failure <au. NS IN>: no signatures from 65.22.197.1 for key au. while building chain of trust
- [1647924337] unbound[95966:0] info: validation failure <ebay.com.au. A IN>: no signatures from 65.22.199.1 for key au. while building chain of trust
- [1647924443] unbound[95966:0] info: validation failure <google.com.au. A IN>: no signatures from 65.22.197.1 for key au. while building chain of trust
- [1647931721] unbound[95966:0] info: validation failure <au. NS IN>: no signatures from 65.22.196.1 for key au. while building chain of trust