tm TLD DNSSEC Outage: 2021-12-27

Date: December 27, 2021

Overview

This page gives some details on the tm (Turkmenistan) TLD DNSSEC outage on December 27, 2021.

Verisign's DNSSEC Debugger

Verisign doesn't archive test results, unlike DNSViz. So here's a screenshot I took on December 27, 2021:

.tm (Turkmenistan) dnssec outage

And here's a snapshot from archive.is.

Timeline / DNSViz

2021-12-27 15:55:57 UTC — No RRSIGs
2021-12-27 15:59:06 UTC — No RRSIGs
2021-12-27 20:00:18 UTC — No RRSIGs
2021-12-27 20:47:03 UTC — No RRSIGs
2021-12-27 22:27:13 UTC — DNSSEC debris
2021-12-27 21:35:48 UTC — last personally observed DNSSEC failure

Here's a screenshot example:

Google Public DNS: with and without DNSSEC

DNSSEC can be disabled in queries via the CD (checking disabled) bit. Let's compare DNS queries with and without DNSSEC. With DNSSEC, DNS queries result in SERVFAIL:

$ dig +dnssec ns tm. @8.8.8.8.

; <<>> dig 9.10.8-P1 <<>> +dnssec ns tm. @8.8.8.8.
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 27790
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 512
;; QUESTION SECTION:
;tm. IN NS

;; Query time: 134 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Mon Dec 27 15:49:45 UTC 2021
;; MSG SIZE rcvd: 31

You have to disable DNSSEC to make DNS work:

$ dig +cd ns tm. @8.8.8.8.

; <<>> dig 9.10.8-P1 <<>> +cd ns tm. @8.8.8.8.
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 32887
;; flags: qr rd ra cd; QUERY: 1, ANSWER: 7, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;tm. IN NS

;; ANSWER SECTION:
tm. 21600 IN NS ns-y1.tm.
tm. 21600 IN NS ns-l1.tm.
tm. 21600 IN NS ns-d1.tm.
tm. 21600 IN NS ns-a2.tm.
tm. 21600 IN NS ns-a1.tm.
tm. 21600 IN NS ns-a4.tm.
tm. 21600 IN NS ns-a3.tm.

;; Query time: 76 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Mon Dec 27 15:49:45 UTC 2021
;; MSG SIZE rcvd: 171

Zonemaster

Zonemaster noted this TLD DNSSEC outage. Check out:

zonemaster.net, which showed no DNSKEY records. (archive.is copy)
zonemaster.iis.se, which showed DNSKEY failures. (archive.is copy)

drill trace

Since DNSSEC contains so much garbage, I put the complete drill trace into its own file with the relevant portion below (emphasis added):

[T] tm. 86400 IN DS 36787 8 2 af4a3a54aa5f73c114f9c6e4d9c3ef0040e387806a9f92ae85276ff38f33bcb5
;; Domain: tm.
;; No DNSKEY record found for tm.
[U] No data found for: tm. type A
;;[S] self sig OK; [B] bogus; [T] trusted

Logfile examples

[1640620182] unbound[64887:0] info: validation failure <tm. NS IN>: No DNSKEY record from 194.0.1.22 for key tm. while building chain of trust
[1640620315] unbound[64887:0] info: validation failure <tm. NS IN>: No DNSKEY record from 74.116.178.22 for key tm. while building chain of trust
[1640640584] unbound[64887:0] info: validation failure <tm. NS IN>: No DNSKEY record from 64.251.31.180 for key tm. while building chain of trust
[1640640854] unbound[64887:0] info: validation failure <tm. NS IN>: No DNSKEY record from 194.0.1.22 for key tm. while building chain of trust
[1640640948] unbound[64887:0] info: validation failure <tm. NS IN>: No DNSKEY record from 74.116.178.22 for key tm. while building chain of trust