.zm (Zambia) DNSSEC Outage: 2019-04-26 to 2019-04-27

Date: April 27, 2019

Overview

This page gives some details on the .zm TLD DNSSEC outage from April 26 to April 27, 2019. Zambia has had multiple DNSSEC outages.

DNSViz / Timeline

(At the time of this writing, DNSViz has been having a multi-week outage. DNSSEC makes its users think downtime doesn't matter.)

DNSSEC Debugger

Unlike DNSViz (well, most of the time), Verisign's DNSSEC Debugger doesn't archive results, so here's a screenshot of my web browser's output from April 26, 2019:

April 26, 2019, .zm (Zambia) DNSSEC outage

Google Public DNS: with and without DNSSEC

DNSSEC can be disabled in queries via the CD (checking disabled) bit. Let's compare DNS queries with and without DNS SEC. With DNSSEC, DNS queries result in SERVFAIL:

$ dig +dnssec ns zm. @8.8.8.8

; <<>> DiG 9.4.2-P2 <<>> +dnssec ns zm. @8.8.8.8
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 17064
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 512
;; QUESTION SECTION:
;zm. IN NS

;; Query time: 39 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Fri Apr 26 14:06:17 2019
;; MSG SIZE rcvd: 31

You have to disable DNSSEC to make DNS queries work:

$ dig +cd ns zm. @8.8.8.8

; <<>> DiG 9.4.2-P2 <<>> +cd ns zm. @8.8.8.8
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 36604
;; flags: qr rd ra cd; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;zm. IN NS

;; ANSWER SECTION:
zm. 21599 IN NS ns2.zamnet.zm.
zm. 21599 IN NS pch.nic.zm.
zm. 21599 IN NS ns-zm.afrinic.net.
zm. 21599 IN NS ns1.zamnet.zm.

;; Query time: 31 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Fri Apr 26 14:06:17 2019
;; MSG SIZE rcvd: 116

Zonemaster

zonemaster.iis.se archived "The apex DNSKEY RRset was not correctly signed."

dns.google.com

dns.google.com reported this DNSSEC failure. A screenshot is below. Please also see this archive.org view of the same data (requires javascript).

April 26, 2019 zm (Zambia) DNSSEC outage, shown by dns.google.com

drill trace

Since DNSSEC contains so much garbage, I put the complete drill trace into its own fil e with the relevant portion below (emphasis added):

;; Domain: zm.
[B] zm. 3600 IN DNSKEY 256 3 8 ;{id = 34653 (zsk), size = 1024b}
zm. 3600 IN DNSKEY 257 3 8 ;{id = 32227 (ksk), size = 2048b}
[B] Error verifying denial of existence for zm. type A: No keys with the keytag and algorithm from the RRSIG found
;;[S] self sig OK; [B] bogus; [T] trusted

Logfile examples

The below log examples come from 3 different Unbound instances all in different geographical regions.