libreswan.org DNSSEC Outage: 2020-09-10 to 2020-09-12
Date: September 12, 2020
Overview
This page gives some details on the libreswan.org DNSSEC outage from September 10, 2020 to September 12, 2020. It was probably related a DNSSEC failure within nohats.ca, which provides DNS service for libreswan.org. This time, affected domains were libreswan.net, libreswan.com, libreswan.org, and nohats.ca.
DNSViz / Timeline
- 2020-09-10 18:22:37 UTC — libreswan.org/AAAA Expired RRSIG
- 2020-09-11 13:29:43 UTC — first personally observed libreswan.org DNSSEC failure
- 2020-09-11 13:30:07 UTC — A, AAAA, NS, TXT Expired RRSIGs
- 2020-09-12 01:23:50 UTC — Expired RRSIGs (archive.is copy)
- 2020-09-12 16:51:41 UTC — last personally observed libreswan.org DNSSEC failure
Here's a screenshot since DNSViz has lost its database multiple times, including an outage that lasted over a year:

DNSSEC Debugger
Verisign's DNSSEC Debugger doesn't archive results, so here are screenshots of my web browser's output from September 12, 2020:

Zonemaster
- zonemaster.iis.se shows this DNSSEC outage for libreswan.org. There's also a copy from archive.is.
- zonemaster.labs.nic.cz also shows a DNSSEC outage for libreswan.org. There's also a copy thanks to archive.is.
- zonemaster.net as well recorded a DNSSEC outage for libreswan.org. And thanks again to archive.is for a copy.
Google DNS: with and without DNSSEC
DNSSEC can be disabled in queries via the CD (checking disabled) bit. Let's compare DNS queries with and without DNSSEC.
With DNSSEC, DNS queries fail:
$ dig +dnssec a libreswan.org. @8.8.8.8
; <<>> dig 9.10.8-P1 <<>> +dnssec a libreswan.org. @8.8.8.8
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 63149
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 512
;; QUESTION SECTION:
;libreswan.org. IN A
;; Query time: 322 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Fri Sep 11 13:29:48 UTC 2020
;; MSG SIZE rcvd: 42
You have to disable DNSSEC to make DNS queries work:
$ dig +cd a libreswan.org. @8.8.8.8
; <<>> dig 9.10.8-P1 <<>> +cd a libreswan.org. @8.8.8.8
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 19906
;; flags: qr rd ra cd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;libreswan.org. IN A
;; ANSWER SECTION:
libreswan.org. 7199 IN A 188.127.201.229
;; Query time: 145 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Fri Sep 11 13:29:48 UTC 2020
;; MSG SIZE rcvd: 58
dns.google.com
dns.google.com is related to but separate from Google Public DNS. During this DNSSEC outage, dns.google.com showed the following for libreswan.org:

This data is also at archive.is.
drill trace
Since DNSSEC contains so much garbage, I put the complete drill trace into its own file, with the relevant portion below (emphasis added):
;; Domain: libreswan.org.
[T] libreswan.org. 3600 IN DNSKEY 257 3 8 ;{id = 2644 (ksk), size = 2048b}
libreswan.org. 3600 IN DNSKEY 256 3 8 ;{id = 28179 (zsk), size = 2048b}
[B] libreswan.org. 7200 IN A 188.127.201.229
;; Error: DNSSEC signature has expired
;;[S] self sig OK; [B] bogus; [T] trusted
Logfile examples
These logs come from different servers in different geographical regions:
- [1599745184] unbound[18103:0] info: validation failure <www.libreswan.org. NS IN>: signature expired from 193.110.157.102 for <sfg24catf8qf47bkdjgsj9sual64dh64.libreswan.org. NSEC3 IN>
- [1599761969] unbound[33360:0] info: validation failure <www.libreswan.org. A IN>: signature expired from 149.56.110.74
- [1599762211] unbound[18103:0] info: validation failure <www.libreswan.org. A IN>: signature expired from 149.56.110.74
- [1599830983] unbound[33360:0] info: validation failure <libreswan.org. A IN>: signature expired from 149.56.110.74
- [1599837147] unbound[18103:0] info: validation failure <libreswan.org. A IN>: signature expired from 188.127.201.225
- [1599930319] unbound[18103:0] info: validation failure <libreswan.org. A IN>: signature expired from 193.110.157.102 for key libreswan.org. while building chain of trust
- [1599930038] unbound[33360:0] info: validation failure <www.libreswan.org. A IN>: no keys have a DS with algorithm RSASHA256 from 188.127.201.225 for key libreswan.org. while building chain of trust