Post #1253116

@markd @dangoodin It is likely happening much more often than people think. In 2023, I had an elusive DNS error on my hands and tracked it down to 2 out of 4 "physical" servers behind 1 out of 13 root server names being out-of-date in a _single_ geography (Japan). Any query hitting the two unfortunate hosts would get out-of-date response, which included an expired DNSSEC signature. With an invalid DNSSEC signature, well-behaved recursive resolvers would throw on the ground any response coming from that server. In other words, DNS resolution was completely broken. Guess what? Almost nobody saw that problem. DNS is so redundant. The query would generally be retried, and then it would hit a healthy root server. Multiple times on the DNS layer, and then potentially more times on the application layer. The rest of the world was oblivious to it, and even in Japan it would have been perceived as connectivity noise. After discovering the issue, I contacted the operators of that root and they resolved it in a few hours. 0 apocalypses took place.