success resolving xxx after disabling EDNS

Mon May 9 11:19:13 UTC 2022

Hello,

Now we are investigating another case:

On our internal DNS server we see :

08-May-2022 20:48:14.248 edns-disabled: info: success resolving 'grid31.physics.uoi.gr/A' (in '.'?) after reducing the advertised EDNS UDP packet size to 512 octets
08-May-2022 20:48:14.249 edns-disabled: info: success resolving 'grid31.physics.uoi.gr/AAAA' (in '.'?) after reducing the advertised EDNS UDP packet size to 512 octets

and on our external cache DNS server we see:

03-May-2022 20:47:40.934 edns-disabled: info: success resolving 'grid31.physics.uoi.gr/AAAA' (in 'physics.uoi.gr'?) after disabling EDNS
03-May-2022 20:47:40.937 edns-disabled: info: success resolving 'grid31.physics.uoi.gr/A' (in 'physics.uoi.gr'?) after disabling EDNS

Looking into our detailed logs, it seems to us that the root cause of the problem for this domain is that the DNS server used by physics.uoi.gr don't reply over ipv6, and that our DNS servers finally get an answer when they query the ipv4 NS.

This seems also to be confirmed by https://dnsviz.net/d/physics.uoi.gr/dnssec/ (really great tool !).

If the problem is simply ipv6, is it correct to say that the BIND messages above are misleading ?
Or is there really a EDNS-related issue ?

Thanks again,
Veronique

> On 05/05/2022 03:01 Mark Andrews <marka at isc.org> wrote:
> 
>  
> > On 5 May 2022, at 00:17, Veronique Lefebure <veronique.lefebure at cern.ch> wrote:
> > 
> > Thanks Greg and Ondrej,
> > 
> > Many thanks for the pointer to DNS Cookies in BIND 9 (isc.org)
> > 
> > I have used https://ednscomp.isc.org/ednscomp/1ba42afa27 to check if  they are compliant, but the answer is ambiguous:
> > 
> > EDNS Compliance Tester
> > Checking: 'sour.woinsta.com' as at 2022-05-04T13:45:39Z
> > sour.woinsta.com.: NS lookup failed
> > Codes
> > 	• ok - test passed.
> 
> Use the zone name ‘woinsta.com’.  The site asks for the zone name not an arbitrary zone name.
> That said we make need to tweak the nameserver settings the tester is using to get the initial
> NS lookup to succeed with broken servers like this.
> 
> > Anyway, from what you have seen you are suspecting that the problem is on the woinsta.com side and not on our side ?
> > 
> > The following indeed indicates a problem related to cookies:
> > 
> > dig @ns1.thednscloud.com. +nocookie sour.woinsta.com A +short 
> > 23.82.12.29 
> > 
> > while 
> > 
> > dig @ns1.thednscloud.com. +cookie sour.woinsta.com A +short
> > ; <<>> DiG 9.11.36 <<>> @ns1.thednscloud.com. +cookie sour.woinsta.com A +short 
> > ; (2 servers found) 
> > ;; global options: +cmd 
> > ;; connection timed out; no servers could be reached 
> 
> The servers will echo back a COOKIE option that has both the client and a server cookie present.
> 
> % dig sour.woinsta.com ns @23.82.12.27 +norec +cookie=44fd7abd853763e60100000062731621e76cd1b746056de3 +qr
> 
> ; <<>> DiG 9.17.22 <<>> sour.woinsta.com ns @23.82.12.27 +norec +cookie=44fd7abd853763e60100000062731621e76cd1b746056de3 +qr
> ;; global options: +cmd
> ;; Sending:
> ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 2072
> ;; flags: ad; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
> 
> ;; OPT PSEUDOSECTION:
> ; EDNS: version: 0, flags:; udp: 1232
> ; COOKIE: 44fd7abd853763e60100000062731621e76cd1b746056de3
> ;; QUESTION SECTION:
> ;sour.woinsta.com.		IN	NS
> 
> ;; QUERY SIZE: 73
> 
> ;; Got answer:
> ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 2072
> ;; flags: qr aa; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1
> 
> ;; OPT PSEUDOSECTION:
> ; EDNS: version: 0, flags:; udp: 1232
> ; COOKIE: 44fd7abd853763e60100000062731621e76cd1b746056de3 (good)
> ;; QUESTION SECTION:
> ;sour.woinsta.com.		IN	NS
> 
> ;; AUTHORITY SECTION:
> woinsta.com.		300	IN	SOA	ns1.emu-dns.com. admin.woinsta.com. 2022020800 86400 10800 604800 300
> 
> ;; Query time: 236 msec
> ;; SERVER: 23.82.12.27#53(23.82.12.27) (UDP)
> ;; WHEN: Thu May 05 10:16:25 AEST 2022
> ;; MSG SIZE  rcvd: 127
> 
> % 
> 
> I suspect a stupid firewall in front of it that thinks it knows what is a valid COOKIE option is
> but really it doesn’t.  The firewall then did the stupid thing of dropping the request.  The correct
> protocol response to an invalid EDNS option is FORMERR with an OPT record present if you understand
> EDNS.  If you don’t understand EDNS the correct response to EDNS is FORMERR *without* an OPT record
> being present or just ignore the OPT record.
> 
> Too many servers don’t know how to construct a valid FORMERR response.  You DO NOT just copy the
> request, set rcode to FORMERR and QR to 1 and send the resulting message.  Why would you send what
> you believe is garbage to the client?  The DNS message has a well defined header which identifies
> the request / response.  Construct a valid response header based on the header in the request and
> set the rcode to FORMERR.  Zero out the response header, copy the ID and OPCODE fields.  Copy any
> header bits that you know are supposed to be copied.  Set the RCODE field to FORMERR.  If it was
> a QUERY, and you understood the QUESTION section, you can copy that as well updating the QUESTION
> count.
> 
> Basically there is a broken firewall and a broken DNS server here.  Both need to be fixed.
> 
> Mark
> 
> > I will try send-cookie no for that server to confirm it is the source of the issue.
> > 
> > Cheers,
> > Veronique
> > 
> >> On 04/05/2022 14:34 Greg Choules <gregchoules+bindusers at googlemail.com> wrote:
> >> 
> >> 
> >> Hi Veronique. 
> >> Every DNS server should support EDNS by now. It has been around for a very long time. Even if it doesn't support EDNS it should ignore it.
> >> 
> >> I made some test queries and packet captures to 23.82.12.28. Whatever this box is, please talk to the manufacturer about EDNS support. 
> >> Or.. it may be that some network infrastructure - firewalls are usually the first place to look - is blocking this traffic.     
> >> 
> >> Whatever is happening at the authoritative end, it needs to be fixed. All modern recursive servers will use EDNS.
> >> 
> >> Cheers, Greg
> > -- 
> > Visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list
> > 
> > ISC funds the development of this software with paid support subscriptions. Contact us at https://www.isc.org/contact/ for more information.
> > 
> > 
> > bind-users mailing list
> > bind-users at lists.isc.org
> > https://lists.isc.org/mailman/listinfo/bind-users
> 
> -- 
> Mark Andrews, ISC
> 1 Seymour St., Dundas Valley, NSW 2117, Australia
> PHONE: +61 2 9871 4742              INTERNET: marka at isc.org