head scratcher: nsupdate, Bind views, and TLSA record updates

Wed Nov 1 00:16:23 UTC 2017

----- Original Message -----
> From: "Warren Kumari" <warren at kumari.net>
> To: "Kevin" <bind-users-ml at thesandiegos.com>
> Cc: "bind-users" <bind-users at lists.isc.org>
> Sent: Tuesday, October 31, 2017 12:47:06 PM
> Subject: Re: head scratcher: nsupdate, Bind views, and TLSA record updates

> So, can you confirm that you are not getting SERVFAIL?
> 
> You really haven't provided enough information (like the actual
> domains!) for people to be able to help you.
> W

Google's DNS servers are returning SERVFAIL, but think this is incorrect. Querying from other off-network locations shows NOERROR.

To de-obfuscate replace example.com with thesandiegos.com.

To further illustrate the issue, when I run the following nsupdate:

server 1.2.3.4
zone thesandiegos.com
key updatekey xyz123...
add 123.testtlsa.mail.thesandiegos.com. 3600 IN TLSA 3 1 1 abc123...
add _25._tcp.mail.thesandiegos.com. 3600 IN TLSA 3 1 1 abc123...
local 127.0.0.1
show
send

The TLSA record 123.testtlsa.mail.thesandiegos.com resolves fine from off-network locations.
$ dig TLSA 123.testtlsa.mail.thesandiegos.com @75.149.33.153 +dnssec +short
3 1 1 E273FDF3D76928F59A11BBD2FB147114A4EE65D3300EAC3945E6B8A8 44079D5F
TLSA 5 5 3600 20171201000743 20171031230743 20103 thesandiegos.com. Leww2La3lbRwChFiTHy6aps6s2tPv5/5490j8owKo1/edq/dGEp4dLSM j6oeEgyKpemm3CGdNIpTU3iAEHbusgYAe2vB/lJUkH6ZRfP8gvu517Yi HRD0tlnpGC2lZav0xf7YL+S+G5w1HCyN6RoZHFswpMP+FVjPZKyIGsUU ooU=

The TLSA record _25._tcp.mail.thesandiegos.com does not.

$ dig TLSA _25._tcp.mail.thesandiegos.com @75.149.33.153 +dnssec +short
<crickets>

I'm really at a loss as to what's going on inside of Bind.

-Kevin

> 
> On Tue, Oct 31, 2017 at 3:39 PM, Kevin via bind-users
> <bind-users at lists.isc.org> wrote:
>> 
>> 
>> ----- Original Message -----
>>> From: "Kevin" <bind-users-ml at thesandiegos.com>
>>> To: "Kevin" <bind-users-ml at thesandiegos.com>
>>> Cc: "Warren Kumari" <warren at kumari.net>, "bind-users" <bind-users at lists.isc.org>
>>> Sent: Tuesday, October 31, 2017 12:33:56 PM
>>> Subject: Re: head scratcher: nsupdate, Bind views, and TLSA record updates
>> 
>>> ----- Original Message -----
>>> > From: "Kevin" <bind-users-ml at thesandiegos.com>
>>> > To: "Warren Kumari" <warren at kumari.net>
>>>> Cc: "Kevin" <bind-users-ml at thesandiegos.com>, "bind-users"
>>> > <bind-users at lists.isc.org>
>>> > Sent: Tuesday, October 31, 2017 12:18:41 PM
>>> > Subject: Re: head scratcher: nsupdate, Bind views, and TLSA record updates
>> 
>>> > From: "Warren Kumari" <warren at kumari.net>
>>> > To: "Kevin" <bind-users-ml at thesandiegos.com>
>>> > Cc: "bind-users" <bind-users at lists.isc.org>
>>> > Sent: Tuesday, October 31, 2017 11:28:58 AM
>>> > Subject: Re: head scratcher: nsupdate, Bind views, and TLSA record updates
>> 
>>> > On Tue, Oct 31, 2017 at 1:50 PM, Kevin via bind-users
>>> > <bind-users at lists.isc.org> wrote:
>>> >> I'm running into an odd issue with Bind 9.9.4 whereby I'm trying to run a
>>> >> scripted nsupdate to rotate TLSA records. I'm running nsupdate via a Bash
>>> >> script that executes the following nsupdate batch commands which are
>>> >> directed to a Bind "view" that is accessible from the wider internet:
>> 
>>> >> server 1.2.3.4
>>> >> zone example.com
>>> >> key updatekey xyz123...
>>> >> update delete _25._tcp.mail.example.com. TLSA
>>> >> local 127.0.0.1
>>> >> show
>>> >> send
>> 
>>> >> And then:
>>> >> server 1.2.3.4
>>> >> zone example.com
>>> >> key updatekey xyz123...
>>> >> update add _25._tcp.mail.example.com. 3600 IN TLSA 3 1 1 abc123...
>>> >> local 127.0.0.1
>>> >> show
>>> >> send
>> 
>>> >> I get the following output, all looks good:
>> 
>>> >> + Updating DNS 1.2.3.4: for ... ok.
>>> >> + Updating DNS 1.2.3.4: for ... ok.
>> 
>>> >> I see the following in /var/log/messages, all looks good (updating the view
>>> >> named "remote", responsible for answering queries from off-network sources):
>>> >> Oct 31 10:28:19 test named[106]: client 127.0.0.1#33710/key updatekey: view
>>> >> remote: signer "updatekey" approved
>>> >> Oct 31 10:28:19 test named[106]: client 127.0.0.1#33710/key updatekey: view
>>> >> remote: updating zone 'example.com/IN': deleting rrset at
>>> >> '_25._tcp.mail.example.com' TLSA
>>> >> Oct 31 10:28:19 test named[106]: zone example.com/IN/remote: sending
>>> >> notifies (serial 165)
>>> >> Oct 31 10:28:19 test named[106]: client 1.2.3.4#38629: view internal:
>>> >> received notify for zone 'example.com'
>>> >> Oct 31 10:28:19 test named[106]: client 127.0.0.1#56323/key updatekey: view
>>> >> remote: signer "updatekey" approved
>>> >> Oct 31 10:28:19 test named[106]: client 127.0.0.1#56323/key updatekey: view
>>> >> remote: updating zone 'example.com/IN': adding an RR at
>>> >> '_25._tcp.mail.example.com' TLSA
>> 
>>> >> But then when I try to do a query from remote, no TLSA record exists.
>> 
>>> >> $ dig @8.8.8.8 TLSA _25._tcp.mail.example.com +dnssec
>> 
>>> >> ; <<>> DiG 9.9.4-RedHat-9.9.4-51.el7 <<>> @8.8.8.8 TLSA
>>> >> _25._tcp.mail.example.com +dnssec
>>> >> ; (1 server found)
>>> >> ;; global options: +cmd
>>> >> ;; Got answer:
>>> >> ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 29421
>>> >> ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
>> 
>>> >> ;; OPT PSEUDOSECTION:
>>> >> ; EDNS: version: 0, flags: do; udp: 512
>>> >> ;; QUESTION SECTION:
>>> >> ;_25._tcp.mail.example.com. IN TLSA
>> 
>>> >> ;; Query time: 74 msec
>>> >> ;; SERVER: 8.8.8.8#53(8.8.8.8)
>>> >> ;; WHEN: Tue Oct 31 10:37:02 PDT 2017
>>> >> ;; MSG SIZE rcvd: 59
>> 
>>> >> Oct 31 10:33:12 test named[106]: query logging is now on
>>> >> Oct 31 10:33:33 test named[106]: client 74.125.80.69#45732
>>> >> (_25._tcp.mail.example.com): view remote: query: _25._tcp.mail.example.com
>>> >> IN TLSA -ED (1.2.3.4)
>>> >> Oct 31 10:33:36 test named[106]: client 1.2.3.4#39184
>>> >> (74.165.37.177.in-addr.arpa): view internal: query:
>>> >> 74.165.37.177.in-addr.arpa IN PTR + (1.2.3.4)
>>> >> Oct 31 10:33:39 test named[106]: received control channel command 'querylog'
>> 
>>> >> I'm at a loss as to what's going on, any ideas?
>> 
>>> > You've elided enough stuff that debugging/helping you is really hard,
>>> > but at least one of the issues is that you are getting back SERVFAIL.
>>> > This is almost defeintely a DNSSEC issue -- I'd suggest looking at
>>> > DNSVIZ to fogure out why...
>> 
>>> > W
>> 
>>> > okay...done.
>> 
>>> > After further debugging, it looks like nsupdate (or Bind) doesn't like
>>> > underscores (that arrive via nsupdate). If I create a TLSA record using the
>>> > same mechanism that doesn't contain underscore characters, all is right with
>>> > the world and remote queries work as expected.
>> 
>>> > However, given that TLSA records use a common record structure that requires
>>> > underscores, this is a problem.
>> 
>>> > Am I missing some obscure named.conf setting that needs to be relaxed to allow
>>> > underscores in dynamically updated records?
>> 
>>> > -Kevin
>> 
>> 
>>> I looked and already had "check-names {master|slave|response} ignore;" set at
>>> the view level.
>> 
>>> I next tried setting these options at the global level and there is no change in
>>> behavior.
>> 
>>> -Kevin
>> 
>> Even more curiously, if I do an "rndc sync" and view the zone's ".signed" file,
>> I see the proper DANE TLSA records with underscores in that file. They just
>> don't seem to be serve-able to remote systems (while test TLSA records that
>> don't contain underscores are resolved).
>> 
>> -Kevin
>> _______________________________________________
>> Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe
>> from this list
>> 
>> bind-users mailing list
>> bind-users at lists.isc.org
>> https://lists.isc.org/mailman/listinfo/bind-users
> 
> 
> 
> --
> I don't think the execution is relevant when it was obviously a bad
> idea in the first place.
> This is like putting rabid weasels in your pants, and later expressing
> regret at having chosen those particular rabid weasels and that pair
> of pants.
> ---maf