Bind 9.11 serving up false answers for a single domain.

[John W. Blue]

I have found to tshark to be useful as well but the failing it has is that it is generally not included in a unix OS distribution.

Assuming anything referred to as "being in production" should also be following good change management protocols it makes sense to be fluent with tools that are readily accessible when a super fun session of all-of-sudden-troubleshooting breaks out.

We recently had issues with a vendor that (I won't mention any names but it rhymes with proofpoint) where they insisted that the error we were having was on our side of the fence.  They were basically unwilling to lift a finger to help.  Color me shocked because I thought that how only Microsoft rolled.  Packet captures proved that it was a load balancing code issue on their side and only then were they compelled to take action.

Being able to correctly examine packets on the wire is a "must have" skillset to be an effective sysadmin.  It can save the day or save your butt.

@OP:  very curious as to where you are at now in your troubleshooting.  Status update?

John

-----Original Message-----
From: Paul Kosinski [mailto:bind at iment.com] 
Sent: Wednesday, February 10, 2021 10:37 PM
To: bind-users at lists.isc.org
Cc: John W. Blue
Subject: Re: Bind 9.11 serving up false answers for a single domain.

I rather prefer tshark to tcpdump: it's essentially the command line version of wireshark, and thus has wireshark's protocol "dissecting" abilities.


On Wed, 10 Feb 2021 22:20:08 +0000
"John W. Blue via bind-users" <bind-users at lists.isc.org> wrote:

> Three words:  tcpdump and wireshark
> 
> It is like peanut and jelly .. hall and oates .. salt and pepper .. ebb and flow .. pen and paper .. I could go on but …
> 
> Know them.  Love them.  They are your newest best friends.
> 
> <grin>
> 
> Using tcpdump IMHO should be the first tool anyone uses when troubleshooting seemly unexplainable DNS weirdness.
> 
> Knowing what is being put on the wire (or lack thereof) is critical since it provides key factual data points that decisions can be made on.  When running tcpdump on the DNS server I personally prefer this command:
> 
> tcpdump -n -i <interface eg eth0> -s 65535 -w <filename.pcap>
> 
> dash n is telling tcpdump that you do not want it to resolve hostnames.  This is an important option when doing DNS troubleshooting because you do not want extra resolutions taking place.
> dash s is saying gimme the full packet.
> dash w is the name of the file you want the output saved in.
> 
> After starting the command, run several queries from a host and ctrl+c to exit.
> 
> Once you get your file into wireshark now you can start slicing n dicing on the data!
> 
> Here is handy wireshark filter:  dns.qry.name == internet-dns1.state.ma.us
> 
> By using a filter of dns.flags.rcode == (number here) you can drive off into the weeds and get super granular with sorting the data.  For example “dns.flags.rcode == 2” will show you all of the server failures for queries.
> 
> It is hard to provide further guidance on what to do since what you find in the pcap is only a starting point.
> 
> Good hunting!
> 
> As an aside I would like to mention that you do not need to travel home to get situational awareness when the diggui.com website can be used instead.
> 
> Also.  For the people running .us tld .. SHA1 for DNSSEC .. really?
> 
> https://dnsviz.net/d/state.ma.us/dnssec/
> 
> John