Recursive bind becomes unresponsive with high load

[Michael Brunnbauer]

Hello Mike,

On Thu, Mar 31, 2016 at 04:05:39PM +0000, Mike Hoskins (michoski) wrote:
> If you are crawling lots of new names, the cache size won't have much
> impact.  Each new query will require recursing vs hitting the cache.  Try
> "rndc recursing" and look at what you have sitting around waiting for
> answers.  Hopefully that provides some clues.  This can be all sorts of
> things like unresponsive auth servers, network issues, firewalls munging
> EDNS, etc causing the recursive client backlog.

Can a "recursive client backlog" be a problem if recursing clients is ca. 1000
while recursive-clients is 6000? If yes, where is the backlog? I can see it
in the syslog when recursive-clients is reached - this does not happen here.

Here are the first 10 lines. The other 995 lines all look like this.

;
; Recursing Queries
;
; client 127.0.0.1#40278: id 13156 'fnnd0u.ciptdd.cn/A/IN' requesttime 1459440503
; client 127.0.0.1#43457: id 30082 '6aj344.iqr8aop.cn/A/IN' requesttime 1459440503
; client 127.0.0.1#55751: id 58170 'g1zdo7.02fucag.cn/A/IN' requesttime 1459440503
; client 127.0.0.1#38696: id 62912 'v6mzb.566095.top/A/IN' requesttime 1459440504
; client 127.0.0.1#38585: id 17254 'l3ay0.688903.top/A/IN' requesttime 1459440504
; client 127.0.0.1#47576: id 24940 '0h8xi.866099.top/A/IN' requesttime 1459440504
; client 127.0.0.1#38195: id 25054 'oipy2.spwgm89.com/A/IN' requesttime 1459440504

There are only 2 requests for .de domains in the queue so the failing requests
for netestate.de cannot be explained by a rate limiting of the .de nameservers.
What are current rate limits for tld nameservers anyway? I wonder how fast
a single bind instance should hammer them.

Our database is cluttered with chinese linkfarms and the DNS queries for them
tend to fail early and often or take a long time. I may be able to address
this in some way so that those queries are reduced but I would also like to
have a DNS server that can handle high load and it seems my current setup is
lacking. 

Regards,

Michael Brunnbauer

-- 
++  Michael Brunnbauer
++  netEstate GmbH
++  Geisenhausener Straße 11a
++  81379 München
++  Tel +49 89 32 19 77 80
++  Fax +49 89 32 19 77 89 
++  E-Mail brunni at netestate.de
++  http://www.netestate.de/
++
++  Sitz: München, HRB Nr.142452 (Handelsregister B München)
++  USt-IdNr. DE221033342
++  Geschäftsführer: Michael Brunnbauer, Franz Brunnbauer
++  Prokurist: Dipl. Kfm. (Univ.) Markus Hendel
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <https://lists.isc.org/pipermail/bind-users/attachments/20160331/b637e17a/attachment-0001.bin>