how to debug "no more recursive clients"

[Mark Andrews]

> 
> We have "forwarding DNS" setup, where all the internal DNS "forward 
> first" to our recursive-only rns1 and rns2 (recursive ns).
> 
> rns1's named process occupies an average of about 170 - 200 MB of 
> memory footprint.
> 
> rns's are running BIND 9.3.1, yesterday upped to 9.3.2 on fbsd 4.x.
> 
> BIND is setup with:
> 
> blackhole { !<mynetworks>; };
> 
> On two occasions, rns1 has apparently stopped resolving recursive 
> queries, with syslog showing many lines of:
> 
> no more recursive clients: quota reached
> 
> "rndc status" reports the quota to be 1000 clients.
> 
> We have been running rndc status every 5 minutes for several days and 
> logging the recursive clients line.  Avg is only 58/1000 recursive 
> clients, with a handful in the 200 - 400 range.
> 
> We also turned on BIND logging and found rns1 sending an average of 
> 30 createfetch/second over several business hours.
> 
> When rns1 quit resolving those two times in a period of about 8 weeks 
> of operation, rebooting the machine and stop/start bind solved the problem.
> 
> We figure that some upstream network problem could cause BIND's 
> createfetch to pile up a lot of timing-out queries in a hurry, but 
> our network monitors showed no networking problems in the two periods 
> where BIND quit resolving.
> 
> What is BIND9's behavior when it hits "recursive clients quota"?  For 
> us, it seems to be that it dies, requiring named to be stop/start.
> 
> Any suggestions beyond the above for debugging this problem?
> 
> Len

	rndc recursing
--
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742                 INTERNET: Mark_Andrews at isc.org