BIND 9.x caching Performance under heavy loads

[Jim Reid]

>>>>> "Srini" == Srini Avirneni <avirsri at gmail.com> writes:

    Srini> We are looking to hear peoples experiences with heavily
    Srini> loaded caching servers (> 2500 queries/Sec) running on
    Srini> linux (RH ES 3.0 Kernel 2.4.x). We have used various builds
    Srini> of BIND 9.x (9.2.3, 9.2.4, 9.3.0, 9.3.1rc1) and also 8.4.5.

    Srini> Our results show in all cases that BIND falls on its face
    Srini> with CPU time increasing from 20% at startup to 80+ % over
    Srini> 24 hour period.  Cache Size typically ranges from 800MB to
    Srini> 1GB. Expermenting with lower cache sizes showed no
    Srini> improvement.

If you restrict the size of the name server's cache, the name server
has to do much more work. That should be obvious. First of all, the
server has to maintain the cache at the appointed (self-imposed) limit.
Secondly, it could well be cache-thrashing: discarding perfectly good
data that it's been forced to throw away only to have to retreive it
again. Finally, the name server is almost certainly having to waste
resources -- bandwidth, CPU, internal memory state for queries under
resolution, etc, etc -- because it's having to resolve more queries
that it might have been able to answer out of cache if it had been big
enough. BTW the size of most caches tends to stabilise in a couple of
days after start-up: TTL values are usually no more than a day or so,
perhaps 1 week at the outside.

    Srini> Any shared experience is welcome.

Please be more specific. You seem to be saying that "bad things happen
when your name servers are under heavy load". This shouldn't exactly
be a surprise. What's the actual problem are you experiencing and/or
trying to solve?

Query rates of 2500 qps are rather high. Is this a real, operational
load or something you've cooked up in a testbed? If it's the former,
you need to look at what's generating all that traffic (and why) and
at why the load isn't spread across more name servers. These kind of
traffic rates can be seen for real at big ISPs, cable operators, DSL
providers and the like. Though they generally deal with this by
spending money on better hardware and software or a combination of the
two.