dnsperf and BIND memory consumption -- FreeBSD port dns/bind95 for AMD64 - memory leak?

[ivan jr sy]

Thanks for the advise.
I'm currently testing the performance of authoritative queries. Test data is 100 zones with a total of 50,000 RRs, using dnsperf and invoked 6 times simultaneously. 

That's 300,000 queries per run of the script. The script was invoked 5 (runs) successively. memory consumption was observed by just using 'top'.

result:

1. 
FreeBSD dns/bind95 with the memory leak issue:
1st run:    30,241QPS - mem: 220MB
2nd run:    28,121QPS - mem: 640MB
3rd run:    14,854QPS - mem: 990MB
4th run:    9,521QPS - mem: 1780MB
5th run:    7,545QPS - mem: 2540MB (1.5 million queries)

note: the physical memory of the system is just 2GIG, swap was 20% during the last test. 

i restarted the box (just to be sure) and... 

2.
FreeBSD dns/bind95 with the patch below. ARCH=x86_64
1st run:    34,213QPS - mem: 65MB
2nd run:    33,505QPS - mem: 65MB
3rd run:    34,251QPS - mem: 65MB
4th run:    34,345QPS - mem: 65MB
5th run:    34,012QPS - mem: 65MB (1.5 million queries)
note: there was no movement in mem as shown by 'top'. not even a single KB

basically, the memory consumption did not move (as this is an isolated test setup)..

i'll be looping this script for a day or two, and after that test it as a recursive server for another day or two, and for another day and two as caching and authoritative at the same time. After that I can then migrate from my old linux box to this freebsd.

my tools are just:
http://www.nominum.com/services/measurement_tools.php


Thanks
- ivan



--- On Fri, 11/28/08, JINMEI Tatuya / 神明達哉 <Jinmei_Tatuya at isc.org> wrote:

> From: JINMEI Tatuya / 神明達哉 <Jinmei_Tatuya at isc.org>
> Subject: Re: dnsperf and BIND memory consumption
> To: ivan_jr at yahoo.com
> Cc: dougb at FreeBSD.org, "Vinny Abello" <vinny at tellurian.com>, "bind-users at isc.org" <bind-users at isc.org>
> Date: Friday, November 28, 2008, 10:09 PM
> At Thu, 27 Nov 2008 23:35:30 -0800 (PST),
> ivan jr sy <ivan_jr at yahoo.com> wrote:
> 
> > so does this memory leak only occur if 
> > @ISC_ARCH_DIR@ is "noatomic" under FreeBSD
> amd64?
> > and not when its "x86_32" ?
> 
> First off, note that I have no explicit evidence of memory
> leak.  But
> *if there is indeed leak in the FreeBSD pthread library*,
> the key is
> "noatomic".  With this configuration named will
> call pthread
> locks/unlocks much, much heavier, so the problem may be
> observable
> more clearly.  named still uses pthread locks Even with
> x86_32, so it
> may just be leaking memory more slowly.
> 
> Again, everything is just a guess and could be wrong.  We
> should seek
> advice from someone who knows FreeBSD library well.
> 
> ---
> JINMEI, Tatuya
> Internet Systems Consortium, Inc.