dnsperf and BIND memory consumption

[Dmitry Rybin]

max-cache-size 64M;
# /usr/bin/limits -v 1200M /usr/local/sbin/named-test -c
/etc/namedb/named.conf

Over 10 minutes of work and core dumped:

(gdb) bt
#0  0x000000000058c3fc in thr_kill ()
#1  0x00000000005c5a68 in abort ()
#2  0x0000000000597af7 in malloc ()
#3  0x000000000056645a in isc_mem_createx2 (init_max_size=0, target_size=0,
    memalloc=0x564400 <default_memalloc>, memfree=0x563320
<default_memfree>, arg=0x0,
    ctxp=0xcb29b978, flags=Variable "flags" is not available.
) at mem.c:790
#4  0x0000000000566730 in isc_mem_create (init_max_size=Variable
"init_max_size" is not available.
) at mem.c:859
#5  0x00000000004d83ae in dns_resolver_create (view=0xca46e800,
taskmgr=0x80828000,
    ntasks=31, socketmgr=Variable "socketmgr" is not available.
) at resolver.c:6514
#6  0x00000000004ee860 in dns_view_createresolver (view=0xca46e800,
taskmgr=0x80828000,
    ntasks=31, socketmgr=0x8082b000, timermgr=0x80829000, options=0,
dispatchmgr=0x8083c000,
    dispatchv4=0x864c9000, dispatchv6=0x864c9800) at view.c:580
#7  0x000000000041bba2 in configure_view (view=0xca46e800,
config=0x80abb4c0,
    vconfig=0x8085ada8, mctx=0x8080d140, actx=0x7ffffeff8860,
need_hints=isc_boolean_true)
    at server.c:1290
#8  0x0000000000420f42 in load_configuration (filename=Variable
"filename" is not available.
) at server.c:3285
#9  0x0000000000422095 in loadconfig (server=0x8082f000) at server.c:4121
#10 0x0000000000422426 in reload (server=0x8082f000) at server.c:4141
#11 0x00000000004225c2 in ns_server_reloadcommand (server=0x8082f000,
args=Variable "args" is not available.
) at server.c:4334
#12 0x0000000000407682 in ns_control_docommand (message=Variable
"message" is not available.
) at control.c:102
#13 0x000000000040a8b7 in control_recvmessage (task=0x80839000,
event=Variable "event" is not available.
) at controlconf.c:456
#14 0x000000000057052c in run (uap=Variable "uap" is not available.
) at task.c:862
#15 0x00000000005868a7 in thread_start ()
#16 0x0000000000000000 in ?? ()
Cannot access memory at address 0x7ffffeff9000



JINMEI Tatuya / 神明達哉 wrote:
> At Wed, 10 Dec 2008 15:50:22 +0300,
> Dmitry Rybin <kirgudu at corbina.net> wrote:
>> JINMEI Tatuya / 神明達哉 wrote:
>>> At Tue, 09 Dec 2008 18:05:27 +0300,
>>> Dmitry Rybin <kirgudu at corbina.net> wrote:
>>>
>>>> I test patch, add to bind95/Makefile
>>>> .if (${ARCH} == "amd64")
>>>> ARCH=           x86_64
>>>> .endif
>>> Future versions of BIND9 will support amd64 in its configure script to
>>> workaround the FreeBSD port for amd64.
>>>
>>> Regarding the memory leak, I believe it's already solved in 9.5.1rc1
>>> (even with threads and without atomic).
>> I just make port bind 9.5.1rc1. It has same problem with memory leak.
>> It grows from 670M on startup, to 1,4Gb after 20 minutes of work.
> 
> Can you first fall back to the vanilla 9.5.1rc1 (i.e., not FreeBSD
> port) so that we can separate FreeBSD-port specific issue and BIND9
> specific leak?
> 
> Second, what if you stop named by 'rndc stop'?  If there's memory leak
> in BIND9, it normally detects it during a cleanup process and
> indicates the bug by aborting (core dumping) itself.
> 
> If it doesn't cause an abort, please then try the diagnosing I
> suggested before:
> http://marc.info/?l=bind-users&m=121811979629090&w=2
>  
> To summarize it:
> 
> 1. create a symbolic link from "/etc/malloc.conf" to "X":
>    # ln -s X /etc/malloc.conf
> 2. - start named with a moderate limitation of virtual memory size, e.g.
>    # /usr/bin/limits -v 384m $path_to_named/named <command line options>
> (note that "384m" should be reasonably large compared with
> max-cache-size.  I'd suggest setting max-cache-size to 128M and
> setting 'limits -v' to 512m).
> 3. Then the named process will eventually abort itself with a core dump
>    due to malloc failure.  Please show us the stack trace at that point.
>    Hopefully it will reveal the malloc call that keeps consuming memory.
> 
> In fact, I myself successfully identified one leak in 9.5.0-P2 with
> FreeBSD port this way.
> 
> ---
> JINMEI, Tatuya
> Internet Systems Consortium, Inc.