bind v9.9.5 becomes unresponsive when using samba4 dlopen driver

Fri Mar 14 20:35:53 UTC 2014

Hello again,

today I reinstalled bind 9.9.5 without threads, but i still have the same
behavior.

My environment was rhel v6.4, bind 9.9.3-P3 and samba v4.1.
I upgraded to rhel v6.5, bind 9.9.5 and samba v4.1.5 but the problem
remains:

after a while bind 9 becomes unresponsive.

I was not able to use gdb, probably I can do it on Monday, in the meanwhile
this is my named.conf, named -V and pstack output captured the moment bind
was unresponsive.

named-checkconf -px
================
options {
directory "/var/named";
dump-file "/var/named/data/cache.dump.db";
listen-on port 53 {
"any";
};
listen-on-v6 {
"none";
};
statistics-file "/var/named/data/named.stats.txt";
tkey-gssapi-keytab "/usr/local/samba/private/dns.keytab";
version "Bind";
allow-recursion {
10.0.0.0/8;
10.1.4.0/24;
"localhost";
"localnets";
};
auth-nxdomain yes;
empty-zones-enable no;
max-cache-size 4294967296;
allow-query {
"any";
};
allow-transfer {
"none";
};
allow-update {
127.0.0.1/32;
"localhost";
10.0.0.0/8;
};
forwarders {
8.8.8.8;
8.8.4.4;
};
notify no;
};
controls {
inet 127.0.0.1 port 953 allow {
127.0.0.1/32;
} keys {
"domain1";
};
};
acl "trusted" {
10.1.4.0/24;
127.0.0.1/32;
};
logging {
channel "basic" {
file "/var/log/named/named.log" versions 3 size 20971520;
severity debug 1;
print-time yes;
print-severity yes;
print-category yes;
};
category "default" {
"basic";
};
};
statistics-channels {
inet 0.0.0.0 port 8060 allow {
"trusted";
};
};
key "domain1" {
algorithm "hmac-md5";
secret "????????????????????????????????????????????";
};
zone "." {
type hint;
file "named.root";
};
zone "localhost" {
type master;
file "master/localhost.zone";
};
zone "0.0.127.in-addr.arpa" {
type master;
file "master/0.0.127.zone";
};
zone "ait.nkm" IN {
type master;
file "dynamic/fz.db.ait.nkm";
};
zone "aka.nkm" IN {
type master;
file "dynamic/fz.db.aka.nkm";
};
zone "axa.nkm" IN {
type master;
file "dynamic/fz.db.axa.nkm";
};
zone "2.1.10.in-addr.arpa" IN {
type master;
file "dynamic/rz.db.2.1.10";
};
zone "7.1.10.in-addr.arpa" IN {
type master;
file "dynamic/rz.db.7.1.10";
};
zone "6.1.10.in-addr.arpa" IN {
type master;
file "dynamic/rz.db.6.1.10";
};
dlz "AD DNS Zone" {
database "dlopen /usr/local/samba/lib/bind9/dlz_bind9_9.so -d 3";
};
server ::/0 {
bogus yes;
};

named -V
=======
BIND 9.9.5 (Extended Support Version) <id:f9b8a50e> built by make with
'--build=x86_64-redhat-linux-gnu' '--host=x86_64-redhat-linux-gnu'
'--target=x86_64-redhat-linux-gnu' '--program-prefix=' '--prefix=/usr'
'--exec-prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin'
'--sysconfdir=/etc' '--datadir=/usr/share' '--includedir=/usr/include'
'--libdir=/usr/lib64' '--libexecdir=/usr/libexec'
'--sharedstatedir=/var/lib' '--mandir=/usr/share/man'
'--infodir=/usr/share/info' '--with-libtool' '--localstatedir=/var'
'--enable-ipv6' '--with-pic' '--disable-static'
'--disable-openssl-version-check' '--with-dlz-ldap=yes'
'--with-dlz-filesystem=yes' '--with-gssapi=/usr/include/gssapi'
'--with-dlopen=yes'
'--with-docbook-xsl=/usr/share/sgml/docbook/xsl-stylesheets'
'--enable-fixed-rrset' '--enable-rrl' '--enable-newstats'
'build_alias=x86_64-redhat-linux-gnu' 'host_alias=x86_64-redhat-linux-gnu'
'target_alias=x86_64-redhat-linux-gnu'
compiled by GCC 4.4.7 20120313 (Red Hat 4.4.7-4)
using OpenSSL version: OpenSSL 1.0.1e 11 Feb 2013
using libxml2 version: 2.7.6

pstack when named was unresponsive
============================
#0  0x00000039bb80e975 in fcntl () from /lib64/libpthread.so.0
#1  0x00007fc753c1b347 in fcntl_lock () from
/usr/local/samba/lib/private/libtdb.so.1
#2  0x00007fc753c1b44f in tdb_brlock () from
/usr/local/samba/lib/private/libtdb.so.1
#3  0x00007fc753c1b919 in tdb_nest_lock () from
/usr/local/samba/lib/private/libtdb.so.1
#4  0x00007fc753c1bf2c in tdb_transaction_lock () from
/usr/local/samba/lib/private/libtdb.so.1
#5  0x00007fc753c213be in _tdb_transaction_start () from
/usr/local/samba/lib/private/libtdb.so.1
#6  0x00007fc753c216d3 in tdb_transaction_start () from
/usr/local/samba/lib/private/libtdb.so.1
#7  0x00007fc74dafa321 in partition_metadata_start_trans () from
/usr/local/samba/lib/ldb/partition.so
#8  0x00007fc74daf5f1f in partition_start_trans () from
/usr/local/samba/lib/ldb/partition.so
#9  0x00007fc7594586e1 in ldb_next_start_trans () from
/usr/local/samba/lib/private/libldb.so.1
#10 0x00007fc74ed23925 in linked_attributes_start_transaction () from
/usr/local/samba/lib/ldb/linked_attributes.so
#11 0x00007fc7594586e1 in ldb_next_start_trans () from
/usr/local/samba/lib/private/libldb.so.1
#12 0x00007fc74d2d40c3 in replmd_start_transaction () from
/usr/local/samba/lib/ldb/repl_meta_data.so
#13 0x00007fc7594586e1 in ldb_next_start_trans () from
/usr/local/samba/lib/private/libldb.so.1
#14 0x00007fc750b83b12 in descriptor_start_transaction () from
/usr/local/samba/lib/ldb/descriptor.so
#15 0x00007fc7594586e1 in ldb_next_start_trans () from
/usr/local/samba/lib/private/libldb.so.1
#16 0x00007fc74ba7f70f in schema_load_start_transaction () from
/usr/local/samba/lib/ldb/schema_load.so
#17 0x00007fc759475eeb in ldb_transaction_start () from
/usr/local/samba/lib/private/libldb.so.1
#18 0x00007fc759d12ea9 in dlz_newversion () from
/usr/local/samba/lib/bind9/dlz_bind9_9.so
#19 0x00000000004720cd in dlopen_dlz_newversion ()
#20 0x00007fc75ac628ec in newversion () from /usr/lib64/libdns.so.100
#21 0x000000000045913f in update_action ()
#22 0x00007fc75a2d94eb in isc__taskmgr_dispatch () from
/usr/lib64/libisc.so.95
#23 0x00007fc75a2deedb in evloop () from /usr/lib64/libisc.so.95
#24 0x00007fc75a2df14a in isc__app_ctxrun () from /usr/lib64/libisc.so.95
#25 0x00000000004273cc in main ()

pstack when named was working
========================
#0  0x00000039bb4e9143 in __epoll_wait_nocancel () from /lib64/libc.so.6
#1  0x00007fc61b96bd0f in isc__socketmgr_waitevents () from
/usr/lib64/libisc.so.95
#2  0x00007fc61b962eaf in evloop () from /usr/lib64/libisc.so.95
#3  0x00007fc61b96314a in isc__app_ctxrun () from /usr/lib64/libisc.so.95
#4  0x00000000004273cc in main ()

Thanks for your time

Nikos Mitas

2014-03-12 22:37 GMT+02:00 Evan Hunt <each at isc.org>:

> On Wed, Mar 12, 2014 at 10:09:44PM +0200, Nikos Mitas wrote:
> > at this point, nslookup and rndc are not working, the only option i have
> is
> > to kill samba and named and start over. But after a while the problem
> > repeats.
> >
> > any help will be very welcome
>
> Offhand I'd guess it's a deadlock.  Does it happen if named is built
> without threads?  Can you attach to the process with gdb and get a
> backtrace?
>
> You can submit a bug report against BIND at bind9-bugs at isc.org -- if
> you do, please include as much information as possible about the system
> you're running on, how you configured the built ("named -V" will tell
> you this), and your named.conf ("named-checkconf -px" dumps a copy of
> your configuration with key secrets omitted).
>
> If the problem's in the samba DLZ module, I probably won't be able to
> help you, but if it's in the dlopen driver, perhaps I can.
>
> --
> Evan Hunt -- each at isc.org
> Internet Systems Consortium, Inc.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.isc.org/pipermail/bind-users/attachments/20140314/6a0eecbd/attachment-0001.html>