Restarting DNS service causes isc-dhcp-server to stop handing out leases

[Glenn Satchell]

Hi Andrew

The dhcpd messages look quite normal. Do you restart the two dhcpd daemons
simultaneously? This may be introducing a subtle race condition in
establishing the failover communications.

If so it might be better to stagger the restarts by about 1 minute to
allow the restarting daemon to establish communication with the peer. It
doesn't really matter which one is restarted first. You will then see some
messages about loss of communication with the peer when the other server
restarts, but this is ok.

It might also be more efficient to check the modification time on
dhcpd.conf, and only restart if this has changed, rather than restarting
every 15 minutes.

regards,
-glenn

On Wed, April 2, 2014 2:50 am, Andrew Martin wrote:
> Hello,
>
> I am running the isc-dhcp-server (4.1.ESV-R4-0ubuntu5.6) for DHCP along
> with
> dnsmasq (2.59-4ubuntu0.1) for DNS on a pair of Ubuntu 12.04 servers:
> server1 - 192.168.0.101
>    - isc-dhcp-server
>    - dnsmasq
>
> server2 - 192.168.0.102
>    - isc-dhcp-server
>    - dnsmasq
>
> I have configured isc-dhcp-server's peer failover mode. I have several
> VLAN
> interfaces configured on both servers, e.g eth0.<vlan>, and have
> configured
> isc-dhcp-server to listen on all of them and hand out leases:
> /usr/sbin/dhcpd -f -q -4 -pf /run/dhcp-server/dhcpd.pid -cf
> /etc/dhcp/dhcpd.conf eth0.2 eth0.3 eth0.4 eth0.5 eth0.6 eth0.7
>
> The isc-dhcp-server has the following config:
> http://pastebin.com/RM6EPshy
>
> Last night, I applied some changes to the dnsmasq config (changing one of
> the
> upstream DNS servers) and then restarted networking and the dnsmasq
> service:
> /etc/init.d/networking restart
> service dnsmasq restart
> Shortly thereafter, the isc-dhcp-server restarted via a cron job that
> restarts
> it every 15 minutes. Here are these syslog messages from these 3 events:
> http://pastebin.com/33NTnMbY
>
> At this point, the isc-dhcp-server seems to have stopped handing out
> leases. The
> cron job continued to restart it every 15 minutes, so it was restarted a
> number
> of times during the night, but none of those restarts appear to have
> corrected
> the problem. All of the restarts logged one of these 3 variations in the
> syslog:
> http://pastebin.com/3x3fWPHW
>
> This morning, I restarted dnsmasq again and then restarted isc-dhcp-server
> after
> doing so, and the problem appears to have been resolved:
> http://pastebin.com/wr9iabAj
>
> Is there a way to obtain more debug information about what caused this, or
> how
> to prevent it in the future? Note that while isc-dhcp-server was in this
> bad
> state, I tried running it in the foreground with debug mode (-f and -d)
> but did
> not receive any additional information.
>
> Thanks,
>
> Andrew Martin
> _______________________________________________
> dhcp-users mailing list
> dhcp-users at lists.isc.org
> https://lists.isc.org/mailman/listinfo/dhcp-users
>