Problems running multiple dhclients at the same time
Glenn Satchell
glenn.satchell at uniq.com.au
Wed Jun 9 00:04:48 UTC 2010
On 06/09/10 06:37, Jeff Haran wrote:
>> -----Original Message-----
>> From: dhcp-users-bounces+jharan=brocade.com at lists.isc.org [mailto:dhcp-users-
>> bounces+jharan=brocade.com at lists.isc.org] On Behalf Of Bob Schatz
>> Sent: Tuesday, June 08, 2010 12:12 PM
>> To: dhcp-users at lists.isc.org
>> Subject: Problems running multiple dhclients at the same time
>>
>> We are having problems running
>> multiple dhclients from the same box and same interface at the same time and
>> could not find a solution on the FAQ.
>>
>> We are using version 4.1.1 of the
>> DHCP server and dhclient and we are only running one DHCP
>> server.
>>
>> The problem is that all 6 dhclients
>> end up with the same IP address even though we specify a different dhcp-client-identifier for each
>> dhclient.
>> This is very reproducible.
>>
>> Are we doing something
>> wrong?
>>
>> The reproduction case
>> is:
>>
>> 1. Create 6 dhclient.config files where
>> each has these contents:
>>
>> request subnet-mask,
>> broadcast-address, time-offset, routers,
>> domain-name, domain-name-servers,
>> host-name,
>> netbios-name-servers,
>> netbios-scope, ntp-servers;
>>
>> pseudo "eth0-vcsX" "eth0" {
>> send dhcp-client-identifier
>> "00:30:48:bf:85:86-eth0-vcsX";
>> script
>> "/sbin/dhclient-script";
>> }
>> timeout
>> 15;
>>
>> where vcsX is one of vcs1, vcs2,
>> etc.
>>
>> For example, the file
>> /tmp/dhclient.vcs1.config would have the contents:
>>
>> request subnet-mask,
>> broadcast-address, time-offset, routers,
>> domain-name, domain-name-servers,
>> host-name,
>> netbios-name-servers,
>> netbios-scope, ntp-servers;
>>
>> pseudo "eth0-vcs1" "eth0" {
>> send dhcp-client-identifier
>> "00:30:48:bf:85:86-eth0-vcs1";
>> script
>> "/sbin/dhclient-script";
>> }
>> timeout
>> 15;
>>
>> 2. Run 6 dhclients in parallel in a
>> loop with:
>>
>> # for I in 1 2 3 4 5
>> 6
>>
>> do
>> dhclient –pf
>> /tmp/pid.$i –lf /tmp/dhclient.$i.lease –cf /tmp/dhclient.$i.config
>> eth0&
>>
>> done
>>
>> 3. What you will see is that all lease
>> files will have the same fixed-lease IP address.
>>
>> We see these messages on the DHCP
>> server:
>>
>> Jun 7
>> 17:54:11 mgraid-qa2-1 dhcpd: DHCPDISCOVER from 00:30:48:bf:b8:88 via
>> eth1
>> Jun 7
>> 17:54:11 mgraid-qa2-1 dhcpd: Abandoning IP address 10.4.253.20: pinged before
>> offer
>> Jun 7
>> 17:54:15 mgraid-qa2-1 dhcpd: DHCPDISCOVER from 00:30:48:bf:b8:88 via
>> eth0
>> Jun 7
>> 17:54:16 mgraid-qa2-1 dhcpd: DHCPOFFER on 10.4.253.21 to 00:30:48:bf:b8:88 via
>> eth0
>> Jun 7
>> 17:54:16 mgraid-qa2-1 dhcpd: DHCPREQUEST for 10.4.253.21 (10.4.253.10) from
>> 00:30:48:bf:b8:88 via eth0
>> Jun 7
>> 17:54:16 mgraid-qa2-1 dhcpd: DHCPACK on 10.4.253.21 to 00:30:48:bf:b8:88 via
>> eth0
>> Jun 7
>> 17:54:16 mgraid-qa2-1 dhcpd: DHCPREQUEST for 10.4.253.21 (10.4.253.10) from
>> 00:30:48:bf:b8:88 via eth1
>> Jun 7
>> 17:54:16 mgraid-qa2-1 dhcpd: DHCPACK on 10.4.253.21 to 00:30:48:bf:b8:88 via
>> eth1
>> Jun 7
>> 17:54:19 mgraid-qa2-1 dhcpd: DHCPDISCOVER from 00:30:48:bf:b8:88 via
>> eth0
>> Jun 7
>> 17:54:19 mgraid-qa2-1 dhcpd: DHCPDISCOVER from 00:30:48:bf:b8:88 via
>> eth0
>> Jun 7
>> 17:54:19 mgraid-qa2-1 dhcpd: DHCPDISCOVER from 00:30:48:bf:b8:88 via
>> eth0
>> Jun 7
>> 17:54:19 mgraid-qa2-1 dhcpd: DHCPDISCOVER from 00:30:48:bf:b8:88 via
>> eth0
>> Jun 7
>> 17:54:19 mgraid-qa2-1 dhcpd: DHCPDISCOVER from 00:30:48:bf:b8:88 via
>> eth1
>> Jun 7
>> 17:54:19 mgraid-qa2-1 dhcpd: DHCPDISCOVER from 00:30:48:bf:b8:88 via
>> eth1
>> Jun 7
>> 17:54:20 mgraid-qa2-1 dhcpd: DHCPOFFER on 10.4.253.22 to 00:30:48:bf:b8:88 via
>> eth0
>> Jun 7
>> 17:54:20 mgraid-qa2-1 dhcpd: DHCPOFFER on 10.4.253.23 to 00:30:48:bf:b8:88 via
>> eth0
>> Jun 7
>> 17:54:20 mgraid-qa2-1 dhcpd: DHCPOFFER on 10.4.253.24 to 00:30:48:bf:b8:88 via
>> eth0
>> Jun 7
>> 17:54:20 mgraid-qa2-1 dhcpd: DHCPOFFER on 10.4.253.25 to 00:30:48:bf:b8:88 via
>> eth0
>> Jun 7
>> 17:54:20 mgraid-qa2-1 dhcpd: DHCPOFFER on 10.4.253.26 to 00:30:48:bf:b8:88 via
>> eth1
>> Jun 7
>> 17:54:20 mgraid-qa2-1 dhcpd: DHCPOFFER on 10.4.253.27 to 00:30:48:bf:b8:88 via
>> eth1
>> Jun 7
>> 17:54:20 mgraid-qa2-1 dhcpd: uid lease 10.4.253.25 for client 00:30:48:bf:b8:88
>> is duplicate on 10.4.253.0/25
>> Jun 7
>> 17:54:20 mgraid-qa2-1 dhcpd: DHCPREQUEST for 10.4.253.22 (10.4.253.10) from
>> 00:30:48:bf:b8:88 via eth1
>> Jun 7
>> 17:54:20 mgraid-qa2-1 dhcpd: DHCPACK on 10.4.253.22 to 00:30:48:bf:b8:88 via
>> eth1
>> Jun 7
>> 17:54:20 mgraid-qa2-1 dhcpd: DHCPREQUEST for 10.4.253.22 (10.4.253.10) from
>> 00:30:48:bf:b8:88 via eth0
>> Jun 7
>> 17:54:20 mgraid-qa2-1 dhcpd: DHCPACK on 10.4.253.22 to 00:30:48:bf:b8:88 via
>> eth0
>> Jun 7
>> 17:54:20 mgraid-qa2-1 dhcpd: DHCPREQUEST for 10.4.253.22 (10.4.253.10) from
>> 00:30:48:bf:b8:88 via eth1: lease 10.4.253.22
>> unavailable.
>> Jun 7
>> 17:54:20 mgraid-qa2-1 dhcpd: DHCPNAK on 10.4.253.22 to 00:30:48:bf:b8:88 via
>> eth1
>> Jun 7
>> 17:54:20 mgraid-qa2-1 dhcpd: DHCPREQUEST for 10.4.253.22 (10.4.253.10) from
>> 00:30:48:bf:b8:88 via eth0: lease 10.4.253.22
>> unavailable.
>> Jun 7
>> 17:54:20 mgraid-qa2-1 dhcpd: DHCPNAK on 10.4.253.22 to 00:30:48:bf:b8:88 via
>> eth0
>> Jun 7
>> 17:54:20 mgraid-qa2-1 dhcpd: DHCPREQUEST for 10.4.253.22 (10.4.253.10) from
>> 00:30:48:bf:b8:88 via eth1: lease 10.4.253.22
>> unavailable.
>> Jun 7
>> 17:54:20 mgraid-qa2-1 dhcpd: DHCPNAK on 10.4.253.22 to 00:30:48:bf:b8:88 via
>> eth1
>> Jun 7
>> 17:54:20 mgraid-qa2-1 dhcpd: DHCPREQUEST for 10.4.253.22 (10.4.253.10) from
>> 00:30:48:bf:b8:88 via eth0: lease 10.4.253.22
>> unavailable.
>> Jun 7
>> 17:54:20 mgraid-qa2-1 dhcpd: DHCPNAK on 10.4.253.22 to 00:30:48:bf:b8:88 via
>> eth0
>> Jun 7
>> 17:54:20 mgraid-qa2-1 dhcpd: DHCPREQUEST for 10.4.253.22 (10.4.253.10) from
>> 00:30:48:bf:b8:88 via eth1: lease 10.4.253.22
>> unavailable.
>> Jun 7
>> 17:54:20 mgraid-qa2-1 dhcpd: DHCPNAK on 10.4.253.22 to 00:30:48:bf:b8:88 via
>> eth1
>> Jun 7
>> 17:54:20 mgraid-qa2-1 dhcpd: DHCPREQUEST for 10.4.253.22 (10.4.253.10) from
>> 00:30:48:bf:b8:88 via eth0: lease 10.4.253.22
>> unavailable.
>> Jun 7
>> 17:54:20 mgraid-qa2-1 dhcpd: DHCPNAK on 10.4.253.22 to 00:30:48:bf:b8:88 via
>> eth0
>> Jun 7
>> 17:54:20 mgraid-qa2-1 dhcpd: DHCPREQUEST for 10.4.253.22 (10.4.253.10) from
>> 00:30:48:bf:b8:88 via eth1: lease 10.4.253.22
>> unavailable.
>> Jun 7
>> 17:54:20 mgraid-qa2-1 dhcpd: DHCPNAK on 10.4.253.22 to 00:30:48:bf:b8:88 via
>> eth1
>> Jun 7
>> 17:54:20 mgraid-qa2-1 dhcpd: DHCPREQUEST for 10.4.253.22 (10.4.253.10) from
>> 00:30:48:bf:b8:88 via eth0: lease 10.4.253.22
>> unavailable.
>> Jun 7
>> 17:54:20 mgraid-qa2-1 dhcpd: DHCPNAK on 10.4.253.22 to 00:30:48:bf:b8:88 via
>> eth0
>> Jun 7
>> 17:54:20 mgraid-qa2-1 dhcpd: DHCPREQUEST for 10.4.253.22 (10.4.253.10) from
>> 00:30:48:bf:b8:88 via eth1: lease 10.4.253.22
>> unavailable.
>> Jun 7
>> 17:54:20 mgraid-qa2-1 dhcpd: DHCPNAK on 10.4.253.22 to 00:30:48:bf:b8:88 via
>> eth1
>> Jun 7
>> 17:54:20 mgraid-qa2-1 dhcpd: DHCPREQUEST for 10.4.253.22 (10.4.253.10) from
>> 00:30:48:bf:b8:88 via eth0: lease 10.4.253.22
>> unavailable.
>> Jun 7
>> 17:54:20 mgraid-qa2-1 dhcpd: DHCPNAK on 10.4.253.22 to 00:30:48:bf:b8:88 via
>> eth0
>>
>>
>> We used tcpdump to catch the packets
>> and Wireshark to review them and it looks like in the DHCPOFFER message the
>> dhcp-client-identifier is not included. Therefore, the dhclient does not know
>> if it should ignore an offer meant for another dhclient currently
>> executing.
>>
>> Also, DHCP sends 1 ACK and 6 NAK
>> packets. According to line 1709 of dhclient.c, the NAK, received by the client,
>> should cause the client to EXPIRE ... which should cause a DHCPDISCOVER...
>> correct? But we see no new DHCPDISCOVER attempts by the remaining “dhclient”
>> instances.
>>
>> Any feedback would be
>> appreciated.
>>
>>
>> Thanks
>>
>
> Just a wild guess, but I ran into a similar problem (quite) a while ago with a different client. I had multiple instances of dhcpcd doing DISCOVERS with different client ID options on the same subnet. Turns out the dhcpcd clients were matching OFFERs to their DISCOVERs using the XID field in the protocol (which I suspect in retrospect is the correct thing to do according to some RFC). The problem was the clients generated XIDs using a hash of the MAC address and current time, so if they started within the same second they all generated the same XIDs, which confused everything, clients and servers alike.
>
> A one line source code fix was to include the process ID (what comes back from a getpid() system call) of the client in the hash that generates the XID. Since process IDs are unique, this got unique XIDs for each client and everything worked.
>
> Again, this was a different client, so I don't know if this will fix the problem on dhclient, but the behavior looks the same.
>
> Jeff Haran
> Brocade Communications
>
In that case a simple test would be to put a sleep in between the
dhclient startups so that time() is different each time. If that "fixes"
it, then adding the getpid() would be a permanent solution.
for i in 1 2 3 4 5 6
do
dhclient –pf /tmp/pid.$i –lf /tmp/dhclient.$i.lease –cf
/tmp/dhclient.$i.config eth0 &
# pause between requests, sleep 1 can be less than 1 second
sleep 2
done
--
regards,
-glenn
--
Glenn Satchell | Miss 9: What do you
Uniq Advances Pty Ltd, Sydney Australia | do at work Dad?
mailto:glenn.satchell at uniq.com.au | Miss 6: He just
http://www.uniq.com.au tel:0409-458-580 | types random stuff.
More information about the dhcp-users
mailing list