Sudden Slow Performance

Mon Dec 15 15:56:24 UTC 2008

I ran in to something similar one time - I spent an entire long weekend
working to fix it.  It turned out that there was a flag set in the
vendor provided dhcp conf (#get-lease-hostnames true;) which tells the
server to do a dns query before handing out an IP.  If I didn't have a
DNS server configured on the server, it was fine.  But once I configured
a DNS server that didn't have an answer, it was causing huge slowdowns,
causing retransmits from the clients.  This caused the DHCP server to
send back packets to a request that was already timed out, and some
really stupid, inconsistant behaviour.

One tool that helped us a lot was "dhcping" I think it was called - let
you form a dhcp request and send it, so yo ucould see what was
happening, and the timing delays.

Hopefully that helps some.

Cheers

Todd.

-----Original Message-----
From: dhcp-users-bounces at lists.isc.org
[mailto:dhcp-users-bounces at lists.isc.org] On Behalf Of Kelvin Williams
Sent: Monday, December 15, 2008 10:48 AM
To: 'Users of ISC DHCP'
Subject: RE: Sudden Slow Performance

I just executed a sniff on the DHCP Server.  This is for ONE client.  

Time	Message	Transaction ID
21s	DISCOVER	0x1179
25s	OFFER		0x1179
26s	DISCOVER	0x117a
34s	OFFER		0x117a
35s	DISCOVER	0x117b
44s	OFFER		0x117b
46s	REQUEST	0x117b
50s	REQUEST	0x117b
54s	ACK		0x117b
58s	ACK		0x117b

The client is getting all messages; but appears to be giving up and
starting again after the wait.

kw

-----Original Message-----
From: dhcp-users-bounces at lists.isc.org
[mailto:dhcp-users-bounces at lists.isc.org] On Behalf Of Tom Greaser
Sent: Monday, December 15, 2008 10:10 AM
To: 'Users of ISC DHCP'
Subject: RE: Sudden Slow Performance

 Have you sniffed / snooped a station to see if its getting this Offer /
ACK ?

>>> "Kelvin Williams" <kelvin-lists at williamschadwell.com> 12/15/08 9:52 
>>> AM
>>> 
The entire network. 

-----Original Message-----
From: dhcp-users-bounces at lists.isc.org
[mailto:dhcp-users-bounces at lists.isc.org] On Behalf Of Tom Greaser
Sent: Monday, December 15, 2008 9:47 AM
To: 'Users of ISC DHCP'
Subject: RE: Sudden Slow Performance

Is this effecting All of your network or just a few subnets ?

 Ive had issues like this.. but only for specific networks.. I had a
switch that would auto prune spanning tree .. Thus i would get the
DHCPDISCOVER  but never the clients would never see the DHCPOFFER.  

I manually made sure that network was never pruned..  

>>> "Kelvin Williams" <kelvin-lists at williamschadwell.com> 12/15/08 9:38 
>>> AM
>>> 
Network good.  No errors on the interfaces.

No disk errors reported.

No patches applied.

CPU and Memory levels are good.

Number of clients always increases--but not a drastic amount.  Maybe 50
new clients.

-----Original Message-----
From: dhcp-users-bounces at lists.isc.org
[mailto:dhcp-users-bounces at lists.isc.org] On Behalf Of Glenn Satchell
Sent: Monday, December 15, 2008 9:25 AM
To: dhcp-users at lists.isc.org
Subject: Re: Sudden Slow Performance

>From: "Kelvin Williams" <kelvin-lists at williamschadwell.com>
>To: <dhcp-users at lists.isc.org>
>Subject: Sudden Slow Performance
>Date: Mon, 15 Dec 2008 08:57:08 -0500
>X-Spam-Level: ***
>X-BeenThere: dhcp-users at lists.isc.org
>
> We've been operating one of our DHCP Servers for approximately 6 
> months without any issues.  This server is on a network with about
2,500 hosts.
> The server has worked flawlessly until last week when it became 
> extremely slow.
> 
> Clients are having to send multiple DISCOVERs before the server will 
> respond, and sometimes the server does not respond timely to a
REQUEST.
> Causing the clients to timeout and stop attempting.
> 
> We've reviewed the server's load, and its very, very light.  vmstat
doesn't
> show any bottlenecks in the processor or disk.  The current version we

> are running is 3.1.1;
> 
> As I stated, this all occurred within the last week.  Can anyone 
> recommend something to check?  Or is there a way to get greater 
> verbosity from the debug mode?
> 
> Many thanks in advance.
> 
> Kw

Have you checked out the hardware, network, etc? Maybe there is a
hardware fault. Consider the slow dhcp response to be a symtim rather
than the fault?

Could be a number of things:

- faulty network. try a different cable and/or switch port and/or nic

- disk errors. check logs for warnings or errors

- operating system patches. done any os patches around the time it went
bad?

- any other system changes at that time?

- rogue process using all available cpu or memory?

- did the number of clients increase that week?

regards,
-glenn

_______________________________________________
dhcp-users mailing list
dhcp-users at lists.isc.org
https://lists.isc.org/mailman/listinfo/dhcp-users

_______________________________________________
dhcp-users mailing list
dhcp-users at lists.isc.org
https://lists.isc.org/mailman/listinfo/dhcp-users

_______________________________________________
dhcp-users mailing list
dhcp-users at lists.isc.org
https://lists.isc.org/mailman/listinfo/dhcp-users

_______________________________________________
dhcp-users mailing list
dhcp-users at lists.isc.org
https://lists.isc.org/mailman/listinfo/dhcp-users

_______________________________________________
dhcp-users mailing list
dhcp-users at lists.isc.org
https://lists.isc.org/mailman/listinfo/dhcp-users

_______________________________________________
dhcp-users mailing list
dhcp-users at lists.isc.org
https://lists.isc.org/mailman/listinfo/dhcp-users

---------------------------------------------------------------------
This transmission (including any attachments) may contain confidential information, privileged material (including material protected by the solicitor-client or other applicable privileges), or constitute non-public information. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this transmission in error, please immediately reply to the sender and delete this information from your system. Use, dissemination, distribution, or reproduction of this transmission by unintended recipients is not authorized and may be unlawful.