How Somebody Helped Kill dhcpd on Our Network

[Darren]

Martin,

I suggest Vlans.  Set up as many Vlans as you can and separate the user 
traffic.  The rogue DHCP server may not have been malicious in nature.  
All someone has to do is hook a linksys router up backwards and this 
will happen.  Its worse when all 10,000 users are in the same broadcast 
domain.  If it was one vlan of 253 users or something, the damage would 
have been minimal.  The rogue DHCP server would not have seen the vast 
majority of user traffic, and therefore would not have been able to 
respond to it with DHCPNAK.


Martin McCormick wrote:
> 	We recently had our dhcp V3.0.3 system crash or, I should
> say, crashed by a denial-of-service scheme in which the miscreant
> ran something on his/her computer that behaved like a dhcp server
> enough to send DHCPNAK's to every dhcp request it saw on the
> VLAN.  This made every client send more traffic to the real dhcp
> server and apparently caused the 1-gig processor with 1 gig of
> RAM to consume all available memory.  For 3 minutes, it generated
> about 2,500 "out of memory" messages and then dhcpd finally
> exited with a 11) SIGSEG and a core dump.  Good old FreeBSD unix
> is pretty bullet-proof, but this was more than the box could
> handle.  The platform continued to operate properly afterwards,
> but dhcpd had to be restarted again.
>
> 	We are in the process of instituting dhcp failover with a
> second server although I suspect that this situation would have
> added maybe another few minutes to the mayhem before it, to,
> succumbed to the hammering since failover isn't going to protect
> against external attacks that hit both servers equally.
>
> 	We are also installing switches with port snooping
> capabilities as funds permit to kill off anybody who aspires to
> run dhcpd on anything other than the proper dhcp server.  This
> brings me to my question.
>
> 	Is there any particular configuration parameter I can use
> in dhcpd to make it rate-limit itself since it would have been a
> much better outcome for it to come up for air, so to speak,
> rather than crash and have to be restarted.
>
> 	This particular event occurred on a Sunday afternoon on
> the weekend that classes finished for the Summer semester and
> campus activity was as near to dead as it ever gets around here
> so we didn't even know anything was wrong for a period of time.
>
> 	Our dhcp server gives service to around 10,000 clients of
> several types and the platform it is on hardly breaks a sweat
> even on the busiest days, but this was too much to handle.
>
> Martin McCormick WB5AGZ  Stillwater, OK 
> Systems Engineer
> OSU Information Technology Department Network Operations Group
>