Strange Issue with Linksys routers

Mon May 18 14:53:55 UTC 2009

You should see what your DHCP relays are doing with those packets....you
shouldn't be getting 4.

Frank

-----Original Message-----
From: dhcp-users-bounces at lists.isc.org
[mailto:dhcp-users-bounces at lists.isc.org] On Behalf Of Ben Wiechman
Sent: Monday, May 18, 2009 9:00 AM
To: Users of ISC DHCP
Subject: RE: Strange Issue with Linksys routers

I must have missed those discussions. 

It was a long week. But speaking of Belkins we found a couple of those in
the network that appeared to be duplicating unicast DHCP requests. Strangest
thing. We were trying to get to the bottom of this issue and doing captures
at an affected customers location and in the data center. We'd see a single
unicast request at the customer location, but 4 in the data center and the
customer obviously got four responses. Doing a packet capture on the network
segment showed a single request with the source mac of the customer's
Linksys router, and 4 duplicate requests from different Belkin routers with
identical information with the exception of the source mac address in the IP
headers. Goofy. 

Ben Wiechman
Network Administrator

> -----Original Message-----
> From: Frank Bulk [mailto:frnkblk at iname.com]
> Sent: Sunday, May 17, 2009 8:49 PM
> To: Ben Wiechman; Users of ISC DHCP
> Subject: RE: Strange Issue with Linksys routers
> 
> Good point -- we've seen brownouts do the same.  Almost always get a rash
of
> helpdesk calls after those.  I would prefer the power company just do a
> 5-second salute. =)
> 
> It has been previously discussed, but it's worth mentioning again the
Belkin
> DHCP issue.  We dealt with most of them a few months ago but still see
them
> pop up once in a while.  Again, a firmware upgrade fixes most of those.
> 
> Sometimes I joke that the SOHO vendors ought to be paying us to do their
> tech support.
> 
> Frank
> 
> -----Original Message-----
> From: dhcp-users-bounces at lists.isc.org
> [mailto:dhcp-users-bounces at lists.isc.org] On Behalf Of Ben Wiechman
> Sent: Sunday, May 17, 2009 8:42 PM
> To: Users of ISC DHCP
> Subject: RE: Strange Issue with Linksys routers
> 
> We've seen similar issues in the past after lightning storms where the
> routers seem to be affected by a brownout or other power issue. Typically
a
> reboot seems to cure those issues. In this case these routers are
scattered
> across several separate power companies and 50-60 miles of geography.
> Rebooting the router does not clear up the issue. Upgrading the firmware
> seems to be the only real solution that has a long term affect.
> 
> Ben Wiechman
> Network Administrator
> 
> > -----Original Message-----
> > From: dhcp-users-bounces at lists.isc.org [mailto:dhcp-users-
> > bounces at lists.isc.org] On Behalf Of Frank Bulk
> > Sent: Sunday, May 17, 2009 8:17 PM
> > To: Users of ISC DHCP
> > Subject: RE: Strange Issue with Linksys routers
> >
> > Ben:
> >
> > We have hundreds of WRT54G's attached to cable modems and not noticed
this
> > issue.  Any chance that there was a common powering/lightning issue that
> > affected these routers' power supplies?  We have seen several cases
where
> a
> > Linksys router acts marginally with a bad power supply.
> >
> > Frank
> >
> > -----Original Message-----
> > From: dhcp-users-bounces at lists.isc.org
> > [mailto:dhcp-users-bounces at lists.isc.org] On Behalf Of Ben Wiechman
> > Sent: Saturday, May 16, 2009 5:21 PM
> > To: Users of ISC DHCP
> > Subject: Strange Issue with Linksys routers
> >
> > This doesn't appear to be an issue that is specific to ISC, but since
this
> > is most readily visible in the dhcp logs maybe someone else here has
seen
> or
> > is seeing this issue as well.
> >
> > We have a number of subscribers on our network (75-100 - possibly more)
> that
> > are running WRT54G/GS routers with older firmware (1.00.6/7, etc.) with
> > ethernet controllers that appear to lock up. When the router is power
> cycled
> > they tend to work properly for several hours and then the ethernet
> > controller appears to lock up again. This keeps repeating.
> >
> > When the condition exists any computers attached to the wired interfaces
> > will eventually report limited or no connectivity. Any traffic sent to
the
> > router on the LAN ports receives no response. However it is possible to
> > connect to the routers using the wireless interface and access the web
> > management interface. With remote management enabled and pings allowed
> they
> > will not respond on the WAN interface when this condition exists. We
have
> > done packet captures to verify that the icmp/ip packets are being
> delivered
> > to the WAN interface, however the router generates no response.
> >
> > This is where it gets weird. The router will continue to send
> > DHCPREQUEST/DHCPOFFER packets but does not appear to receive the
response.
> > This is how we initially noticed the issue. Large numbers of routers
were
> > hammering our dhcp server hundreds of times every hour with DHCPDISCOVER
> > broadcasts.
> >
> > When the router is power cycled it will broadcast a DHCPDISCOVER packet
to
> > the dhcp server, receive the offer, broadcast the request and receive
the
> > ack. Our default lease time is 12 hours. Normally the router would send
a
> > unicast DHCPREQUEST to the server half way through the lease time and
> > receives a unicast DHCPACK. Under normal conditions this would simply
> > repeat. Here the ethernet controller appears to lock up at some point.
So
> > the router will send the DHCPREQUEST packet at the midway point, then
with
> > increasing frequency as the end of the lease period nears. Doing a
packet
> > capture will show the DHCPACK is received at the WAN interface of the
> > router. In the last couple of minutes before the lease expires the
router
> > will broadcast a series of DHCPREQUEST packets and receive broadcast
> > responses from the server. Once the dhcp lease expires the router will
> > continue to broadcast a series of DHCPDISCOVER messages every minute or
so
> > and receives the DHCPOFFERs in return. This will repeat until the router
> is
> > power cycled. Even if the ethernet ports are disconnected the lost link
is
> > not detected.
> >
> > The dicover/offer cycle can be tripped by logging into the router via
the
> > wireless interface and changing the hostname. This causes the router to
> send
> > a DHCPRELEASE request, followed by a DHCPDISCOVER. It receives a
> DHCPOFFER,
> > however does not appear to process the offer and once again enters a
loop
> > where it continues to broadcast a series of DHCPDISCOVER packets every
> > minute or so.
> >
> > We have not seen this on WRT54GL routers to this point. It appears to
have
> > begun at a very defined point on Monday 5/11. Is this some new exploit?
We
> > have yet to track down anything that appears to trigger this condition.
> >
> > Has anyone seen anything like this in the past?
> >
> > Ben Wiechman
> > Network Administrator
> > Wisper High Speed Internet
> >
> >
> >
> >
> > _______________________________________________
> > dhcp-users mailing list
> > dhcp-users at lists.isc.org
> > https://lists.isc.org/mailman/listinfo/dhcp-users
> >
> > _______________________________________________
> > dhcp-users mailing list
> > dhcp-users at lists.isc.org
> > https://lists.isc.org/mailman/listinfo/dhcp-users
> >
> 
> 
> 
> _______________________________________________
> dhcp-users mailing list
> dhcp-users at lists.isc.org
> https://lists.isc.org/mailman/listinfo/dhcp-users
> 

_______________________________________________
dhcp-users mailing list
dhcp-users at lists.isc.org
https://lists.isc.org/mailman/listinfo/dhcp-users