Max number of subnets DHCP can handle

[David W. Hankins]

On Sat, Jan 26, 2008 at 02:44:33PM -0600, Chris Fabri wrote:
> standalone, 3.0.1 takes a while to load, 3.1.0 loads quickly.    But 
> when in failover, both DHCP processes start, but the primary appears to 
> simply spend all it's time calculating lease pool balance - the log just 
> scrolls through constantly with pool balance numbers, the secondary 
> server eventually gets a failover timeout.    The secondary will answer 
> and fulfill requests, but the primary doesn't.

that's a strange situation to be in.  it sounds like the secondary
thinks the pools are "panic unbalanced", but the primary does not
think it needs to transfer any leases.

i'd make sure the config files are consistent (including "failover
peer" lines in the pools), and then compare the post-failover lease
databases to see what's going wrong.  if anything were wrong, i would
probably wind up "faulting" one of the lease databases so that it is
reloaded off the peer.  since this sounds like a test enviroment,
maybe you could just cut to that chase and try that.  just shutdown
one daemon, delete dhcpd.leases and touch a new file, then restart the
daemon; it will load the db off the peer, enter a RECOVER-WAIT state
for MCLT, then resume normal operations (this isn't a bad test to
perform either).

3.0.1 also has several known failover flaws.  run 3.0.6 at least if
you want to evaluate 3.0.x.

this sort of "pool rebalance cpu starvation" due to the peer sending
never-ending POOLREQ messages has been addressed in our CVS repository
for the next release; we now implement a hold-down timer on POOLREQs.

from 3.1.next's RELNOTES;

- A bug in failover pool rebalancing that caused POOLREQ message ping-pongs
  was repaired.

- A flaw in failover pool rebalancing that could cause POOLREQ messages to
  be sent outside of the min-balance/max-balance scheduled intervals has
  been repaired.

but it's hard to say if the above would resolve your secondary's
condition that's causing it to send the POOLREQs in the first place;
e.g. any lease db inconsistency.

> In failover in 3.1.0, it also tries to pre-populate the leases file, but 
> it only gets through a couple subnets.    3.0.1 doesn't do this.    I 

pre whatsitwhosits?  in normal operation, "virgin" (never before used)
leases are not written to the lease db, to save time and space.  in
failover (of any version), some of these will change state to 'backup'
(to be allocated by the secondary), and so will be recorded in the
lease db.

this is normal in any version.

> have less.   I guess I'm looking for either some way to tweak the config 
> to deal with my situation, or an idea of what number of subnets is going 
> to cause the problem.   Load time is a concern, right now it's about 2 

essentially;

leases are ordered in memory on a hash table which (as of 3.1.0) is
not nearly so pessimal as 3.0.x.  several little missteps were fixed.
i think 4.0.x is supposed to be faster still, but i can't remember
why.

shared networks are ordered in memory on a linear list, and subnets
are attached on a linear list beneath them.  it is possible to find a
subnet from a lease, but for DISCOVER or SELECTING/INIT-REBOOT REQUEST
processing, these lists may be traversed in order to calculate the
client's point of network attachment (seeking a subnet that contains
the giaddr, no seek is necessary if giaddr is not set since the daemon
uses one socket per interface, and can just record what socket the
packet came in on).


so whereas lease lookup performance is (hopefully) O(1), although
realistically O(leases) on 3.0.x and sufficiently large numbers of
leases (it starts around 16k I think, from memory), subnet lookup
performance is O(subnets) always.

so for each lease you add, it's a no-op.  for each subnet you add,
you're using a teensy bit more CPU for this processing.

the question is whether or not you have enough cpu for the subnets you
want.

short of rewriting the sources (to do a quicker subnet lookup), there
is no workaround.

-- 
Ash bugud-gul durbatuluk agh burzum-ishi krimpatul.
Why settle for the lesser evil?	 https://secure.isc.org/store/t-shirt/
-- 
David W. Hankins	"If you don't do it right the first time,
Software Engineer		     you'll just have to do it again."
Internet Systems Consortium, Inc.		-- Jack T. Hankins