No Free Addresses?

[Bob Proulx]

Simon Hobson wrote:
> Bob Proulx wrote:
> > > When the communication between them breaks they do not automatically
> >> assume the other server is down,
> >
> >Right.  It could be a network split.  Each server might continue to be
> >up and online on an isolated segment of the network.
> 
> More seriously, there are people running failover pairs with out of
> band communication - ie the server-server traffic follows a
> different route to server-client traffic. In this situation (and one
> or two other corner cases such as routing errors) it's possible to
> have both servers still able to serve clients ON THE SAME SUBNET
> while unable to communicate with each other. You wouldn't want
> either of them going into partner down state in that situation, so
> it's always been left to the admin to decide.

All in complete agreement.  I have no arguments with the way dhcpd
implements failover.  It all seems perfectly reasonable to me so far.
It seems a clever implementation and handles problems well enough.

> If you wanted it to be automatic, you could always script a "is
> partner there ? If not put myself into partner down state" process -
> external to the DHCP service itself.

I apparently haven't been clear in my words.  My bad.  Let me be
clear.  I *never* want to put a server into partner down.  I don't
want to do it automatically.  I don't want to do it manually.  I want
enough IP addresses in each pool so that each server can operate
independently and continue to support the network without manual
interaction needed when the other server has failed.

Putting a server into partner-down requires manual interaction.  That
negates a primary benefit of having redundant servers.  If failover
required manual interaction then it would be a less useful concept.
Fortunately I am pretty sure that isn't the case and failover actually
does work just fine regardless of what this thread would have someone
believe.

It was asked if I *had* put the server into partner-down.  I am sure
it was asked with the simple idea that it would increase the IP space
by reclaiming that given to the other half of the split and workaround
the no free lease problem.  I don't think it was suggested as a
solution to not having enough free IP addresses but just as workaround
to keep things moving.  That was a fine question.  But please don't
get the idea that it is a desirable state.  Needing partner-down is an
indication that you don't actually have redundancy.  It really is only
good as a workaround to not having enough IPs in the address pool.
But the better answer is to have enough IPs in the address pool to
begin with so that it isn't needed.

> > > it is your responsibility to tell the remaining server that the
> >> other is really dead, not just unable to communicate.
> >
> >It died today.  But it will be back tomorrow.  I am provisioning a new
> >server for it right now.  It isn't permanently gone.  It isn't even
> >going to be gone long term.
> 
> Doesn't matter - just put the remaining one into partner down state.

Sorry but no.  If you *need* to put a server into partner-down then
you do *not* have a redundant dhcpd server.  If you are operating in
that mode then you might as well not be using failover at all.

> When the other recovers (or is replaced) they will sort themselves
> out automatically.

Yes of course.  But that is unrelated.

> > > Until the server gets the information that its partner really is dead,
> >> it will not hand out leases belonging to the partner -> effectively you
> >> are missing half your address space until then.
> >
> >Correct.  That is why you need twice the available address pool.
> 
> You don't have to, it's a design decision you've made that imposes
> that.

Right.  I have made the design decision that I want the dhcp service
to keep working in the face of a single server failure.  I don't think
that is an unreasonable decision.  I am going to keep moving forward
to make sure it works.

> Running a network is always a matter of being pragmatic about
> these things - at least with RFC1918 addresses there are plenty to
> go around.

Of course there are an infinite number of numbers.  And with IPv6
there are even more. :-)  But /24 networks are very common and there
really aren't an infinite number of IP addresses available on every
subnet.  I already had 250 extra IP addresses available.  How many is
enough?  If 3.5 times as many isn't enough then how many is needed?
Are 10x needed?  Are 100x needed?  Are an infinite number needed?

> Another thought on this. If the OP doesn't want to deal with putting a
> failover server into partner down mode, and so configure both servers
> with enough addresses to serve all clients ...

That was exactly my original question.  I had both servers configured
with what should have been enough IP addresses.  There were 501
configured with 147 used.  Surely that should have been enough?  That
was the design, the goal and the implementation.  But it failed.
There weren't enough IP addresses.  It needed more than what seemed
like it should be needing.  I couldn't account for the loss.

> Then failover isn't actually going to offer much benefit. Two
> independent servers, each with enough addresses (in non-overlapping
> ranges) to serve all clients, would probably work just as well.

Two independent servers with enough addresses should operate very
similarly to two servers with enough addresses operating in a failover
mode.  And then there isn't a reason to stop at two.  You could use
three in that case.  But there are disadvantages too.  Obviously there
was enough disadvantages that the failover mode was written and it is
being used to advantage by many sites.  The major advantage is that in
failover mode it can continue renewing addresses for hosts that
previously aquired one on the failed server.  Active connections will
not get killed because the IP address won't change out from under it.

And in my case I believe that two independent servers with independent
pools would have failed in exactly the same way of not having enough
free IP addresses.  It is still the same dhcpd software whether it is
failover or not and when it thinks an address is not free to give out
then it will still think the address is not free to give out.

And so once again I am back to the question of how many are enough and
what did it do with the ones that it had.  Several people have posted
some good suggestions for monitoring software.  I am looking into
those.  Thanks to all who pointed those out.

Bob