[Kea-users] Weird stats from a shared database

[Tomek Mrugalski]

W dniu 12.12.2017 o 15:48, Munroe Sollog pisze:
> Let me know if this should be a bug, but I have noticed some weird stats
> when running two kea-1.3 DHCP servers from the same mysql database
> (using Galera).
> 
> I have included a screen grab of the stats.
> 
> Between noon and 2pm yesterday we when I migrated rogi from the memfile
> to the mysql database.  I migrated all of the existing leases from the
> CSV to mysql and started rogi.
> 
> Igor from around 2:30pm until about 7:45am the next day it steadily
> declines all the way to -436 leases.  How can it possibly have
> *negative* leases?
Rasmus is right. Running more than one Kea server using the same
database is not officially supported.

Here's what is likely to happen: each Kea instance allocates leases to
clients. For each allocation, the statistic is increased. The statistic
is observed on each instance. It is likely to be incorrect as there is
another instance that also allocates leases.

Now, unless you took extra steps to disable lease expiration on one
instance and keep it running on another, there are two instances
periodically looking for leases that are expired. Depending on how many
leases are expired during exact moment when the expiration triggers, one
server may get more expired leases to process than the other. Only that
server will decrease the statistic.

Finally, I don't know how you set this up, but I presume that the server
that allocated a lease will send its own server-id and thus the release
messages will be processed only by that server. So this shouldn't
contribute to the confusion, unless you did some clever things with
server-id.

You may perceive it as a bug. It's a valid point of view. But I see it
as Kea being run in a configuration that is not officially supported.
There's nothing wrong with it. We're happy it provides service and
generally works. It's just there are quirks like this.

We do have recountLeaseStats4 and recountLeaseStats6 method, but it is
only used internally. I suppose we could expose it as a command that you
could call. Kea instance would then consult the database and recalculate
the values.

As Rasmus mentioned, we do plan to improve the situation significantly
in 1.4. We want to provide a high availability solution, but also
improve many aspects of running multiple Kea servers at the same time.

I don't have any specific solution for you right now, just some things
to consider. Kea doesn't have any notion (at least not yet) of a server
instance owning a lease. You could try generating the statistic by
pooling both servers and adding the values together. Consider it an
experiment. It may or may not work. I'd love to hear about the results.

I'd like to ask you a favour. Can you describe how you did set up Galera
for MySQL on kea wiki? There are installation instructions here:
http://kea.isc.org/wiki/Install I was thinking about something similar,
but with detailed instructions how to set up Galera cluster. This would
be useful for two reasons. First, other users could set it up in similar
fashion. Second, one of ISC engineers will get to look at this problem
one day. It will be very helpful to have an instruction to replicate
your environment.

Finally, can you submit a bug for this? It would great if this bug
report had a link to the installation instruction.

Hope that helps,
Tomek