dhcp fails with big dhcpd.leases

[dorian]

sthaug at nethelp.no wrote:
>> I do not remember whole dhcp protocol. So I don't know what it is really
>> exchanged between client and server.
>> But according my -maybe naive consideration- the host should be able to
>> ask for a quite new IP without querying for assigning the "old" one.
>> Especially when it tries to get dhcp data connecting  totally fresh
>> network: there is no address to ask about.
>>     
>>> If there is no Request, then the server has nothing to Ack.
>>>       
>> Ok. I undestand - the DHCPACK is posted only when the host asks about
>> the IP address and the IP is confirmed.
>>     
>
> You need to understand the basic DHCP protocol. This is vitally important
> if you run a reasonably large DHCP installation. How many clients do you
> have?
>
> The basic message exchange (assuming the server has an address to offer):
>
> Client:	  Discover
> Server:	  Offer
> Client:	  Request
> Server:	  Ack
>
> *Later*, when a client wants to renew the same IP address it already  has,
> the typical sequence is:
>
> Client:   Request
> Server:   Ack
>
>   
Thanks a lot for explanation. It makes me to recall a little bit about dhcp.
I've learned the protocol years ago...
My problem with linux is that I am learning something, next I am
starting the service and - since it is working stable for a months - I
am forgetting the basis.
>>> But deleting the leases file DOES fundamentally break your server config.
>>>       
>> Well. So, was Sten Carlsen wrong writing " The leases file is a log file
>> - the server only ever appends to it, and during operations it never
>> reads from it." ?
>>     
>
> Sten Carlsen was right. The file is not read during normal *running*
> operation of the dhcpd process. However, the leases file is needed for
> *restarts* of the dhcpd process - this is when the DHCP server is
> expected to "regain" previous knowledge about leases. You are breaking
> this.
>
>   
I thought it worked that way.
And I am breaking this since there is the _only solution I have found_
to fix my problem and makes the dhcp server to serve dhcp data again.
>> Because if he was right deleting lease file (during server run time)
>> should not break the server - or there is a bug in the software as
>> writing to opened (i.e using file handle to) file  which has been
>> removed should be detected in the software.
>>     
>
> If you delete the leases file while the dhcpd process is running, I would
> expect the following to happen:
>
> - The space is not freed, because the dhcpd process still has the file
> open.
> - Each hour the dhcpd process rewrites the leases file - which in your
> case means that the file would appear to be recreated.
>
>   
>> I know that noone assumes such stupid user action but for the services
>> running 24/24 everything may happen - the file system can crash (or
>> whole HDD having the partition with this file can corrupt) so such
>> service should "behave correctly" and report the error in other way.
>>     
>
> The dhcpd process is based on having enough space for the leases file,
> and being able to *read* leases data from this file when it starts. You
> are fundamentally breaking this. As long as you keep breaking this, you
> will continue to have problems.
>   
As I wrote above.
The main reason I have to delete lease file is that the server _stops_
serving data.
An this is _the only solution I have found_.
Therefore I posted this thread to the mailing list - maybe there is
anybody who knows this software so deeply that could explain me _what is
the reason_ of the problem.

The server worked fine for around 1.5 month (continuously) and the lease
file size became bigger and bigger.
My observations are that when it became over some size the next clients
did not get the leases. This is the second time I've got dhcp server to
be dysfunctional.
And there was nothing unusual in the log file then.
Everything looked "normal" although clients could not to connect the
network.

I realize that "big" lease file  "translates" into the "big" dhcp server
internal structures in the RAM (or vice versa).
So probably if the structures became too large the server behaves
incorrectly (maybe the offers are sent too slow, maybe none is sent).

But I do not see server's internal structures - I can see only the size
of the lease file.
And if the lease file exceeds some size (it is very difficult for me to
state which is the exact 'critical' size) the server became dysfunctional.

Another matter (which could be ultimately accepted) is the restart time.
10 minutes for parsing the 893116-bytes text file (= 3893 lease entries)
looks very strange for me.
> There are lots of us running DHCP services 24 x 7 without problems. We
> do this by making sure that the DHCP servers have enough disk space, and
> by carefully monitoring the DHCP services. If you want to ensure that 
> you cannot run out of disk space for the leases file: Put the file on
> its own file system partition (and make sure that partition is several
> times as large as the largest lease file expected).
>
>   
please look below:

#ls -l /var/lib/dhcp
total 981
-rw-r--r-- 1 dhcp dhcp  50212 09-01 09:59 dhcpd.leases
-rw-r--r-- 1 root root 893116 08-31 15:33 dhcpd.leases-
-rw-r--r-- 1 dhcp dhcp  49584 09-01 09:58 dhcpd.leases~

notes:
the dhcpd.leases- is a copy of the dhcpd.leases which I've made before
removing dhcpd.leases
the dhcpd.leases is the newly created (by dhcp server) file after
previous one has been removed.

and

#df
....
/dev/sda6              4000000    151856   3848144   4% /var

I believe there is enough space on HDD?
> Steinar Haug, Nethelp consulting, sthaug at nethelp.no
>
>