32.5M dhcpd.conf with 108000 classes is parsed for 40 minutes. Is this ok?

[Glenn Satchell]

I think the problem is with the subnet declaration:

subnet 0.0.0.0 netmask 0.0.0.0 {

I can't believe you're serving the whole IPv4 address space :)
100k classes with 1 IP each is only a few /16 subnets.

But seriously, I remember previous posts where allocating whole class A
or class B address spaces cause the hash table generators to go
balistic. So first step is to specify the subnet(s) correctly, and in
smaller sizes.

There was a series of recent posts (last couple of months?) about
tuning the sizes of hash tables to optimise for large ranges. I
recommend searching the archives and looking at those values too.

Next up you could make your matching statements simpler. On the
individual class it probably doesn't make much difference, but multiply
it by 100k times and it mounts up:

  binary-to-ascii(10, 8, "", suffix(option agent.circuit-id,1)) = "1"

can avoid an expensive binary-to-ascii and be written as

  suffix(option agent.circuit-id, 1) = 1

Note no quotes around the right hand side, as we're matching a binary 1
here, rather than a string; likewise

  binary-to-ascii(10, 8, ".", packet(24, 4)) = "10.102.100.100"
  
can be redone as a sequence of hex digits (I assume you have a script
to generate this file, hence hex shouldn't be a problem):

  packet(24, 4) = 0a:66:64:64

man dhcp-eval explains the syntaxes for strings or binary values.

Finally you have 38000 real IPs with a max lease time of 180 seconds.
Typical clients renew approximately half way through, so you need
enough grunt to handle around 400 requests per second.

You may want to look at the "spawn with" statement, which allows
classes to be generated dynamically. This may work a whole lot better
for you.

The overall number of IPs is not so large, there are many on this list
with 100k leases, but your way of managing them is overly complex.

regards,
-glenn

 
>Date: Fri, 15 May 2009 13:05:42 +0800
>Subject: 32.5M dhcpd.conf with 108000 classes is parsed for 40 minutes. Is  
this ok?
>From: michael kapelko <kornerr at gmail.com>
>To: dhcp-users at isc.org
>X-BeenThere: dhcp-users at lists.isc.org
>
>Hi.
>I have generated dhcpd.conf from DB with real classes (~38000) and
>faked ones (~70000, to test for user count growth) resulting in a
>config with ~108000 classes.
>It looks like this:
>
>ddns-update-style none;
>min-lease-time 120;
>default-lease-time 150;
>max-lease-time 180;
>local-address 10.102.101.13;
>subnet 0.0.0.0 netmask 0.0.0.0 {
>
>- - - - many lines - - - -
>
>class "10.102.100.100:1" {
>	match if (binary-to-ascii(10, 8, "", suffix(option agent.circuit-id,
>1)) = "1") and
>	         (binary-to-ascii(10, 8, ".", packet(24, 4)) = 
"10.102.100.100");
>}
>pool {
>	range 10.102.100.177;
>	option routers 10.102.100.1;
>	option subnet-mask 255.255.255.0;
>	allow members of "10.102.100.100:1";
>}
>
>class "10.102.100.100:2" {
>	match if (binary-to-ascii(10, 8, "", suffix(option agent.circuit-id,
>1)) = "2") and
>	         (binary-to-ascii(10, 8, ".", packet(24, 4)) = 
"10.102.100.100");
>}
>pool {
>	range 10.102.100.162;
>	option routers 10.102.100.1;
>	option subnet-mask 255.255.255.0;
>	allow members of "10.102.100.100:2";
>}
>
>class "10.102.100.100:3" {
>	match if (binary-to-ascii(10, 8, "", suffix(option agent.circuit-id,
>1)) = "3") and
>	         (binary-to-ascii(10, 8, ".", packet(24, 4)) = 
"10.102.100.100");
>}
>pool {
>	range 10.102.100.14;
>	option routers 10.102.100.1;
>	option subnet-mask 255.255.255.0;
>	allow members of "10.102.100.100:3";
>}
>
>class "10.102.100.100:4" {
>	match if (binary-to-ascii(10, 8, "", suffix(option agent.circuit-id,
>1)) = "4") and
>	         (binary-to-ascii(10, 8, ".", packet(24, 4)) = 
"10.102.100.100");
>}
>pool {
>	range 10.102.100.193;
>	option routers 10.102.100.1;
>	option subnet-mask 255.255.255.0;
>	allow members of "10.102.100.100:4";
>}
>
>class "10.102.100.100:48" {
>    match if (binary-to-ascii(10, 8, "", suffix(option
>agent.circuit-id, 1)) = "48") and
>             (binary-to-ascii(10, 8, ".", packet(24, 4)) = "10.102.100.100");
>}
>pool {
>    range 10.102.100.124;
>    range 10.102.100.125;
>    option routers 10.102.150.1;
>    option subnet-mask 255.255.255.0;
>    allow members of "10.102.100.100:48";
>}
>
>- - - - many lines - - - -
>
>}
>
>Class matches a rule when a DHCP message has giaddr field setup and
>Relay agent info option present. We want to use it for a network of
>D-Link's.
>The compressed dhcpd.conf can be downloaded from here [750K]:
>http://kornerr.alfamoon.com/download/dhcpd.conf.bz2
>I wonder if it's normal for such a config to be parsed for such a long
>time on [Core2Duo 1.8GHz / CentOS 5 Final / DHCP Server V3.0.5-RedHat]
>Thanks.
>_______________________________________________
>dhcp-users mailing list
>dhcp-users at lists.isc.org
>https://lists.isc.org/mailman/listinfo/dhcp-users