DHCP server assigned its own address

[Larry Apolonio]

All,

I have a weird problem that I am trying to solve.

In short, for those who don't want to read the details, I am trying to 
figure out why the DHCP server assigned its own IP address to another 
device.


My dhcp server is running on CentOS 6.10 and is the regular RPM that 
comes with that distribution dhcp-4.1.1-63.P1.el6.centos.x86_64.

What is a little unusual is that webmin is used to manage the dhcp 
server, for the most part it works for our environment.

Yesterday, I got a nagios alert that the server was no longer available. 
  This nagios server is on the same subnet as the server so there was no 
weird firewall routing issues involved.  With the help of the networking 
guys, we found that another machine took the IP address of our DHCP 
server.  This happened late July this year and it ended up being a human 
error, the person spinning up a machine on this network assigned a 
static IP address to their machine that was the same IP as our server, 
so we thought someone did it again.

The difference this time is that it seems like the DHCP server itself 
assigned its own IP address

Here is a sample of that subnet declaration, with IPs changed to protect 
the innocent

# XXXXXX Subnet
subnet 192.168.11.0 netmask 255.255.255.0 {
         range 192.168.11.10 10.254.11.10;
         option subnet-mask 255.255.255.0;
         default-lease-time 28800;
         option broadcast-address 192.168.11.255;
         option routers 192.168.11.254;
         option domain-name-servers 208.67.222.222 , 208.67.220.220;
         option domain-name "example.local";
         }

The IP address of the DHCP server is 192.168.11.10, I personally would 
not do this, I would have not even had the DHCP server IP address in 
that range.  But please read on

This is a rarely used subnet, so a machine appearing on this subnet is 
rare, in fact I thought this subnet did not have a dhcp declaration 
prior to me looking in to it.  Doesn't this log entry in 
/var/log/messages confirm it? (hostname was changed in this paste)

Sep 12 10:02:12 linuxdhcpserver dhcpd: No subnet declaration for eth0 
(no IPv4 addresses).
Sep 12 10:02:12 linuxdhcpserver dhcpd: ** Ignoring requests on eth0.  If 
this is not what
Sep 12 10:02:12 linuxdhcpserver dhcpd:    you want, please write a 
subnet declaration
Sep 12 10:02:12 linuxdhcpserver dhcpd:    in your dhcpd.conf file for 
the network segment
Sep 12 10:02:12 linuxdhcpserver dhcpd:    to which interface eth0 is 
attached. **

When the service was restarted 3 hours later, that same message about no 
subnet declaration for eth0 did not appear.

One reason we use webmin is so that non-linux folk (AKA people without 
the root password) can log in to an easy web interface is to manage the 
service that the Linux server does, in this case dhcp.

But it also logs what they did, up to a certain point, I can tell who 
edited which subnet declarations but not the exact changes they did.

 From the webmin logs, until yesterday this subnet was not changed.

 From the command line I also ran last to see who logged in, it was 
either root, or a proper Linux server admin, and I admit that someone in 
this group could be holding back, I don't think we did anything via CLI.

So I am at a loss, trying to figure out why a DHCP server would assign 
its own IP address (it is pingable, no iptables rules blocking ICMP), I 
thought conflict resolution would prevent it. If I am reading RFC1541 
section 2.2 correctly.

Did someone do a good job at cleaning up their tracks?  I don't think 
the effort or skill was there.  It would be easier to just admit they 
made a mistake.

Was webmin not logging correctly?  I really dont recall this subnet 
being on this server, because I do recall seeing that message in the 
logs regarding no subnet declaration in the past.

Couple solutions were proposed so this would not happen again, the 
biggest one is putting this server and its big brother nagios server on 
its lonesome VLAN/subnet and restrict anything else from being on this 
subnet.  Seems overkill but this IP hijack happened twice within 60 days 
when it has been fine for years.

Thank you,

Larry Apolonio

Although I have been speaking English for a while now, I still have 
problems articulating my thoughts, thank you for your patience.