[Kea-users] HA lease updates not working after upgrade to 1.8

[Christian Schuldt]

Hey,

I updated one of our Kea hot-standby HA pairs from 1.6 to 1.8 and now
HA lease-updates to the standby server no longer work.
The only config change was the required move of the logging section
into the dhcp4-server section.

Error on primary:
2020-11-04 13:48:48.754 INFO  [kea-dhcp4.leases/3244.139646677208128]
DHCP4_LEASE_ALLOC [hwtype=1 00:01:02:81:41:ed], cid=[no info],
tid=0x1ec6987e: lease 192.168.100.211 has been allocated for 86400
seconds
2020-11-04 13:48:48.760 WARN  [kea-dhcp4.ha-hooks/3244.139646677208128]
HA_LEASE_UPDATE_FAILED [hwtype=1 00:01:02:81:41:ed], cid=[no info],
tid=0x1ec6987e: lease update to ns2-kea (http://10.1.0.254:8080/)
failed: unable to update lease for address 192.168.100.211 as it does
not exist, error code 1

Error on standby:
2020-11-04 13:48:48.759 ERROR
[kea-dhcp4.callouts/15648.140137833094208] HOOKS_CALLOUT_ERROR error
returned by callout on hook 1 registered by library with index
$lease4_update (callout address 0x7f74577f0d40) (callout duration 1.262
ms)

I already purged both kea and mariadb (leases and reservations) and 
thenreinstalled, syncing reservations and then trying again with no
change.

I previously tried to manually update a lease on the standby with a
query extracted from traffic which also doesn't work. The error seems to
originate from the lease_commands hooks:
curl -XPOST http://localhost:8080 -H  "Content-Type: application/json"
\
  -d '{ "arguments": { "expire": 1604579347, "force-create": true,
"fqdn-fwd": false, "fqdn-rev": false, "hostname": "pc.", "hw-address":
"00:01:02:73:93:b7", "ip-address": "192.168.210.148", "state": 0,
"subnet-id": 124, "valid-lft": 86400 }, "command": "lease4-update",
"service": [ "dhcp4" ] }'
[ { "result": 1, "text": "unable to update lease for address
192.168.210.148 as it does not exist" } ]

However when immediately trying "lease4-add" instead of update, it
fails with "IPv4 lease already exists."
A manual update also doesn't work on the primary.

When deleting the leases4 table contents on the standby server, initial
HA lease synchronization runs without issues, but afterwards updates
don't work. Restarting after initial synchronization causes the same
lease update errors as during "normal" operation.

The servers configuration is the same (synchronised via git) with the
necessary exceptions of interfaces and HA config. Subnets have fixed IDs
for database reservations.
HA config only differs in "this-server-name".

Hooks config:

"hooks-libraries": [
  {
    #" lease cmds are required for ha",
    "library":
"/usr/lib/x86_64-linux-gnu/kea/hooks/libdhcp_lease_cmds.so",
    "parameters": { }
  },
  {
    "library": "/usr/lib/x86_64-linux-gnu/kea/hooks/libdhcp_ha.so",
    "parameters": {
	  "high-availability": [
	    {
		  "this-server-name": "ns2-kea",
		  "mode": "hot-standby",
		  "heartbeat-delay": 4000,
		  "max-response-delay": 15000,
		  "max-ack-delay": 5000,
		  "max-unacked-clients": 0,
		  "sync-page-limit": 5000,
		  "sync-timeout": 30000,
		  "peers": [
		    {
			  "name": "ns1-kea",
			  "url": "http://10.1.0.253:8080/",
			  "role": "primary",
			  #"role": "standby",
			  "auto-failover": true
		    },
		    {
			  "name": "ns2-kea",
			  "url": "http://10.1.0.254:8080/",
			  #"role": "primary",
			  "role": "standby",
			  "auto-failover": true
		    }
		  ]
	    }
	  ]
    }
  }
]

Any help would be appreciated.

Best Regards
Christian Schuldt


www.studiofunk.de  


This email may contain material that is confidential, privileged and/or
attorney work product for the sole use of the intended recipient. Any
review, reliance or distribution by others or forwarding without express
permission is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.  Studio Funk
GmbH & Co. KG · Geschäftsführer: Klaus Funk, Markus Weber · Sitz der
Gesellschaft: Hamburg · Handelsregister-Nr.: 67553


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.isc.org/pipermail/kea-users/attachments/20201104/5fd723ff/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: IMAGE.gif
Type: image/gif
Size: 2107 bytes
Desc: CompuServe GIF graphic
URL: <https://lists.isc.org/pipermail/kea-users/attachments/20201104/5fd723ff/attachment.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: IMAGE.gif
Type: image/gif
Size: 2545 bytes
Desc: CompuServe GIF graphic
URL: <https://lists.isc.org/pipermail/kea-users/attachments/20201104/5fd723ff/attachment-0001.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: IMAGE.gif
Type: image/gif
Size: 758 bytes
Desc: CompuServe GIF graphic
URL: <https://lists.isc.org/pipermail/kea-users/attachments/20201104/5fd723ff/attachment-0002.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: IMAGE.gif
Type: image/gif
Size: 733 bytes
Desc: CompuServe GIF graphic
URL: <https://lists.isc.org/pipermail/kea-users/attachments/20201104/5fd723ff/attachment-0003.gif>