[Kea-users] HA heartbeat communications failure
Darren Ankney
darren.ankney at gmail.com
Wed Dec 27 11:18:25 UTC 2023
Hi,
If I may ask, what version of Kea are you using? Some defaults have
changed across versions.
Thank you,
Darren Ankney
On Tue, Dec 26, 2023 at 4:31 PM CS <cs.temp.mail at gmail.com> wrote:
>
> >Please describe what you mean by "it doesn't work".
> I mean I get a pretty useless error: "Unable to connect to Kea Control Agent."
>
> > it might be be best to ask Men & Mice about "micetro" and how best to set things
> I will at some point, when I find a resource with them. But there are two players in this and since kea isn't behaving as expected like you, I and the docs said. I'm starting here.
>
> >It actually SHOULDN'T work
> That's my read on it too. But here's proof. The CA config for one server. It matches for the other server except certs and ip addresses obv.
>
> "Control-agent": {
> "http-host": "xxx.xx1.xxx.xxx",
> "trust-anchor": "Certificate_Autority.pem",
> "cert-file": "ca1_cert.pem",
> "key-file": "ca1_key.pem",
> "cert-required": true,
> "http-port": 8000,
> "authentication": {
> "type": "basic",
> "realm": "kea-control-agent",
> "clients": [{
> "user": "baduser",
> "password": "badpassword",
> }]
> },
>
> And the dhcp4 config, likewise only the small differences between the two servers
>
> "hooks-libraries": [{
> "library": "/usr/lib/x86_64-linux-gnu/kea/hooks/libdhcp_lease_cmds.so",
> "parameters": {}
> },{
> "library" : "/usr/lib/x86_64-linux-gnu/kea/hooks/libdhcp_ha.so",
> "parameters": {
> "high-availability": [{
> "this-server-name": "server1.org.org",
> "mode": "load-balancing",
> "heartbeat-delay": 10000,
> "max-response-delay": 60000,
> "max-ack-delay": 5000,
> "max-unacked-clients": 0,
> "require-client-certs": true,
> "trust-anchor": "Certificate_Autority.pem",
> "auto-failover": true,
>
> "peers": [{
> "name": "server1.org.org",
> "url": "http://xxx.xx1.xxx.xxx:8000/",
> "cert-file": "dhcp1_cert.pem",
> "key-file": "dhcp1_key.pem",
> "basic-auth-user": "baduser",
> "basic-auth-password": "badpassword",
> "role": "primary",
> },{
> "name": "server2.org.org",
> "url": "http://xxx.xx2.xxx.xxx:8000/",
> "cert-file": "dhcp2_cert.pem",
> "key-file": "dhcp2_key.pem",
> "role": "secondary",
> "basic-auth-user": "baduser",
> "basic-auth-password": "badpassword",
> }]
> }]
>
> low and behold it runs. The same nature of daemon status and logs on the other server.
>
> $ sudo systemctl restart isc-kea-ctrl-agent.service isc-kea-dhcp4-server.service
> $ sudo systemctl status isc-kea-ctrl-agent.service isc-kea-dhcp4-server.service
> ● isc-kea-ctrl-agent.service - Kea Control Agent
> Loaded: loaded (/lib/systemd/system/isc-kea-ctrl-agent.service; enabled; vendor preset: enabled)
> Active: active (running) since Tue 2023-12-26 20:57:29 UTC; 11s ago
> Docs: man:kea-ctrl-agent(8)
> Main PID: 1393724 (kea-ctrl-agent)
> Tasks: 5 (limit: 19052)
> Memory: 2.5M
> CPU: 26ms
> CGroup: /system.slice/isc-kea-ctrl-agent.service
> └─1393724 /usr/sbin/kea-ctrl-agent -c /etc/kea/kea-ctrl-agent.conf
>
> Dec 26 20:57:29 kea1 systemd[1]: Started Kea Control Agent.
>
> ● isc-kea-dhcp4-server.service - Kea DHCPv4 Service
> Loaded: loaded (/lib/systemd/system/isc-kea-dhcp4-server.service; enabled; vendor preset: enabled)
> Active: active (running) since Tue 2023-12-26 20:57:29 UTC; 11s ago
> Docs: man:kea-dhcp4(8)
> Main PID: 1393730 (kea-dhcp4)
> Tasks: 9 (limit: 19052)
> Memory: 4.5M
> CPU: 96ms
> CGroup: /system.slice/isc-kea-dhcp4-server.service
> └─1393730 /usr/sbin/kea-dhcp4 -c /etc/kea/kea-dhcp4.conf
>
> Dec 26 20:57:29 kea1 systemd[1]: isc-kea-dhcp4-server.service: Deactivated successfully.
> Dec 26 20:57:29 kea1 systemd[1]: Stopped Kea DHCPv4 Service.
> Dec 26 20:57:29 kea1 systemd[1]: isc-kea-dhcp4-server.service: Consumed 1min 28.504s CPU time.
> Dec 26 20:57:29 kea1 systemd[1]: Started Kea DHCPv4 Service.
>
> $ tail -n10 /var/log/kea/kea-ctrl-agent.log
> 2023-12-26 20:59:53.827 INFO [kea-ctrl-agent.ctrl-agent/1393724] CTRL_AGENT_COMMAND_RECEIVED command ha-heartbeat received from remote address xxx.xxx2.xxx.xxx
> 2023-12-26 20:59:53.828 INFO [kea-ctrl-agent.ctrl-agent/1393724] CTRL_AGENT_COMMAND_FORWARDED command ha-heartbeat successfully forwarded to the service dhcp4 from remote address xxx.xx2.xxx.xxx
> 2023-12-26 21:00:03.843 INFO [kea-ctrl-agent.auth/1393724] HTTP_CLIENT_REQUEST_AUTHORIZED received HTTP request authorized for 'baduser'
> 2023-12-26 21:00:03.843 INFO [kea-ctrl-agent.commands/1393724] COMMAND_RECEIVED Received command 'ha-heartbeat'
> 2023-12-26 21:00:03.843 INFO [kea-ctrl-agent.ctrl-agent/1393724] CTRL_AGENT_COMMAND_RECEIVED command ha-heartbeat received from remote address xxx.xxx2.xxx.xxx
> 2023-12-26 21:00:03.844 INFO [kea-ctrl-agent.ctrl-agent/1393724] CTRL_AGENT_COMMAND_FORWARDED command ha-heartbeat successfully forwarded to the service dhcp4 from remote address xxx.xxx2.xxx.xxx
> 2023-12-26 21:00:13.859 INFO [kea-ctrl-agent.auth/1393724] HTTP_CLIENT_REQUEST_AUTHORIZED received HTTP request authorized for 'baduser'
> 2023-12-26 21:00:13.859 INFO [kea-ctrl-agent.commands/1393724] COMMAND_RECEIVED Received command 'ha-heartbeat'
> 2023-12-26 21:00:13.859 INFO [kea-ctrl-agent.ctrl-agent/1393724] CTRL_AGENT_COMMAND_RECEIVED command ha-heartbeat received from remote address xxx.xxx2.xxx.xxx
> 2023-12-26 21:00:13.860 INFO [kea-ctrl-agent.ctrl-agent/1393724] CTRL_AGENT_COMMAND_FORWARDED command ha-heartbeat successfully forwarded to the service dhcp4 from remote address xxx.xxx2.xxx.xxx
> $ tail -n10 /var/log/kea/kea-dhcp4.log
> 2023-12-26 20:58:53.728 INFO [kea-dhcp4.commands/1393730] COMMAND_RECEIVED Received command 'ha-heartbeat'
> 2023-12-26 20:59:03.745 INFO [kea-dhcp4.commands/1393730] COMMAND_RECEIVED Received command 'ha-heartbeat'
> 2023-12-26 20:59:13.762 INFO [kea-dhcp4.commands/1393730] COMMAND_RECEIVED Received command 'ha-heartbeat'
> 2023-12-26 20:59:23.777 INFO [kea-dhcp4.commands/1393730] COMMAND_RECEIVED Received command 'ha-heartbeat'
> 2023-12-26 20:59:33.793 INFO [kea-dhcp4.commands/1393730] COMMAND_RECEIVED Received command 'ha-heartbeat'
> 2023-12-26 20:59:43.811 INFO [kea-dhcp4.commands/1393730] COMMAND_RECEIVED Received command 'ha-heartbeat'
> 2023-12-26 20:59:53.827 INFO [kea-dhcp4.commands/1393730] COMMAND_RECEIVED Received command 'ha-heartbeat'
> 2023-12-26 21:00:03.844 INFO [kea-dhcp4.commands/1393730] COMMAND_RECEIVED Received command 'ha-heartbeat'
> 2023-12-26 21:00:13.859 INFO [kea-dhcp4.commands/1393730] COMMAND_RECEIVED Received command 'ha-heartbeat'
> 2023-12-26 21:00:23.875 INFO [kea-dhcp4.commands/1393730] COMMAND_RECEIVED Received command 'ha-heartbeat'
>
> And changing the CA or the server HA paramersts to port 8001 without changing the other (and the other server results in "connection refused" logs. It obv wants the CA port to match the HA parameters port despite what we and the documentation suggests...
>
> CS, cs.Temp.Mail at gMail.com
>
>
> On Mon, 25 Dec 2023 at 02:45, Darren Ankney <darren.ankney at gmail.com> wrote:
>>
>> Hi,
>>
>> It actually SHOULDN'T work to set your control agent and
>> multi-threaded HA listener to the same port as only one of the
>> applications should be able to setup a listener on that port. Please
>> describe what you mean by "it doesn't work". I'm thinking it might be
>> be best to ask Men & Mice about "micetro" and how best to set things
>> up there.
>>
>> Thank you,
>>
>> Darren Ankney
>>
>> On Thu, Dec 21, 2023 at 6:47 PM CS <cs.temp.mail at gmail.com> wrote:
>> >
>> > Hi all,
>> > Moving on from my failure to start and logging issues (thank you for your help btw!) I now don't have my heartbeat/control_agent working correctly.
>> >
>> > It works fine so long as I set the ports of my control agents and ha hook parameters to be the same (IE 8000 or 8001)
>> >
>> > However I am unable to tie the tiny cluster into micetro, probably because the CA port is occupied with HA heartbeats?
>> >
>> > Looking to these examples:
>> > https://github.com/isc-projects/kea/tree/master/doc/examples/template-ha-mt-tls
>> >
>> > Documentation points out
>> > //This specifies the port CA will listen on.
>> > // If enabling HA and multi-threading, the 8000 port is used by the HA
>> > // hook library http listener. When using HA hook library with
>> > // multi-threading to function, make sure the port used by dedicated
>> > // listener is different (e.g. 8001) than the one used by CA. Note
>> > // the commands should still be sent via CA. The dedicated listener
>> > // is specifically for HA updates only.
>> >
>> > However, how to have a dedicated port for HA and a different one for CA escapes me.
>> >
>> > CS, cs.Temp.Mail at gMail.com
>> > --
>> > ISC funds the development of this software with paid support subscriptions. Contact us at https://www.isc.org/contact/ for more information.
>> >
>> > To unsubscribe visit https://lists.isc.org/mailman/listinfo/kea-users.
>> >
>> > Kea-users mailing list
>> > Kea-users at lists.isc.org
>> > https://lists.isc.org/mailman/listinfo/kea-users
>> --
>> ISC funds the development of this software with paid support subscriptions. Contact us at https://www.isc.org/contact/ for more information.
>>
>> To unsubscribe visit https://lists.isc.org/mailman/listinfo/kea-users.
>>
>> Kea-users mailing list
>> Kea-users at lists.isc.org
>> https://lists.isc.org/mailman/listinfo/kea-users
More information about the Kea-users
mailing list