[Kea-users] HA heartbeat communications failure
CS
cs.temp.mail at gmail.com
Wed Dec 27 12:16:18 UTC 2023
Kea 2.4.0
On Wed, Dec 27, 2023, 03:18 Darren Ankney <darren.ankney at gmail.com> wrote:
> Hi,
>
> If I may ask, what version of Kea are you using? Some defaults have
> changed across versions.
>
> Thank you,
>
> Darren Ankney
>
> On Tue, Dec 26, 2023 at 4:31 PM CS <cs.temp.mail at gmail.com> wrote:
> >
> > >Please describe what you mean by "it doesn't work".
> > I mean I get a pretty useless error: "Unable to connect to Kea Control
> Agent."
> >
> > > it might be be best to ask Men & Mice about "micetro" and how best to
> set things
> > I will at some point, when I find a resource with them. But there are
> two players in this and since kea isn't behaving as expected like you, I
> and the docs said. I'm starting here.
> >
> > >It actually SHOULDN'T work
> > That's my read on it too. But here's proof. The CA config for one
> server. It matches for the other server except certs and ip addresses obv.
> >
> > "Control-agent": {
> > "http-host": "xxx.xx1.xxx.xxx",
> > "trust-anchor": "Certificate_Autority.pem",
> > "cert-file": "ca1_cert.pem",
> > "key-file": "ca1_key.pem",
> > "cert-required": true,
> > "http-port": 8000,
> > "authentication": {
> > "type": "basic",
> > "realm": "kea-control-agent",
> > "clients": [{
> > "user": "baduser",
> > "password": "badpassword",
> > }]
> > },
> >
> > And the dhcp4 config, likewise only the small differences between the
> two servers
> >
> > "hooks-libraries": [{
> > "library":
> "/usr/lib/x86_64-linux-gnu/kea/hooks/libdhcp_lease_cmds.so",
> > "parameters": {}
> > },{
> > "library" :
> "/usr/lib/x86_64-linux-gnu/kea/hooks/libdhcp_ha.so",
> > "parameters": {
> > "high-availability": [{
> > "this-server-name": "
> server1.org.org",
> > "mode": "load-balancing",
> > "heartbeat-delay": 10000,
> > "max-response-delay": 60000,
> > "max-ack-delay": 5000,
> > "max-unacked-clients": 0,
> > "require-client-certs": true,
> > "trust-anchor":
> "Certificate_Autority.pem",
> > "auto-failover": true,
> >
> > "peers": [{
> > "name": "server1.org.org
> ",
> > "url": "
> http://xxx.xx1.xxx.xxx:8000/",
> > "cert-file":
> "dhcp1_cert.pem",
> > "key-file":
> "dhcp1_key.pem",
> > "basic-auth-user":
> "baduser",
> > "basic-auth-password":
> "badpassword",
> > "role": "primary",
> > },{
> > "name": "server2.org.org
> ",
> > "url": "
> http://xxx.xx2.xxx.xxx:8000/",
> > "cert-file":
> "dhcp2_cert.pem",
> > "key-file":
> "dhcp2_key.pem",
> > "role": "secondary",
> > "basic-auth-user":
> "baduser",
> > "basic-auth-password":
> "badpassword",
> > }]
> > }]
> >
> > low and behold it runs. The same nature of daemon status and logs on the
> other server.
> >
> > $ sudo systemctl restart isc-kea-ctrl-agent.service
> isc-kea-dhcp4-server.service
> > $ sudo systemctl status isc-kea-ctrl-agent.service
> isc-kea-dhcp4-server.service
> > ● isc-kea-ctrl-agent.service - Kea Control Agent
> > Loaded: loaded (/lib/systemd/system/isc-kea-ctrl-agent.service;
> enabled; vendor preset: enabled)
> > Active: active (running) since Tue 2023-12-26 20:57:29 UTC; 11s ago
> > Docs: man:kea-ctrl-agent(8)
> > Main PID: 1393724 (kea-ctrl-agent)
> > Tasks: 5 (limit: 19052)
> > Memory: 2.5M
> > CPU: 26ms
> > CGroup: /system.slice/isc-kea-ctrl-agent.service
> > └─1393724 /usr/sbin/kea-ctrl-agent -c
> /etc/kea/kea-ctrl-agent.conf
> >
> > Dec 26 20:57:29 kea1 systemd[1]: Started Kea Control Agent.
> >
> > ● isc-kea-dhcp4-server.service - Kea DHCPv4 Service
> > Loaded: loaded (/lib/systemd/system/isc-kea-dhcp4-server.service;
> enabled; vendor preset: enabled)
> > Active: active (running) since Tue 2023-12-26 20:57:29 UTC; 11s ago
> > Docs: man:kea-dhcp4(8)
> > Main PID: 1393730 (kea-dhcp4)
> > Tasks: 9 (limit: 19052)
> > Memory: 4.5M
> > CPU: 96ms
> > CGroup: /system.slice/isc-kea-dhcp4-server.service
> > └─1393730 /usr/sbin/kea-dhcp4 -c /etc/kea/kea-dhcp4.conf
> >
> > Dec 26 20:57:29 kea1 systemd[1]: isc-kea-dhcp4-server.service:
> Deactivated successfully.
> > Dec 26 20:57:29 kea1 systemd[1]: Stopped Kea DHCPv4 Service.
> > Dec 26 20:57:29 kea1 systemd[1]: isc-kea-dhcp4-server.service: Consumed
> 1min 28.504s CPU time.
> > Dec 26 20:57:29 kea1 systemd[1]: Started Kea DHCPv4 Service.
> >
> > $ tail -n10 /var/log/kea/kea-ctrl-agent.log
> > 2023-12-26 20:59:53.827 INFO [kea-ctrl-agent.ctrl-agent/1393724]
> CTRL_AGENT_COMMAND_RECEIVED command ha-heartbeat received from remote
> address xxx.xxx2.xxx.xxx
> > 2023-12-26 20:59:53.828 INFO [kea-ctrl-agent.ctrl-agent/1393724]
> CTRL_AGENT_COMMAND_FORWARDED command ha-heartbeat successfully forwarded to
> the service dhcp4 from remote address xxx.xx2.xxx.xxx
> > 2023-12-26 21:00:03.843 INFO [kea-ctrl-agent.auth/1393724]
> HTTP_CLIENT_REQUEST_AUTHORIZED received HTTP request authorized for
> 'baduser'
> > 2023-12-26 21:00:03.843 INFO [kea-ctrl-agent.commands/1393724]
> COMMAND_RECEIVED Received command 'ha-heartbeat'
> > 2023-12-26 21:00:03.843 INFO [kea-ctrl-agent.ctrl-agent/1393724]
> CTRL_AGENT_COMMAND_RECEIVED command ha-heartbeat received from remote
> address xxx.xxx2.xxx.xxx
> > 2023-12-26 21:00:03.844 INFO [kea-ctrl-agent.ctrl-agent/1393724]
> CTRL_AGENT_COMMAND_FORWARDED command ha-heartbeat successfully forwarded to
> the service dhcp4 from remote address xxx.xxx2.xxx.xxx
> > 2023-12-26 21:00:13.859 INFO [kea-ctrl-agent.auth/1393724]
> HTTP_CLIENT_REQUEST_AUTHORIZED received HTTP request authorized for
> 'baduser'
> > 2023-12-26 21:00:13.859 INFO [kea-ctrl-agent.commands/1393724]
> COMMAND_RECEIVED Received command 'ha-heartbeat'
> > 2023-12-26 21:00:13.859 INFO [kea-ctrl-agent.ctrl-agent/1393724]
> CTRL_AGENT_COMMAND_RECEIVED command ha-heartbeat received from remote
> address xxx.xxx2.xxx.xxx
> > 2023-12-26 21:00:13.860 INFO [kea-ctrl-agent.ctrl-agent/1393724]
> CTRL_AGENT_COMMAND_FORWARDED command ha-heartbeat successfully forwarded to
> the service dhcp4 from remote address xxx.xxx2.xxx.xxx
> > $ tail -n10 /var/log/kea/kea-dhcp4.log
> > 2023-12-26 20:58:53.728 INFO [kea-dhcp4.commands/1393730]
> COMMAND_RECEIVED Received command 'ha-heartbeat'
> > 2023-12-26 20:59:03.745 INFO [kea-dhcp4.commands/1393730]
> COMMAND_RECEIVED Received command 'ha-heartbeat'
> > 2023-12-26 20:59:13.762 INFO [kea-dhcp4.commands/1393730]
> COMMAND_RECEIVED Received command 'ha-heartbeat'
> > 2023-12-26 20:59:23.777 INFO [kea-dhcp4.commands/1393730]
> COMMAND_RECEIVED Received command 'ha-heartbeat'
> > 2023-12-26 20:59:33.793 INFO [kea-dhcp4.commands/1393730]
> COMMAND_RECEIVED Received command 'ha-heartbeat'
> > 2023-12-26 20:59:43.811 INFO [kea-dhcp4.commands/1393730]
> COMMAND_RECEIVED Received command 'ha-heartbeat'
> > 2023-12-26 20:59:53.827 INFO [kea-dhcp4.commands/1393730]
> COMMAND_RECEIVED Received command 'ha-heartbeat'
> > 2023-12-26 21:00:03.844 INFO [kea-dhcp4.commands/1393730]
> COMMAND_RECEIVED Received command 'ha-heartbeat'
> > 2023-12-26 21:00:13.859 INFO [kea-dhcp4.commands/1393730]
> COMMAND_RECEIVED Received command 'ha-heartbeat'
> > 2023-12-26 21:00:23.875 INFO [kea-dhcp4.commands/1393730]
> COMMAND_RECEIVED Received command 'ha-heartbeat'
> >
> > And changing the CA or the server HA paramersts to port 8001 without
> changing the other (and the other server results in "connection refused"
> logs. It obv wants the CA port to match the HA parameters port despite what
> we and the documentation suggests...
> >
> > CS, cs.Temp.Mail at gMail.com
> >
> >
> > On Mon, 25 Dec 2023 at 02:45, Darren Ankney <darren.ankney at gmail.com>
> wrote:
> >>
> >> Hi,
> >>
> >> It actually SHOULDN'T work to set your control agent and
> >> multi-threaded HA listener to the same port as only one of the
> >> applications should be able to setup a listener on that port. Please
> >> describe what you mean by "it doesn't work". I'm thinking it might be
> >> be best to ask Men & Mice about "micetro" and how best to set things
> >> up there.
> >>
> >> Thank you,
> >>
> >> Darren Ankney
> >>
> >> On Thu, Dec 21, 2023 at 6:47 PM CS <cs.temp.mail at gmail.com> wrote:
> >> >
> >> > Hi all,
> >> > Moving on from my failure to start and logging issues (thank you for
> your help btw!) I now don't have my heartbeat/control_agent working
> correctly.
> >> >
> >> > It works fine so long as I set the ports of my control agents and ha
> hook parameters to be the same (IE 8000 or 8001)
> >> >
> >> > However I am unable to tie the tiny cluster into micetro, probably
> because the CA port is occupied with HA heartbeats?
> >> >
> >> > Looking to these examples:
> >> >
> https://github.com/isc-projects/kea/tree/master/doc/examples/template-ha-mt-tls
> >> >
> >> > Documentation points out
> >> > //This specifies the port CA will listen on.
> >> > // If enabling HA and multi-threading, the 8000 port is used
> by the HA
> >> > // hook library http listener. When using HA hook library with
> >> > // multi-threading to function, make sure the port used by
> dedicated
> >> > // listener is different (e.g. 8001) than the one used by CA.
> Note
> >> > // the commands should still be sent via CA. The dedicated
> listener
> >> > // is specifically for HA updates only.
> >> >
> >> > However, how to have a dedicated port for HA and a different one for
> CA escapes me.
> >> >
> >> > CS, cs.Temp.Mail at gMail.com
> >> > --
> >> > ISC funds the development of this software with paid support
> subscriptions. Contact us at https://www.isc.org/contact/ for more
> information.
> >> >
> >> > To unsubscribe visit https://lists.isc.org/mailman/listinfo/kea-users
> .
> >> >
> >> > Kea-users mailing list
> >> > Kea-users at lists.isc.org
> >> > https://lists.isc.org/mailman/listinfo/kea-users
> >> --
> >> ISC funds the development of this software with paid support
> subscriptions. Contact us at https://www.isc.org/contact/ for more
> information.
> >>
> >> To unsubscribe visit https://lists.isc.org/mailman/listinfo/kea-users.
> >>
> >> Kea-users mailing list
> >> Kea-users at lists.isc.org
> >> https://lists.isc.org/mailman/listinfo/kea-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.isc.org/pipermail/kea-users/attachments/20231227/b775aa9d/attachment-0001.htm>
More information about the Kea-users
mailing list