Kea and NETCONF in 2018: Towards Automated Configurability

ISC has a long history of participating in hackathons, starting a long time ago at Gdansk University, and continuing with IETF Hackathons. Every time the goal was similar: spend a day or two playing with a new technology, concept, or idea to see whether it makes sense, understand how to possibly extend Kea to support it properly, and gain some experience. This time the goal was to revisit the concept of YANG models and the NETCONF interface. See our earlier hackathon report from 2016.

IETF Hackathon in London

Hacking software in London: IETF 101 Hackathon

Since we did some experiments with YANG and NETCONF in an earlier IETF hackathon, the obvious first goal was to revisit what’s been going on in the Sysrepo project. They’ve been busy for the last two years and the code made a lot of progress. Implementing the software was fast and reasonably easy using their new Docker image. The availability of C++ bindings was a very welcome discovery.

Compared to our attempts two years ago, another aspect that matured quite significantly was the IETF YANG model for DHCPv6. While several issues were discovered, the overall state of the model is quite good and we were able to extract it in YANG model and load it into sysrepo.

We managed to define and load a simple but full-featured configuration that covered DHCPv6 server configuration. It covered the server’s duid configuration, quite a few options (including the simple ones, like DNS servers, but also complex ones, like lightweight 4over6 options with containers and v4-v6 bindings). After some tweaks, we managed to load that configuration and make it available via NETCONF interface.

With that infrastructure in place we moved on to the major part of the hackathon, which was to get the information extracted and actually used by Kea.

Kea-netconf daemon

The problem of retrieving a configuration from NETCONF is complex. First, a new Kea instance that starts for the first time must be able to retrieve its configuration. Second, once the configuration is retrieved and applied, it needs to listen on any changes that may become available and then apply them internally.

The diagram above shows the overall architecture of the solution. Sysrepo software is used to store the YANG model, configure based on that model, and provide a remote NETCONF interface. This provides a standard frontend to the Kea instance that can be used remotely.

We spent some time listing out the tasks the kea-netconf daemon would have to conduct: connect to sysrepo, retrieve the initial configuration, translate it to JSON syntax that can be understood by Kea servers, then send that configuration to Kea. A significant challenge will be to be able to use callbacks to listen on specific changes and act if specific parts of the whole configuration changes. The tricky part here is to apply just the small part of the configuration that has changed, without reapplying the whole configuration. Think about a case of adding 1001st subnet when you already have 1000 subnets configured.

The problem here is how to deal with the complexity of the translation process.

Translation process

The DHCPv6 server YANG model itself is complex. Its definition takes 18 pages of text and uses many dependencies on other models. Any attempts at converting the whole model in its entirety are clearly a recipe for disaster. Even if we somehow could pull this off, maintaining such a large monolithic code would be a trauma we’d have to endure on a daily basis.

Fortunately, the solution to this problem appeared. We came up with the concept of translators. A translator is a self contained piece of code that is able to take a relatively small part of the configuration in YANG syntax and convert it to JSON format that is understandable by Kea. Obviously, Kea must be able to provide the necessary commands to process such chunks, but fortunately many of these commands are already supported: subnet4-list, subnet6-list, subnet4-add, subnet6-add, subnet4-del, subnet4-get, subnet6-get, reservation-add, reservation-get, reservation-del and many more.

The prototype framework written during the hackathon is far from being complete, but it laid out a good foundation. It is able to load multiple translators and register callbacks for each of them. With more development effort spent on it, it could become a fully featured solution.

So far we developed a prototype for TranslatorNetworkRanges that covers network ranges, or using Kea nomenclature, Subnets and Pools. There is also a skeleton for TranslatorOptions that will one day be able to handle options. TranslatorInterfaces and TranslatorDuid were considered.

So where does this lead us?

Long-term goal: centralized configuration storage

Right now, Kea 1.3 is a nice stand-alone server that does its job. But network engineers expect more than a single server. They want multiple instances running in parallel, they want to deploy new servers with a push of a button, they need centralized configuration for huge networks, instantaneous configuration updates and more.

The long-term goal for Kea is to provide a solution that will offer the ability to store all of its configuration and runtime state in a database. With this capability available, it will be possible to spawn new Kea instances and simply point them to a database: “there’s everything you need, now go do your job.” We are not there yet and there is a long road ahead, but the ultimate goal is clear.

This objective is not going to appear in one step. We need to get there in smaller, more manageable steps. Kea already is able to use REST interface. Extending it to also use NETCONF interface will add to the flexibility.

Turning this vision into reality

ISC already decided to implement the ability to be able to store essential parts of the configuration to databases. Kea 1.3 can store host reservations in a separate backend database. With Kea 1.5, we will extend this to subnets and options. More configuration elements will come in future releases.

We have not made a decision about NETCONF yet. We would love to hear from network operators, ISPs, and other interested people about their thoughts about NETCONF. If you are interested, please share your use cases. What the the most important things you’d like to do with a NETCONF interface? Finally, are you willing to help funding the development of this work? If you can, share your opinion on the kea-users list. If you prefer to share your thoughts more privately, please send an e-mail to Vicky Risk and Tomek Mrugalski.

Thank you to Razvan Becheriu, Andrei Pavel, Mislav Novakovic, and Luka Perkov from Sartura, Normen Kowalewski, and Ian Farrer. Some of you participated and wrote code, others helped with configuration and various tips or donated other essential pieces of configuration. Thanks!