Comparative Performance Results of BIND Versions in Authoritative Configurations - September 2023

We are only one quarter away from producing the next stable branch of BIND, and from ending maintenance of BIND 9.16 versions. This is a good time to look at the performance of BIND 9.18 vs 9.16 as many users have not yet adopted 9.18. The bottom line is, BIND authoritative server performance is remarkably stable, and users updating from 9.16 to 9.18 should not anticipate a performance impact. There are, however, a number of other changes to be aware of when moving from BIND 9.16 to 9.18, which are explained in this Knowledgebase article.

As part of the development cycle of BIND, we carry out a set of performance tests. Our performance laboratory system for DNS authoritative testing (somewhat unimaginatively called “perflab”) continually cycles through all configurations registered with it, building BIND and running performance tests. (Details of the system have been presented elsewhere), so they will not be described here.) In this way, as the development branches are updated with changes, the effect of those changes on performance is quickly visible. We use a completely different methodology for testing DNS resolver operations, because that is impacted by factors on the live Internet, and tests with canned queries are a poor predictor of real-world performance. Resolver performance results will be presented soon in a separate blog post.

Among the configurations tested, the following are especially significant:

• One million zones: These tests simulate the environment of a web hosting provider, whose server might be serving zones for a number of customers. Each of the zones in the test is very small, comprising just an SOA, NS, A, and AAAA record.
• One million delegations: This configuration simulates a large top- (or second-level) domain, where the zone comprises almost exclusively delegations to the zones of subdomain owners.
• One million resource records: This is the opposite extreme to the one million delegations test. In that test, BIND returns referrals in response to queries; in this test BIND returns resource records from within the zone.
• Root zone: As some of the twelve root server operators run BIND on at least some of their servers, and given the crucial importance of the root zone to the global DNS infrastructure, we test the performance of BIND when serving a copy of the root zone.

For each scenario, the query set is one that will result in about 5% NXDOMAIN responses (with the exception of the root zone test, where we would expect a much higher rate of NXDOMAINs).

To obtain a figure for the query rate that BIND can handle, perflab starts up the server under test. It then fires queries at the server for 30 seconds and calculates a query rate. This 30-second measurement is repeated 30 times and a figure for the query rate (and standard deviation) is calculated. The first measurement is discarded from calculations, to allow the server to settle down, i.e., for all data structures to be created and reach their final size. The chart above shows the latest value, but there was virtually no measurable variation in the runs for 1K and 1M zones and very little variation for the other scenarios over the trailing month.

We also have a new metric, startup time, which measures the number of seconds from starting BIND until it prints “running”. Here we see a slight drop in 9.18, but in absolute terms it is a drop of only a few seconds.

We hope that this information helps alleviate concerns about a performance regression from the list of considerations when planning an update to BIND 9.18.