Notice of plan to deprecate map zone file format

[Ondřej Surý]

I am just going to point out that the 10 second speed up was measured on .net zone. Thus the calculations provided below have no practical impact because if anybody is loading 100k .net-sized zones on a single server they will have a different problem that the loading speed…

The goal of the initial email was to get feedback from serious users of the feature (if there are any). It seems that I need to say that we constantly look and work on improving BIND 9, and while we are going to drop the map masterfile format, the ad how internal benchmarks already show significant speed up when loading many zones between 9.11 and 9.16.

The map masterfile format was a smart idea at the time, but it’s extremely fragile, and it’s not a good use of the engineering time to spend time caring about it. I would rather make BIND 9 faster for everybody than fine tune extreme edge case.

Ondrej
--
Ondřej Surý — ISC (He/Him)

My working hours and your working hours may be different. Please do not feel obligated to reply outside your normal working hours.

> On 10. 9. 2021, at 19:44, Timothe Litt <litt at acm.org> wrote:
> 
> 
> I'm not a consumer of this and agree that it's up to users to speak up, so I'll stop here - with one final observation.
> 
> The issue comment containing the benchmarks includes:
> 
>> Speedup provided by the map format does not seem significant enough to warrant the complexity of map format, especially when we take into account that the difference measured in terms of "real time" is in order of 10s of seconds.
> 10s of seconds per zone can certainly add up.  Call it 10 secs/zone * 100,000 zones = 1M sec / 3600 = 278 hrs saved.
> Suppose loading zones is not disk limited, and cores scale linearly (e.g. no lock conflicts & an index lets each core find a zone to work on for free).  So give it 16 cores (each taking on one complete zone), and it's still 17 hrs saved.  Real life won't be that efficient - meaning cores won't help that much.
> 
> A new memory mapped data structure that didn't require "updating node pointers" (e.g. that used offsets instead of pointers) may be worth considering.  In current hardware and with a decent compiler and coding, the apparent cost of this over absolute pointers may well be vanishingly small.
> 
> OK, that was two.
> 
> Timothe Litt
> ACM Distinguished Engineer
> --------------------------
> This communication may not represent the ACM or my employer's views,
> if any, on the matters discussed. 
> On 10-Sep-21 12:56, Victoria Risk wrote:
>> 
>>>>> After all the "other improvements in performance" that you cited, what is the performance difference between map and the other formats?
>>>> 
>>>> I don’t know that, to be honest. We don’t have the resources to benchmark everything. Maybe someone on this list could?  We would also like to be able to embark on a wholesale update to the rbtdb next year and this is the sort of thing that might complicate refactoring unnecessarily. 
>> 
>> 
>> I was wrong, and in fact we have benchmarked it. See https://gitlab.isc.org/isc-projects/bind9/-/issues/2882 for details. Map format is still faster than raw, but not so much faster that it warrants retaining it, given it is riskier, harder to maintain and we have no feedback from users that it is important to them.  It also seems not to work with large numbers of zones, (>100K) at least in current versions of 9.11 and 9.16, which is further indication that it isn’t in wide use or we would have had complaints. 
>> 
>> We also have discussed internally that there are other factors, other than loading the zone files, that may have more impact on the time it takes a BIND server to restart.
>> 
>> If anyone out there is using it successfully, and wants us to keep this feature, this would be the time to speak up!
>> 
>> Thank you,
>> 
>> Vicky
> _______________________________________________
> Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list
> 
> ISC funds the development of this software with paid support subscriptions. Contact us at https://www.isc.org/contact/ for more information.
> 
> 
> bind-users mailing list
> bind-users at lists.isc.org
> https://lists.isc.org/mailman/listinfo/bind-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.isc.org/pipermail/bind-users/attachments/20210910/c3761e91/attachment.htm>