High memory consumption in bind 9.18.2

Wed Aug 17 15:26:31 UTC 2022

Hello,

Reading release notes, 9.16.25 is the point release where the default 
memory allocator was switched:
https://downloads.isc.org/isc/bind9/9.16.32/doc/arm/html/notes.html#notes-for-bind-9-16-25

The issue linked https://gitlab.isc.org/isc-projects/bind9/-/issues/2398 
is giving lots of interesting informations.

The only thing explaining your observation is that you test machine is 
very low cpu count powered and that the trades of made by the new 
allocator is negative (from mem consumption point of view) in this 
specific artificial corner case : very low powered machine with very 
high zone/record count. Useless in real world scenario.

You could do the test yourself with command-line option -M internal with 
 >= 9.16.25

For the other parameter I recalled from memory, it was 
--with-tuning=large/--with-tuning=small but was large since 9.16.0.

Emmanuel.
Le 08/08/2022 à 12:29, Dmitri Pavlov a écrit :
> Hi again Ondřej, Emmanuel,
>
> I hope you enjoyed the weekend ?!
>
> Just trying to summarize where we after running a few tests against "zoo of machines" and comparing to your lab results.  Please, see the attached "Symmary.jpg"
>
> What we see is the RAM utilization before "<" 9.16.25  (regardless of kernel version and GCC) lower than in >= 9.16.25 including 9.18.5 and 9.19.3. Pre 9.16.x results are without jemalloc.  And the results are pretty much consistent before 9.16.25 and after. It is 2 GB higher in 9.18.x with ~ 100 mil records data set. I understand your previous notes about ridiculousness of the given scenario, patch levels, bugs in earlier releases etc.... But anyway, sorry , the question is still open for us -> is there really a change in 9.18.x codebase that allows lower RAM utilization as compared vs pre 9.16.25 versions https://kb.isc.org/docs/bind-memory-consumption-explained? Or maybe particularly in this scenario, when we adding 100 mil records into  to the cache, does not fall under KB scope?
>
> Your response about the KB correctness will help to deliver a better optimized business decision.
>
> Thank you very much in advice,
> Dmitri.
>
>
>
>
> -----Original Message-----
> From: bind-users <bind-users-bounces at lists.isc.org> On Behalf Of Ondrej Surý
> Sent: Thursday, August 4, 2022 7:48 PM
> To: Emmanuel Fusté <manu.fuste at gmail.com>
> Cc: bind-users at lists.isc.org
> Subject: Re: High memory consumption in bind 9.18.2
>
> What Emmanuel said…
>
> --
> Ondřej Surý — ISC (He/Him)
>
> My working hours and your working hours may be different. Please do not feel obligated to reply outside your normal working hours.
>
>> On 4. 8. 2022, at 19:15, Emmanuel Fusté <manu.fuste at gmail.com> wrote:
>>
>> Le 04/08/2022 à 17:48, Dmitri Pavlov a écrit
>>> Therefore, a very small request. Would it be possible on your side to run the same experiment as with (BIND 9.16.32 / BIND 9.18.6 / BIND 9.19.4) one more time but with BIND 9.16.21 (or any other version in 9.16.x <25 range )?
>>>
>>>
>> Why not the opposite ? Why do you insist to run obsolete/inferior patch level version ? Who want to run something older than the latest patch release of one maintained version and even more a more than ten patch level apart ?
>> The memory consomption diff is not an argument as it is simply ridiculous vs the used scenario.
>> Reproduce the 9.16.32 scenario, and if it reproduce Ondřej result, the conclusion will be evident : bugs in the older patch level as you clearly reproduced the 9.18 usage which you could surely reproduce with the 9.19 series too.
>> Do you really prefer to run buggy but less memory hungry version ?
>>
>> I understand that you want to have answer to you questions. Simply do the complete exercise and you will have answers. Don't ask people to do them for you.
>>
>> Emmanuel.
>>
>> PS: there where a switch on the default runtime config switch to "big server" mode sometimes during the 9.16 series if I recall correctly. It perhaps explain the diff.
>> --
>> Visit https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.isc.org%2Fmailman%2Flistinfo%2Fbind-users&data=05%7C01%7Cdpavlov%40perforce.com%7Cbf656a835be74993002e08da76391910%7C95b666d19a7549ab95a38969fbcdc08c%7C0%7C0%7C637952284930827181%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Kirvq3EkQe239keUeA18xvG%2FOxY8tAhs2yF9AxaoR0Y%3D&reserved=0 to unsubscribe from this list
>>
>> ISC funds the development of this software with paid support subscriptions. Contact us at https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.isc.org%2Fcontact%2F&data=05%7C01%7Cdpavlov%40perforce.com%7Cbf656a835be74993002e08da76391910%7C95b666d19a7549ab95a38969fbcdc08c%7C0%7C0%7C637952284930827181%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=V1HTBX29PPdDrjG6wHaJ6YU5BrHNymn1oNoaiidh6dk%3D&reserved=0 for more information.
>>
>>
>> bind-users mailing list
>> bind-users at lists.isc.org
>> https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.isc.org%2Fmailman%2Flistinfo%2Fbind-users&data=05%7C01%7Cdpavlov%40perforce.com%7Cbf656a835be74993002e08da76391910%7C95b666d19a7549ab95a38969fbcdc08c%7C0%7C0%7C637952284930827181%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Kirvq3EkQe239keUeA18xvG%2FOxY8tAhs2yF9AxaoR0Y%3D&reserved=0
> --
> Visit https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.isc.org%2Fmailman%2Flistinfo%2Fbind-users&data=05%7C01%7Cdpavlov%40perforce.com%7Cbf656a835be74993002e08da76391910%7C95b666d19a7549ab95a38969fbcdc08c%7C0%7C0%7C637952284930827181%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Kirvq3EkQe239keUeA18xvG%2FOxY8tAhs2yF9AxaoR0Y%3D&reserved=0 to unsubscribe from this list
>
> ISC funds the development of this software with paid support subscriptions. Contact us at https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.isc.org%2Fcontact%2F&data=05%7C01%7Cdpavlov%40perforce.com%7Cbf656a835be74993002e08da76391910%7C95b666d19a7549ab95a38969fbcdc08c%7C0%7C0%7C637952284930827181%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=V1HTBX29PPdDrjG6wHaJ6YU5BrHNymn1oNoaiidh6dk%3D&reserved=0 for more information.
>
>
> bind-users mailing list
> bind-users at lists.isc.org
> https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.isc.org%2Fmailman%2Flistinfo%2Fbind-users&data=05%7C01%7Cdpavlov%40perforce.com%7Cbf656a835be74993002e08da76391910%7C95b666d19a7549ab95a38969fbcdc08c%7C0%7C0%7C637952284930827181%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Kirvq3EkQe239keUeA18xvG%2FOxY8tAhs2yF9AxaoR0Y%3D&reserved=0
>
>
> CAUTION: This email originated from outside of the organization. Do not click on links or open attachments unless you recognize the sender and know the content is safe.
>
> This e-mail may contain information that is privileged or confidential. If you are not the intended recipient, please delete the e-mail and any attachments and notify us immediately.
>