Recommended setup with large cache memory

[Brad Knowles]

At 1:17 PM +0200 2005-09-09, Attila Nagy wrote:

>  I guess I won't understand the code in the next ten minutes, so could
>  you please help understanding the situation?

	I don't know the code that well myself, but I don't think that 
there's anything useful about the code that I could possibly teach to 
anyone in ten minutes.

>  - if a query happens and it's in the cache, but it has been expired,
>  remove it (it seems you know the code, is it currently implemented
>  this way?). This could give some level of auto cleaning.

	Query comes in.  Look up the answer in the cache.  Answer is 
there, but expired.  The server will go ahead and re-query for that 
information, give you the answer that comes back, and store that in 
the cache.  This is the way it works today, but there is no memory 
recovered.

>  - if the memory is full, simply drop, or overwrite the "older" entries.
>  For example if you have 1000 "slots" if the number of entries reach 1000,
>  the first will be the next available slot.

	How do you know which ones are oldest?  You'd have to search the 
entire in-memory database to find that out, which is the largest part 
of what the standard cache cleaning process is already doing.

>  - somehow make the expire process adaptive or "rate limited". It seems
>  a little bit odd, to have very degraded performance while the expire runs.

	Garbage collection in any application is almost always an 
expensive process, unless you can design the system so that a little 
bit is pretty much constantly being done, so that you never have a 
large build up that you have to then spend a lot of time to go 
through and clean out.

>  I will try threading enabled to see whether it improves this 
>situation or not.

	If you're running on FreeBSD, Jinmei's report has demonstrated 
that turning on threading is not likely to help you.

>  And what do you think about the cyclic reuse of slots?

	There are no "slots".  There is no "circular linked list".  Try 
taking a look at some programming books on data structures before you 
start making pronouncements of how you should solve these problems. 
Then at least take a brief look at how the code actually works.

	In particular, my understanding is that BIND uses a red/black dbtree.


	The problem here is that you're making sweeping proclamations 
regarding how BIND should be improved, based on assumptions you're 
making regarding the operations of BIND, but you're working from 
incomplete and inaccurate information.  Can you see the flaw in your 
logic?

	Myself, I know that my knowledge is incomplete, but hopefully not 
too inaccurate.  But I also know enough about how BIND works to know 
that I am not qualified to be able to make a whole lot of sweeping 
pronouncements as to how the code should be fixed.

-- 
Brad Knowles, <brad at stop.mail-abuse.org>

"Those who would give up essential Liberty, to purchase a little
temporary Safety, deserve neither Liberty nor Safety."

     -- Benjamin Franklin (1706-1790), reply of the Pennsylvania
     Assembly to the Governor, November 11, 1755

   SAGE member since 1995.  See <http://www.sage.org/> for more info.