Performance Test Metrics for dns server performance.

[Kevin Darcy]

I never said performance didn't matter. In fact, I clearly said that
performance *did* matter, but it's query performance to the *clients* that
matters, not just queries/sec on the nameserver side. Your original message
seemed rather obsessed with the queries/sec metric, where in actual fact
intelligent placement and distribution of nameservers in the enterprise, good
namespace design, master/slave dependency graphs, Refresh/Retry/TTL tuning etc.
are often much more important in terms of actual performance than the query/sec
capacities of individual servers or nameserver implementations. To paraphrase a
classic phrase about software efficiency: "the fastest query response is the
one that isn't sent", i.e. because the query was already answered by a
nameserver closer to the client, straight out of its cache or authoritative
data.

Besides, if you care about high performance, why did you run your tests on a
toy Intel-based box? What does that prove? That a pig can sing?

As for your comparison between automobile maintenance and DNS maintenance,
well, I might know a thing or two about the automotive world as well :-) If
you're willing to outsource all of your DNS maintenance the way you
"outsource" your automobile maintenance to your local auto mechanic, and money
is no object, then sure, maybe the relative maintainability of djbdns and
BIND doesn't matter to you. You're basically just slapping a fixed cost on your
TCO and letting someone else deal with the headaches. But that's not the way
most folks operate, not even large ISPs with more money than brains. When
you're maintaining your own stuff, then tools *do* matter. A lot. And they have
a large effect on the TCO of a particular solution.

And, why the hell are you getting calls from your junior admins about missing
dots or un-incremented serial numbers? If you'd done your job correctly as a
senior admin (I assume that's what you consider yourself), the update process
would be automated and such problems wouldn't exist. Manual update of serial
numbers is for Mom&Pop shops, not "enterprises".


- Kevin

Matt Simerson wrote:

> > -----Original Message-----
> > From: Kevin Darcy [mailto:kcd at daimlerchrysler.com]
> > Sent: Friday, May 04, 2001 4:09 PM
> > To: 'bind-users at isc.org'
> > Subject: Re: Performance Test Metrics for dns server performance.
> >
> > I guess I'm not sure what you're really trying to accomplish
> > here. "Building an enterprise DNS solution". Been there, done
> > that.
>
> My Enterprise != Your Enterprise.
>
> < snipped lots of useless drivel stating that server performance isn't
> important>
>
> >From previous email:
>    Clue #1: 90,000 IP's in ONE NOC
>    Clue #2: Clue 1 implies multiple NOC's
>    Clue #3: 45% of IP's reverse.
>    Clue #4: I wouldn't be doing this if our current
>             solution wasn't ready to burst at the seams.
>    Clue #5: I'm testing performance.
>
> Maybe in your world you can stick a few little servers in corners of your
> enterprise and solve your DNS problems. I used to own a small world like
> that. Two name servers, two T1's, 4 class C's, IPFW & IOS Access lists, and
> nary a problem a problem in 6 years, but enough about my house, this is a
> work problem. In this world, I have 5 NOC's, hundreds of thousands of zone
> files, millions of RR's, and thousands of servers, many of which do very DNS
> intensive things like mail servers, mailing lists, etc. DNS is a big deal,
> and having a solution that scales across my enterprise means building it
> right from the ground up. Performance does matter.
>
> > After all, what's important to a DNS client in some far-flung
> > corner of your enterprise is not how many queries/sec some distant
> > mega-nameserver
>
> What's important to each of the thousands of machines in each corner of my
> world is that they have an array of caching servers that can quickly answer
> lots of queries. What's important to our bottom line is whether we need to
> stick 10 load balanced caching resolvers in there or 4. What affects the
> Dual OC3's is how much effort I expend into keeping the caches organized
> hierarhically. Performance does matter.
>
> > can theoretically process,
>
> We aren't talking about theory, we're talking reality. I have one caching
> name server right now that's answering an average of 1270 queries per
> second. That server only has a few hundred machines using it for resolution.
> What about the thousands of other machines? Should I choose a name server
> than can only handle 10% of that and buy ten times as many machines? That
> sounds like a dot.gone mentality. Performance does matter.
>
> > but how quickly *their*particular*query* gets answered, and the latter
> > depends on a variety of factors besides the capacity of the
> > central server or the efficiency of the
> > nameserver implementation it's running; factors like network
> > latency, number of hops, etc.. The distributed approach also
> > tends to fare  better in the face of network outages and isolation
> situations.
>
> Duh.
>
> > And don't overlook the cost of ongoing maintenance.
>
> And what makes you think one costs any more to maintain than the other? I've
> used BIND for, oh golly, almost a decade now. I know how much time and
> effort it takes to admin, I've trained quite a few jr. sysadmins in the ways
> of BIND. I've gotten plenty of phone calls because "sysadmins" forgot a dot
> in their zone file or didn't increment a serial number. I've been there pal,
> I've done all that too. It costs money to own either one. Assuming one is
> better or worse without fairly comparing them is akin to burying ones head
> in the sand.
>
> > There is
> > a wealth of tools available to maintain BIND nameservers,
> > but next to nothing for djbdns.
>
> There's also a wealth of tools available for rebuilding the 350 engine in a
> 1976 GM pickup. It's even one of the least expensive engines to rebuild
> because of it's popularity. That still doesn't make it a good choice for
> driving me and 3 friends out to the pass to go skiing. I have to keep a
> mechanic in business just to own it. It's price/performance ratio is
> terrible when you figure in the Total Cost of Ownership. My shiny new TDI
> Jetta doesn't have many tools for it either but I know darned well that it
> won't need much more than oil changes, a sponge, and a vacuum for the first
> 10 years of it's life.
>
> > What
> > good is it to your enterprise to achieve some theoretical queries/sec
> > threshold, but have to hire half a dozen more administrators
> > just to keep those nameservers running?
>
> You're making some might big assumptions there that you have evidence to
> support.
>
> > Generally speaking,
> > people time costs a lot more than hardware.
>
> Maybe I should just buy ten thousand Mac IIci's with NetBSD on them. Yeah,
> that'll work. :-P  I better warn the NOC guys that I'm going to need a few
> hundred more racks.
>
> Matt