BIND Server Considerations (was Re: bind with mysql)

[Chris Buxton]

Many of the constraints mentioned can be worked around using a  
management system, such as Men & Mice Suite or any other DNS/IPAM  
management system. If the number of zones, for example, is really a  
constraint, then Men & Mice Suite solves this problem; the specific  
mechanism used in the back-end to update and maintain the data that  
BIND is using (static file-based zones, dynamic zones, DLZ) makes very  
little difference in this case.

> Nowadays, dynamic DNS is commonly used to insure that a single  
> master and one or more slave DNS server are kept in sync.

No, DNS notify takes care of this. Support for dynamic update has  
nothing to do with this.

> First, although manually managing DNS data is still commonly used,  
> it is sort of passe.

We see very large enterprises still using vi as their primary DNS  
management tool, along with an IPAM database in the form of an Excel  
spreadsheet. Often such enterprises are unhappy with these tools,  
which is why they come to us, but such tools do work with sufficient  
care, and their use is very common.

The OP's question about a web interface and use of MySQL demonstrates  
a typical web application developer's approach to the problem, without  
understanding the performance impact (as noted below). However, that's  
not to say it's necessarily the wrong approach. Use of the notify and  
zone transfer mechanisms for replication, instead of implementing DLZ  
on each server, and use of a hidden master (the server with DLZ built  
in) can readily solve the performance problem.

However, a good management system with a web interface that does not  
rely on DLZ is no more complex - probably less so, actually - and  
would very likely require less work on the part of the system  
integrator or web app developer. It sounds like the OP's needs would  
mandate a custom-built web interface, but the interface that comes  
with the management system could be used as a starting point.

Chris Buxton
Professional Services
Men & Mice

On Apr 3, 2008, at 11:03 AM, Bill Larson wrote:
> On Apr 3, 2008, at 9:28 AM, J. Peng wrote:
>
>> On Thu, Apr 3, 2008 at 11:11 PM, Kevin Darcy <kcd at chrysler.com>  
>> wrote:
>>
>>> If your main problem is slow reload times, perhaps you should
>>> consider
>>> using Dynamic Update to maintain your DNS records. This would
>>> obviate
>>> the constant reloads whenever you change something.
>>
>> Do you prefer using traditional zone files?
>> But if we provide business domain hosting, it's maybe not convenient
>> to manage BIND with that way.We want to control the DNS records  
>> with a
>> web interface, like the one Godaddy is using.
>> So, how about the solution for my situation? Thanks again.
>
> You are asking to compare three possible scenarios for you to
> maintain your DNS data.
>
> 	1.	Editing standard text files to manage DNS data
> 	2.	Using Dynamic DNS to manage DNS data
> 	3.	Using a database to manage DNS data
>
> You have a few constraints imposed by you and your customers that you
> need to understand.
>
> 	1.	How "dynamic" is your DNS data?  How often do you expect it to
> change
> 		(any add/delete/modify)?
> 	2.	How often do you expect to add/delete zones?
> 	3.	How many zones are you expecting to be responsible for?
> 	4.	How large are your zones?  How many resource records?
> 	5.	What database system are you willing to use?  (There are other
> 		databases supported by BIND-DLZ than just MySQL.)
> 	6.	How many queries to second do you expect to answer, both
> 		internally and from the Internet?  (Raw DNS system/server
> performance.)
> 	7.	How quickly do you need to make this new data available to the
> Internet?
> 		(DNS data replication time.)
> 	8.	How many DNS servers are you willing to provide/support?
> 	9.	How good is your connection to the Internet?
>
> Take the constraints first, not in any specific order.
>
> If your DNS data is very dynamic, adding/deleting/modifying data very
> often, then managing this data via standard text files is going to be
> difficult.  Remember that you are not talking about just the zone
> files here, but the BIND configuration file also where you add and
> delete zones that you are authoritative for.  Adding or deleting a
> zone definition in your BIND configuration file must be considered.
> Modifying DNS zone data with either dynamic DNS or some database
> system can be quite quick.
>
> If you are expecting to host a large number of zones (for some
> "large" number) then maintaining the contents of your BIND
> configuration file may become a limiting part.  If you have 10,000
> zones that you are authoritative for, then you are talking about a
> large number of "zone" definition entries that you have to manage.
>
> BIND-DLZ supports a large number of different database systems.  Some
> of these are traditional database systems like, MySQL and
> PostgresSQL.  Others are system specific, such as LDAP and filesystem
> based.  Other databases are designed for high performance, such as
> the Berkeley DB system, especially the HPT schemes.  What database
> are you willing to use with BIND-DLZ?  This isn't limited to just  
> MySQL.
>
> Now, you have to consider the performance of each of these database
> systems.  For more information  about this, I am looking at the BIND-
> DLZ performance tests described at http://bind-dlz.sourceforge.net/.
> The baseline performance of straight BIND supports 16,000 QPS.  If
> you are using MySQL or Postgres for maintaining your data, then you
> are looking at 500-700 QPS, which is about a 20x performance
> degradation over straight BIND.  Now, if you are using the HPT BDB
> drivers, then they saw 8,000-12,000 QPS.  This is a 1.5-2x
> degradation over straight BIND, significant but not necessarily a
> killer.  The performance using LDAP and filesystems is so poor that
> it isn't worth considering in any production environment.
>
> So, what does this performance testing mean?  If you are looking at
> receiving 1000 QPS (not a huge number!) for all of the zones that you
> are authoritative for, then one server will not be able to provide
> this level of performance.  You will have to add multiple servers.
> Ok, you need multiple servers anyway, two MySQL servers can provide
> this capability easily.  But what happens when you need to take one
> server down?  Now you are back to a single server and it has already
> been shown that one server can't support this rate of queries.  But
> one normal BIND server can handle this level by itself.  So, if you
> are going to use MySQL as the back end for your DNS data, then you
> need more than two servers to handle even 1,000 QPS at all times.
> (Please be aware that BIND-DLZ using the HPT DBD system can support
> these levels.  If I were to consider using BIND-DLZ, I would
> definitely be using DBD and not MySQL, just for performance
> consideration.)
>
> Now, as a DNS data provider to the Internet, you need to try and
> follow the suggestions provided in the RFCs.  In particular, look at
> RFC2182/BCP16, "Selection and Operation of Secondary DNS Servers".
> Your infrastructure needs to include DNS servers on separate networks
> with multiple connections to the Internet with no single point of
> failure.  Some large organizations do this by using multiple data
> centers.  Others provide this by having other organizations provide
> slave DNS service for them.  Either way, you avoid single points of
> failure.  As a small organization, having multiple data centers is
> extremely expensive.  But using another organization to provide slave
> DNS service requires coordination between the organizations and
> standardization on DNS server configuration/operation.
>
> DNS data replication time is a strong consideration.  Back in the
> days with simple zone transfers using polling (controlled by the SOA
> refresh/retry parameters), once or twice a day was considered
> acceptable, and if a server was down this could stretch into a day or
> more.  DNS data wasn't considered that dynamic.  Nowadays, dynamic
> DNS is commonly used to insure that a single master and one or more
> slave DNS server are kept in sync.  The DNS protocol, and BIND
> itself, provides for this synchronization without any additional work
> on your part.  This synchronization occurs within a short period of
> time, seconds or minutes at worst.  (Anyone have an idea as to the
> dynamic DNS update performance on BIND?  How many updates per second
> can be handled?  How quickly is dDNS data propagated?)
>
> If you use multiple master DNS servers, which is what you would have
> with multiple BIND-DLZ servers, then the synchronization of the data
> must be handled by an external mechanism.  This would be thru some
> database replication mechanism since BIND-DLZ doesn't have a
> mechanism to replicate the data itself.  Please note that having a
> single database feeding multiple DNS servers creates a single point
> of failure that should NOT be tolerated in a properly managed DNS
> system.
>
> Now, there is still one other possibility (at least).  You could have
> the master server fed from a database system and the slave(s) storing
> their data in standard BIND zone data files.  I don't believe that
> BIND-DLZ supports dynamic DNS updates, since the data store is a
> database, but I know that BIND-DLZ supports zone transfers (and
> hopefully incremental zone transfers).  This means that you could
> have a BIND-DLZ database driven master server and a standard BIND
> slave.  If someone very familiar with BIND-DLZ could address these
> points it would be very helpful.
>
> So, what does this mean for the scenarios originally listed.
>
> First, although manually managing DNS data is still commonly used, it
> is sort of passe. It works, it is well understood, there are lots of
> tools available for doing this, but dynamic DNS and/or BIND-DLZ seems
> to be the future for organizations needing to manage large amounts of
> DNS data.
>
> DNS zone data files can updated using dynamic DNS.  This is very well
> defined by DNS RFCs and well implemented by BIND.  Dynamic DNS is
> reasonable quick and reliable.
>
> Dynamic DNS doesn't result in a "flat" data file in that the updates
> are initially recorded in a journal file which is periodically
> incorporated into the base zone data file.  This is one reason that
> they say to NOT directly edit zone files that can be updated with
> dDNS while the server is running.  Just as in a normal database
> system, you have to insure that the system is quiescent while working
> with the database files, you need to insure that the DNS zone files
> are only written using a single mechanism to insure data integrity.
>
> Using a database driven DNS server is a very nice concept, but it
> isn't directly supported by BIND (as supplied by ISC).  This means
> that although it does work, and work well, it doesn't support all of
> the possibilities provided by the DNS protocol.  If it works, great,
> but if you need dynamic DNS support, then it won't work for you.  You
> also have to consider the infrastructure support requirements that
> are built into standard BIND but not included in BIND-DLZ.  The major
> concern here is DNS data replication between the DNS servers.  Also,
> no matter how you look at it, the query performance of BIND-DLZ is
> worse than straight BIND.  And, if you feel that you need to use an
> SQL database for maintaining your DNS data, then the performance will
> be extremely poor.
>
> So, if you feel that you MUST use a MySQL driven DNS system, then you
> need to closely consider:
>
> 	How are you going to replicate the DNS data between your servers.
>
> 	How are you going to ameliorate the poor performance of a MySQL
> driven DNS server.
>
> Obviously this doesn't address your original desire for a web based
> DNS management system, which has additional concerns itself.  With
> any web based system, my primary concerns are reliability (can you do
> something that will crash the whole system) and security (can someone
> do something with it that I don't want).  Both of these issues with
> your web based system could impact the operation of your DNS servers.
>
> Your original "So, how about the solution for my situation?" entails
> a very good understanding of your situation that only you can
> provide.  Even the "BIND & BIND" bible only provides suggestions and
> not solutions.  You need to identify what solution is right in your
> situation.
>
> Anyway, you questions are generating more questions/concerns by me
> about DNS operations.  Some are simple, others difficult, but the
> topic is a good one for this forum.  I realize that my response went
> well beyond your original question, which is why I changed the
> subject line.
>
> Bill Larson
>