- By Adib Behjat on January 4, 2011
The exhaustion of IPv4 space from IANA is coming as soon as February (yes, next month!) and the reserve held by the RIRs will be running dry shortly thereafter. The ability to provide (and use) IPv6 infrastructure is no longer optional; it is a requirement.
ISC, unlike others who may talk the talk in regard to IPv6 experience, has been walking the walk with a thorough understanding of the operational aspects of IPv6 since 2002 when we began testing IPv6 in the F-root DNS nameservers running BIND, ISC’s DHCP product added IPv6 support in 2007. ISC’s AFTR [router software designed specifically to ease transition to an IPv6 backbone network], was released in 2009. SNS – ISC’s DNS hosting service – has supported IPv6 since its inception in 2010. We’re ready and willing to share our knowledge.
ISC is continuing to show leadership and innovation in the IPv6 arena, coming out with valuable training in how to implement IPv6. Everything from an introduction to the IPv6 protocol to specific deployment information via hands-on labs and interactive discussion. The class will also include new technologies for bridging the chasm between IPv4 and IPv6 deployments including information on ISC’s AFTR implementation.
Our initial class will be held at our Redwood City office January 19 – 21. This is the first run of the class, and as such, we are declaring it to be a “beta” — it is 3-days in length where we expect the following IPv6 classes to be 2 days in length. Beyond that difference, I can say that the class will be the same high quality that is expected from all ISC training.
For years, we’ve heard about IPv6 and the need for deployment. For years, we’ve heard that IPv4 space is running out. It’s true. The time is now, so come and get trained.
- By Adib Behjat on December 3, 2010
ISC has been working with Tridge from the Samba team to make it easier to configure BIND 9 to use GSS-TKEY. GSS-TKEY is used to allow Windows clients to securely update DNS zones using dynamic DNS, primarily in an Active Directory environment.
These changes may be coming as early as BIND 9.8.0, which is scheduled to be released in late January, 2011. They were proposed and written by the Samba team in order to more easily integrate BIND 9 with Samba 4.
The first change allows for automatic configuration of many GSSAPI configuration parameters which previously had to be specified in
Now, all you must do is point the
tkey-gssapi-keytaboption to the Kerberos 5 keytab which contains the credentials the server has access to. This will cause the Kerberos 5 keytab to be searched for the appropriate match based upon the client’s request.
This change also means named does not rely on system environment settings such as KEYTAB_FILE or KRB5_KTNAME. Putting the keytab name in the configuration file makes it easier to reliably start named from system startup scripts and manually, and do so consistently.
Another upcoming change is to prevent performing tests on the Kerberos 5 KDC (key distribution center) during named startup. Doing these tests caused an artificial system startup requirement where the KDC must be running for BIND to start, but the KDC may rely on DNS queries to start.
With this change in place, BIND will only need the KDC to be operational during the authentication phase rather than at all times. This change only affects automatic configurations, so previous behavior is unchanged.
Join The Global Passive DNS (pDNS) Network Today & Gain Effective Tools To Fight Against Cyber CrimeBy Adib Behjat on December 3, 2010
Why contribute passive DNS data to ISC?
ISC - the Public Benefit Company that works to sustain the spirit of the Internet – is expanding the capacity of our Passive DNS System. Passive DNS provides the industry greater insight into how the cyber-criminals are using DNS to violate the Internet.
Vetted organizations are invited to join the pDNS network by configuring their DNS infrastructure to be a passive DNS sensor (pDNS). Once you join, your system becomes a part of the global pDNS network, helping to fight against cybercrime gaining you access to new and effective tools.
Passive DNS is a very scalable network design and has minimal operational impact. As an additional bonus for participating, all vetted organizations that contribute Passive DNS will have access to the DNS Database (DNSDB) at the ISC Security Information Exchange (SIE) – an investigative tool that we use to analyze the cyber-criminal’s use of DNS. By participating in this effort, you are expanding the data collected, thereby enabling greater insights into how the cyber-criminals are using DNS to exploit the Internet.
What is Passive DNS?
“Passive DNS” or “passive DNS replication” is a technique invented by Florian Weimer in 2004 to opportunistically reconstruct a partial view of the data available in the global Domain Name System into a central database where it can be indexed and queried.
Passive DNS databases are extremely useful for a variety of purposes. Malware and e-crime rely heavily on the DNS, and so-called “fast flux botnets” abuse the DNS with frequent updates and low TTLs. Passive DNS databases can answer questions that are difficult or impossible to answer with the standard DNS protocol, such as:
· Where did this domain name point to in the past?
· What domain names are hosted by a given nameserver?
· What domain names point into a given IP network?
· What subdomains exist below a certain domain name?
Joining the network by operating a passive DNS sensor is an effective and easy way to contribute to the global online anti-abuse effort. The data is ‘aggregated’, thus not linkable to the specific devices making the query.
Passive DNS Data Collection
Passive DNS only replicates the inter-server traffic between caching recursive nameservers and authoritative nameservers. Data capture only occurs when a recursive DNS cache experiences a cache miss and must query in order to obtain needed data.
Passive DNS replication is inherently both efficient and privacy preserving. Only a small portion of the network traffic generated by a DNS server needs to be captured and backhauled. This is more efficient than the traditional “DNS logging”.
No client IP addresses are ever captured in the data requested by the recursive DNS cache. The lack of the client IP addresses preserves privacy. In addition, the DNS cache data is reused. Large, busy DNS caches are thus naturally very effective at protecting the privacy of individual users.
ISC has produced an easy to install passive DNS sensor program, which can be installed directly on either production DNS servers or a monitoring server connected to a “tap” or “SPAN” port. The sensor uses libpcap to collect the upstream packets generated by the recursive DNS server and then periodically uploads the data in compressed form to collection servers operated by ISC’s Security Information Exchange (SIE) project. Extremely busy DNS servers typically produce only 1-5 megabytes of data per minute.
DNS Database (DNSDB)
Passive DNS data is uploaded into the SIE network, where it is distributed to vetted security researchers that maintain contractual relationships with SIE that prohibit unauthorized redistribution. ISC has invested a considerable amount of resources into analyzing and aggregating the large volumes of passive DNS data submitted to SIE. One of the results of this effort is the ISC Passive DNS Database (DNSDB), a database cluster that stores unique DNS records witnessed in the passive DNS data.
In exchange for providing passive DNS data to the ISC DNSDB project, vetted operators of passive DNS sensors are entitled to no-fee access to ISC DNSDB’s easy-to-use web search interface.
As can be seen in figure 2, DNSDB provides insight to criminal activity. Using a domain from SPAM E-mail, DNSDB can trace the DNS activity to uncover associated domains which have been or will be used for future SPAM.
How to Participate?
If you are interested in joining and operating a passive DNS sensor on behalf of ISC or have questions that are not answered by this document, please send email to firstname.lastname@example.org.
The ISC SIE website:
The ISC DNSDB website:
SIE DNS sensor packages:
Red Hat / CentOS and Debian:
Papers and Materials:
Florian Weimer’s original paper:
- By Adib Behjat on November 30, 2010
Yesterday I blogged about how ISC has been changing our internal development practices for BIND 9. Today, with the release of several security patches, I wanted to talk a bit on how they have helped us already.
In many projects, and previously in BIND 9, tests were written after the code was working. This left writing automated tests as an afterthought at best, and meant our tests were not as robust. One common problem was that the test didn’t actually test what we thought it did.
Now BIND 9 developers at ISC write a test first, which highlights the defect. Once a patch is found, the test should begin to pass. This confirms that we have properly repaired the defect and we have a test to prove it.
Filling in tests
“Untested code is buggy code.” This is a reality in any project regardless of size. We’ve started doing something about our untested code.
One of the security flaws found today was a product of this back-fill of test coverage. We started with the most important types of features (ACLs in this case) and started writing tests based on documented behavior. The output of this is either a bug fix or a documentation fix, when an error is found.
As we continue to modernize and improve our development practices within ISC, expect us to find more bugs. We prefer to find them before our customers do, and not only when they are security related. We also hope to catch more before they get out the door.
- By Adib Behjat on November 30, 2010
ISC has begun implementing several methodology changes relating to BIND 9 development. The goals of these changes is to increase our software quality and relevance to you, our customers. Some of these are more internal, but we hope the outcome of these changes are that the effects are positive and noticed by those outside of ISC.
As with all changes, we’ll get some of it right and some we’ll have to revisit and modify as we learn. We welcome any feedback about where we are now, where you would like us to be, and as we progress along our path.
We hope to have a discussion with our customers rather than present our plans. If we are not getting something right, the sooner we know the better we are able to correct our path. Our customer base is very diverse so it will be difficult to implement each and every desired feature immediately. If we know of them and why they are important we have a better chance at making people happy. We cannot implement features or correct problems we do not know about.
We have begun some changes around August 2010, so some might already see a positive effect. We’re excited about what we have planned for 2011!
We have identified several primary goals and specific paths forward for each. The goals are measured against various metrics described for each goal.
The metrics we plan on using here are two-fold. The first metric is the feedback from customers and the direction it takes. The second metric will be comparing the list of problems known to us and, when we publish a release with what we think is an improvement, looking for a reduction in problem reports in that area.
Robustness encompasses the reliability and scalability of our product, as well as other things that make BIND 9 “just work.”
BIND 9 is a complicated project, both in how our code is written and the number of features this product contains. BIND 9 is a tool for general-purpose DNS service, and many times a new feature will cause unexpected interaction with existing features. Sometimes a specific feature may not meet up to customer requirements for performance.
In the past, ISC has focused more on the DNS protocol and related technology. This means we sometimes surprise a DNS administrator due to unexpected restrictions or protocol strictness. We know BIND 9 lives in the real world where things are not perfect, and we hope to increase our reliability when interacting with an authority server or client that is not strictly protocol compliant.
We have also recently had some problems in the quality of our releases. We have already altered some of our internal QA processes which we believe will have an immediate effect. Other short and long term plans will be described in our development methodology changes.
Usability includes operator workflow, “do what is expected”, and documenting how features work clearly. It is the “does what I want” as well as “does it how I want it” side.
Some of the usability issues we have had were because of our development model. We tend to have long release cycles where an alpha, beta, and release candidate are released late in our cycle. This means features are considered done before customers even see them.
We have chosen a development methodology which we believe will help address these issues immediately and help to improve customer involvement in the development of our tools and features.
Functionality encompasses the specific check-box items that are often needed: specific protocols, in-server functionality, and command line tools.
A product must not only add new features. It must also improve on existing features and remove features which are not used. Sometimes these changes will cause operators to need to change configuration files to start the server. We intend to introduce these operation-affecting changes using three steps: announcing the upcoming change in release notes, logging deprecation warnings when configuration files are loaded, and at some point later making the warnings into errors.
We hope most users will never encounter the warnings. This is a key point on when to give us feedback: if we are planning the removal of a feature you are using to effect, let us know.
Sustainability relates to how easy or difficult it is for ISC to maintain BIND 9 as a product, provide quality and effective support to our support customers, and to maintain a healthy pace for our employees.
While many of these are internal concerns, we hope they are still visible externally in ways that are not always contributed to this type of goal. An example of this is that a healthy development pace should reduce errors caused by rushing development or compressing schedules.
BIND 9 has historically been a difficult to approach open source project. Some of this is unchangeable because of how we got to where we are today, while some of this can and should change. We have some specific process changes which are intended to open up the project to more developers over time. It is unlikely we will attain the same level of openness which BIND 10, our eventual replacement for BIND 9, has had since its inception. BIND 10 and other very open projects are how we measure BIND 9’s success in this area.
Core development methodology change
ISC has chosen an Agile methodology known as Scrum. Scrum’s details are topics for books, and won’t be discussed here in detail. Some of the expected results of this process are important to note, however.
Customer interaction with the development process is purposeful rather than accidental. It is desired and sought rather than ad-hoc.
Features are presented earlier, perhaps before they are complete. This does not mean they are not usable, but they may not be perfectly polished. By using feedback on partially implemented features we hope to end with a better result than we could get to ourselves, or after much time has been invested in fully implementing a feature that is not as usable.
Shorter development cycles strung together, rather than a single, long, closed development cycle. Each new version release is broken into a number of smaller “sprints” where specific features are selected from a large pool of potential features. These are focused on, designed, and implemented during this sprint. The sprint duration recommended for Scrum is two to four weeks. We have chosen two weeks for BIND 9 sprints.
Development focus. During a sprint, engineers are protected from distractions whenever possible. We recognize some distractions are important enough to bring to the attention of a developer immediately, while others are less critical to address in an immediate urgent manner. We do not intend to delay support issues, security issues, or other truly urgent matters. We will have to feel out where the line lies. Some issues really need to interrupt a sprint, some can wait until the next sprint, and some can go into later sprints.
A sprint is a mini-release. The end of each development sprint is a potential release. This means that we could release code at the end of each sprint and those wishing to live on the edge and run the latest code with the newest features may do so. This does not mean we will do a full, formal release for every sprint, so don’t think we are asking people to upgrade every two weeks. We will continue to keep our ESV-style “stable” releases stable, and our current numbering scheme with around 3-4 month releases. Those who wish to remain fully on a supported stable release train are encouraged to do so. This new concept allows for those who also wish to live closer to the edge to do so as well.
Openness is more than just saying what we are doing or what we did. It is also about opening the doors to those not currently part of the development process and encouraging a vibrant community involvement. We see three areas of improvement which we believe will have a positive effect in terms of project openness.
More direct access to our source code. Today, we release snapshots of our code at points in time. These snapshots are usually associated along our release process: alpha, beta, release candidate, and release. A logical place to add more frequent releases is at sprint boundaries. While this helps, it is not our end goal. We hope to provide for public read-only access to our code at any point in the development cycle. Changes we make should not wait weeks before being visible.
Encourage development outside of ISC by providing a writable repository for non-ISC developers. Access to make changes in this repository should be a light-weight process. Official ISC releases are put in this repository as well as the public read-only repository.
We also recognize that we have not been able to work well with non-ISC developers in the past. Part of this can be solved by describing how to submit code that is likely to make it into BIND 9, and what the restrictions are on code we may accept. If we simply accepted whatever was thrown at us no one would like the outcome. However, we tended to reject all but the most simple of change, and that is a disservice as well.
Open up the bug ticketing system. Specifically, this means the open-submission ticketing system. Our concern over opening up our list of bugs to the public is primarily concern over security issues, as well as the current history of bugs in our queue.
To address these two issues, we do not plan on marking existing bug reports for public view. New submissions may be marked if they are not security related. If the person submitting the bug report marks a ticket as security, or if we believe it has security impact, we may choose to not open that ticket.
Increased BIND Forum involvement. Our BIND Forum has been around for some time, but it has not become the forum we had hoped for. We believe it will benefit from our general plans to open our development processes, but we also seek ways in which we can provide a framework in which forum members may more easily discuss topics with ISC staff and one another.
We have outlined some ambitious goals for 2011. We believe progress on each is attainable in a measurable way and will lay out a more detailed plan to attain these goals in the future.
To help us in this, please feedback early, and feedback often.