ISC has begun implementing several methodology changes relating to BIND 9 development. The goals of these changes is to increase our software quality and relevance to you, our customers. Some of these are more internal, but we hope the outcome of these changes are that the effects are positive and noticed by those outside of ISC.
As with all changes, we’ll get some of it right and some we’ll have to revisit and modify as we learn. We welcome any feedback about where we are now, where you would like us to be, and as we progress along our path.
We hope to have a discussion with our customers rather than present our plans. If we are not getting something right, the sooner we know the better we are able to correct our path. Our customer base is very diverse so it will be difficult to implement each and every desired feature immediately. If we know of them and why they are important we have a better chance at making people happy. We cannot implement features or correct problems we do not know about.
We have begun some changes around August 2010, so some might already see a positive effect. We’re excited about what we have planned for 2011!
We have identified several primary goals and specific paths forward for each. The goals are measured against various metrics described for each goal.
The metrics we plan on using here are two-fold. The first metric is the feedback from customers and the direction it takes. The second metric will be comparing the list of problems known to us and, when we publish a release with what we think is an improvement, looking for a reduction in problem reports in that area.
Robustness encompasses the reliability and scalability of our product, as well as other things that make BIND 9 “just work.”
BIND 9 is a complicated project, both in how our code is written and the number of features this product contains. BIND 9 is a tool for general-purpose DNS service, and many times a new feature will cause unexpected interaction with existing features. Sometimes a specific feature may not meet up to customer requirements for performance.
In the past, ISC has focused more on the DNS protocol and related technology. This means we sometimes surprise a DNS administrator due to unexpected restrictions or protocol strictness. We know BIND 9 lives in the real world where things are not perfect, and we hope to increase our reliability when interacting with an authority server or client that is not strictly protocol compliant.
We have also recently had some problems in the quality of our releases. We have already altered some of our internal QA processes which we believe will have an immediate effect. Other short and long term plans will be described in our development methodology changes.
Usability includes operator workflow, “do what is expected”, and documenting how features work clearly. It is the “does what I want” as well as “does it how I want it” side.
Some of the usability issues we have had were because of our development model. We tend to have long release cycles where an alpha, beta, and release candidate are released late in our cycle. This means features are considered done before customers even see them.
We have chosen a development methodology which we believe will help address these issues immediately and help to improve customer involvement in the development of our tools and features.
Functionality encompasses the specific check-box items that are often needed: specific protocols, in-server functionality, and command line tools.
A product must not only add new features. It must also improve on existing features and remove features which are not used. Sometimes these changes will cause operators to need to change configuration files to start the server. We intend to introduce these operation-affecting changes using three steps: announcing the upcoming change in release notes, logging deprecation warnings when configuration files are loaded, and at some point later making the warnings into errors.
We hope most users will never encounter the warnings. This is a key point on when to give us feedback: if we are planning the removal of a feature you are using to effect, let us know.
Sustainability relates to how easy or difficult it is for ISC to maintain BIND 9 as a product, provide quality and effective support to our support customers, and to maintain a healthy pace for our employees.
While many of these are internal concerns, we hope they are still visible externally in ways that are not always contributed to this type of goal. An example of this is that a healthy development pace should reduce errors caused by rushing development or compressing schedules.
BIND 9 has historically been a difficult to approach open source project. Some of this is unchangeable because of how we got to where we are today, while some of this can and should change. We have some specific process changes which are intended to open up the project to more developers over time. It is unlikely we will attain the same level of openness which BIND 10, our eventual replacement for BIND 9, has had since its inception. BIND 10 and other very open projects are how we measure BIND 9’s success in this area.
Core development methodology change
ISC has chosen an Agile methodology known as Scrum. Scrum’s details are topics for books, and won’t be discussed here in detail. Some of the expected results of this process are important to note, however.
Customer interaction with the development process is purposeful rather than accidental. It is desired and sought rather than ad-hoc.
Features are presented earlier, perhaps before they are complete. This does not mean they are not usable, but they may not be perfectly polished. By using feedback on partially implemented features we hope to end with a better result than we could get to ourselves, or after much time has been invested in fully implementing a feature that is not as usable.
Shorter development cycles strung together, rather than a single, long, closed development cycle. Each new version release is broken into a number of smaller “sprints” where specific features are selected from a large pool of potential features. These are focused on, designed, and implemented during this sprint. The sprint duration recommended for Scrum is two to four weeks. We have chosen two weeks for BIND 9 sprints.
Development focus. During a sprint, engineers are protected from distractions whenever possible. We recognize some distractions are important enough to bring to the attention of a developer immediately, while others are less critical to address in an immediate urgent manner. We do not intend to delay support issues, security issues, or other truly urgent matters. We will have to feel out where the line lies. Some issues really need to interrupt a sprint, some can wait until the next sprint, and some can go into later sprints.
A sprint is a mini-release. The end of each development sprint is a potential release. This means that we could release code at the end of each sprint and those wishing to live on the edge and run the latest code with the newest features may do so. This does not mean we will do a full, formal release for every sprint, so don’t think we are asking people to upgrade every two weeks. We will continue to keep our ESV-style “stable” releases stable, and our current numbering scheme with around 3-4 month releases. Those who wish to remain fully on a supported stable release train are encouraged to do so. This new concept allows for those who also wish to live closer to the edge to do so as well.
Openness is more than just saying what we are doing or what we did. It is also about opening the doors to those not currently part of the development process and encouraging a vibrant community involvement. We see three areas of improvement which we believe will have a positive effect in terms of project openness.
More direct access to our source code. Today, we release snapshots of our code at points in time. These snapshots are usually associated along our release process: alpha, beta, release candidate, and release. A logical place to add more frequent releases is at sprint boundaries. While this helps, it is not our end goal. We hope to provide for public read-only access to our code at any point in the development cycle. Changes we make should not wait weeks before being visible.
Encourage development outside of ISC by providing a writable repository for non-ISC developers. Access to make changes in this repository should be a light-weight process. Official ISC releases are put in this repository as well as the public read-only repository.
We also recognize that we have not been able to work well with non-ISC developers in the past. Part of this can be solved by describing how to submit code that is likely to make it into BIND 9, and what the restrictions are on code we may accept. If we simply accepted whatever was thrown at us no one would like the outcome. However, we tended to reject all but the most simple of change, and that is a disservice as well.
Open up the bug ticketing system. Specifically, this means the open-submission ticketing system. Our concern over opening up our list of bugs to the public is primarily concern over security issues, as well as the current history of bugs in our queue.
To address these two issues, we do not plan on marking existing bug reports for public view. New submissions may be marked if they are not security related. If the person submitting the bug report marks a ticket as security, or if we believe it has security impact, we may choose to not open that ticket.
Increased BIND Forum involvement. Our BIND Forum has been around for some time, but it has not become the forum we had hoped for. We believe it will benefit from our general plans to open our development processes, but we also seek ways in which we can provide a framework in which forum members may more easily discuss topics with ISC staff and one another.
We have outlined some ambitious goals for 2011. We believe progress on each is attainable in a measurable way and will lay out a more detailed plan to attain these goals in the future.
To help us in this, please feedback early, and feedback often.