For the past year, the ever-growing DMDirc code base has been managed using the Git version control system. We still mirror our code to an SVN repository, and a couple of our automated tools still use the SVN version of things, but we’re slowly and surely modifying, upgrading or scrapping these in favour of Git-based solutions.

Recently we began a trial of using Gerrit to manage commits made to the main DMDirc repositories for the client and the parser. Gerrit is a code review tool, which developers submit their changes to (instead of pushing them directly to the Git repositories); other developers can then review the changes, test them, and leave feedback, before ultimately approving or rejecting the change. Only once a change has been approved is it allowed to go into the repository.

This has helped us greatly improve the quality of the code that makes it into the client. Another advantage to using Gerrit is that it now allows us to open up the repositories a bit more and allow third-party contributions with much more ease than before: it is now possible for anybody to sign up to Gerrit and begin committing to DMDirc right away! We even have a guide on our Git wiki page detailing how to do just that.

If you would like to contribute to DMDirc, either by committing (via Gerrit!) or in other ways, we suggest paying a visit to #DMDirc on either Quakenet, Freenode, OTFC, Coldfront or Chatspike! (Don’t worry: we have a system in place to link all the channels together!)

Once again, we are looking at improving the DMDirc release cycle. First, a bit of history. When we first started with DMDirc back in 2007 we had very regular releases, but these have slowly started getting longer and longer:

• 0.1 was released on the 1st of March 2007
• 0.2 quickly followed 5 days later (6th March)
• 0.3 was 1 month, 18 days after that (24th April)
• 0.4 followed after 1 month, 18 days again (11th June)
• 0.5 then came 3 months, 15 days later (26th September)
• 0.5.1 was then released as a bug fix a mere 11 days after that (7th October)

So that was six releases in just over 7 months. We were happy to be doing regular releases as we felt it was more likely to keep users with us as the client evolved quickly. However, this pace soon died down and 3 months, 23 days later (which was at the time the longest we’d gone without a release), with no sign of 0.6 being finished any time soon, we made the decision to release 0.5.5. This was intended as a half-way point between 0.5 and 0.6 to try and keep the releases regular, with the idea that 0.6 would follow in a similar time scale. This was not quite the case, as it took us a further 7 months, 12 days before 0.6 was released!

Shortly before the release of 0.6 we looked at the time between releases, and took the decision to split our workload for 0.7 into 3 separate parts (as described here) in order to try and bring the release time down to a more reasonable time scale. After some time it became obvious from the amount of outstanding issues for 0.6.3 that this again wouldn’t work, as six months in we still had a large number of major features outstanding. We therefore decided once again to split the release further into a set of “milestone” releases — 0.6.3m1 (which we’ve just released), 0.6.3m2 (up next) and then probably 0.6.3 proper. These milestone releases were supposed to be point in time where we thought the client was in a good state that we would finish anything we were working on, tidy up the code and release.

Well, 9 months, 16 days after the release of 0.6 we released the long-awaited version 0.6.3m1. We once again took a hard look at this and have come to the conclusion that this simply isn’t acceptable, despite two attempts at breaking the release down it still took us 9 months to do what essentially boils down to 1/9th of the 0.7 release we had originally planned (assuming that 0.6.6 and 0.7 will also be split into two milestone releases).

So now we are going to introduce yet another change to our release cycle. We are going to continue with 0.6.3, 0.6.6 and 0.7 as separate release targets, but instead tweak how we handle milestone releases. Currently we decide to release a milestone randomly when we feel that the client is stable, instead we are going to try and stick to a reasonably fixed release cycle of 4 months in between releases. Approximately 3 months after the last release we are going to start the process of preparing for a milestone release. This will entail a feature freeze and the bumping of any non-essential issues back to the next milestone. Following this we will finish any currently outstanding issues (or bump them if there is still too much to do), a reasonably quick QA cycle (during which we open up the UNSTABLE updater channel to allow for testing of the upcoming release) and then release after the 4th month and repeat.

So with this new release cycle we hope to go back to our original faster-paced release strategy without losing any of the quality we like in a release, and should be looking at releasing the next milestone on or around the 1st of November, so watch out during October for news of the pending release of 0.6.3m2!

One of the things that makes DMDirc good (in my opinion) is that the three main developers actively use it pretty much all day, every day. We are all very, very heavy IRC users (unlike the author of mIRC for instance, who admits to only occasionally using IRC), which is what led to us [re]starting DMDirc development — when we all switched to Linux, we couldn’t find an existing client that any of us really liked.

From the start we wanted DMDirc to be an excellent client, and we quickly got it into a usable state (I have logs showing me using it in early May 2007). To help us do this we registered with Google Code, which provided us with a Subversion repository, download hosting and an issue tracker.

In addition to the tools on Google Code we also wrote a service that received error reports from DMDirc clients, and allowed developers to browse them using a web interface. This system never really worked too well; it meant that we in effect had two systems for tracking errors, the automated error system (which was checked manually, and therefore quite infrequently) and the user-facing Google Code issue tracker. The Google Code issue tracker also had quite a few issues of its own [ba-dum-tish] that posed problems for us: we didn’t have much control over it, Google were slow to improve it, and it didn’t allow for dependencies or private issues. The lack of private issues was the reason we felt the need to have two separate systems at the time — automatically reported errors may in some cases contain private information, so we can’t show them publicly.

Sometime just before DMDirc 0.5.5 was released, we decided to start using Mantis as our primary issue tracker. Mantis, whilst not perfect, gave us the control we wanted — it’s written in PHP (which, like Java, we all know very well), and is easy enough to modify. We imported all of our existing issues from Google Code, using a Python script (yay for multi-language proficiency!) to screenscrape the website and build an XML file, and then a PHP script to read the XML file and import the issues into Mantis. With the issues imported, we set about modifying Mantis to better suit our needs.

We (well, Chris and I) had got quite used to using the issue ‘grid view’ offered by Google code, so one of the first things we did when switching to Mantis was implementing our own version of it. We also improved the error reporting backend to raise new Mantis issues (marked as private, of course). Later on, we also enabled SVN integration, which initially meant writing a script to parse e-mails from Google Code (where our SVN repository was still hosted) and passing the information on to Mantis for it to use. After a while, we modified the integration to include links to our Fisheye instance, and also map Subversion users to Mantis users in order to add comments with the appropriate username instead of just ‘SVN user’.

More recently, we’ve improved the Fisheye aspect of the integration dramatically; we now access Fisheye and include its representation of the changes directly in Mantis, allowing you to see a list of affected files and even view diffs of the changes without leaving the issue page. We also parse any stack traces in the issue description, and automatically link relevant lines to the Fisheye view of the specified file and revision, if the information is available. You can see an example of both of these modifications in this issue.

Now, as I mentioned above, we are all heavy IRC users so having to periodically check Mantis for new bugs wasn’t very appealing to us, so we modified it some more so that it communicated with an IRC bot we have which is designed to relay notifications. Mantis tells us everything that happens — new issues, edited issues, property changes on issues — anything that happens there, we instantly know about it.

This setup worked well for us for about 4 months, after which we decided to switch over to using a self-hosted SVN Repository, which opened up even more possibilities. We added post-commit hooks to replace the email-parsing hacks for Mantis support, enabled reporting of commits to IRC, and created a pre-commit hook to make sure that every commit was linked to an issue on Mantis.

Not long after that we discovered and implemented Bamboo into our development process. Bamboo is a Continuous Integration server, which means that every time we commit something to the trunk (or active branches), it will rebuild the project on our server, run our collection of unit tests and then publish and store the results. Bamboo also notifies us via Jabber if you cause a build to fail. Of course this wasn’t quite perfect, and after a short while we discovered a plugin that can execute post-build scripts. One Bash script later and Bamboo was reporting to IRC like everything else. We also, of course, integrated Bamboo with Mantis. If you check this issue again you will see that the results of the build from Bamboo are also reported under the the commit that triggered it.

As mentioned before, we have a collection of unit tests to make sure things work as expected. Every so often we generate Clover reports to show how much code coverage we have, and in the last month we have also discovered and began using the FEST Swing library to allow us to start unit testing the DMDirc GUI.

So as you can see, what we have now is a very tightly integrated system to help ensure that DMDirc is a quality product. Everything reports to IRC so we know as soon as things happen, our issue tracker is almost omnipotent in that it can tell nearly everything about an issue from being raised to being fixed, and to top it all off, we have Nagios running and monitoring it all for us (and of course, this reports to IRC also!)

Of course this setup wouldn’t be possible if it wasn’t for a few choice products provided for free — Mantis, Fisheye, Bamboo, Clover and many others. We would like to thank the developers of these products for helping us make DMDirc great.

Hi everyone,

As a newcomer to the DMDirc project, I’d thought I’d introduce myself. I’m Chris ‘laser’ Northwood, and just before the releases of the 0.5 release candidates, I was asked by the other developers on DMDirc to become the QA Lead.

My approach will hopefully become fully implemented for the 0.6 release, and we’ll hopefully see this by having fewer bugfix releases (particularly, less release candidates), as I did not have enough time to implement this for the 0.5 release.

My experience in testing is mainly in functional testing (I’m currently working as a Test Engineer for Sophos, a computer security company), so I’m going to be mainly concerning myself with functional testing, leaving the current unit testing approach to be worked on by the other developers.

The biggest problem I’ve found since joining the DMDirc team is a lack of documentation, which is essential to ensure a good set of test coverage. Greboid has so far developed a pretty good functional specification for the UI, which I can work from, however there are a few other changes I’m hoping to make to ensure the focus on QA. First off is a “Requirements Dashboard”, matching new, changed and existing functionality to a specific requirement, allowing us to check dev status and test case coverage for a specific requirement (see, this mockup dashboard).

Each requirement should have a test case to verify the functionality before testing can begin to ensure a good testing strategy to increase quality of the DMDirc releases.

We currently use a handful of different tools (specifically, Google Code Issue Tracker and Litmus) for QA, which although are good on their own, do not necessarily integrate together well, which is where my dashboard idea is hoping to pull them all together.

I’m also hoping to change the release strategy of DMDirc. Currently, testing is continuous on feature incomplete nightly builds, and release candidates are generated as soon as DMDirc is feature complete. Some of my changes are mainly semantic, but should hopefully clarify the process.

• Nightly builds are still generated, but these are mainly for quick testing and developers rather than the more extensive general testing.
• Once all features for that milestone are implemented (i.e., the dev status on the dashboard is all green), and all defects raised against the last release of DMDirc fixed, a Feature Complete build is released with which the DMDirc QA team can run test cases. Additionally, any defects raised against the last release of DMDirc are verified at this point.
• Any raised bugs are then fixed by the dev team until all test cases have been executed, and raised defects fixed, when the developers release second build to the DMDirc testers, who verify the defects raised against the previous build.
• When the testers are satisfied that there are no unverified raised defects, and nothing caught in testing, a release candidate is published, which will allow DMDirc to be tested in user environments.
• If any issues are raised by the release candidate, a new one is released until these are resolved.
• When a release candidate has been released and no issues identified with it, the release candiate becomes a released final build, and developers can move on to developing new features for the next milestone.

I’ve created a workflow to hopefully show how I intend this to work:

This is not too dissimilar to the current process, just slightly formalised.

Hopefully I’ll be able to implement this with no real trouble, which means that come 0.6, you’ll be able to see the benefits in a more bugfree release.

Chris Northwood,
QA Lead