DMDircthe intelligent IRC client

One of the things that makes DMDirc good (in my opinion) is that the three main developers actively use it pretty much all day, every day. We are all very, very heavy IRC users (unlike the author of mIRC for instance, who admits to only occasionally using IRC), which is what led to us [re]starting DMDirc development — when we all switched to Linux, we couldn’t find an existing client that any of us really liked.

From the start we wanted DMDirc to be an excellent client, and we quickly got it into a usable state (I have logs showing me using it in early May 2007). To help us do this we registered with Google Code, which provided us with a Subversion repository, download hosting and an issue tracker.

In addition to the tools on Google Code we also wrote a service that received error reports from DMDirc clients, and allowed developers to browse them using a web interface. This system never really worked too well; it meant that we in effect had two systems for tracking errors, the automated error system (which was checked manually, and therefore quite infrequently) and the user-facing Google Code issue tracker. The Google Code issue tracker also had quite a few issues of its own [ba-dum-tish] that posed problems for us: we didn’t have much control over it, Google were slow to improve it, and it didn’t allow for dependencies or private issues. The lack of private issues was the reason we felt the need to have two separate systems at the time — automatically reported errors may in some cases contain private information, so we can’t show them publicly.

Sometime just before DMDirc 0.5.5 was released, we decided to start using Mantis as our primary issue tracker. Mantis, whilst not perfect, gave us the control we wanted — it’s written in PHP (which, like Java, we all know very well), and is easy enough to modify. We imported all of our existing issues from Google Code, using a Python script (yay for multi-language proficiency!) to screenscrape the website and build an XML file, and then a PHP script to read the XML file and import the issues into Mantis. With the issues imported, we set about modifying Mantis to better suit our needs.

We (well, Chris and I) had got quite used to using the issue ‘grid view’ offered by Google code, so one of the first things we did when switching to Mantis was implementing our own version of it. We also improved the error reporting backend to raise new Mantis issues (marked as private, of course). Later on, we also enabled SVN integration, which initially meant writing a script to parse e-mails from Google Code (where our SVN repository was still hosted) and passing the information on to Mantis for it to use. After a while, we modified the integration to include links to our Fisheye instance, and also map Subversion users to Mantis users in order to add comments with the appropriate username instead of just ‘SVN user’.

More recently, we’ve improved the Fisheye aspect of the integration dramatically; we now access Fisheye and include its representation of the changes directly in Mantis, allowing you to see a list of affected files and even view diffs of the changes without leaving the issue page. We also parse any stack traces in the issue description, and automatically link relevant lines to the Fisheye view of the specified file and revision, if the information is available. You can see an example of both of these modifications in this issue.

Now, as I mentioned above, we are all heavy IRC users so having to periodically check Mantis for new bugs wasn’t very appealing to us, so we modified it some more so that it communicated with an IRC bot we have which is designed to relay notifications. Mantis tells us everything that happens — new issues, edited issues, property changes on issues — anything that happens there, we instantly know about it.

This setup worked well for us for about 4 months, after which we decided to switch over to using a self-hosted SVN Repository, which opened up even more possibilities. We added post-commit hooks to replace the email-parsing hacks for Mantis support, enabled reporting of commits to IRC, and created a pre-commit hook to make sure that every commit was linked to an issue on Mantis.

Not long after that we discovered and implemented Bamboo into our development process. Bamboo is a Continuous Integration server, which means that every time we commit something to the trunk (or active branches), it will rebuild the project on our server, run our collection of unit tests and then publish and store the results. Bamboo also notifies us via Jabber if you cause a build to fail. Of course this wasn’t quite perfect, and after a short while we discovered a plugin that can execute post-build scripts. One Bash script later and Bamboo was reporting to IRC like everything else. We also, of course, integrated Bamboo with Mantis. If you check this issue again you will see that the results of the build from Bamboo are also reported under the the commit that triggered it.

As mentioned before, we have a collection of unit tests to make sure things work as expected. Every so often we generate Clover reports to show how much code coverage we have, and in the last month we have also discovered and began using the FEST Swing library to allow us to start unit testing the DMDirc GUI.

So as you can see, what we have now is a very tightly integrated system to help ensure that DMDirc is a quality product. Everything reports to IRC so we know as soon as things happen, our issue tracker is almost omnipotent in that it can tell nearly everything about an issue from being raised to being fixed, and to top it all off, we have Nagios running and monitoring it all for us (and of course, this reports to IRC also!)

Of course this setup wouldn’t be possible if it wasn’t for a few choice products provided for free — Mantis, Fisheye, Bamboo, Clover and many others. We would like to thank the developers of these products for helping us make DMDirc great.

As previously mentioned, once Apple released Java6 for OS X I set about updating the scripts we use for building the installers and nightlies so that we could support OS X officially for DMDirc 0.6.

To do this we needed to be able to create a DMG file to distribute it in, and we wanted this to be done by our current build scripts. As you may be aware, we do all of our development on Ubuntu Linux (which we also use on the DMDirc build/web server) so our current build scripts are a bunch of bash scripts that build the various installers (.run files for Linux, and .exes for Windows).

Other projects exist which distribute their cross-platform releases on Linux, OS X and Windows, so I looked at the build processes for both Firefox and Inkscape which both have apple releases. Unfortunately both of these projects appear to use an Apple machine to create the releases. This roughly translated to the following:

	# DMDirc-dmg contains the files we want to put in the dmg
	hdiutil create -volname "DMDirc" -fs HFS+ -srcfolder DMDirc-dmg -format UDRW DMDirc.RW.dmg
	hdiutil attach DMDirc.RW.dmg
	bless -openfolder /Volumes/DMDirc
	hdiutil detach /Volumes/DMDirc
	hdiutil convert DMDirc.RW.dmg -format UDZO -imagekey zlib-level=9 -o DMDirc.dmg

Which, while it does work (and is what we use if the installer is being built on OS X), wasn’t an acceptable solution as it relied on an OS X install.

I then discovered a wiki page which describes a method that works under Linux. After compiling a Linux-compatible version of Apple’s mkhfs program we could now do something along the lines of:

	# DMDirc-dmg contains the files we want to put in the dmg
	SIZE=$((`du -sb DMDirc-dmg | awk '{print $1}'`  + 10))
	if [ ${SIZE} -lt 4194304 ]; then
		echo "Size is less than 4MB"
		SIZE=4194304;
	fi;
	dd if=/dev/zero of=DMDirc.img bs=${SIZE} count=1
	mkfs.hfsplus -v 'DMDirc' DMDirc.img

	mkdir DMDirc.img-mounted
	mount DMDirc.img-mounted
	MOUNTRES=${?}
	if [ ${MOUNTRES} -ne 0 ]; then
		# Try using full parameters - could be running as root.
		mount -t hfsplus -o loop DMDirc.img DMDirc.img-mounted
		MOUNTRES=${?}
	fi;
	if [ ${MOUNTRES} -ne 0 ]; then
		echo "Unable to mount, need root access or an fstab entry like:"
		echo "${PWD}/DMDirc.img ${PWD}/DMDirc.img-mounted auto users,noauto,loop 0 0"
		exit 1;
	fi;
	mv -fv $DMDirc-dmg/* DMDirc.img-mounted
	mv -fv DMDirc-dmg/.[A-Za-z]* DMDirc.img-mounted
	umount DMDirc.img-mounted
	mv DMDirc.img DMDirc.dmg

This was an improvement, as images could now be created under Linux, however it still also had lots of problems:

• images have to be at least 4MB (compared to 1.3mb when created using the OS X method),
• they’re created read-write (compared to read-only when created using the OS X method), and
• they do not auto-open when mounted (the OS X method can “bless” the image).

After a short while, I discovered the gentoo wiki had a page on hfsplus, which had a version of diskdev-cmds that allowed for images that were as small as 512kb. Using this, I was able to edit the first part of above code to something like:

	# DMDirc-dmg contains the files we want to put in the dmg
	SIZE=$((`du -sb DMDirc-dmg | awk '{print $1}'`  + 10))
	if [ ${SIZE} -lt 524288 ]; then
		echo "Size is less than 512kb"
		SIZE=524288;
	fi;
	dd if=/dev/zero of=DMDirc.img bs=${SIZE} count=1
	mkfs.hfsplus -v 'DMDirc' DMDirc.img
	# however older versions of mkfs.hfs will only create 4mb+ sized images :/
	if [ $? -eq 1 ]; then
		if [ ${SIZE} -lt 4194304 ]; then
			echo "Size is less than 4MB"
			SIZE=4194304;
		fi;
		dd if=/dev/zero of=DMDirc.img bs=${SIZE} count=1
		mkfs.hfsplus -v 'DMDirc' DMDirc.img
	fi;

This still produced 3.8mb images, so basically had all the same problems as the old method. The actual code we used was also a lot less tidy as sometimes the size returned by du was too small, so the code looped increasing the size until it got a successful image.

However I recently discovered a much tidier method, which we now use. mkisofs has the ability to create and bless hfs images:

	# DMDirc-dmg contains the files we want to put in the dmg
	mkisofs -V 'DMDirc' -no-pad -r -apple -o "DMDirc-dmg" -hfs-bless "/Volumes/DMDirc" DMDirc.dmg

This code has the advantages of:

• being much tidier than the old code,
• not requiring a loop to get a correctly-sized image,
• being able to bless the image,
• creating a read-only image, and
• not requiring a patched version of Apple’s diskdev-cmds

All that was left now was to fix the file size (which was still 3.8 mb). To do this, I found a new project (libdmg-hfsplus) which is a DMG manipulation library for Linux capable of producing compressed DMG files. I jumped at this, downloaded and compiled it, and came up with:

	# DMDirc-dmg contains the files we want to put in the dmg
	mkisofs -V 'DMDirc' -no-pad -r -apple -o "DMDirc-dmg" -hfs-bless "/Volumes/DMDirc" DMDirc.dmg.pre
	./dmg dmg DMDirc.dmg.pre DMDirc.dmg

I copied this image to an OS X machine and tried to open it and was greeted by a “CRC Mismatch” error. After digging around a little I found the cause of this (The code generates a “Master Checksum” which always seems to generate the same for converting DMGs using the code above, but OS X doesn’t like it), and produced a patched version of libdmg, which can be found on my personal website: http://shanemcc.co.uk/libdmg/, along with a script that will patch it for you if you want to compile your own.

With this newly patched version of the DMG binary from libdmg-hfsplus we are now able to produce DMDirc.dmg on both OS X and Linux with similar results (read-only, blessed and compressed).

It is also worth pointing out that Java 6 for OS X is no longer a “developer preview” only, and can be obtained by anyone with a 64-bit Intel-based Mac running 10.5.2 or later from the Apple website. I would imagine that it is also available using the software update facility in the Apple menu.

OS X nightly builds of DMDirc are available on the nightly builds page of the website.

Now that Apple have re-released their ‘developer preview’ of Java 6, we can begin work on improving DMDirc’s compatibility with OS X. Currently the release only works on 64-bit Intel processors (Xeons or Core2Duos - not CoreDuos, CoreSolos or PowerPCs) running Leopard. They had previously released a developer preview for Tiger, but we do not have access to it, so are unable to tell how well it works with DMDirc (if at all).

If your machine fits the bill, you can download the developer preview from the Apple Developer site:

• Go to http://www.apple.com/java
• Click on “Java Downloads”
• Click on “Java SE 6 Developer Preview” — This will redirect you to the ADC site.
• Login to the ADC site (you may need to register first.)
• Click on “Downloads”
• Click on “Java” in the sidebar
• Download the “Java SE 6 Developer Preview”
• Install

Once you have the developer preview installed, you can download a nightly build of DMDirc from http://www.dmdirc.com/nightlies or you can build your own by checking out the source code from our subversion repository (if you have the developer tools installed) by running the following commands:

svn checkout http://dmdirc.googlecode.com/svn/trunk/ dmdirc
cd dmdirc/trunk/installer
./release.sh --target osx this
mv output/DMDirc.dmg ~/Desktop/DMDirc.dmg


Now you can mount the image, and drag DMDirc to the applications folder to install it.

However, at the moment there is a bug with the “First Run Wizard” that we use to extract plugins and actions for the first time, so you will need to run the following in a terminal prior to running the client to make it work. This assumes that you installed DMDirc into the Applications folder and not somewhere else.
Edit: This bug has been fixed.

mkdir -p ~/Library/Preferences/DMDirc/
cd ~/Library/Preferences/DMDirc/
unzip /Applications/DMDirc.app/Contents/Resources/Java/DMDirc.jar plugins/*.jar com/dmdirc/actions/defaults/*
mv com/dmdirc/actions/defaults actions
rm -Rf com
echo "general.firstRun=false" >> dmdirc.config
echo "general.addonrevision=10" >> dmdirc.config
echo "updater.enable=true" >> dmdirc.config
echo "general.submitErrors=true" >> dmdirc.config


The last two lines can be omitted if you do not wish to receive update notifications, or automatically send error reports to the developers. We would however prefer you to have them both enabled.

Now you can enjoy DMDirc on OS X :)

Feel free to raise any issues on our bug tracker, leave a comment here, or contact us in #DMDirc on Quakenet

In DMDirc 0.5 we introduced a feature described in the release notes as “intelligent tab completion for commands” (introduced in this blog post), which allowed commands to tell the tab completer about the arguments that they’re expecting. In that incarnation, commands can specify a bunch of extra words to add, and can toggle whether the default tab completion targets (nicknames, channel names, command names) are used.

In DMDirc 0.6, we’ve taken this one step further. Commands can now specify exactly what group(s) of targets should be used - so, for example, the first argument of the /ctcp command will now only tab complete to nicknames and channel names (but not command names, as it would have done in DMDirc 0.5).

We’ve also added support for what I call “deferred intelligent tab completion”. This comes in to play with the half-dozen or so commands that take another command as one of their arguments, such as the /alias command, which takes an alias name and then a command to associate with that alias. So, say you want to create an alias to start a timer that will CTCP PING a certain person after five seconds. You type: /ali<tab> ping /tim<tab> 1 5 /ctcp Nickname <tab>. When you hit that final tab, DMDirc suggests eight standard CTCP types (PING, FINGER, VERSION, etc…), even though the CTCP command is in the middle of a much larger command. How’s that for intelligent tab completion?

As far as I’m aware, no other IRC client comes close to the intelligence of DMDirc 0.5’s tab completer, so DMDirc 0.6’s puts us leagues ahead of the competition in that respect. But, as ever, we’re eager to hear any feedback or suggestions that you might have - feel free to leave a comment here, drop by #DMDirc on Quakenet to talk to us, or use the “Send feedback” feature (available in the help menu) of DMDirc 0.5.5.

The latest major change we’ve added to the 0.6 branch is a revamp of the preferences system. The original purpose of the revamp was to shift a large amount of logic out of the UI and into the main client core, so that if/when we have additional user interfaces written, they don’t have to duplicate several hundred lines of code to produce a preferences dialog.

We took this opportunity to move plugin configuration settings into the main preferences dialog. This makes the configuration options for individual plugins much easier to find, and reduces the amount of clutter in the Settings menu somewhat. Here’s a screenshot of the new prefs dialog — you can see the configuration categories for all of the plugins I have installed, as well as the plugin manager itself in its newly integrated state:

We still have some more improvements planned for the preferences dialog in 0.6, including a new component to allow easier editing of duration fields (no more entering times in milliseconds!), and better management of optional colour fields.

In other news, DMDirc 0.5.5’s release is drawing very close. DMDirc 0.5.5 has been the most tested release to date, and we’re working out the last few problems now. You can view the current changelog or the draft release notes if you’re interested in the changes since DMDirc 0.5.1. If you want to try it for yourself, or want to help us test it, you can try an unstable release of DMDirc.