Thoughts on Developer-based Testing

Written by Dean Edis.

Developers write code. Code contains bugs. As a developer, there is a small chance that the code that you have just written is perfect, bug free, and works every time on every person's machine. If you're like me, there is an even better chance that this is not the case.

In my early days as a professional software developer I lost count of the number of times I confidently passed a feature over to the QA team, only for them to find a bug within a handful of minutes. Now, after 15 years of experience, I have built up a small arsenal of techniques to try and reduce this situation. The motivation for this? Well obviously it is so I can be an effective member of the team, creating complex and STABLE applications... But mostly it is because it's also very annoying, as usually by the time a bug is found I am well into thinking about the next developer task - Putting this on hold to revisit old code can be a major interruption. But to be honest my main motivation is to save personal embarrassment.

At "work" I have the luxury of depending on a small QA team to help test features. At home, this is not the case. Gimpy Software is a small team with no dedicated QA personnel, making the emphasis on developer testing much more important.

Over the years I have known a handful of developers who state "I do not know how to test", or "it is not my job to test". Anyone in the latter camp either needs a serious wake-up call, or a career change. For those in the first camp there is definitely hope. Like any skill, testing is something that can be learnt, and even a rudimentary level of test knowledge can make a big difference in the quality of your output.

Test Automation
As a developer, I love to code. I would much rather be writing code than testing a product, but it is important to understand the two are not mutually exclusive. Unit tests, and particularly test driven development, are excellent ways of crossing the bridge between development and test. They can also prevent bugs from creeping in in the first place - Always a welcome bonus!

When writing our ZX Spectrum emulator Speculator (http://gimpysoftware.com/index.php/apps/4-speculator) app, unit tests played an absolutely key role in its development. The Z80 CPU emulation code alone had several hundred unit tests. I made sure they all ran in a few seconds, making it easy to identify regressions in behaviour, but more importantly giving me no excuse not to run them regularly!

For testing at a higher level, consider writing UI-based tests with automation tool, or test scripts (batch files, Perl script, et cetera). These sorts of test can be automated and repeated, and can run many many times faster than any manual test. There are many good quality tools out there to help with this, a lot of them free.

Whether or not automated tests are applicable to you, there is often no substitute for 'hands on' testing.

How a developer can help themselves
The earlier bugs are found, the better - Bugs found during the coding cycle can often be fixed an order of magnitude times faster than those caught later. For a start the code will still be 'fresh' in your mind. Secondly, if they can be found before being 'officially' tested by a QAer (If you're lucky enough to have one!), perfect. That way there is no 'red tape' involved in writing up test steps, taking screenshots, testing on multiple OSes, etc - Much easier on everyone.

So, some nuggets of advice...

RESERVE TIME FOR TESTING
Always include time for your own testing in any estimates you need to give. 'Development' is not just about typing in code. It needs to include time for your own testing, code reviewing, checking builds aren't broken, ...

REVIEW YOUR OWN CODE
...whether or not your code is peer reviewed! It's all too easy to leave in 'debug' code, 'todo' comments, commented-out code, typos, etc. Assuming you're using a form of source code control (Which I'd recommend, even for personal projects), running a quick 'diff' on your changes is always worth while.

PUT YOURSELF IN 'USER-MODE'
As a developer you're probably used to working with complicated applications. IDEs, debuggers, source code control - All can present complicated UIs which you might not have difficulty understanding. Your users will probably not be so familiar with applications. Users might not know how to use keyboard shortcuts, even for common operations. Try to look at your application keeping a 'typical' user in mind. With Lunar Panda we know that many of the users will be children, who either can't read or don't like to read! Therefore we deliberately make the HUD large and clear, and favour use of icons over words.

When writing the Lunar Panda collision detection code I was fairly certain I had it working 100%. My 3 year old daughter started playing and, as most 3 year olds would, got distracted and just started clicking all over my Windows desktop. Within 30 seconds - BUG! She had temporarily minimized the game window and after I restored it Mr Panda had successfully passed through the biggest lightning obstacle in the game! It turns out my collision detection code relied on actual DRAW calls being performed, which were skipped by the framework when the window is minimized. Whoa. It didn't even occur to me to test that...

DON'T BE TOO DEFENSIVE
If you do get feedback from other users about a bug, don't be too defensive about it. It's not the tester's fault! It might be a pain to go back and fix it, it might even require a rewrite of most of your changes, but even 'niggly' bugs can have a big impact on how a users feel about your product.

LEARN FROM YOUR TESTERS
If you are lucky enough to have access to a professional tester, try to learn from them. For those developers who "Don't know how to test" - Learn! Pair up with a tester for at least a few hours. Watch how they test. Makes notes if necessary. In my experience most testers love it when people show an interest in their work - Especially developers!

So there we go - A few ideas to consider when working on your next coding session. Hopefully you found some of it useful!