In Summer 2013 I made the difficult decision to move away from my beloved Cardiff to live in Yorkshire with my (now-) wife. During my 6 month job hunting period I blogged about my frustrations with Jobcentre Plus and shared my advice for dealing with recruiters. Dozens of applications and 3 job interviews later, I found a new career as a Web Tester for Numiko, a digital agency in Leeds. Like many others, testing wasn’t a career path I planned, but it had always interested me so I jumped at the chance to try it. As well as a switch from marketing to testing, this was also a change in company type (tiny SME to medium-sized agency), industry sector (desktop software to web development) and location (Cardiff to Leeds)! In October 2015 I joined Byng as their first test engineer.  This is my first blog post since switching careers – it’s been a busy 3 years, but I’ve learned a lot. Here are my top three lessons from this time:

1. Testing is an invisible output of software development.

Testing is a very strange profession. As a professional critic of other people’s work, I rarely have much to show for my efforts. In fact I’d argue that a tester’s main deliverable at the end of a project (or sprint/iteration, if you’re working in an agile context) is a product that is lacking in bugs (or at least has fewer bugs than before testing began). The users (and sometimes the client) have no idea how much testing was done, the testing approach used, or indeed if the product was really tested at all! The main tangible artefacts of my work are:

• bug reports
• screenshots and videos
• reports about the current state of the product
• miscellaneous documentation

And, let’s be honest, who cares about any of these after a project/sprint/iteration is finished? The users certainly don’t; they just want the product to work well and do what they want it to. The client probably doesn’t care either (for similar reasons). Even our colleagues will rarely have need of these artefacts after they have been acted upon. Yet without the work involved in creating these artefacts – and the action that was taken as a result – the product would almost certainly be in a worse state. If we accept that the above testing artefacts have little long-term value, then it follows that testing has no real tangible deliverables. For the sake of our own career development and the growth of the testing profession as a whole, it’s up to us to make sure that stakeholders are aware of (and value) our contributions.

2. Testing is not (just) checking!

To the uninformed, testing seems like a linear task. Here’s the specification, here’s the product, does it work or not? The term ‘Quality Assurance‘, often synonymous with testing, doesn’t help this perception. Much of testing is a series of checks, there’s no denying that. But it’s the design, planning and execution of these checks that gives them value. A tester should also ask difficult questions and help predict likely edge cases. To put it another way: testing is not checking, checking is just one of its outputs. Getting this message across to project managers, developers and clients is vital if you want your work to be valued. Understanding what a tester actually does is equally vital for planning and budgeting – if the only testing time scheduled is for checking after development has been done, much of the value is lost.

In an agency environment I work with multiple PMs (depending on the project), and in the past I’ve found that different PMs have a different view of what testing involves. When a PM thinks of testing as post-development checking, they tend to allocate testing resources sparingly. The resulting lack of testing time often means that I’m forced to operate on a ‘works as expected’ basis. There probably won’t be any opportunity to scrutinise requirements, play around with work in progress, or suggest alternative solutions before a decision is made! Conversely, PMs who understand that testing is much more than checking, then allocate resources accordingly, should find that testing provides them with greater value. Once again, it’s up to us as testers to demonstrate our value by educating our colleagues about what testing is, and taking the initiative to get involved in every stage of the development process.

3. Testing performance cannot be measured.

Many software development teams attempt to measure performance in some way. This is often linked to salary or bonus. Unfortunately, there doesn’t seem to be a way to measure the performance of testers. Almost all of the testing metrics that could possibly be measured are also affected by the project and the developers who worked on it:

• Bugs found – the more complex the project, the more bugs there are likely to be. Even if you control for this, this number is directly linked to the quality of the original code!
• Bugs reported by end users  – again, this depends on the quality of the code, not just the quality of testing. Even then, users’ technical competency can greatly affect their ability to find bugs.
• User satisfaction – assuming you find a way to measure this, you can’t directly link the user’s happiness to the quality of testing, for the reasons mentioned above.
• Client satisfaction – as above.
• Time spent testing – as well as depending on project complexity, it doesn’t really tell you anything. A good tester will adapt their test approach to the time they have available, and not enough time spent testing will inevitable produce a poorer result.

All of the above boils down to two key conclusions:

• Quality cannot be objectively measured. You can track metrics relating to quality, but a subjective judgement still has to be made. Even then, the perceived quality of a product varies depending on who you ask.
• Testing can’t be measured in isolation. Almost everything a tester does depends on other people’s work.

So, what’s the solution to this dilemma? No idea, sorry. Much has been written about this topic, with no clear answers. But the conclusion I’ve reached so far is that, since quality cannot be measured or directly linked to a tester’s skill, their performance has to be judged subjectively. Are projects smoother when a tester is involved? Are there fewer rounds of revisions because the tester is catching issues earlier? Is the PM spending less time focusing on minute technical details and more time managing the project and the people involved? If the answer to these and similar questions is yes, your tester is performing well. Reward them appropriately!

Conclusion

If these three lessons have anything in common, it’s that testing’s value is not always obvious to those who have little direct knowledge of what it involves. To be successful, engaging proactively with colleagues and stakeholders must be part of our role. Being a good tester isn’t enough on its own!

I expect fellow newbie testers would recognise some, if not all, of these lessons, although I’m sure some will disagree with my conclusions. I’d love to hear what others have learnt early on in their careers and how their experiences have differed from mine.