Categories
Quality

Programming is the number one quality assurance activity

By Jim Grey (about)

Programming is the number one quality assurance activity.

Or maybe it’s design. Or maybe it’s writing good user stories. Or maybe it’s having good ideas for things to build in the first place. Or maybe it’s paying down technical debt.

Test

But it sure as hell isn’t testing.

When I talk to engineering leaders at small software development shops and they find out I make my living in testing, many of them admit that they don’t have any testers yet. They’re sheepish about it. It’s as if I’ll be offended!

I tell them to rock on, and to delay hiring that first tester for as long as they can.

And then I ask them about the quality challenges they have. Too many bugs? Won’t scale? Bogs down under load? Don’t fall prey to the gut reaction “oh my god we need to test,” as if testers are a magic filter through which perfect software passes.

Because if you respond by hiring testers, you’re likely to end up with testing theater. Your testers will do testery things and find bugs, sometimes even good ones, bugs that let you sleep better at night.

But they can’t fix your quality problems. Only your developers can do that. Instead of letting your developers do whatever it is they do and hope a tester can find everything that’s wrong, challenge your developers to get better.

I ask them these questions:

Do your developers have the skills needed to build the software you’re asking of them? If not, help them build those skills or, gulp, replace them with developers who have them.

Are you following good development practices? Test-driven development, pairing, and code reviews. Not only do they promote solid code, they help create a culture of quality among your developers.

Is your team writing lots of automated unit tests and and acceptance tests? (By acceptance tests, I mean thin functional tests at the API or controller level.) Do they run on every commit? This traps basic problems as soon as they’re introduced.

Do you have a well-functioning delivery methodology, at the right scale for your organization? If you’re two developers, that might be a kanban board drawn on a whiteboard. If you’re ten developers, you might use a tool like Pivotal Tracker and have a couple defined but light rules and ceremonies. If you’re 100 developers, it might be some scaled agile methodology like SAFe, backed up with a tool like JIRA, guided by a program management office. Whatever it is, are you following it and is work flowing smoothly through the delivery pipeline?

Are you giving your developers the time they need to do thoughtful work? To design resilient software that performs? To architect it for long-term growth?

Do all of these things before you hire your first tester. Your developers will give you a level of quality that makes it hard for testers to find low-level bugs. Testers will then have time to find more interesting and valuable bugs because basic stuff is seldom broken. This makes testing a lot less expensive, by the way. You need way fewer testers when you deliver software to them where core functionality works.

And then your testers, instead of feverishly testing the basics, can contribute at a higher level. Testers bring a different mindset to your teams, one of critical thinking about how the software is deployed and used. When you harness that mindset, you can improve your designs and your architecture before you write that first line of code.

Categories
Quality Testing

Fast failure recovery lets you take more risk and increase speed

By Jim Grey (about)

I can just imagine the “oh no” moment that rippled through Feedly last week when they learned that their big new release for iOS 9 crashed on launch for iOS 7 and 8 users. Their first response: add a hastily-written warning to the App Store description.

IMG_4066

Their second response: fix the bug, fast. It was ready the next day.

IMG_4069

Apparently, Apple has a way for developers to rush the very occasional critical fix into the App Store, sidestepping the normal, weeks-long approval queue.

Apple’s App Store fast track provides a safety net, and I don’t blame them for using it. Because Feedly could recover fast, hardly anybody will remember this gaffe, and it won’t cost the company very much.

I actually feel slightly bad talking about it, because it keeps the memory of this bug alive. But it illustrates such a simple equation: the faster you can recover from failure, the less perfect your product has to be when you ship it, and therefore the less expensive it is to build and support, and the faster your company can move.

I’m not advocating for delivering buggy product on purpose. Follow good development practices and test for important risks to deliver the best software you can as fast as you can. But when you inevitably deliver a bad bug, being set up to recover fast means you can deliver without worry.

I remember when the best way to get software to users was to mail it to them on CDs. A bug of this magnitude was a much bigger deal then. It was harder to warn users of the problem, and lots slower and more expensive to correct it. In that world, Feedly’s bug could have damaged their reputation for a long time. So back then it made sense to test more thoroughly — and therefore spend more time and money before releasing.

But today, in a world of Web software and 24-hour emergency App Store turnaround, you’ll deliver faster and with less expense when you set yourself up for fast failure recovery. Continuous integration and continuous delivery are usually a part of that strategy.