By Jim Grey (about)
Where I live and work, it’s difficult to find experienced developers and testers. Demand outpaces supply. So one strategy I’ve been using to grow my testing team is to hire people who have never tested.
Obviously, that means they need to be taught how to test. It so happens I like doing that! But this means I have to be able to identify non-traditional candidates who are apt to be good students.
I look to my company’s customer-service team for candidates, as those people know the software and understand the customer. But that’s not enough; I need to know whether they can be taught to test.
Fortunately, I’ve figured out four key traits someone needs to be able to learn to test and to enjoy it. The new tester must have a knack for troubleshooting, which means they are curious and persistent, can create mental models of the system they’re examining, and can develop and investigate theories of why things are broken.
Be curious: A tester needs to notice when something is wrong and feel compelled to investigate. “Wait, what? Why did the software do that? I’d better dig in and find out.”
Be persistent: But sometimes the investigations take a while. A tester needs to keep feeling compelled, even when the investigation stretches on. (Bonus points if they recognize when they’ve reached a point of diminishing returns, and choose a different avenue of exploration.)
Create mental models: Building a mental model of a system, even if it’s incomplete or partially inaccurate, helps a tester orient themselves to a problem and generate ideas on how to work through it.
Develop a theory of error: Thinking about how the current problem is creating or exposing a flaw in the system informs the steps taken to prove or disprove the theory — and isolate bugs.
A person who’s not curious won’t troubleshoot. A person who’s not persistent won’t keep going when the troubleshooting gets rough. Troubleshooting doesn’t get very far if you can’t see how the thing you’re troubleshooting is a system and imagine reasons why it’s not working.
So when I interview testers, I ask this question, which gets at all of these things:
Think of a time you had a problem with software or electronic equipment when you tried to solve the problem. It might have been trouble with your home network, some difficulty upgrading your PC or your phone to a new OS, a misbehaving printer, or anything that went wrong with technology you use. Tell me how you assessed the problem and tried to figure out what was wrong, and what steps you took to try to fix the problem.
I ask candidates to tell me two or three stories in answer to this question. Many people can come up with one story, but someone who enjoys, or at least doesn’t mind, getting to the bottom of technology problems will have more than one story.
I’m discouraged when I hear these things in answers:
- Trying nothing beyond obvious troubleshooting steps, such as rebooting
- Giving up after a couple steps: “I tried a couple things and then just called support”
- Trying things randomly in hopes one of them works
- Fear of trying things because they might break things worse
I’m excited when I hear these things in answers:
- Developing a theory of how the system works, including applying knowledge from other systems — “I wonder if this thing I’m trying to figure out is like that other thing I know very well”
- Some logical, systematic method for exploring the problem, generally starting from the simplest, most probable causes and moving stepwise through successively less likely and more complicated possible causes
- Research, usually via Internet search but also by asking someone who might know
- Evidence that they learned something about how the system worked, and felt some satisfaction in that
It’s fine if the candidate didn’t successfully resolve the problem. What I’m listening for is whether they systematically eliminate possibilities and move toward finding root cause.
Good answers can be mundane. One candidate told me about sitting down one night to watch Netflix, but his movie kept failing to stream. He was used to experiencing a little of that on Friday and Saturday night; he knew Netflix drew a lot of Internet traffic on those nights. But because it was a weeknight he wondered whether the problem might be within his home. He knew that rebooting the Roku usually solved connectivity problems, but it didn’t work this time. So he went over to his AT&T Uverse modem to check out its blinking lights. “I don’t know much about those lights,” he said, “but I do know that if most of them aren’t blinking, especially the INTERNET light, there’s a problem.” Sure enough, the INTERNET light was out. So he tried rebooting the modem, but the INTERNET light wouldn’t come back on. He found his modem’s manual, but it didn’t cover the situation. He tried a couple Google searches for this condition, but found no useful information. Out of ideas, he called AT&T, which reported intermittent outages in his area.
The candidate’s answer showed that he created a mental model of Netflix and the Internet work and tried the most likely causes first. I especially liked that he confidently used his incomplete technical knowledge to narrow down the problem. And I didn’t mind it a bit that he ended up calling for help — he’d first done all he reasonably could.
The rest of my interview questions look to gauge technical experience and fit for the company and the team. It’s pretty standard stuff. But without good answers to the troubleshooting question, the candidate goes no further.
So look for these four traits. They give you trainable team members.
Thanks to Rick Grey for his valuable feedback that helped shape this post.