Renewing the network

I had lunch with a favorite colleague recently, someone I worked with several years ago and think the world of. She’s among the best at what she does, and is fun and warm and genuine while she does it. I’d work with her again in a heartbeat.

She moved here from elsewhere to take her current job, and finds herself wishing to be better connected in our city’s software industry. So she has been talking to everybody she knows to find out who the movers and shakers are, and try to get introductions to them.

She ran through the litany of people she has met and wanted to meet, a dozen names or more. We were both astonished that I had worked at some point with all but one of them, most of them as they were on their way up to enormous success.

Angel lighting the way

Not this kind of angel investor

I told her funny stories about some of these people. One fellow owned a software company where I was a first-level manager. He was a veteran bathroom talker — you’d be standing there draining the tank and he’d come up next to you and start a conversation. And it wasn’t always small talk. You could end up making important business decisions right there at the urinals. Men everywhere know that this breaks the Guy Bathroom Code: no eye contact and absolutely no talking. In, out, move on. But not this CEO.

He sold his company and started a venture capital firm. Through his investments sits on the board of pretty much every growing software company in town. I hadn’t seen him in four or five years when he spotted me in a restaurant last year and came over to say hello. And my colleague said he was hard to meet because he’s so sought after.

I’ve worked with scores of people in my 25-year career, but keep up with just a handful. If I used to work with you and sometimes meet you for lunch, you’re not just a colleague, but a friend.

But lunch with this friend and colleague made me realize that I am very well connected — and I’ve let those connections languish. It’s a shame not only because these people might help me into a better gig someday, but primarily because they’re interesting people who do big things.

When I worked with many of them, there was no way to know that they would do so well. Most people I’ve worked with have not risen so far, actually; most of them are still writing code, or testing, or writing documentation, or leading teams. But so many of them are interesting, too.

But I’m an introvert of working-class roots — a fellow who prefers to keep to himself and let his work speak for itself. At least that’s what I tell myself so I don’t feel so bad about not reaching out to the good people I know. But if I could go back to the beginning of my career and give myself one piece of advice, it would be to not lose contact with the people I enjoyed or admired.

Fortunately, even though a quarter century has slipped by, it’s not too late to start now. I can reach out to people I’ve worked with recently, and rekindle some long-ago connections. I’d like to build a habit of keeping in touch. Maybe I’ll be able to help someone along in their career, or maybe someone will help me along in mine. If not, simply catching up and swapping stories will be ample reward.

Twenty-five years in the software salt mines

Tomorrow it will have been 25 years since I started my career in the software industry.

It might seem odd that I remember the day only until you know that I started work on Monday, July 3, 1989, making my second day a paid holiday. The office was nearly deserted on my first day. My boss regretted not having me start on July 5 so he could have had an extra-long weekend too.

I was 21 years old when I joined that little software company in Terre Haute. I’m 46 now. I have worked more than half my life in and around the software industry.

taught myself how to write computer programs when I was 15. When I was 16, my math teacher saw some of my programs and praised my work. He encouraged me to pursue software development as a career. He began to tell me about this tough engineering school in Terre Haute.

I graduated from that tough engineering school hoping to find work as a programmer. Jobs were hard to come by that year, so when a software company wanted to hire me as a technical writer I was thrilled just to work. And then it turned out I had a real knack for explaining software to people. I did it for twelve years, including a brief stint in technology publishing and five years managing writers.

I then returned to my technical roots, testing software and managing software testers. I learned to write automated functional and performance tests – code that tests code – and it has taken me places in my career that I could never have imagined.

Office

My office at one of my career stops

I’ve worked for eight companies in 25 years. The longest I’ve stayed anywhere is five years. I left one company in which I was a poor fit after just 14 months. I’ve moved on voluntarily seven times, was laid off once, and was fired and un-fired once (which is quite a story; read it here). Changing jobs this often isn’t unusual in this industry and has given me rich experience I couldn’t have gained by staying with one company all this time.

I’ve worked on software that managed telephone networks, helped media buyers place advertising, helped manufacturers manage their business, run Medicare call centers, helped small banks make more money, enabled very large companies to more effectively market their products, and gave various medical verticals insight so they can improve their operations and their business.

Some of these companies were private and others were public; so far, I’ve liked private companies better. Some of them made lots of money, some of them had good and bad years, and one of them folded. Some of them were well run and others had cheats and liars at the helm. Some were very difficult places to work, but those were crucibles in which I learned the most. Others have brought successes beyond anything I could have hoped for a quarter century ago.

I did, however, hope for a good, long run in this industry, and I got it. But I’m also having a hard time envisioning another 25 years. It’s not just because I’d be 71 then. I really like to work, and – right now at least – I plan to do so for as long as I am able. But I’m starting to have trouble imagining what mountains I might yet climb in this career. Maybe that’s part of reaching middle age – indeed, many of my similarly aged colleagues, some with careers far beyond mine, have gone into other lines of work. I’m still having a lot of fun making software, though. I currently manage six software testers, one test-automation and performance-test developer, and one technical writer. I get to bring all of my experience to bear, and encourage my teams to reach and grow. I don’t want to stop just yet.


If this sounds familiar, it’s because it’s an update of an eariler post. Cross-posted to my personal blog, Down the Road.

A paean to BASIC

It’s the BASIC programming language’s 50th birthday. The first BASIC programs ran at Dartmouth College on May 1, 1964.

BASIC proliferated in large part thanks to early personal computers, so much so that for a long time to be able to operate a personal computer meant to program it in BASIC. Like so many geeks my age, my first encounter with a personal computer (a Commodore PET) brought me face to face with BASIC. I was drawn in. My dad bought me a Sinclair ZX-81 so I could teach myself the language. I later moved on to a Commodore 64 and an Apple II.

Photo: old-computers.com

Photo: old-computers.com

I spent untold hours writing BASIC code. I fell in love with being able to make a computer do the things I wanted it to do. I wrote whatever felt good to me, programming just for the joy of it. I mostly wrote simple games, but I even once wrote a reduced version of BASIC in BASIC just to see if I could do it.

When the geometry teacher got wind that I had written a program that drew any regular polygon on the screen, he asked me to demonstrate it to his classes. He liked my work so much that he suggested that I could study this in college and do it for a living.

“Wait – what?” I thought. “People do this for a living?” It may seem astonishing now that the idea hadn’t occurred to me, but in the early 1980s software development was still an unusual career choice. His encouragement got me to apply to engineering school, where I studied mathematics and computer science, which led to my first job working for a software company. That was 25 years ago and I’m still in the industry.

It was easy to criticize BASIC back then because it was so rudimentary. But all of us who first learned to code using it built upon that knowledge as we explored more powerful languages. I next learned Pascal, and then C++, and later a little Visual Basic. All of those languages featured complexities that were easier for me to scale because I had already mastered standard control structures in little old BASIC.

It’s been a long time since I’ve written any code. I fell into the testing side of software development and I’ve happily made my home there for a long time now. But I’ll forever be grateful for BASIC for being my guide into this world.

More reading: a Time Magazine article about this anniversary (here), and Dartmouth’s celebratory site “BASIC at 50″ (here).

Turdpolishing

I used to think that writing a fat suite of automated regression tests was the way to hold the line on software quality release over release. But after 12 years of pursuing that goal at various companies, I’ve given up. It was always doomed to fail.

In part, it’s because I’ve always had to automate tests through a UI. When I did straight record-and-playback automation, the tests were enormously fragile. Even when I designed the tests as reusable modules, and even when I worked with a keyword-driven framework, the tests were still pretty fragile. My automation teams always ended up spending more time maintaining the test suite than building new tests. It’s tedious and expensive to keep UI-level test automation running.

But the bigger reason is that I’ve made a fundamental shift in how I think about software quality. Namely, you can’t test in quality – you have to build it in. Once code reaches the test team, it’s garbage in, garbage out. The test team can’t polish a turd.

Writing an enormous pile of automated tests through the UI? Turdpolishing.

I’ve worked in some places where turdpolishing was the best that could be done. Company leadership couldn’t bear the thought of spending the time and money necessary to pay down years of technical debt, and hoped that building out a big pile of automated tests would hold the line on quality well enough. I’ve led the effort at a couple companies to do just that. We never developed the breadth and depth of coverage necessary to prevent every critical bug from reaching customers, but the automation did find some bugs and that made company leadership feel better. So I guess the automation had some value.

But if you want to deliver real value, you have to improve the quality of the code that reaches your test team. Even if the software you’re building is sitting on a mountain of technical debt, better new code can be delivered to the test team starting today. I’m a big believer in unit testing. If your software development team writes meaningful unit tests for all new code that cover 60, 70, 80 percent of the code, you will see initial code quality skyrocket. Other practices such as continuous integration, pair programming, test-driven development, and even good old code reviews can really help, too.

But whatever you do, don’t expect your software test team to be a magic filter through which working software passes. You will always be disappointed.

Three-point estimation is a critical skill you should build right now

Maybe you’re one of the lucky ones.

Maybe you work in a shop that executes scrum well. Your team keeps the backlog well groomed, understands its velocity, and delivers working software sprint after sprint. Or maybe you work in an open research-and-development shop where the journey is as important as the result, and projects are always open-ended.

Madison Bank clockIf, like me, you have to deliver software on deadline using a less-than-perfectly executed methodology, this post is for you. It’s going to give you a technique for knowing where you stand against your deadlines – a very useful skill that will reduce the chaos you experience and help your boss steer projects more successfully, both of which are good for you. It is a method for implementing continuous reestimation, which I wrote about in this post. Read it to know why this will make you and your boss more effective.

Teams that have worked for me and have estimated using this technique have seen their projects come in within estimate about 80% of the time. The other 20% of the time, they learned important things about their work that let them estimate more accurately the next time.

Here’s the technique:

  1. Break down your work into tasks.
  2. Estimate the tasks using a three-point formula.
  3. As you work, reestimate every day.

Break down your work into tasks

You might resist this step because in your core you know you will miss some tasks. It’s okay. This process honors that you don’t know what you don’t know. Just outline the tasks as best you can.

But think small, because you’ll be more accurate when you estimate more small things than fewer big things. Break things down as small as you reasonably can.

Estimate the tasks using a three-point formula

For each task, then, think of the following:

  • If everything goes right on that task, how long will it take? This is the best case, B.
  • If everything goes wrong on that task, how long will it take? This is the worst case, W.
  • If a reasonable amount of things go wrong on that task, how long will it take? This is the likely case, L.

Think for a minute or two about each task, but any more than that is probably overthinking it.

Important: Calculate your estimates in hours. Not days, not weeks, hours. There’s too much slush in days and weeks. Besides, in an 8-hour day you probably work just 6 hours on project tasks. You spend the other two hours in meetings, having hallway conversations, using the restroom, and being interrupted.

Here is where some magic comes in. Just go with me on this. For each task, drop B, W, and L into this formula and calculate the answer.

\displaystyle Estimate=\frac{B+4L+W}{6}

You’re weighting the likely case, but honoring the best and worst cases – and in one deft motion you take fear out of your estimates. What makes estimates bad is either unexpected things happening or overpadding for unexpected things happening. Thinking best/likely/worst helps you be more objective. And when you add up these three-point estimates for all your tasks, you get a reasonable cushion for the things that could go wrong.

Divide the number of hours by the number of hours per day or week you will work on the project. Lay that number of days out on a calendar; tell your boss that’s when you’ll be done.

As you work, reestimate every day

You are going to discover tasks you didn’t think of at first. And some tasks are going to take longer than you originally thought. When this happens, estimate the new tasks and/or reestimate the task that’s taking longer. Recalculate your end date and break the news to your boss.

Your boss ought to kiss you right then and there, by the way, because you gave him or her information early enough to do something about it – adjust scope, add resources, move the date, or tell you that unfortunately you are looking at some late nights. Hopefully the answer will be a, b, or c more often than d.

And every time you have to reestimate, you’ve learned something about your work that helps you estimate more accurately in the future.

If you want to ship software, stay in touch with how much you suck

My colleague Matt Block recently posted on his blog a link to an article about how a software shop’s business model affects how well agile scrum works for them. It breaks business models down into emergent, essentially meaning that the company builds product to meet goals such as selling ads or driving traffic, and convergent, essentially meaning that the company builds product that directly serves a target market. The article argues that agile is made for emergent and is a poor fit for emergent. That’s just a sketch of the article; go read it to get the full flavor.

I’ve always worked for companies following convergent business models. We’ve made our money by selling the software we created, which made it always important to deliver a certain scope by a certain time. When those companies implemented agile scrum, they could never fully adapt a key principle of it: when it’s time to ship, you ship whatever is built. In a convergent world, scope is king; you ship when everything specified is built.

I e-mailed my brother, Rick Grey, a link to this article. It’s great to have a brother who does the same thing I do for a living as we can talk endlessly about it. I thought we’d have a conversation about how to scope an agile project, but instead he had a brilliant insight: What if agile is good for convergent-model companies because it tells you sooner how much your project is off track? He gave me permission to share his e-mailed reply, which I’ve edited.

What if the companies we’ve worked for and all the other convergent-model teams of the world are doing agile just fine? By “just fine” I mean “as good as they do waterfall,” which may not be “just fine,” but we’ll get to that in a minute. Meanwhile, consider:

Long waterfall project:

    • No one pays real attention to progress (there’s always next month to catch up)
    • Engineers go dark, checking out huge sections of the codebase and not merging them back for long periods
    • Engineers (who are notoriously poor estimators) claim 50% done when it’s really about 25% – and then, as the code-complete milestone nears, they (usually innocently) claim 90% done when it’s really 70%
    • A couple of days before the code-complete milestone, engineering finally acknowledges they won’t hit the milestone and delays delivery to QA – “but we’re 95% done, for sure”
    • Under the pressure of already having missed a deadline, developers quietly take shortcuts to make it possible to hit the new QA delivery date
    • Weeks and months of unmerged changes come crashing in, creating conflicts and compile/deploy problems, further delaying delivery to QA
    • QA, now staring with a multiple-week handicap on an already-too-aggressive schedule, quietly takes its own shortcuts
    • QA finds hairy showstopper bugs and so the ship date gets moved
    • Management is livid, so QA goes into confirmatory testing mode just to get it out the door

Agile project of the same size:

    • Much of the above happens at a smaller scale, one iteration at a time
    • You fail to deliver everything planned starting with the first sprint
    • Instead of spending 80% of the project thinking you don’t suck as an organization and the last 20% realizing that you do, agile lets you feel like you suck every step of the way
    • Takeaway for management: “agile sucks” and/or “we suck at agile”

I assert that most teams are bad at delivering under a convergent business model.  The hallmark pathologies of software delivery under a convergent model are too numerous and powerful for most teams to overcome, but their struggles are masked by waterfall until the end. Agile surfaces the problems every iteration. You feel like a loser by week 4 instead of week 40.

But this is actually a win. You get better project visibility and a tighter feedback loop, meaning you’ve got a better chance to make adjustments earlier to get the most out of your team you have. Embrace the feedback loop as a chance to make things better, and learn not to view it as proof of how much you (collectively) suck.

I will add that agile also helps you keep resetting expectations within your organization, because it makes it standard practice to keep reestimating what it will take to finish everything. This is just what I was talking about in my last post (read it here).

The power of continuous reestimation

Not long ago a software developer I work with suggested that building software could not, and so should not, be managed to a delivery date. “You can’t know everything you’re going to encounter while you build it,” he said, “and so it just takes the time it takes to get it done.”

I said that I agreed: you don’t know what you don’t know, and the work takes the time it takes. But I couldn’t agree that projects should be open-ended, finishing when they finish.

Capitalism interferes. Pesky, pesky capitalism. Most of us make software that achieves some corporate goal about making piles of money. Because the company has plans for that money, it needs to know when it will start rolling in. The money can’t start rolling in until you ship.

Or maybe you’re making a prototype that will be demoed at a trade show in hopes of attracting investors, or you’re fixing a batch of bugs that have angered a key customer who is threatening to walk, or you have to deliver some fixes to comply with some new regulation that’s about to go into effect. Point is, your company has good reasons to ask when you’ll ship.

So you need to set expectations about the size and duration of your work. But fortunately, these guesses don’t have to lock you in and enslave you to your keyboard. That’s because every day you can check your progress, adjust for the things you’ve learned, and see if you need to set new expectations.

You’re driving to a destination

Getting your work done is a lot like taking a road trip. I love road trips! You never know what you’re going to encounter. Road construction, bad weather, mechanical failure, and even fun roadside attractions can all slow you down. You have to steer your way through all of these things.

The road to MadisonWhen you’re on the road, it’s pretty easy to know at any time about when you’ll get where you’re going. It involves answering these three questions:

  • How will you know when you get where you’re going? On the road, you just recognize your destination. But I’m going to fit this metaphor to your work, where it isn’t always as clear what it means to arrive.
  • How far do you have left to go? Watch the signs; they’ll tell you.
  • How fast are you going? Just check your speedometer.

Every time you see a sign, divide the number of miles to go by your average speed, and you have the best case for how long it will take to arrive. You can’t control what might go wrong on the road ahead, of course. But after something on the road does slow you down, you can just recalculate and get a new arrival time.

You can manage your own work in much the same way. But it means you have to build some skill at estimating.

“Uh, two weeks”

Here’s how not to estimate. My first post-college job was as a technical writer in a software company. My first assignment was to convert a thick product manual from an old publishing platform to a new one. It was hardly a mission-critical project, but I was green.

When the boss asked when I’d be done, I had no idea, so I shrugged and said, “Uh, two weeks.” Then the task turned out to be enormous and tedious. I had to put in a ton of extra time to finish in two weeks.

When my boss asked “how long” on my next assignment, I guessed a number – and then doubled it to account for the unknown. Success! I worked no extra time and finished my project before the deadline.

Except that I hadn’t learned anything of value. My “guess” estimating technique became “guess and pad like crazy.” And I still had the problem that if I didn’t pad enough, my only recourse was to work tons of extra hours. I had no way of knowing on any given day whether I was on track to finish when I said I would. I just had to work hard and hope.

It’s okay that you don’t know what you don’t know

If I could tell my 21-year-old self something, it would be this: Ask the boss if you can get back to him in two days. Then use your two days to do this:

  • Think about what done looks like. What must you do to finish the work? How will you know when you’ve gotten through it all? In this case, I had about the easiest possible situation: “Done” meant running out of pages to convert. You may have a number of requirements to code, or a batch of test charters to execute, or a set of stories to complete – there are any number of ways to chunk the work.
  • Do some of the work. Pay attention to how many chunks you complete in your two days. Then note how many chunks you have left. There’s your speedometer and your miles-to-go sign. Divide to get your rate: chunks per hour or chunks per day.
  • Calculate how long the rest of the work will take. Divide your rate into the number of remaining chunks. Lay that number out on a calendar to find your finish date. Tell that to the boss.

And then as you work, recalculate every day based on how much you got done that day. If you need to put in a few extra hours to stay on track, just do it. But when the amount of extra effort needed to stay on track starts to edge toward heroic, it’s time to negotiate.

That’s right, negotiate. You are the master of your own work, after all; nobody knows the reality of it better than you. Your estimate is what you believe it takes to reach Done, and you need to stand up for it. This is true, by the way, even when your boss has given you a due date.

So go to the boss and ask which is more important: the due date or this body of work? In other words, can you change what it means to be done, removing enough work to hit the date? Or can you move the date out to get all the work done? Or can you do a little of both?

Sometimes the answer to this question doesn’t go far enough to make the remaining work match the due date. So ask yourself if you can give some of the work to someone else, enough to hit the due date. If so, ask for people to help you.

What you’re doing is giving the boss and all of your project’s stakeholders as much runway as possible to understand the probability that you’ll deliver the originally defined Done on time. Should that probability drop, your negotiations give them the opportunity to change course, change expectations, or both.

Sometimes, you’ll get no answers to all of these questions. Sometimes, you’ll get yes answers to some or all of them and it still won’t be enough. But asking these questions is your best chance of avoiding working heroic hours.

An estimate should not be a commitment

If no is always the answer, or your bosses just expect you to rise heroically to every occasion, or your initial estimate is interpreted as a contract or an iron-clad promise, you just might be an in environment with dysfunctional beliefs about accountability. Consider finding another place to work.

Your estimate is only your best (and, hopefully, educated) view into when you will finish. It assumes that you don’t know what you don’t know. When you learn more, your estimate is obviously going to change. You’re better off working someplace that has at least an elementary understanding of this.

And now you understand it. Estimate based on what you do know. Go ahead and pad a little (a little!) for the unknown.

But then steer your work to successful completion.

I thank my brother, Rick Grey, who tests software for his supper, for contributing several key concepts to this post.