TDD is hard, but it's tough to beat well-tested code

I just spent a couple of weeks doing TDD with somebody at work. He knew about TDD and unit testing in general, but he wasn't necessarily convinced about the trade-offs of TDD, especially for somebody with no previous experience doing it.

It takes time to learn TDD well. The initial feeling is that you can go ten times faster if you didn't have to write those darn tests. I get those that stop doing TDD after trying for just a few days; it's hard to envision the long-term gains while you are feeling so much pain to make any meaningful progress.

If you are going through this, don't stop. TDD is hard, but not harder than the other million things that you already learned. TDD will set apart your work; you'll be the one doing what the other hundred aren't just because they didn't take the time to learn it correctly. You can do better by sticking with it and overcoming the challenging phase. Eventually, TDD becomes second nature, and you'll be ripping off the benefits of your hard work.

The fact is that it's tough to beat well-tested code. There'll always be a thousand justifications to avoid unit testing, but at the end of the road, the better code always wins.

When the estimates are too high

It will happen more often than not: after you finish estimating the work, you realize the estimates are higher than what everyone was expecting.

An estimate is an unbiased prediction of how long a project will take or cost, regardless of what is the specific target that you want to accomplish. You can't just reduce your estimates, so here are some suggestions to keep in mind whenever your estimates seem too high:

  • Don't reduce estimates that came directly from the developers. They tend to provide estimates that are too optimistic already, so further reducing them will not help your chances of success.

  • Don't cut the estimates down without discussing the consequences and a plan to mitigate them. You can negotiate (and reduce) commitments, but you can't negotiate estimates.

  • Try to use different estimation techniques to validate your previous results. If these estimates agree, trust them.

  • An excellent way to improve the accuracy of the estimates is using group reviews. Wideband Delphi is a structured group-estimation technique that produces very good results.

  • Look at the least important features, and negotiate them out of the scope. Find the ones with the most uncertainty and start a conversation about them. Keep notes of every assumption you make to reduce the estimate.

  • Always remember that if you torture the data long enough, it will confess to anything. Don't push it.

Right at the intersection

Great, capable individuals can definitively make a difference. I know a lot of them; sharp, hardworking people that could easily impress you with their knowledge after a 5-minute conversation with them.

Sadly, you need something else to build a team. It's not enough to be great at what you do; you also need to be good at working with others. I've seen time and time again talented developers that have no clue about being part of something bigger than themselves.

Just like a baseball team, it's not about packing as many stars as possible under the same roof, but about finding the right balance and chemistry that will lead to winning games.

With every new interview, I've found myself asking more questions to uncover that side of the candidates. I don't stop anymore as soon as I realize how great developer you are, but I try to dig deeper to make sure you'll be a good fit for the rest of the orchestra.

The secret is right at the intersection of a great developer and a great team player.

Things to consider before sending your next estimate

Before sending an estimate for your next project, take some time and consider the following questions:

  • How long will it take you to clarify all the requirements, document them, and prepare the backlog of the project?
  • How long will it take to prioritize and size that backlog?
  • If you need to put together an initial release plan, are you accounting for the time to do it?
  • Are there any third-party integrations that will require further review and you aren't planning on it yet?
  • Do you need to schedule any time to think about performance, scalability and the security of the project?
  • Are you planning enough time to deploy the product?
  • How much time do you need to transition the project to the client?
  • Do you need to do any user training before handing over the project?
  • Is there any data migration that you should include in your estimate?
  • Is there any sort of warranty period that should be planned and estimated?

I keep this list handy, and it's very helpful every time I have to think about a new project. You can expand it to include additional activities (I removed some from the list above because they are unique to my company) and make sure you don't forget to account for that time again.

Padding

If you do a quick Google search for how to get better at estimating software tasks, you'll find multiple people recommending the way they do it. Some of these recommendations are some variation of the following ideas:

  • Come up with your estimate and then multiply this number by 2 (or 3, or even 4!)
  • Come up with your estimate and increase the unit of time you used. For example, if you came up with 2 days, your final estimate should be 2 weeks. If you came up with 4 weeks, your final estimate should be 4 months, and so on.

The principle behind these techniques (and any of their variations) is to add padding to the estimate to cover for unknowns. Despite well intentioned, doing this doesn't really make you better at estimation. It doesn't make your estimates more accurate either, and there's probably an argument about whether this is even ethical.

Would your client understand and agree with the way you are deciding how long things will take? If you feel the need to hide from your customer the way you come up with estimates, then you probably have more work to do on this front.

3-point estimates

It's being a long time since I estimated something using a single value. Providing a range is usually a better practice, and forces a different thought process that's extremely valuable.

What I do is to think about three different values: the optimistic case, the most likely case, and a pessimistic case. I'll talk about PERT in the future, but for now, let's keep the idea as simple as possible.

In order to build that feature, it will approximately take 5 days if I'm lucky. Most likely will take me 7 days, but depending on this and that, it could take as long as 12 days.

Thinking about the optimistic and pessimistic scenarios in addition to your most likely case forces you to answer very important questions:

  • What happens if I end up not needing that difficult integration after all?
  • Am I considering the dependencies to finish this task?
  • What are the risks that could get in my way?
  • What happens if some of the assumptions that I'm making don't pan out as planned?
  • Am I sure that documentation will be clear enough to finish the work?

Considering the impact of different factors on your estimate is extremely valuable, and I've found that it rarely happens when you think about a single value. But even when you think about all of this, it is hard to represent the results within a single number. 3-point estimates will tell the story that you need your audience understanding: big gaps between estimates will immediately point out to risks, dependencies, and unknowns that we should clear up to reduce uncertainty.

From this point, you can use your 3 values in different ways:

  • Provide a range using the optimistic and pesimistic values (5 - 12 on the example above).
  • Provide a range using the most likely and pesimistic values (7 - 12 on the example above).
  • Get a little bit more fancy, and compute the expected duration using the formula explained here. (This would be my recommendation, but it is also more time-consuming, so I'll discuss more about it in a future post.)

At the end, even if you are forced to use a single value as your estimate, the exercise of coming up with 3 different cases will play a huge role in making your estimate as accurate as possible.

One hundred 1-hour tasks or one 100-hour task

Are these the same really? I know that mathematically they are, but do they represent an equivalent effort for your product?

Most of the time they don't.

On one hand, having multiple low-complexity tasks is good to keep a large team busy. You will usually find ways to parallelize the work by having different people work on different tasks at the same time. There's no reason you can't do this with only one big task, but the former is usually simpler.

Smaller estimates will also tend to be more accurate than bigger estimates, where there's a lot of hidden complexity that we can't understand until we get close enough to the work.

On the flip side, one hundred tasks are just too many opportunities to make a mistake, to harbor scope creep, to have too many balls in the air at the same time. More moving things represent more complexity.

Make sure you consider everything at play before you let math play tricks with your estimates.

Estimates vs. Process

Where would you rather invest your time? Upfront to try and come up with a more accurate estimate, or throughout the project to manage around an inaccurate estimate?

In other words, what would you prefer? A good estimate paired with a flawed project management process or a poor estimate together with a robust process?

Coming up with good estimates is something that we all should work to get better, but we need to make peace with the fact that there's no such thing as "a perfect estimate," and the only solution to an imperfect estimate is a strong process to manage projects.

That's what I'd do: invest as much as possible into building a strong project management foundation that helps navigate our imperfect estimates. This formula will certainly win every time.

Prioritization

"Prioritization" as in having a conversation regarding all the tasks assigned to the team and coming up with the best plan to achieve our goals.

After we know what we need to be working on, there are too many times when we pick up something completely random and focus on it. Some people do it by the size of the task; some people take whatever is at the top of their list; some others can't even tell what method they use.

There's a better way; before anything else, you should set up a strategy to tackle your list. The plan will always vary, but it will certainly run laps around the lack of one.

Here are some different questions that you can ask when determining the order of your list:

  1. Is there anything that could potentially change the direction we are going?
  2. Do we need early feedback about anything?
  3. Is there anything that we can quickly complete and forget about it?
  4. Is there a way we can organize the work to maximize the skills of our team?
  5. Is there anything that excites us more than everything else?
  6. Is there anything that will unblock another task?
  7. Is there anything that will unblock another team member?
  8. Is there anything that will facilitate our work on something else?
  9. Is there anything that will increase our confidence going forward?
  10. Is there a way we can organize the work to increase the collective ownership of the project?

Prioritizing work is like solving a puzzle; the order in which you put new pieces down matters a lot.

Averages

There are three people in the room helping you out with an estimate. You discuss the next feature, then ask for everyone's opinion. A guy and a girl think it will be two days. The other guy thinks it will be five.

Alright, then (2 + 2 + 5) / 3 = 3 days. You average the estimates out, write down the result and move on. As easy as that.

Except that's an awful idea.

Averaging estimates is even worse than picking the lowest or the highest estimate. When taking a side, at least you are giving credit to someone, but by using the average, you are forsaking and distorting everyone's opinion.

What should you do? Start a discussion. Ask why it should take five days. Ask why it could be as easy as two days. Find out what's making different people give different estimates. The conversation will likely float hidden requirements or misunderstandings about the work. That's what you need to happen before moving on.