A better way to do Agile Estimates

The standard agile estimation process falls into typical estimation traps that make creating accurate estimates difficult. Major traps are  relying solely on expert judgement,  rushing the team to estimate in a high pressure setting, like a sprint planning meeting, and we often estimate work without clearly defining the scope.

When estimating, we want to prioritize Counting, Computing and Judgement, in that order. The best way to estimate something is to have a record of how long things took historically, then create a complete set of stories for the work. To create the estimate you simply count up the stories. In construction you might have a certain number of feet of drywall you need put up in your house. To estimate the cost you would multiply the number of feet by the going rate per foot of drywall and end up with a pretty good estimate. 

In software we work with myriad heterogeneous tasks that we don’t have industry standard data for. So we have to rely on Computing and Judgement to create our estimates. To compute an estimate you could do something like reasoning that since your historical rate of completing stories is 2 days for the company, we will create an estimate based on (2 days * #stories). That is computation will create a rough estimate at best so it would be worthwhile to round it to one significant digit. What you should not do is use your judgement to tweak the estimate after you compute it. Don’t say to yourself, “Our team is the best in the company we will only take 1.5 days per story” if you don’t actually have data to back up that rate of 1.5 days per story.

Our last resort in estimation is to use the judgement of experts to tell us what they think is a reasonable estimate. The problem is that in Agile we rely on Software Engineers who have no training in estimation and our estimating process encourages them to take shortcuts when creating estimates. 

The standard agile process today is to put a bunch of engineers in a room and have them play ‘planning poker’ where they are expected to use expert judgement on a short timeframe, without access to any historical data. 

We want to avoid pressuring people to ‘rush’ when estimating. If data is available we should have time to look it up and compare with previous projects. 

I propose a tweak to the standard sprint planning meeting called ‘Serious Agile Estimating’. Before sprint planning most teams will have a backlog grooming session. After that meeting prepare a spreadsheet of every story that your team needs estimates for. Next fill out the estimation form with the stories for the next sprint and send it out to each team member before the sprint planning meeting. 

The sprint planning meeting will be split into 2 phases, estimation and discussion. The estimation phase will take up the first 30 minutes of sprint planning and should be carried out in silence. Each team member should have a laptop and the estimation form. Team members should take this time to read through the stories creating best, most likely and worst case estimates. After 30 minutes, ask everyone to submit their forms online. 

The discussion phase of the meeting is focused on discussing the differences between people’s estimates. Was there work that we forgot to break up into stories? Do members of the team have differing opinions on how much work different tasks are? What data did team members use to support their estimates? It is important to remember that we do not change estimates during the discussion phase. 

Never make estimates off the cuff or during a meeting while people are talking. Having a hard rule of ‘never make estimates during a meeting’ would be the best. But in my experience it is more reliable to block off focused time in a meeting than to have everyone do it beforehand. 

Serious Estimating Steps

  • Create a list of groomed stories to estimate ahead of time
  • Give each team member a checklist and estimation form (See the example form included at the bottom of this post.)
  • Have each team member estimate stories individually then meet to compare your estimates. 
  • Do not make any estimates during the discussion
  • Require team members to submit a complete set of estimates before the meeting. 
  • Don’t just average the estimates and call it good enough. You need to discuss the differences among individual results, don’t just take the calculated average automatically. 

The big difference is that team members create estimates on their own with ample time to think. No count downs, no poker, estimates are written down and then discussed. 

Finally, I strongly recommend reading Software Estimation Demystifying the Black Art by Steve McConnell which is the source of many of the ideas in this post. 

The book can be found here: https://www.construx.com/books/software-estimation-demystifying-the-black-art/

Google Docs Link:


Bookshelf Post

I have gotten back into reading Software Engineering books. It has been a good change of pace to study a bit in the mornings. My current focus is on Estimation. I want to come up with a better more accurate way for us to estimate software projects in agile environments. 

These two books are great references. Designing Data-Intensive Applications is a good read and believe it or not my workplace still uses Java 8 so Java Performance is not out of date.

I got these two management books back when I was a manager for a few months. But I haven’t dug into them yet. 

Agile Software Development was a textbook back in University. It is a funny book since it has about as much space devoted to Agile as it does to Object Oriented programming. The Mythical Man-Month is fun because of all the references to technology in the old days before the modern web. 

Building Products for the Enterprise is a product book, but I really like how it really explains what those Product Owners are up to all day. 

How much does using Haskell instead of Java cost your company?

Following on my earlier post “How much is turnover costing your company?” this post is about how much using non-standard technology stacks costs your company. Lets say company A is high performance company with a custom functional programming stack that uses Haskell and Miso for development. Company A doesn’t ship a lot of bugs to production, in fact they run CI/CD with deployments to prod everyday. Their SAAS product is one of the best in the industry and is growing at a good rate. 

Now Company A has a competitor in the form of Company B. In this case B does stand for Boring. Company B offers a SAAS product that is almost identical to the one Company A sells but Company B’s has more bugs because they use Java. In fact Company B uses Java with Spring Boot and ReactJS. Both Company A and Company B started in 2018 when React was already mainstream. Company B also uses CI/CD deployments but they can only deploy to production once a week because there are so many bugs. Company B also hired a QA Engineer to help support the pipeline because of an embarrassing outage that CompanyA used in a humiliating Ad. 

In January 2019 both companies raise a round of founding for 10 million dollars. The mandate from the VCs is to 10x the company in size and destroy the competition. Both companies need to start hiring fast. Throughout 2019 CompanyA has an advantage, there are a bunch of passionate Haskell engineers that want to work on their product. Both Company A and B reach their growth targets and hit 100 engineers that year. 

Fortunately, things go well for both companies in 2020. The market is growing and revenues are increasing. The VCs call for more growth. Now we want to grow engineering to 500 engineers. 

Now Company A has a problem. They already hired all the passionate Haskell+Miso engineers they could find. Any new engineers will take 12 months to train on their technology stack. CompanyB doesn’t have this problem everyone still remembers how ReactJS and Spring Boot work even though they are no longer the top technologies. Company B only needs 6 months to train a new engineer on their technology stack. 

Lets run the numbers. How much is it going to cost to onboard 400 new engineers for each company?

We will keep our 2 year average tenure estimate and use 12 months of onboarding for company A. Each engineer has a total cost of 200k including salary, equity, benefits, taxes and insurance. 

Each engineer is going to cost around 100k to onboard at Company A. 

Each engineer is going to cost about 50k to onboard at Company B.

Now the next question, how long will it take each company to get all 400 new engineers fully operational? 

My assumption is that it takes 1 fully onboarded engineer to train a new engineer. Each company has 100 engineers to start so at the beginning the first 100 engineers will train the next 100. Then each company will have 200 engineers to train the next 200. Then things calm down a little and we have 400 engineers to train the last 100. 

After 12 months Company B has double the trained engineers Company A has. At 18 months Company B is finished onboarding and can have all 500 engineers focus on the product. Company A won’t catch up for another 18 months.

Most people will agree that Company A’s functional programming stack is not going to make up for the extra engineer years that Company B has at this point. 

Estimating in Agile

I have been reading Software Estimation (https://www.amazon.com/Software-Estimation-Demystifying-Developer-Practices/dp/0735605351) again after a recent conversation with one of the product owners on my team. The Senior engineer on my team has pushed us into a ‘pointless’ estimating process where the estimate is essentially the count of the subtasks in the story. 

The Product owner wanted us to provide time or t-shirt size estimates earlier in the process before we did the work to break up the story into subtasks. This discussion broke down into people arguing over agile… 

Personally, I am quite disillusioned with how we create estimates a part of the Software Development lifecycle. The standard  estimation technique in my experience is to compare a new project with a past project based solely on personal memory and gut feel. Then to double your estimate from what ever you came up with. The computer science and engineering classes I took did not cover anything related to estimating projects. Which is a little strange since assumed the other engineering discipline like Mechanical or Electrical engineering would cover estimates. 

The business executives need estimates so that they can coordinate work, sign deals and generally make money. If you promise a client something and the schedule slips 6 months that is a problem. If you sign a 10 million dollar advertising deal and the delivery schedule slips 3 months its a problem. So the business needs estimates that are accurate to plan things. But if we are just guessing as the engineering team what is the point? If our estimates are constantly wrong and the levels above us ignore them why do we need to waste our time ‘voting’ and playing ‘planning poker’? 

Then there is the issue that in agile we are very loose with stories. Subtasks are added and removed with little oversight. After a story is estimated things can change and suddenly the story goes from 3 subtasks to 10.  The next day 8 of those subtasks are moved into another story. What were we estimating again? 

I will be writing a bunch more on estimation, but some of the things I have gained so far are that expert judgement is considered the weakest means of getting an estimate. The best way is to count something whether stories, square feet of drywall or some other work component. The second means is to use heuristics to compute your way into an estimate. The last resort is expert judgement or gut feel. 

Count > Compute > Judgement

The usefulness of linking email threads in stories and tickets.

Have you ever had a great email thread about a feature? You and your manager hashed out a solution with the product team and everyone is in agreement. All that is left to do, is to create a JIRA ticket and get working. Now wouldn’t it be nice if you could just LINK THE EMAIL THREAD IN THE STORY!

I have seen email archives, I know they exist. Linux has them. My employer even has email archives. But how can I link to the archive from Outlook? There should be a way to do this super simple thing to share my emails in a story. 

I am going to investigate how the email archives work at my employer.