by Jerad Bitner on October 31, 2012 // Short URL

An Update on the Art of Estimation

If you haven't read it already, there is a lot of really good meat in the first article on this topic by Seth Brown. Seth does an excellent job of digesting our process of working with our clients during the decomposition of a project to figure out just how much this baby is gonna cost.

Estimating

One thing that has evolved since the writing of Seth's article, is the spreadsheet that we use for our work breakdown, and the exact method in which we go about estimating the various deliverables. The spreadsheet is full of notes on how we go about our various estimates, and the things what we calculate automatically.

new-spreadsheet-img

First we have at least three developers who are going to be working on the project conduct a blind estimation line by line with white text on a white background and then do a reveal all at once. During the reveal, if we find that there ends up being large discrepancies in the estimates, we talk about the assumptions each developer made in order to make sure we're all on the same page with what is expected for said deliverable and give them a chance to adjust their estimates given any new information. We then average these as the total estimated developer hours.

There is also an added "pm factor" for our project manager which is a .25 multiplier. This multiplier is based on Lullabot's historic data for our projects where we've found that project managers end up spending about one quarter of the time developers spend on a project overall.

Our updated spreadsheet also introduces a concept of a risk or uncertainty factor which is helpful in calling out the unknown aspects of a project. The higher the risk, the more unknown that element of the project is. Measuring risk also helps measure the difference between a fixed-bid VS a time and materials approach to the project. Risk is measured on a scale from 0-5 and we calculate additional developer hours by taking the sum of the developer average plus the pm factor and multiply by the risk rating. Finally we multiply again by point one (=sum(Average+PM Factor)URating.1). We're basically asking our developers how uncertain they are about their estimates and why. It's usually because of some unknown factors, and we try to determine if these factors are something that the client can help us aleviate. If so we go back to make sure that expectations are clearer and give our developers the opportunity to reduce the risk and adjust their estimates. But sometimes it's a factor that is simply not controllable and if the risk remains high it increases the overall estimated hours.

Resourcing

We've also added a new sheet which helps us to figure out what resources we need on the project in order to accomplish the estimated deliverables and to help us to align our resources with the timeline of the project and the estimated amount of work. This helps our project mangers to get an idea of which deliverables they should schedule for each sprint. For instance, if they know that they have 80 man hours during Sprint 1, they also know that they can schedule X amount of deliverables for that first sprint based on how many hours were estimated.

resource-img

This also gives us a chance to think about dates going forward and account for any scheduled vacation times for the developers who are on the project and holidays can also be factored into the timeline. In the case of a fixed bid project, we can either add resources or if we have no more available resources, advise the client that we'll need to cut scope in order to meet their budget.

We continue to revise our approach to this process as we find better ways of trying to come up with more accurate estimates in order to meet our deadlines, budgets and to set client expectations appropriately. So far our latest version feels like the most accurate we've gotten to date, but we'd love to hear what you think about it! How has project estimation changed for you?

Jerad Bitner

Sr. Technical Project Manager

Want Jerad Bitner to speak at your event? Contact us with the details and we’ll be in touch soon.

Comments

Anonymous

How do you guys actually bill

How do you guys actually bill for PM? Is it per hour spent, which invariably doesn't work, because you can't track for emails and tiny things, or do you just apply a per-project line item?

Reply

sirkitree

The spreadsheet

The spreadsheet tells it all. We have a column for PM time which is typically 25% of our average estimate. We probably do not need to do this on a line by line basis in most cases, but there have been times when we can look at some particular deliverable and say that there will not be as much time involved form a project management standpoint and so we have the ability to adjust accordingly.

Reply

Owen Barton

Thanks for sharing - it's

Thanks for sharing - it's always great to see how others work.

You can see some of the evolution of our estimating template since we first shared it in 2009 at http://civicactions.com/estimating-worksheet - some of the main changes also include resource planning, per-sprint estimates, split estimates (the "RFP" estimate of what is requested, and a "recommended" estimate) and easier to copy-paste summaries. The latest version is now maintained only in Google Docs (https://docs.google.com/spreadsheet/ccc?key=0AvXTeygXd3c2dEtrVHJSTy1vaXN...) and is CC licensed of course, so feel free to use, adapt and borrow. At some point I should probably do a screencast that walks through how it works in a bit more depth!

Reply

vijaycs85

Thank you for sharing

Hello,
Sounds another best of way of estimating (against poker points). Do you note down the risk areas(like more complex component) or dependencies(3rd party service provider blocking our development work or waiting for another component from team mate) as well?. In my experience, we had few place where we had to re-create whole component that we built on previous sprint because of the additional element to the component or client feedback after sprint demo.

Also do you try POC (proof of concept) to prove the assumption are valid? if so, how the estimation for POC would work?

Reply

sirkitree

We do note the risk area, but

We do note the risk area, but I'll admit that we've had a difficult time in accounting for these things on a continuing basis during the project. So while we're estimating the risk, we're still learning how to manage the risk throughout the life of the project.

Any thoughts on doing that?

We have also had quite a few projects where we do POCs first and in those cases we typically make it it's own contract and it's own phase of work. It's like a hands on discovery in which we really figure out what the unknowns are, get a better grasp of the risks involved, and end up being able to have an intelligent conversation with our client about the product we're building. This typically has a particular time cap on it of two weeks, or maybe a month depending on the scope of work and is typically time and materials.

I'll also note that projects that make an allowance for a proof of concept like this tend to be much more successful in terms of getting the client what they need and being able to hit our deliverables in a timely fashion. It's definitely our preferred method!

Reply

Johan Falk

The uncertainty factor?

Thanks for sharing – interesting read!

To me it seems a bit weird that you use a linear uncertainty factor. That is, if two deliverables have the same time estimate, a deliverable with an uncertainty of "2" means twice as much time as one with "1" and half of that of a "4". And if the uncertainty is a "0" it would take no time at all, which seems a bit optimistic.

Have you considered a non-linear uncertainty scale, for example like this:

* no worries: x 1
* small uncertainty: x 1.5
* medium uncertainty: x 2
* high uncertainty: x 5

With such a scale, it would also increase the desire to remove "high uncertainty" elements by reducing them to smaller and more known parts. That kind of work takes time, but hopefully clients would be interested in paying for pre-studies if they can see that the estimated development time can be reduced.

Reply

sirkitree

Good idea!

I hadn't considered this before now, but I think it's worth discussing with Seth! :) Thank you!

Reply