Studies within the last few years have shown that a great deal more money is often spent on projects than is initially anticipated. IBM's Consulting Group did a survey of 24 leading companies in 1994 and found that 55% of the software developed cost more than the initial cost estimates (Hussein, 2002a). The Standish Group also did a study in 1994 of 8,380 projects in the United States and found that 53% of the software projects that were completed cost 189% of the original estimates (Hussein, 2002a). Although these numbers are from a few years ago, they likely have not changed much recently (Hussein, 2002b). The problem of making accurate cost estimates can be attributed to a number of reasons: the wrong cost estimation processes may be used, no processes may be used, or the nature of the problem may not allow for accurate cost estimation. Cost estimation is an often overlooked project management practice. < To Top 2.1 Defining Cost Estimation Cost estimation can be defined as the approximate judgement of the costs for a project. Cost estimation will never be an exact science because there are too many variables involved in the calculation for a cost estimate, such as human, technical, environmental, and political. Futhermore, any process that involves a significant human factor can never be exact because humans are far too complex to be entirely predictable. Furthermore, software development for any fair-sized project will inevitably include a number of tasks that have complexities that are difficult to judge because of the complexity of software systems. Cost estimation is usually measured in terms of effort. The most common metric used is person months or years (or man months or years). The effort is the amount of time for one person to work for a certain period of time. It is important that the specific characteristics of the development environment are taking into account when comparing the effort of two or more projects because no two development environments are the same. A clear example of differences in development environments are the amount of time people work in different countries; the typical workweek in North America is 40 hours per week, while in Europe the typical workweek is 35 hours per week (Londeix, 1987). Thus, when comparing a project from North America with a project from Europe, a conversion factor would have to be used to all for an accurate comparison. Different variables can be used for cost estimation, which leads to a difficulty when comparing projects if standard models or tools are not used. For example, a cost estimate can include factors from management, development (e.g., training, quality assurance), and other areas specific to an organization. < To Top 2.2 Cost Estimation and Project Planning Cost estimation is an important tool that can affect the planning and budgeting of a project. Because there are a finite number of resources for a project, all of the features of a requirements document can often not all be included in the final product. A cost estimate done at the beginning of a project will help determine which features can be included within the resource constraints of the project (e.g., time). Requirements can be prioritized to ensure that the most important features are included in the product. The risk of a project is reduced when the most important features are included at the beginning because the complexity of a project increases with its size, which means there is more opportunity for mistakes as development progresses. Thus, cost estimation can have a big impact on the life cycle and schedule for a project. Cost estimation can also have an important effect on resource allocation. It is prudent for a company to allocate better resources, such as more experienced personnel, to costly projects. Manpower loading is a term used to measure the number of engineering and management personnel allocated to a project in a given amount of time. Most of time, it is worse for a company if a costly project fails than if a less costly project fails. When tools are used for estimation, management and developers can even experiment with trading off some resources (or factors) with others while keeping the cost of the project constant. For example, one tradeoff may be to invest in a more powerful integrated development environment (IDE) so that the number of personnel working on a project could be reduced. Cost estimation has a large impact on project planning and management. < To Top 2.3 Cost Estimation During the Software Life Cycle Cost estimation should be done throughout the entire life cycle. The first time cost estimation can be done is at the beginning of the project after the requirements have been outlined. Cost estimation may even be done more than once at the beginning of the project. For example, several companies may bid on a contract based on some preliminary or initial requirements, and then once a company wins the bid, a second round of estimation could be done with more refined and detailed requirements. Doing cost estimation during the entire life cycle allows for the refinement of the estimate because there is more data available. Periodic re-estimation is a way to gauge the progress of the project and whether deadlines will be able to be met. Effective monitoring and control of the software costs is required for the verification and improvement of the accuracy of the estimates. Tools are available to help organize and manage the cost estimates and the data that is captured during the development process. People are less likely to gather data if the process is cumbersome or tedious, and so using tools that are efficient and easy to use will save time. It is not always the most expensive tool that will be the best tool to buy, but rather the tool that is most suited to the development environment. Some thought should be given to the level of detail at which the metrics will be gathered, as well as planning for what metrics may be used in the future for comparison with other projects. The metrics that are gathered will be highly dependent upon the organization's development and organizational practices. The success of a cost estimate method is not necessarily the accuracy of the initial estimates, but rather the rate at which the estimates converge to the actual cost. An organization that does a great deal of contract work would place more importance on the initial estimates. However, in general, the method will be better if it converges quickly to the actual cost of the project. At the end of the project, all estimation methods have the opportunity to converge to the actual cost because enough information is available. < To Top 2.4 The Estimator The people who do the cost estimates could be either directly or indirectly responsible for the implementation for a project, such as a developer or manager, respectively. Someone who has knowledge of the organization and previous projects could use an analogy-based approach to compare the current project with previous projects, which is a common method of estimation for small organizations and small projects. The historical data is often limited to the memory of the estimator. In this case, the estimator would need to be experienced and would likely have been with the company for awhile. Some people believe it is better if the estimates are done by outsiders so that there is less chance of bias. It is true that people outside an organization will likely have to deal with fewer company politics than people within the organization. For example, the developer for a company may want to please the manager and so give an estimate that is overly-optimistic. The disadvantage of having an outside estimate is that the person would have less knowledge of the development environment, especially if the person is from outside the company. An empirical method of estimation would then be required, such as the Constructive Cost Model (COCOMO), which is discussed in more detail in section 5. Empirical methods of estimation can be used by all types of estimators. There may be some resistance to using an empirical method of estimation because there may be some question on whether a model could outperform an expert. People who are accurate estimators are rare in our experience, and so it is best to get the opinion of several people or tools. < To Top 2.5 General Steps and Remarks To give the reader a better idea of how software cost estimation fits into the development process, we will outline the general steps for doing cost estimation. The steps are not numbered because they are not completely discrete from one another. As well, although they generally follow a logical order, some of the steps can fit into several parts of the development process. Although this may at first seem to be confusing, the steps are straightforward enough that there should not be any difficulty in envisioning how they fit into the development process. The first and most important step is to establish a cost estimate plan (Pressman, 2001). In this plan, it should be stated what data will be gathered, why the data is being gathered, and the goal for doing the cost estimation process. Determining which data is to be gathered is essentially stating the level of detail of the metrics. This decision can influence the amount of decomposition for the tasks. There is obviously no point in gathering data that will not be used. This will seem unnecessary, and require more work, for the people who have to collect and manage the data. Although it may seem like a good idea to gather metrics that will not be used in the near future, but could possibly be used in the future, this is a waste of resources at the time. A fair amount of thought should be put into the cost estimation plan, much like the requirements for a project. The second step is to perform a cost estimation based on the requirements. Decomposition of the project can be done at this time if a lower level of abstraction is needed for the data. Keep in mind that it is important to use more than one method of estimation because there is no perfect technique. If there are wide variances in the estimates of the methods, then the information used to make the estimates should be re-evaluated (Humphrey, 1990). During the lifecycle, re-estimates should be done to allow for refinement of the cost estimates. The re-estimates could be done at major milestones during the project, or at specific time intervals. This decision will depend on the situation. Changes may have to be made to the project if the cost estimates either increase or decrease. At the end of the project, a final assessment of the results of the entire cost estimation process should be done. This allows a company to refine the estimation process in the future because of the data points that were obtained, and also allows the developers to review the development process. < To Top 2.6 Document Overview The remainder of this document consists of the following sections. The cost estimation process is outlined in section 3, which includes two different views of the estimation process. Six methods of doing cost estimation are described in section 4, along with the advantages and disadvantages of each method. A popular empirical method of estimation is discussed in section 5. Finally, section 6 includes a summary of some of the main issues of the cost estimation process.