Construction and evaluation of a tool for quantifying uncertainty of software cost estimates

Abstract

Software development effort estimation is a continuous challenge in the software industry. The inherent uncertainty of effort estimates, which is due to factors such as evolving technology and significant elements of creativity in software development, is an important challenge for software project management. The specific management challenge addressed in this thesis is to assess the uncertainty of effort required for a new software release in the context of incremental software development. The evaluated approach combines task-level estimates with historical data on the estimation accuracy of past tasks for this assessment, by creating effort prediction intervals. The approach was implemented in a web-based tool, and evaluated in the context of a large Norwegian software project with estimation data from three contracted software development companies. In the evaluation we compared the approach to a simpler baseline method, and we found that our suggested approach more consistently produced reasonably accurate prediction intervals. Several variants of the basic approach were investigated. Fitting the historical data to a parametric distribution consistently improved the efficiency of the produced prediction intervals, but the accuracy suffered in cases where the parametric distribution could not reflect the historical distribution of estimation accuracy. Clustering tasks based on size had a positive effect on the produced effort intervals, both in terms of accuracy and efficiency. We believe the suggested approach and tool can be useful in software development project planning and estimation processes providing useful information to support planning, budgeting and resource allocation.