Performance Metrics for Agile SCRUM Process


Performance Metrics for Agile SCRUM Process

Joseph Vargheese PMP CSM CSP, joevgh@gmail.com 

(Currently looking for consulting opportunities within USA.  Please contact joevgh@gmail.com)

  • The metrics below focus on 5 different areas, Productivity, Quality, Effectiveness of SCRUM, Earned Value and Predictability of the SCRUM.
  • Metrics listed below will help track overall health of agile scrum process on enterprise level.
  • Normal Agile metrics are limited to commitments with in sprint like sprint burn down and stories dropped. Customer satisfaction is an important metrics.
  • Employee satisfaction indexes will help to realize true values of agile process and then work as an enabler for creating high performance sprint teams.
  • Though SCRUM ensure quality and releasable product after every sprint, quality will still be a  concern due to the nature of changes allowed with in a sprint and hence those metrics are very important.

 

1.     Introduction

A process is an enabler for software development. Process improvement requires quantitative measurements. Following information covers some of measurement techniques effective in agile Scrum process. These metrics can be further subdivided to get more accurate measurements

All of the below information are related to Agile SCRUM process only.

2.     Measurements in any process

  • Productivity
  • Quality
  • Effectiveness of process
  • Earned Value
  • Predictability of the process

3.     Units of Measurements

  • KPI is a metric with strategic objective of performance and should have at least one time bound value
  • Metrics is any measure of a component
  • SLA is a contract on meeting certain KPI between two parties.
  • Metrics will lead to define KPI, and then further lead to define SLA

 Advertisement

4.     Productivity

4.1   SCRUM Team Productivity (KPI) respective to peer teams

SCRUM Productivity is relative measurement between teams.  One of the easy measures is number of hours taken  between teams on Small/Medium/Large story types.  Story Points will be better to remove confusion of hours.

Example based on average hours by each story type over 1 year time

Story T-shirt Size # of Stories Analyzed Average Hours/story of 4 Teams Sprint Team 1(hours) Sprint Team 2(hours) Sprint Team 3(hours) Sprint Team 4 (Hours)
Small

420

14

10

16

12

18

Medium

800

27

22

28

22

35

Large

350

59

45

68

49

72

Extra Large

80

102

85

102

95

125

Example based on average Story Points by Sprint team per year

Average Story Points/team/sprint Sprint Team 1(SP) Sprint Team 2(SP) Sprint Team 3(SP) Sprint Team 4 (SP)
Story Points 70 90 65 98 55

 

4.2 Test case automation velocity within a Sprint (KPI)

It is often referred as number of test cases automated with in a Sprint/per day/per tester

4.3 Test case execution velocity within a Sprint (KPI)

It is often referred as number of test cases executed with in a Sprint/per day/per tester

5.    Quality

5.1  Pre-Release Bugs (KPI)

This is a measure of defects introduced by Sprint development measured by the amount of hours spend to correct the issues tracked by team.

Examples based on Rework hours introduced by each team

Sprint Team 1(hours) Sprint Team 2(hours) Sprint Team 3(hours) Sprint Team 4 (Hours)
Release 1.0 104 140 55 77
Release 1.1 140 154 77 86
Release 1.2 76 175 45 95

5.2 Post-Release Bugs (KPI)

The defects slipped into production from sprint measured by sprint team introduced the defects and hours spend on fixing the defects.

Examples based on Rework hours introduced by each team

Sprint Team 1(hours) Sprint Team 2(hours) Sprint Team 3(hours) Sprint Team 4 (Hours)
Release 1.0 14 23 19 14
Release 2.0 0 14 23 0
Release 3.0 0 19 0 23

5.3  Number of issues found during code review process with in a sprint (KPI)

5.4  Number of defects found with in a story with in a sprint (KPI)

This metrics is an indicator of the clarity of requirement or quality of resources.  Both reasons could result in unexpected defects within a story. Appropriate remediation can be implemented based on metrics

 

6     Effectiveness of SCRUM

6.1 Capacity available (KPI)

Sum of original capacity available in a Sprint based resource availability and sprint duration

6.2 Velocity (KPI)

Sum of original estimates of all accepted work

6.3 Estimate Expansion (KPI)

Sum of work found during sprint work

6.4 Amount of work dropped out of Sprint (KPI)

6.5 Accuracy of Story Estimation (KPI)

6.6 Capacity utilization (KPI)

This is a very loosely used term with Sprint process.  This forces the team member to report all of the time he worked in a specific project. There is no specific rule of thumb here.  It is very easy to get 85% of time reported in any reporting system.

 

7    Earned Value

Earned value is an important part of waterfall process.  Earned value is a direct measurement in Waterfall process based on hours worked to date and the features completed.

7.1 Number of features released (KPI)

7.2 Number of Story Points released (KPI)

7.3 Cost incurred per release (KPI)

7.4 Number of Story Points remaining for a release (KPI)

7.5 Money required to complete a release (KPI)

7.6 Number of story points accepted by Product Owner (KPI)

 

 

8    Predictability of the SCRUM

8.1 Sprint burned down chart (KPI)

Sprint burn down chart will help to predict health of the sprint as well as the possibility completion of stories committed with in sprint.

8.2 Accuracy of Story Estimation (KPI)

Story estimation expansion is a common problem during the sprint.

8.3 Amount of work dropped out of Sprint (KPI)

8.4 Hours lost because of Impediments with in Sprint (KPI)

8.5 Number of queries originated due to unclear requirements (KPI)

8.6 Estimation expansion due to changing requirements with in sprint (KPI)

 

9    Customer Satisfaction (KPI)

It is measured using the input from the customer. Usually a set of questions will be answered by the user about both qualitative and quantitative satisfaction of the customer.

 

10  Employee Satisfaction (KPI)

Similarly for employee satisfaction is measured directly from employee user anonymous surveys, peer reviews or one-on-one sessions.  It is important to understand the employee satisfaction.  It often is a sizeable factor in creating high performance SCRUM team.

11  Conclusion

Above defined KPIs are only few high level KPIs.  These KPIs can be further subdivided into more granular level ones, to make to 1000 KPIs

Author: Joseph Vargheese  joevgh@gmail.com

Advertisements

About Joseph Vargheese
email:joevgh@gmail.com.

2 Responses to Performance Metrics for Agile SCRUM Process

  1. Pingback: Roles and Responsibilities in Off-shore Scrum | josephvargheese

  2. Pingback: Countries and Rates for off-shore software development | josephvargheese

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: