Performance Metrics for Agile SCRUM Process

Performance Metrics for Agile SCRUM Process

Joseph Vargheese PMP CSM CSP, 

(Currently looking for consulting opportunities within USA.  Please contact

  • The metrics below focus on 5 different areas, Productivity, Quality, Effectiveness of SCRUM, Earned Value and Predictability of the SCRUM.
  • Metrics listed below will help track overall health of agile scrum process on enterprise level.
  • Normal Agile metrics are limited to commitments with in sprint like sprint burn down and stories dropped. Customer satisfaction is an important metrics.
  • Employee satisfaction indexes will help to realize true values of agile process and then work as an enabler for creating high performance sprint teams.
  • Though SCRUM ensure quality and releasable product after every sprint, quality will still be a  concern due to the nature of changes allowed with in a sprint and hence those metrics are very important.


1.     Introduction

A process is an enabler for software development. Process improvement requires quantitative measurements. Following information covers some of measurement techniques effective in agile Scrum process. These metrics can be further subdivided to get more accurate measurements

All of the below information are related to Agile SCRUM process only.

2.     Measurements in any process

  • Productivity
  • Quality
  • Effectiveness of process
  • Earned Value
  • Predictability of the process

3.     Units of Measurements

  • KPI is a metric with strategic objective of performance and should have at least one time bound value
  • Metrics is any measure of a component
  • SLA is a contract on meeting certain KPI between two parties.
  • Metrics will lead to define KPI, and then further lead to define SLA


4.     Productivity

4.1   SCRUM Team Productivity (KPI) respective to peer teams

SCRUM Productivity is relative measurement between teams.  One of the easy measures is number of hours taken  between teams on Small/Medium/Large story types.  Story Points will be better to remove confusion of hours.

Example based on average hours by each story type over 1 year time

Story T-shirt Size # of Stories Analyzed Average Hours/story of 4 Teams Sprint Team 1(hours) Sprint Team 2(hours) Sprint Team 3(hours) Sprint Team 4 (Hours)





















Extra Large







Example based on average Story Points by Sprint team per year

Average Story Points/team/sprint Sprint Team 1(SP) Sprint Team 2(SP) Sprint Team 3(SP) Sprint Team 4 (SP)
Story Points 70 90 65 98 55


4.2 Test case automation velocity within a Sprint (KPI)

It is often referred as number of test cases automated with in a Sprint/per day/per tester

4.3 Test case execution velocity within a Sprint (KPI)

It is often referred as number of test cases executed with in a Sprint/per day/per tester

5.    Quality

5.1  Pre-Release Bugs (KPI)

This is a measure of defects introduced by Sprint development measured by the amount of hours spend to correct the issues tracked by team.

Examples based on Rework hours introduced by each team

Sprint Team 1(hours) Sprint Team 2(hours) Sprint Team 3(hours) Sprint Team 4 (Hours)
Release 1.0 104 140 55 77
Release 1.1 140 154 77 86
Release 1.2 76 175 45 95

5.2 Post-Release Bugs (KPI)

The defects slipped into production from sprint measured by sprint team introduced the defects and hours spend on fixing the defects.

Examples based on Rework hours introduced by each team

Sprint Team 1(hours) Sprint Team 2(hours) Sprint Team 3(hours) Sprint Team 4 (Hours)
Release 1.0 14 23 19 14
Release 2.0 0 14 23 0
Release 3.0 0 19 0 23

5.3  Number of issues found during code review process with in a sprint (KPI)

5.4  Number of defects found with in a story with in a sprint (KPI)

This metrics is an indicator of the clarity of requirement or quality of resources.  Both reasons could result in unexpected defects within a story. Appropriate remediation can be implemented based on metrics


6     Effectiveness of SCRUM

6.1 Capacity available (KPI)

Sum of original capacity available in a Sprint based resource availability and sprint duration

6.2 Velocity (KPI)

Sum of original estimates of all accepted work

6.3 Estimate Expansion (KPI)

Sum of work found during sprint work

6.4 Amount of work dropped out of Sprint (KPI)

6.5 Accuracy of Story Estimation (KPI)

6.6 Capacity utilization (KPI)

This is a very loosely used term with Sprint process.  This forces the team member to report all of the time he worked in a specific project. There is no specific rule of thumb here.  It is very easy to get 85% of time reported in any reporting system.


7    Earned Value

Earned value is an important part of waterfall process.  Earned value is a direct measurement in Waterfall process based on hours worked to date and the features completed.

7.1 Number of features released (KPI)

7.2 Number of Story Points released (KPI)

7.3 Cost incurred per release (KPI)

7.4 Number of Story Points remaining for a release (KPI)

7.5 Money required to complete a release (KPI)

7.6 Number of story points accepted by Product Owner (KPI)



8    Predictability of the SCRUM

8.1 Sprint burned down chart (KPI)

Sprint burn down chart will help to predict health of the sprint as well as the possibility completion of stories committed with in sprint.

8.2 Accuracy of Story Estimation (KPI)

Story estimation expansion is a common problem during the sprint.

8.3 Amount of work dropped out of Sprint (KPI)

8.4 Hours lost because of Impediments with in Sprint (KPI)

8.5 Number of queries originated due to unclear requirements (KPI)

8.6 Estimation expansion due to changing requirements with in sprint (KPI)


9    Customer Satisfaction (KPI)

It is measured using the input from the customer. Usually a set of questions will be answered by the user about both qualitative and quantitative satisfaction of the customer.


10  Employee Satisfaction (KPI)

Similarly for employee satisfaction is measured directly from employee user anonymous surveys, peer reviews or one-on-one sessions.  It is important to understand the employee satisfaction.  It often is a sizeable factor in creating high performance SCRUM team.

11  Conclusion

Above defined KPIs are only few high level KPIs.  These KPIs can be further subdivided into more granular level ones, to make to 1000 KPIs

Author: Joseph Vargheese


India and China IT Resource Challenges, 2012 Contemplation

  • India and China provides the largest pool of outsourcing software development talents to the global market.
  • Even though a million graduates come out every year in India and China, very few are globally employable.
  • More than 80% of developed nations’ talents  are globally employable where as China is 10% and India is 25%
  • Rapid increase of need in India and china forces employers to train talents outside of the traditional supply of talents.
  • Growth of local and global markets makes even lower skill level resource in high demand.
  • Today’s scarcity in India and China will be tomorrow’s resource “threat” to talents in developed countries because of the sheer volume of  resources now getting trained.


There is no doubt that India and China provides largest pool of outsourcing software development talents to global market. With 1.3+ billion people in India or China, thousands of universities and tens of thousands of technical institutes/colleges, one would imagine that getting right IT technical talents in these countries is very easy.  But truth of the matter, it is extremely difficult to get to right talent.  This article will list some of the parameters causing this issue and solutions for it. It has lot to do with university systems/training programs in these countries.

Resource Threat to Rest of the World

But how can these resources be a threat to resources in other countries, if there is scarcity in India and China.  Important point is that if companies need resources and if the university system did not train them, these organizations will train these resources to make them globally competitive for multinational employers to attract such talent.

Choice of Higher Education in India and china

  India China
Universities 348 2884
Regular Colleges 17625 – 
Engineering Colleges 3573 – 
Total Yearly Graduates 847000 1,900,000
Computer/Engg Grads 497000 763000
Globally employable talent % 25% 10%
Globally  employable talent /year 124000 76300

As per 2011 UGC of India statistics, India has 348 universities with 17625 colleges which include 3573 engineering colleges and 828 colleges educating Computer science/Computer Application post graduates, accounting total of 847000 graduates, in which around 497,000 graduates are trained to work in computer related industries. With 2884 universities and several thousands of colleges china produces 1.9 million graduates including 763000 engineering graduates (3 year and 4 year) trained to work in computer industry.

My Experience

During last two years, I interviewed more than 250 computer professionals of varied skill sets from India and China for different types of openings.  Results were not encouraging.  Most of them are not trained enough to work for global market place, atleast not to the extent that western universities train resources.   Very small percentages of them are globally employable.

Following difference will explain why these resources are globally deployable.

Education Streams India and China

Even though years of education are very similar, talents coming out of these universities are not same skill level. Most cases they possess far below skill set than western graduates.

USA Equivalent qualification in India
Associate Degree Graduate Degree Post-Graduate Degree
Diploma – Engineering  Diploma (3 years) BE– Bachelor of engineering(4 years) MS – Master of Science in engineering(2 Years)
BSc– Bachelor of Science (3 years) B.Tech – Bachelor of  Technology(4 Years) ME – Master of Engineering (2 years)
BA – Bachelor of Arts(3 years) MSc – Master of Science(2 years) M.Tech – Master of Technology (2 years)
BCA -Bachelor of Computer Application (3 years) MCA – Master of Computer Application(3 Years)  
PGDCA – Post Graduate Diploma in Computer Application(PGDCA)    


USA Equivalent Qualifications in China
Associated Degree Graduate Degree Post-Graduate Degree
Associated Degree of Science (3 years) BS – Bachelor of Science – (4 to 5 years) MSc – Master of Science 2 to 3 years)
Associated Degree of Engineering(3 years) BE – Bachelor of Engineering (4 to 5 years)  ME – Master of Engineering (2 years)
Associated Degree of Art (3 years) BA– Bachelor of Arts (BA)(4 to 5 years)  MA – Master of Arts(2 years)


Innovative vs. Transaction Talents

Education system in India and china produces majority of transaction based talents. Transaction based talents are well known for their ability to understand certain defined problem domain and find solution for problem.  These talents are very good in repetitive tasks and often provide higher productivity as they attain more experience.

Those who can bring innovation to engineering process are often called as innovative talents.  Innovative talents are abstract thinkers and great problem solvers.  They also work well within a team and possess strong personal skills. Many universities in India and china do train such talents, but volumes of talents are very less.  IIT/NIT programs in India and “C9 league” programs in China are best examples of such training programs. Lately several private universities in India train and produce top talents.

Choosing education selection stream

There is a major difference on how students choose education stream in India and China compared western world.  Most part of western world students chooses education stream based on their interest.  But in these Asian countries, education selection is driven by earning potential at large.  Most often these programs are selected by their parents or relatives for them. School system is not equipped to guide this student to right career paths. Most often lot of students end up in careers they are not passionate about.  Students with interest may not be able secure admission because the competition from the above group.  Situation is slowly changing in India and China with emergence of several alternate education streams now to train these resources in IT field.

Skills gap in India and china

Here are major difference between western world and Asian countries. Some of them are part of culture and some are part of education.

English: English knowledge is not wide spread in all of China and rural India. India still have higher % of English speaking population due to English being medium of instruction in most of the Indian universities. Besides, British ruled India almost 2.5 centuries, and left English awareness all across the country. On the other hand china started these programs probably only few years ago. Their medium instruction is still in Chinese. China introduced the CET (College English Test) levels (4, 6) in recent years as part graduation process. But these exams only guarantee writing skills and emphasize on spoken language. Younger population in china speaks better. During my 2010 visit to China, I spoke to several students in university and schools system and they seem to have higher skills in English and thus future looks promising to China.

Team work: Team work: Ability to work in team comes natural to graduates of several western universities.  It is a premium skill set for lot of graduates in India and China. This is due to education systems, training of these graduates. Several of education curriculums in India and china give higher importance to academics and often forget to impart enough training in team building and leadership activities. We are seeing big leap in India, in this area from private universities and premium private colleges. They are showing active interest in training such talent using varied facets of skill development programs.

Technical skillsAverage technical skill level of equivalent degrees in India and China is far below expectation from average western university graduates. This is due to the lack of practical problem driven training coupled with excellent lab facilities and access to best teachers.

Some other areas of challenge are leadership and problem solving skills.


With the increased demand of IT talents in India and China, these talents with lower skill level is also put to work.  Many of these talents are getting trained and over period of time these skill gaps will be filled with job based training.  It is interesting to see how these talents will fill the global demand of skilled workers as world is turning out to be more and more one single open society.


Author: Joseph Vargheese  



Practical Approach in Creating Agile Test Cases

  • Test case discovery happens throughout the sprint process.
  • Exploratory testing will uncover the hidden cases and expand to better coverage.
  • Overall, finding test cases in agile is a challenging process.
  • Definition of “done” of a story/feature is not clearly quantifiable.
  • Start with happy path and take following step by step approach to expand coverage.
  • Full step by step test case documentation is overkill, rather use summary test cases.
  • Most part, acceptance criteria defines the “done” for a Sprint story.


Identifying test cases in Agile is a somewhat difficult task.  In traditional waterfall or RUP, there will always be galore of documents to deduce this information from and in most cases every test case can be traced from some kinds of documented business/ system/ technical/ functional requirements.  Agile by its nature is very lean in documentation. This will create an issue in defining completion point for a feature distributed across several stories. This article defines the process of finding most test cases documented and undocumented with in a feature/story.

Step 1: Input points for test case scenarios

Traditional Agile scrum has following input points where test case scenarios can be uncovered. Every input provides different information. The list below is not conclusive.

  • Story documentation: Generally very slim.  Intention is to start conversation and get confirmation on requirements and definition of “done”.
  • Acceptance criteria for story: Talks about final intent of the story and definition of what makes story “done”.
  • Use cases /Requirement docs (if any)/UI specification (if any): Most cases use cases can be converted to test cases. Well written requirement will be very rare in agile stories.  If it exists, mapping every requirement to test case is an easy task.
  • Sprint planning discussion: Discuss almost all flows including happy path, alternate flows and exceptions flows. Discussion also includes boundary conditions, business rules, existing functionality changes etc.
  • Sprint standup: Discuss daily issues.  Always listen to change in flows as well as corner conditions.
  • Discussion with Developer/Product Owner: Works as a confirmation on numerous information received.
  • Sprint retrospect: Why test coverage is not enough for stories? Test case modification happens after the fact.
  • Exploratory testing: Testing through system without a specific test case.  This is the most effective method to find missing test case. This is method has been very successful with several agile teams.

 Step 2: Collecting Data

  • Tester is involved in all steps of sprint process. Following information should be sourced during all meetings in step 1.
  • Change in any existing functionality: This will results in modification/adding of existing test cases.
  • Happy path:  Tester will hear this several times during the process. Similarly alternate path/flow is also important.  Identify the number of alternate flows and make sure all flows are working.
  • Path of failures: Popularly called exception flow, flow terminated due to errors
  • Guard conditions: Example, login time out
  • Boundary Conditions: Also called as corner conditions. This will make sure that application will work for intended range of input data.
  • Business Rules: Group the business rules and test for one group at a time
  • Event based notifications: Any trigger point for any of the available functionality
  • Non-Functional requirements: Like performance or load of the system
  • Data mapping documents: Test cases can directly be derived from data mapping document.
  • Assumptions
  • Constraints
  • External Dependency: Is this dependent of any external interface for data pull or data push.

Even though all of these conditions (if exits) will result in test cases, every story will not have all of the above scenarios

Step 3:  Organizing the test cases

Since agile input points are many, it is possible that test cases closely related to each other will duplicate across locations.  Always organize test cases by functionality if possible.

Step 4:  Examining coverage

Most testers like to have 100% test coverage. Exploratory testing is the most viable solution to reach 100% test coverage. Since traditional RTM is missing in agile, acceptance criteria will also determine the coverage of test cases required.  Creating an “Acceptance Traceability Matrix” will help to cross check the coverage.

Step 5: Review

After completing test cases, always make sure to be reviewed by product owner or scrum master/developer.  Any coverage miss derived during the review process will be added/updated to test cases.

Step 6: Continuous integration from defects and enhancements

Product defects as well as enhancements will add/modify test cases. Defects generally indicates overlook of test coverage during the sprint testing.

More details contact author: Joseph Vargheese PMP CISA CSM,,

Is your off-shore engagement profitable

Is your off-shore engagement profitable

Joseph Vargheese PMP CISA CSM

Atlanta GA USA

Dated: 2012-03-09

 (This article is a qualitative assessment, so please don’t quote anywhere)



Most companies do off-shore software development to save cost.  But in most cases it is not effective to have off-shore environment unless closely monitored. Even some of best run environment are not as profitable as perceived. Following method will help to measure off-shore effectiveness.  This model used to measure T&M offshore model. It can also be extended to use for outcome based as well as fixed cost models.


Perception is deceiving.  Attraction of lower cost compared to onsite cost alone does not determine the cost effectiveness. There are overhead costs, not figured into this perception.

Early warning signs

  • Are you hearing your manager’s complains about off-shore management difficulties?
  • Are you hearing that your team members are not happy with off-shore software delivery?
  • Are you hearing off-shore communication difficulties?

It is the time to measure your offshore productivity.

Understand variables

The first step is to understand following variables

  1. Communication overhead (Communication overhead is due to time zone and cultural difference  generally calculated at 10% of cost. This is the time spend by your onsite team to communicate work required from offshore)
  2. Capacity utilization (Billed hours vs actual productivity hours reported against task assigned)
  3. Onsite governance cost (Cost of onsite governance to verify the deliverables produced from offshore environment estimated as high as 10%)
  4.  Employee Productivity (Productivity of the offshore employee compared productivity of onsite employee)
  5. Software rework cost on offshore created issues (case by case basis)

Measuring these variables

                Client should start measuring these variables on regular basis.

Average offshore operations is at 6$ loss per resource hour

How is that possible?  Let us examine some average numbers.

Capacity Utilization: 75%

Resource Productivity: 75%

Communication overhead: 10%

Onsite Governance: 15%

These numbers look very attractive to any normal user.

Let us assume your onsite cost per hour is 75$ your offshore cost is 25$ per hour. 

Your overall productivity is 75% * 75% which is approximately 50%

Your earned value is at 75$ * 50% = 37.5$

Your expense is 25$ + 75$ * (10%+15%) Onsite governance cost = 43.75$

In effect your offshore center is at loss of 6$ per hour of a resource

Steps to improve

  1. Negotiate reasonable offshore rates.
  2. Improve capacity by scheduling available capacity and tracking on regular basis (Target: 90%)
  3. Improve your productivity by having KT programs and frequent onsite and offshore travel programs (Target: 85%)
  4. Reduce onsite governance cost to 5% (Convert your offshore coordinator to a contributing resource)
  5. Reduce communication overhead by having overlapping working hours (5%)
  6. Reduce or avoid rework by putting together KT and close feedback mechanisms


Client required to have constant monitoring of these numbers to ensure minimum variance on achieved performance.  It is recommended to add a part-time role of offshore performance manager to monitor and govern the progress achieved.

Countries and Rates for off-shore software development

Countries and Rates for off-shore software development

Joseph Vargheese PMP CISA CSM, Atlanta GA USA,

Dated: 2012-03-09

(Please let me know, if you need help in locating a vendor with lowest rates below in respective countries)

1.    Introduction

It is somewhat confusing for a company to do offshoring these days. With the entry of Latin American companies, it is somewhat confusing for a client to decide a vendor. Every country has certain strength and weakness. There are always some hidden factors in off-shore software development which will not be known to organization seeking services till engagement starts.

Please check here for Salary of Software Engineers Globally

2.  India (Tier-1 Cities) –Hourly Rate: 20$-40$


  1. Natural choice
  2. Long term player
  3. Strong knowledge in offshore software delivery models
  4. Strong process framework knowledge especially agile processes.
  5. Highly experienced and large supply of experienced workers
  6. Higher English language fluency (both written and oral)
  7. Higher technical skill level and wide array of skill sets
  8. Very good infrastructure



  1. Time zone difference  of 10-12 hours
  2. Offshore operation cost is rising
  3. Resource attrition is high as 25%

3.  India (Tier-2 Cities) – Hourly Rate: 12$-30$

Many Indian companies started operation tier-2 and tier-3 cities.  Apparent cost benefits are most often not transferred to organization seeking services.

Most of recent start-up in software industry originated from Tier-2 cities and there are variety of vendors offering niche services from these cities at lower cost.

Considering ROI on variety of services, Tier-2 cities have better ROI than Tier -1 and Tier 3 Cities.

One of challenge is lower supply of highly skilled man power.   But not every part of software development requires skilled resources.

4.  India (Tier-3 cities and below) – Hourly Rate: 12$-25$

One of challenge is lower supply of highly skilled man power. But not every part of software development requires skilled resources.

But attrition is also equally lower compared Tier-1 and Tier-2 cities.

Suitable for creating small web-site support and BPO operations.


5.  China (Tier -1) – Hourly Rate: 15$-40$


  1. Low cost operation suitable for software testing
  2. Highest productivity on highly documented process driven work items
  3. Large supply of experienced workers
  4. Higher technical skill level on many areas
  5. Very good infrastructure


  1. Time zone difference  of 10-12 hours
  2. Lack of wide array of technology skill set
  3. Lower oral English language skill set
  4. Week software  development process  framework
  5. Lack of knowledge in offshoring delivery model
  6. Resource attrition is high as 25%

6.     China (Tier -2 and below) – Hourly Rate: 10$-25$

One of the advantages on moving to tier-2 cities in China is the lower operation cost. Tier-2 cities in China are 30% cheaper than Tier-2 cities.

Large supply of small companies in tier-2 cities in China. But the disadvantage is English speaking skills. If communication can be limited to email and instant messenger, these companies can do excellent delivery of services.

7.      Brazil – Hourly Rate: 30$-50$


  1. Same time zone as North American clients
  2. Lower cost operation suitable for infrastructure outsourcing
  3. Moderate supply of workers
  4. Moderate technology skill set
  5. Moderate oral English language skill set


  1. Strong labor union presence in software service sector force lower productivity
  2. Scarcity of trained supply for resources and lack of wide array of technology skill set
  3. Week software  development process  framework
  4. Lack of knowledge in off-shoring delivery model
  5. English language is still a challenge
  6. Need strong training division for resources
  7. Unstable governments and inadequate IP laws

8.     Vietnam – Hourly Rate: 10$-20$


  1. Moderate English language skill set both oral and written
  2. TBC…


  1. Time zone difference  of 10-12 hours
  2. Lack of experienced workers
  3. Lack of wide array of technology skill set
  4. Week software  development process  framework
  5. Lack of knowledge in offshoring delivery model

 9.     Russia – Hourly Rate: 5$-20$


  1. Lowest cost of operation
  2. TBC…


  1. Mostly email only communication
  2. Lack of experienced workers
  3. Lack of wide array of technology skill set
  4. Week software  development process  framework
  5. Lack of knowledge in offshoring delivery model
  6. Lack of English language skill set both oral and written

 10. Argentina-Hourly Rate: 30$-55$


  1. Same time zone as North American clients
  2. English language skill set both oral and written
  3. TBC…


  1. Strong labor union presence in software service sector force lower productivity
  2. Lack of experienced workers
  3. Lack of wide array of technology skill set
  4. Week software  development process  framework
  5. Lack of knowledge in offshoring delivery model
  6. Scarcity of trained supply for resources.
  7. English language is still a challenge
  8. Need strong training division for resources
  9. Unstable governments and inadequate IP laws


11. Chile – Hourly Rate: 15$-25$


  1. Same time zone as North American clients
  2. TBC…


  1. Highly labor union driven economy
  2. TBC…

 12. Costa Rica – Hourly Rate: 25$-50$


  1. Same time zone as North American clients
  2. English language skill set both oral and written
  3. TBC…


  1. TBC…

 12. Mexico – Hourly Rate: 35$-55$


  1. Same time zone as North American clients
  2. TBC…


  1. Labor union driven economy (not so much in software)
  2. English language is still a challenge
  3. Need strong training division for resources
  4. Unstable governments and inadequate IP laws

High Performance Software Development Teams

High Performance Software Development Teams

Joseph Vargheese PMP CISA CSM

Atlanta GA

Dated: 2012-03-03



Creating a high performance team often comes as a dream to several software development managers and senior management, because they often produce as high as 5 times the output of a normal team.  But it is not easy to put together a high performance team.  This article presents some steps to create and maintain high performance teams.

Accidental exposure

Three times in my 16 years of career, I had opportunity to work or manage high performance teams.  Almost all three times, evolving of these teams was very accidental. I was able to maintain the tempo of team and productivity level. I like to share my experience with these three very different environments.

Small and Nimble

Teams were small and nimble in all three instances with size of 4 or 5.  It make cross communication channels to minimum.

 Visionary Leadership

My first exposure to a high performing team comes from a small company.  I was a team member that time.  My manager was initially an individual high performer then built a small team of high performers. He understood the strength of every team member and proactively guide in the areas team member may face issues. Idea is, not to waste time. He encourage people to help each other even before other member seek for help.  I felt as if he could predict everything going to happen in the team which was always amazing to me. He was a visionary leader.  If we think outside the box, almost all of high performance companies are built by visionaries and so are the teams

 Co-dependent Members

Second exposure was in large insurance company where I was a manager.  I was asked to deliver mission critical project in small time frame and given freedom to pick the team members. Application has components from PC to web to mainframe and data return in the same path. I never looked at individual high performers instead selected performers who work together very well. To my surprise team worked very closely and came up with three different solutions.  Every member of the team was dependent on other’s skillset wherever applicable to speed up the process. We selected two solutions and worked in parallel sharing some of efforts on both solutions.  First solution was a performance disaster and second solution was a huge success.  Application was delivered on time with minimal or no defects. Same team was kept intact for several assignments and every one of them delivered on time.

 Identity and positioning

Imagine that your company wants to build a very high performance team.  You have selected 5 of your high performers and put in one team.  I can almost guarantee that it will be a disaster and high conflict of interest. My reasoning is that every member of the team should find their identity within the team and like to get that respect and positioning from others. It looks as if an invisible hierarchical system is being developed organically within the team and members understand their position within the team.

This is same reason why hiring decision are important to these teams.  Let me explain two of hiring mistakes. I hired highly knowledgeable person (He wrote several books as well) to high performance team and It was a disaster. He could not find the identity within the team.  Another example is of hiring resources with several awards and she also could not find identity within the team as well.

Challenges and common goal

High performers require challenges and recognition of their importance within the team. I have an experience to share here. A company was starting a very important initiative to go to Web visibility of application.  A meeting was called with few of the best minds in the company and I was the project manager in that effort.  Meeting also attended by few senior management folks because of the strategic importance. One of the team member slept in that meeting. After the conclusion of the meeting one of senior manager called me and asked me to officially warn the person slept in that meeting.  I thought that was not the right approach, instead I called resource and challenged him to make this project successful and promised to write great review based on the success and given the freedom to make any decisions.  He did that project successfully and several projects thereafter. He got high visibility within the company which excited him.  He created a self-goal to work as a change agent within company and act as catalyst to bring new ideas to company.  He also excited people around him to have similar goals.


Communication is the key in success of any high performance team.  That is why expected team size is 4-5, which makes smaller number of communication channels. Prompt and two way interactions resolve issues very quickly and make email communication to minimum.

Off-shore software development vendor selection process

Off-shore software development vendor selection process

Joseph Vargheese PMP CISA CSM

Atlanta, GA, USA

Dated: 2011-July-5


1.     Initial planning

  • Number of resources and budget
  • Type of project
  • Complexity
  • Development process
  • Data Security
  • Compliance

2.     Countries of preference

  • India (Very high process maturity)
  • China (Very cheap testing resources. Very good for Japan and Korea with no language barrier)
  • Brazil (same time zone  with North America, Good for Infrastructure)
  • Vietnam (Very good rates for medium skillset)
  • Russia (Very cheap single digit per hour rate for email only communication)
  • Costa Rica (Same time zone.   Medium skill set)

3.     Preference parameters 

  • Quality of deliverables vs. cost
  • Quality of manpower available
  • Communication Overhead
  • Overlapping hours of operation
  • Process framework
  • Information and Data Security infrastructure

4.     Primary screening process sources

5.     Right sizing a target vendor 

  • Above 10 Millions :  Large companies above 50000 people
  • 4 to 10 Million :  Companies with 10000 to 50000 people
  • 1-4 Million Companies with 1000 to 10000 people
  • < 1 Million: Companies below 1000 people

6.     Pre-screening process

6.1 Domain Expertise

Does the vendor have organization has expertise in the domain? example insurance, banking, healthcare etc…

6.2 Technology Expertise

Does vendor organization have expertise in the technology domain   Example: mainframe, client server, Web, Database etc..

6.3 Process Expertise

Does vendor organization have expertise in the process framework and exposure the development process used by client whether it Agile, Waterfall etc.

7.     Development methodology alignment

  • Agile Development
  • Waterfall Development
  • RUP Development

8.     Cultural alignment
Please make sure cultural alignment is good with destination vendor organization including fast paced environment vs. highly process environment

9.     Training and development

Continuous infrastructure for training and development is key factor the success of the offshore environment due high employee turnover as high as 20%

10. Full RFP process

Full RFP process is recommended only for engagement above 2 million.  Engagements below 2 million a short RFP process will suffice.

11. Vendor presentation preparation

An agenda with short set of slides should be provided in advance for streamlining presentation material.

12. Global rate card

This point client should start collecting all possible needs for next two years with in organization and start getting pricing for the same.  It will be a tough negotiation  to get a better rate on a later time.

13. Vendor contract components

If it is not black and white, it can’t be enforced.

  • Intellectual property rights
  • Communication channels
  • Overlapping hours
  • Governance model
  • Escalation Process
  • Penalty for not performing
  • Exit Criteria
  • Offshore performance management
  • Rate lock

14. Managing a vendor 

  • Governance model  Implementation
  • Escalation Process
  • Formal issue reporting and tracking system
  • Continuous staff training programs
  • Key employee backup program
  • KPIs
  • Tuning of delivery model and process framework

Hidden facts of an off-shore software vendor

Hidden facts of an offshore software vendor

Dated: 12/28/2011

Joseph Vargheese 

Atlanta GA USA

This article mainly reference operations in India and china, and assumes minimum knowledge of offshore operation in China, India, Brazil, Costa Rica, Russia and Vietnam.

Location of your ODC

Location of youroffshore software development (ODC) environment determines offshore vendor price point as opposed to single rate for a country. Resource cost attributes to biggest expense in an ODC. But those salaries are determined by the location. For example average software engineer salary in major cities is around 14000$ in china and 16000$ in India (as of 2011). As operation move from one city to other cost also changes. For example salary in Dalian (second  tier city in China) is 25% cheaper than Beijing (first  tier city in China) or Shanghai or Shenzhen. There will be another 20% resource cost advantage if company move to a city like Jinan, which happens to be located few hours from Beijing. Similarly in India moving from Bangalore/Chennai/Hyderabad to second tier city like Bhopal/Kochi/Mysore may have 30-40% cost advantage.

Not all cities are same

Not all cities are good for sourcing highly skilled software professionals. Companies outsourcing to second  tier cities have to think about the type of manpower they can source from that city. Generally second  tier cities have lower supply of highly skilled man power compared to first  tier cities. If your operation is a heavy R&D based, it will be advisable to stay within a first  tier city.

Challenge in English

Recognize the fact that English speaking resource is a challenge in many outsourced countries. It is very challenging to recruit English speaking resources in China and Russia. Unfortunately the client will have direct feel of the problem only when contract is done and the staff selection has started. To some degree, vendor could find English speaking resources in India/Vietnam/Brazil/Costa Rica because of the higher supply.

Infrastructure and Operations cost

Infrastructure and Operations cost are not always a component in many countries. It is not the same level playing field between outsourced countries. China has high federal subsidies to operate within a software technology park owned by government, including subsidized tax, electricity, water, rent for several years. Brazil, Vietnam and Russia have some subsidies. But operating cost in these countries is much lower than India and China. India has subsidies for operations mostly driven by state governments to attract companies to that state. It includes subsidized rent, electricity and water. The high federal subsidies make China’s operation cheaper than other countries.

Experience and Salary

Experience determines the salary not the technology. Many  vendors sell CRM and ERP skilled offshore technology worker for a higher price. But for most of these countries, including India/China, resource pricing is driven by YOE. That means SAP professional and QA tester will only have  a small difference in remuneration for same years of experience.

Employee turnover rate

Most of the time, “stated employee turnover rate” does not include first year. Most of the time, vendor will state a yearly attrition rate excluding employees leaving in the first year of their employment, unless specifically asked. That number alone could be an additional 10%.

Overlapping hours

Practically overlapping working hours are hard to realize. Many  vendors provide overlapping hours coverage with their USA counterparts as work from home option or office. Most often employees can’t stay during night time due to family commitment or transport etc. In case of  work from home, most of the Indian or Chinese homes are not equipped with internet and phone facilities. Yet another note is that, in large cities these professionals are staying with an extended family in a 2 or 3 bed room apartment, which will not give them enough privacy required to function.

Many cities in India and china get cold during winter time. Life becomes almost stand still after 8:00 PM. That means transportation is very minimal as well as safety become issue for female employee to travel alone.

One size may not fit all

A high performing vendor in one area may not be suitable for other type of efforts. Vendors like to get every type of business available within a company. Every vendor engagement is a standalone unit. Most people do not understand that it is very challenging to bring another horizontal knowledge base to that engagement. For example you have very successful .NET based engagement and suddenly an SAP requirement comes. Natural choice will be the vendor performing currently. But most often it will be as challenging as starting a new engagement.

Disappearing resources

It is usual to exchange experienced resources for other projects. As engagement gets matured, vendor will start pulling experienced resources stating various reasons. Reason is that they are more valuable somewhere else in their organization. This will reduce the quality of your engagement. Client should consider contractually binding these resources.

Plan for the worst, expect the best

Terminating the contract and knowledge transfer to another source is always a sticky point during the contracting process. As time goes your offshore vendor can grow significantly, which make your current operation very small for them. On the contrary, client could grow fast which warrants a better offshore partner to cover client’s needs. Client’s outsourced contract should be able to terminate any time and always add an option to do a knowledge transfer to another vendor. It should be at the discretion of the client to decide on the notice period and not all resources are required for knowledge transfer.

Bring Competition

Competition brings best out of everyone. Always consider  two vendors for your engagement, because inherent competition always brings better value for the client.

Reverse knowledge transfer

It is a challenging issue to bring back knowledge from outsourced engagement to onsite. The challenge will become very high if the resources can’t speak English. This issue  has to be addressed during vendor goverance process.

My special thanks to Rajesh Philip for reviewing this article