Why we launched outsourcing vendor Agile maturity assessment
Published by Guntis Urtans on Friday, August 30, 2013 11:00
When you need to select a company for your project you are dealing with a set of requirements and selection criteria. Technical capabilities of the prospective team can be assessed based on resumes and interviews with candidates. Domain experience can be rated by the list of project references. What if Agile experience is one of project requirements? How to compare Agile experience and maturity of your potential service providers?
What if Agile experience is one of project requirements?
How to compare Agile experience and maturity of your potential service providers?
When we first faced this requirement, our immediate reaction was – you should count the Certified Scrum Masters (CSMs) – since Agile is pretty much Scrum in most cases.
But does the number of CSMs provide you with useful information? – Well, it basically tells you how many people were sent to a few day training sessions. It does not mean these people have done or will be doing any Scrum project activities, it just tells they attended classes.
Your next attempt could be to identify the number of people with Agile skills. This would give you more information – but you may end up with a large number of people who passed the training a few years ago and have never had even a single daily stand-up looking better than a relatively small group of people using most Agile practices systematically.
Selecting agile framework
After some attempts to come up with a set of measurable items summarizing Agile maturity, we understood that such assessment requires an Agile maturity framework. And we found one - Roadmap for Agile success.
Roadmap for Agile success provides several areas of Agile practices which should be taken into account and describes particular practices associated with 3 levels of Agile maturity.
Building assessment process
Our next step was to build a whole assessment process to evaluate against the selected assessment framework.
What were our requirements?
- It should be efficient and fast - we should be able to get reliable assessment results within hours and days rather than months and years
- It should cover framework requirements in the form of quantitative measurement
- Results should reflect actual experience rather than the creativity of companies’marketing departments
So we came up with the idea that all Agile Team leads in the company to be assessed should fill in a survey on practices used in one of their recent projects.
Benefits of the survey
Accountability – we are collecting information about actual practices within a specific project reported by a team lead. If you are interested in understanding how your prospective project manager was handling the previous engagement – you can get his/her individual results as well as a comparison to the company and industry averages.
Assessment of several areas and categories – some may be more relevant to you than others – focus on those that you find essential and important
Numeric results – you can see average values as well as top and bottom values for all categories and compare competitors side by side.
Usage density – you can find out how widely Agile is used in an organization and to what extent. It tells you much more than any pass/fail certification.
Simplicity - it is easy to run. Just get the list of vendor’s Agile Team Leads, make sure they fill in their survey (which takes 1-2 hours on average) and get your VAMI results.
Cost - the survey is free.
So now when we have Agile experience as a requirement and selection criteria for a project, it is quite simple to get the evaluation done – we request participants to pass VAMI assessment and compare their results.
And you can do it as well.