There are better ways to judge a consultancy than win rates

Goodhart’s law (from the impeccable source of Wikipedia) is that “when a measure becomes a target, it ceases to be a good measure.” The anecdote often given to illustrate this is the ‘Cobra Effect’ in colonial India.

In the 19th century, the government was concerned about the large number of venomous cobras in Delhi. So they introduced a bounty for every dead cobra brought to them. Naturally you get the benefit initially (fewer live cobras), but then as the bounty became a target, people began to game the system. Some people started breeding cobras, with the inevitable result (loads more live cobras).

The measure (dead cobras) ceased to be a good indicator of the actual objective (public safety and reduced cobra population).

So, what could this mean for win rates as a metric? Potential unintended consequences:

• Client selection bias: the company may become overly selective in choosing clients, focusing only on those with a high likelihood of success in the narrow field of specific opportunities. They will not be willing to take risks with clients looking to change how they do things or move into high-risk but high-potential markets. Low hanging fruit.
• Selective reporting: having a fast and loose relationship with the data to produce the desired result. Perhaps that reported rate is not an accurate reflection of anything.
• Potential conflicts of interest: if the primary goal is to increase clients’ win rates, there could be a risk of recommending or pursuing strategies that are not in the best interest of clients but instead serve to merely boost their win rates. Sounds great in the short term but could have a knock-on with the customer relationships (selling something you can’t deliver) or over-reliance (not helping you improve because they want to keep selling the same service).

This last point is critical – does this company want to help you improve, or do they want a reliable stream of income for years to come?

Advertised win rates do not provide context on the scope of the consultancy’s involvement or the types of bids they worked on. A high win rate may be impressive, but it may not reflect the level of effort or expertise that was required to achieve it or the relative position the client was in before the consultant rocked up. Clients deciding which consultancy to approach should consider the specific services provided by the consultancy and the complexity of the bids worked on, rather than relying solely on win rates. It’s an easy metric to feel like you’ve done your due diligence, but you know it’s too simple.

Let’s say that Bidder A and Bidder B both submit proposals for a project. Bidder A has a win rate of 80%, while Bidder B has a win rate of 30%. However, upon closer examination, it becomes clear that Bidder A has only submitted five bids in the past year, while Bidder B has submitted 50 bids in the same period.

In this scenario, Bidder A’s high win rate may not necessarily be indicative of their quality or value as a bidder. It is possible that they only bid on projects that are a good fit for their capabilities and have a high likelihood of success. Conversely, Bidder B’s lower win rate may be a result of their willingness to bid on a wider range of projects in line with a specific growth and relationship development strategy, some of which may have been less of a good fit.

Or another way of looking at it is the involvement of the consultancy – can we claim a win on Deal A when we held one pre-mortem? Yes, it may have increased the probability of a win, but it’s not the only factor. Or Deal B where we provided win strategy work, evaluation and scoring, negotiation strategy workshops, wrote the Solution Overview, and generally provided coaching for the wider team in the room? Again, you have to hope we had an impact, but we didn’t set the price, had no influence over the past experience the client could present, or agreed the risk profile of the commercials, etc.

Ultimately, the goal of bid consulting should not solely be to increase win rates, but to provide value to clients by helping them improve their capability. Who wants to be reliant forever on a standard bidding service?

When you’re evaluating a prospective supplier, here are some key questions they should consider asking:

Referrals rate and repeat business rate: How do you measure success beyond just win rates, and what metrics do you use to evaluate the impact of your services – how many of your clients ask you for more and how many clients come from a referral from someone in another company? Do people want to keep working with this company.

Industry experience: What is your experience in our industry, and do you have any relevant case studies or success stories? I know the standard line is that bidding is agnostic, but it’s also not fully. Bidding to Virgin Atlantic is different experience needing different skills than doing so to DIO. You also want to know if they have worked on deals of the same complexity – have they worked a Competitive Procedure with Negotiation in a leadership role?

Scope of services: Can you provide references from previous clients, and what was the scope of the services you provided? Good for weeding out the people that did one workshop and claim the win as having anything to do with their involvement.

Flexibility: What is your approach to <insert your requirement here> project, and how do you ensure that your approach is tailored to our specific needs? Too many out-of-the-box methodologies and processes are foisted on clients. They end up back in a box on a shelf, unused.

Willingness to coach on the job: How do you ensure that your services are aligned with our long-term objectives, beyond just winning bids?

Delivery teams: What team will be delivering and what are their skills and experience? Beware of the experienced lead consultant being replaced by a crew of overworked juniors.

Governance: How do you handle conflicts of interest, and what steps do you take to ensure that you maintain confidentiality and impartiality throughout the process?

By asking these kinds of questions, you can gain a deeper understanding of the prospective supplier’s approach, expertise, and value proposition, and make an informed decision about whether they are the right fit for your needs.

Yes, it’s more involved that comparing a single metric, but your business development effectiveness is core to your business success. Be pragmatic. But don’t rely on a single metric that’s pretty bad at measuring impact.