How to Evaluate Ideas

By Jeffrey Baumgartner

Organisational innovation is not just about generating creative business ideas. It is also about reviewing ideas in order to identify those which are most likely to become successful innovations. Unfortunately, many organisations make mistakes in their idea review processes that result in rejecting the most potentially innovative ideas in favour of less innovative ones.

In some instances, the idea review process is a simple matter of a manager reading through a batch of ideas and selecting those she believes will work best for her firm. This is most often the case in smaller firms run by a single owner and manager. In most medium to large businesses, however, a structured evaluation process is necessary in order to:

• identify the ideas that are most likely to succeed as innovations for the company.

• ensure that complex ideas are reviewed by people with the appropriate expertise necessary to understand what would be necessary to implement the idea – and what might go wrong.

• enable a middle manager to defend the idea to senior management, stakeholders, and financial officers who may need to grant budgetary approval of the idea.

• make it possible to review a large number of ideas in a resource efficient manner.

• improve the idea by identifying potential implementation problems and preparing suitable actions to overcome those problems. Sadly, this last aspect is often lost in formal idea review procedures.

Commonplace Methods

There are all kinds of idea review methods. We will look at three methods that we use, pass-fail evaluation, evaluation matrices and SWOT analysis.

Pass-Fail Evaluation

If there are a large number of ideas that need to be reviewed –- for instance, you have run a company-wide ideas campaign on a popular issue and have generated hundreds of ideas – a simple pass-fail evaluation is often essential to bring the idea pool down to manageable levels.

It is best to start with a simple criterion for determining whether an idea will go on to a more in-depth evaluation. This criterion might be related to budget, time-frames, fit with company culture or just practical viability. Whatever the criterion, it should be made clear. If a colleague later asks what happened to her idea, you can explain why it did not pass this initial stage. Knowing that her idea did not get implemented for a pragmatic reason – such as being too expensive to implement – is more reassuring than having the idea rejected for no apparent reason.

It is also important to be careful that you are not too quick to reject ideas which, with modification, might meet the pass-fail criterion. For example: a very creative idea that does not meet your budget criterion ought nevertheless to pass. You may be able to determine a means of implementing the idea at lower cost.

If there are few ideas, the pass-fail evaluation is often not necessary. It is easier to move on to the more sophisticated evaluation matrix.

Evaluation Matrix

The evaluation matrix is a simple array in which experts compare an idea with a set of criteria. In my experience, five criteria is best as it allows for a rounded review without bogging the evaluators down in unnecessary detail. The evaluator ranks how well the idea meets each criterion (I recommend a scale of 0-5 points for each criterion). Evaluators are also encouraged to provide comments elaborating on their ratings and, in particular, suggesting how the idea might be improved to overcome weaknesses.

The evaluation matrix provides a criterion by criterion score as well as an overall score for each idea. Assuming several ideas, focusing on a particular problem or business issue, are being evaluated at the same time, these scores can be compared and the highest scoring ideas can be selected for further review. However, it is important to look at the evaluators' comments. An idea with a low score might be vastly improved following minor changes.

In my idea management days, I recommended (and included on the idea management software I developed, but which is no longer on the market) the evaluation matrix as the primary idea review approach because it is simple to set up, requires a minimum amount of time for idea review, enables comparative idea review and makes it easy to identify the most promising ideas within a large collection of ideas. That said, the evaluation matrix in itself is not usually sufficient for making a final decision on an idea that may cost millions of Euro to implement. But it helps you select ideas for more detailed review, thus making the review process more efficient.

Incidentally, there is a simple Idea Evaluation tool on the CreativeJeffrey web site. Give it a try!

SWOT Analysis

An analysis of Strengths, Weaknesses, Opportunities and Threats (SWOT) is an old marketing stand-by and as such is a useful follow up to an evaluation matrix. In the unlikely event you are unfamiliar with SWOT analysis, it is a simple form in which reviewers indicate the potential strengths, weaknesses, opportunities and threats of an idea. Because the SWOT analysis looks at an idea from different perspectives, it provides a more rounded review of an idea than some methods.

Be sure that reviewers performing the SWOT analysis do not just list weaknesses and threats, but make suggestions on overcoming them. Applying these suggestions can turn a weak idea into a strong one.

 

Criticism Versus Improvement

Over the years, I have noticed that business analysts tend to be overly critical of new ideas. This is understandable, they are tasked with managing and minimising risk. And creative ideas tend to be the riskiest. As a result, many evaluators stress weaknesses and threats. On one hand, this is understandable. Your company should not be implementing ideas that are likely to be costly failures. But, many weaknesses can readily be improved. An idea that would be very expensive to implement may, with minor changes, be implemented at far lower cost. And by improving a creative idea's weaknesses, you may be turning a costly failure into a profitable success!

Evaluation Teams

Evaluations should be performed by a team of people with relevant expertise. Ideally, that expertise should be varied. For instance, if you are evaluating new product ideas for an electronic gadget, your experts might include engineers, marketing people, retailers (who would sell the product) and one or more people representative of the consumers expected to buy the new products.

Evaluator Agendas and Prejudices

A particular benefit to having teams review ideas is that while individual evaluators are prejudiced, a varied team is likely to cancel such prejudices out.

For example: an engineer trained in an older technology may well be reluctant to give a high evaluation score to an idea that uses a new technology with which she is not familiar. The success of such an idea might well threaten her job! A jealous manager might not like the fact that her subordinates are more creative than her and so might give poor evaluation scores to creative ideas. At the other end of the spectrum, creativity and innovation people like you and I are often too enthusiastic about the most creative ideas and so give overly high scores for creativity. Sometimes, a less creative idea might prove to be the more innovative (in terms of being profitable).

 

Unusual, Playful Idea Review Methods

The following two idea review methods are less commonplace, but effective and particularly suitable for big, creative ideas that may not entirely suit your standard evaluation methods. Better still, they are good fun!

Role play

Create scenarios that involve your idea. For example, if you have an idea for a new service, create a scenario in which two people play act out the role of delivering the service and two others play the role of not-easily satisfied customers. Act out the scenario and discuss. What went well? What did not go so well? What surprised you?

If you have a prototype of a new product, even a primitive one, create role plays that involve using the prototype. In my experience in role play workshops, what seems like a good idea in concept is awkward in practice. Fortunately, one you identify potential issues, you can take action to correct them.

Another effective role play for a new product or service idea is to have two people play the role of the sales team and two others play the roles of customers. Ideally, your sales people would play the role of customers and non-sales people would play the role of sales people. This is because your sales people will be familiar with the kind of objections and preferences customers have during a sales pitch. Have each team prepare, then have the sales actors attempt to sell the product to the customer actors who have been instructed to raise as many objections as possible.

Again, discuss the results afterwards.

I have written about role plays in creativity in some depth here...

Idea Trials

An idea trial is a variation of a role play in which you subject an idea to trial by jury.

Divide your team into two groups: idea advocates who defend the idea and prosecutors who attempt to have the idea banished from your organisation. If possible, bring in an unbiased outsider, or a small team of outsiders, to act as the judges. A group of customers would be ideal, but three colleagues who are not in your team would also suffice.

Once the teams are prepared, create a courtroom where the advocates present the idea and argue in its defense while the prosecutors argue against the idea. Feel free to invite witnesses and experts to support arguments. At any time, a judge may also ask questions.

The judges should not be limited to accepting or rejecting the idea. They might also make recommendations to find a compromise, accept a part of the idea or to explore a related concept that might be more promising than the original idea.

While idea trials are more time and resource consuming than traditional evaluation methods, this approach guarantees an in-depth analysis of each idea in a dynamic setting.

 

Idea Development

Once an idea passes these initial hurdles, it may be ready for implementation or it may require more detailed testing. We call this phase of idea review: “idea development” as it is no longer a process of evaluating an idea so much as a method to develop it for implementation.

Idea development may include business case preparation, prototype development, project management initiation or test marketing. There is not room enough in this issue of Report 103 to go into idea development in detail – so we'll save it for another issue. Moreover, how a firm develops an idea depends on the nature of the idea, the nature of the firm and existing processes for implementing ideas.

 

Good Evaluation Methods Can Save Good Ideas

When ideas are approved or rejected based on the whim of a manager, the most creative ideas often lose out. They are perceived as being too different, too risky and requiring too much change. This is why you need to have good tools that evaluate ideas on their merit. Not only can they help you determine which innovative ideas have the most potential to succeed, but they can identify elements of the idea which may need improvements in order to increase the likelihood of success.

 

I Can Help: Facilitated Role Plays and Idea Trials

Managing role plays and idea trials is not as easy as it seems devising the right scenarios, briefing teams and preparation for these activities takes time and requires experience. If you are keen to try them out, get in touch. I would be delighted to help. Contact me and let's talk about it!

 

 

R103/201705117

 

---

Recent Articles

Leading Diverse Teams
Filed under: Business Innovation
Diverse teams are more innovative and smarter than homogeneous ones. But, they are also harder to manager. Here are some tips. By Jeffrey Baumgartner -- Read the article...

 

Questions you should ask when an innovative project fails
Filed under: Business Innovation
You can learn a lot from the failure of an innovative project, but you need to ask the right questions. Here are those questions. By Jeffrey Baumgartner -- Read the article...

 

Unmarketing the Competition
Filed under: Business Innovation
A look at creative, but unethical dirty trick marketing campaigns designed to damage the competition By Jeffrey Baumgartner -- Read the article...

 

Imaginativefulness and the Fisherman
Filed under: Creativity
What does a fisherman wearing a cycling helmet have to do with imaginativefulness? Quite a lot, it seems. By Jeffrey Baumgartner -- Read the article...

 

Actually, Criticising Ideas Is Good for Creativity
Filed under: Creativity
People have long assumed criticising ideas in a brainstorm inhibits creativity. Research and experience shows that is wrong By Jeffrey Baumgartner -- Read the article...

 

Imaginativefulness
Filed under: Creativity
Imaginativefulness is a state of heightened imagination in which your mind allows thoughts, memories and ideas to play with each other freely. By Jeffrey Baumgartner -- Read the article...

 

Why and How to Exploit Alternative Uses for Your Products
Filed under: Business Innovation
Discovering new ways customers use, misuse and could use your products can inspire innovation. Jeffrey Baumgartner explains. By Jeffrey Baumgartner -- Read the article...

 

The Cost of Not Innovating
Filed under: Business Innovation
If your company fails to innovate, you pay a steep price in terms of loss of leadershop, tight margins, missed opportunities and more. By Jeffrey Baumgartner -- Read the article...

 

Don't Trust the Status Quo
Filed under: Creativity
Jeffrey Baumgartner has never trusted the status quo. He explains why this is so and why you should also not trust the status quo By Jeffrey Baumgartner -- Read the article...

 

Index of all creative articles...