Sunday, June 29, 2008

The Three Primary Value Positionings For Idea Management Software Tools

Jeffrey Phillips has added more to his recent critique on idea gathering tools such as Dell's IdeaStorm and's SalesForce Ideas. In a recent article posted to the Innovation Tools website, Phillips argues that
  • the ideas generated need processing, and for Dell's IdeaStorm with over 9000 ideas submitted a brief cursory evaluation of 5 minutes per idea would take 450,000 minutes or 750 man-hours - a third of a year FTE. In addition, ideas from customers or partners may need legal review for IP ownership, adding to the time involved.
  • Social Networking or Crowdsourcing sites do not provide workflow processes for idea evaluation and selection beyond "a simple voting or rating process."
  • Social Networking and Crowdourcing approaches are claimed to generate less relevant ideas as:
. . . these programs primarily generate very incremental ideas, and since these approaches are very open, collaborative and web-based, they expose ideas to a large number of people. The larger the group, the more the thinking and ideas will revert to the mean. So you can't expect really insightful or disruptive ideas from this approach
I think that Phillip's key message is that an Idea Management system is more than a front end for capturing ideas, it is also a back end for efficiently and effectively processing and filtering those ideas and commercialising or investing in the best ones - for a significant commercial return on innovation investment.

I think that anyone involved with Idea Management Systems and indeed innovation generally will easily find consensus on this key message.

But I don't think that that one point alone is a damning critique of Dell's approach. Let's grant that assessing Dell's existing ideas costs 1/3 of a FTE resource? I suspect Dell could afford it. Indeed they could also afford additional extra resources to develop the criteria against which to assess ideas and to give relevant training to their staff involved in assessing the 9000 ideas and afford a team of reviewers and evaluators. The key issue here should not be the cost of evaluating ideas, but the net return on innovation investment after the most promising ideas have been selected and developed and commercialised.

More to the point from my point of view is that Dell, like many organisations, will already have systems in place for New Product Development. Large organisations will typically already have significant investment in software, systems and processes for New Product Development, process improvement, and input into strategic direction and performance improvement - systems which may have already been trialled 'proven' and committed to to a greater or lesser extent. In addition, organisations may have commitments to existing IT infrastructure such as portals and other communications technologies. The question arises therefore of where Idea Management Solutions 'sit' in relation to these existing systems and processes.

There seem to me to be 3 clear positionings for Idea Management Software solutions:
  1. An Idea Management System is expected to generate and capture ideas and feed them into existing systems and processes which will then filter and assess them.
    In this case the Idea Management tool itself is not expected to provide significant initial business evaluation and filtering of the ideas beyond peer review or a very rapid initial assessment (e.g. a go/no-go decision for promotion to the next stage or a selection from a list of defined options) by an assigned business representative
  2. An Idea Management System is expected to integrate with existing systems or processes by providing suitably qualified ideas to the appropriate entry point.
    In this case the Idea Management tool is expected to provide significant initial business evaluation and filtering of the ideas, but more extensive evaluation and development of opportunities is to be fulfilled by other organisational systems and processes.
  3. An Idea Management System is expected to support and facilitate a complete end-to-end stage-gate process from idea generation to idea commercialisation and value realisation
Depending on what the organisation wants, there are distinct value criteria to assess an Idea Management System against.
  • In case 1 above, the system must generate an appropriate volume of reasonable quality ideas.
  • In case 2 the system must not only generate ideas, it must put them through an appropriate process of qualification and evaluation to filter the best ideas to forward to the next stage in the system.
  • In case 3 the system must be capable of stewarding ideas through a complete stage-gate process from idea generation to commercial realisation.
The question of whether and how an Idea Management System integrates with existing processes for market testing, developing a business case, prototyping, project and portfolio management and other activities is an interesting one, and different vendors have positioned their software in different ways to address this question.

It seems to me that most Idea Management software tools are focused around cases 1 or 2, for the primary reason that organisations do typically already have significant investments in existing systems and processes for activities such as NPD, process improvement, strategy and performance management.

In the case of Dell, they appear to have clearly positioned their Idea Management tool in the first category. My reading of Dell's Idea Management process is that what Dell primarily wanted was a front end for generating ideas to feed a stream of ideas into their existing relevant organisational processes such as NPD.

While I agree with Phillips that an Idea Management tool can provide additional value by providing focus to generate more better quality and more relevant ideas and can further develop that value through efficient initial back-end processes for evaluating and filtering ideas, I don't think that focusing the tool on the front-end of idea capture is inherently a fault. The question is whether the back-end processes at Dell efficiently pick up the slack and process the ideas efficiently, whether enough appropriate evaluation and selection is incorporated into the front-end tool, and most importantly whether the system generates enough commercial value and return on innovation investment. These are, I expect, questions that Dell will eventually answer for themselves from their own experience.

An additional point to raise in relation to Phillips' critique is that we need to think carefully about what value an organisation is seeking when it engages employees, customers or partners for ideas. In addition to the commercial value of evaluated, selected and implemented ideas, there is value in engaging customers and employees with the company, in capturing market intelligence and trends, and in general developing a closer relationship with customers, employees and stakeholder groups. For example, the value of social networking tools for enhancing the relationship with customers is described in a recent MIT Sloane Management Review article through the concept of Virtual Customer Environments, summarised here and here. So the possibility exists that regardless of the success of social networking or crowdsourcing tools for generating commercial value as a front end for innovation, such tools may generate commercial value in themselves through enhancing the relationship with key stakeholder groups such as employees or customers.

Wednesday, June 25, 2008

e-week on's WebStorm 5.0

e-week recently reviewed's version 5.0 release of their WebStorm product.

Webstorm seems to be a front end Idea Capture system modelled on Corporate Social Networking principles that can be used either as a front end to's Idea Management System or independently as a collaboration and Idea Capture tool to harness customer or employee input and sentiment.

e-week write:

OVO Innovation CEO Discusses Idea Management at Dell and

OVO Innovation VP Sales and Marketing Jeffrey Phillips recently posted on Dell's and's initiatives in relation to Idea Management (Idea Storm and SalesForce Ideas respectively). Phillips argues that capturing ideas in a corporate "social networking" space is the easy part - back end management processes to evaluate test, develop and commercialise the best ideas is the hard part. Phillips writes:
". . . anyone who works in innovation will tell you that idea generation is easy - managing, evaluating and maturing ideas is the hard part. We think this is where the actual value in innovation resides - having a process and team that can consistently manage ideas and convert them into new products and services."
Phillips argues that open suggestion models such as enterprise social networking or crowdsourcing approaches:
". . . are interesting but will ultimately run into many of the same problems that doomed the physical suggestion box:
  1. Too many ideas are submitted for the teams to manage
  2. There is no downstream process for managing ideas successfully
  3. The ideas address too many different challenges and issues to manage effectively
  4. The ideas usually don't address issues the management team considers strategic
  5. There are concerns about the ownership and legality of the ideas"
Phillips, of course, makes a very solid point - effective back end processes to evaluate and filter ideas and invest in the best ones is fundamental to any effective Idea Management System. However, it is not so clear at this stage that the criticism applies forcefully to Dell. Dell seem to argue that they already have effective back-end management processes for assessing ideas, and IdeaStorm is simply a front end for feeding ideas into that system. Dawn Laccalade from Dell talks about Dell's backend management processes for Idea Management in IdeaStorm in a recent podcast (start about 1:50 in to the talk). Whether Dell's backend management processes can scale effectively to deal with the more than 9000 ideas generated is the key question for Dell's implementation.

One of the most high profile implementations of SalesForce Ideas for Idea Management is through Starbucks' my starbucks idea portal (see here and here for some brief information on Starbucks's solution). Dawn Laccalade (above) notes that when Starbucks implemented their system, they were quite surprised by the number of people they needed evaluating ideas (around 30 people FTE).

Imaginatik CEO interviewed by Podcast

On the topic of Imaginatik, CEO Mark Turrell was also interviewed earlier this year in a podcast. The 15 minute interview covers topics including a case example of Idea Management drawn from Imaginatik's experience using Idea Central, differences between online collaboration in the Enterprise and the personal social networking space, some tips for small companies to start capturing and assessing ideas and a discussion of cost constraints bearing on what size companies typically adopt full-strength Idea Management tools.

Imaginatik CEO Speaks on Innovation and Idea Management on YouTube

Imaginatik have been putting quite a bit of good content on Innovation and Idea Management up on YouTube.

This video was posted recently. In this video, Imaginatik CEO Mark Turrell talks about how he captures ideas personally from his daily life and argues that Idea Management Systems used by Dell and Starbucks are not as effective as they could be because they focus on the 'front end' of Idea Capture and do not develop the back end processes of assessing and filtering the ideas as strongly as other products in the Idea Management space.

Part II of the discussion can be found here. In Part II, some of the interesting points include the centrality of effective processes to effective organisation and emphasises the importance of software support for people implementing idea assessment and selection processes to realise a high return on investment

Monday, June 23, 2008

The Idea Hunter on Sustaining the Innovation Process

Andrew Greaves from the Idea Hunter blog recently posted an article on sustaining the innovation process.

Andrew proposes 18 relevant steps for sustaining innovation, which are also highly relevant also for organisation implementing an innovation initiative from scratch.

I won't reproduce Andrew's list here, as - due to the power of the internet - his list is only a click away. However I will comment there would be a few elements I would add to his list:

1. Strategic alignment

Align the innovation initiative with the organisational strategic objectives from the outset. What is the organisation committed to achieving? Ensure that the innovation agenda and infrastructure is aligned with this.

2. Use project management and change management methodologies to initiate and embed innovation structures, processes and culture

Implement the innovation initiative as related projects with appropriate change management to integrate an innovation mindset and behaviours into the organisation. Reinforce, sustain and reward the new culture and behaviours.

3. Tailor the innovation initiative to the specific organisation

Every organisation is different. The leadership is different, the culture is different, and across different industries or different competitive positioning the dynamics of an organisation and its relationship to the market and competitors may be different. While some of the structure of Idea Management may be generic in its general features, think carefully about your needs and tailor your innovation solution to meet your needs and your culture.

K-Community hosts a focus group on Idea Management

K-Community, a Knowledge Management professional group based in Bangalore, India recently hosted a focus group on Idea Management systems attended by representatives from organizations including Capgemini, TCS, Wipro, Honeywell, Robert Bosch, and Accenture.

The discussion was reported by Vinay Dabholkar, and some of the key points raised in the discussion were:
  • "All organizations use some tool or the other for idea management." Some of the organisations present used Idea Management tools from a vendor, others used Idea Management tools developed in-house. None of the participants expressed a sentiment that their tool was a limiting factor
  • Most organisations were putting an effective in putting appropriate supporting organisational structure around the Idea Management initiative
  • One of the challenges was identifying criteria for ideas that were strong enough to guide development of useful ideas but not so strong as to be limiting
  • A second challenge was that may ideas are by their nature cross functional, and the challenge is when to involve a cross functional group in the discussion in the ideation phase
  • A third challenge was obtaining the right mix of radical and incremental (or at least less radical) innovations in the portfolio mix
  • A fourth challenge was "sustaining the innovation engine" over time. If sustainability is perceived as an extra task then motivation for innovation by participants may vary over time.
  • The final challenges were integrating Knowledge Management with Idea Management and maintaining the "soul" of the innovation process while developing processes and structure around it.

McKinsey on Online Collaborative Technologies for New Product Development

Global Management Consultancy McKinsey recently published an article the next step in open innovation arguing that the next step in innovation is the use of online technologies to allow companies and outside groups such as customers or suppliers to come together and "co-create" new products and services.

McKinsey write:
If a company could use technology to link these outsiders into its development projects, could it come up with better ideas for new products and develop those ideas more quickly and cheaply than today?
McKinsey discuss a number of case examples, such as
LEGO, for instance, famously invited customers to suggest new models interactively and then financially rewarded the people whose ideas proved marketable . . . shirt retailer Threadless sells merchandise online - and now in a physical store . . . that is designed interactively with the company's customer base.
A number of challenges exist for successful co-creation:
  • Building communities and attracting participants
  • Breaking down complex problems into an appropriate form where members of the community can work on different parts and put them back together
  • Governance mechanisms
  • Maintaining or identifying quality
McKinsey conclude that "companies should experiment with this new approach to learn both how to use it successfully and more about its long term significance."

Wednesday, June 18, 2008

Brainstorming - Is It Worth The Effort, And Does Online Software Do It Better?

In Adrian Furnham's article "The Brainstorming Myth", Furnham reviewed literature on Brainstorming over the last 50 years. Furnham argued that:
  1. The experimental evidence for brainstorming does not support the notion that face to face brainstorming sessions are as effective as they are claimed to be
  2. Online brainstorming tools (such as those found in Idea Management Systems) facilitate more effective brainstorming sessions than face-to-face group brainstorming sessions
  3. Face-to-face brainstorming sessions can be improved for better outcomes
Furnham's theme is explored and elaborated at greater length in Mongeau and Morr's 1999 article "Reconsidering Brainstorming".

These articles make interesting reading.

A key piece of research cited by both Furnham and by Mongeau and Morr and underlying a stream of subsequent research is some (1958) research from Donald Taylor (et al).

Taylor's research team argued that it is inappropriate to compare the brainstorming output of a group of people with with that of an individual, as Alex Osborn had done in support of his brainstorming methodology. In order to determine if group synergies had added value, the brainstorming output of a group of X people should instead be compared with the aggregated brainstorming output from X individuals. Taylor termed the latter aggregation a "nominal group", and proceeded to compare the brainstorming performance of actual brainstorming sessions with that of nominal groups. The expectation was that if Alex Osborn's theories regarding Brainstorming were correct, the actual brainstorming group sessions should outperform the nominal group.

Taylor's team found that the converse was the case - it was found that nominal groups outperformed actual brainstorming groups.

As detailed in the above mentioned articles, Taylor's research has been supported by later research. This body of research has suggested that one key exception is when the group members are interacting or brainstorming together online through online brainstorming software (such as Idea Management Software).

But these results need to be understood in context. Did Taylor (and later researchers) really evaluate the "brainstorming" methodology as Osborne had introduced it? And are the findings in relation to the value of face-to-face brainstorming as damning as they might first appear?

Were the sessions run by skilled group facilitators?

It is pertinent to observe firstly that in Taylor's research, the participants in the study were 96 undergraduate "juniors and seniors" at Yale, and the facilitators for both the group and individual sessions were two "advanced graduate students in psychology." The total prior experience of the facilitators in brainstorming was obtained during "pre-testing" of the problems to be posed to the study groups by the research team, just prior to running the study. In addition, the facilitators were not focused on one discussion session, but apparently divided their time between two brainstorming groups simultaneously. The length of the brainstorming sessions was limited to only12 minutes, this length being chosen on the basis of being around the estimated time people could keep suggesting ideas without a significant pause.

The level of experience in group facilitation in general and brainstorming facilitation in particular during the study is important.
The level of facilitation skills demonstrated has a potential impact on possible "process loss" in the group. "Process loss" is the loss of productivity that can arise in group sessions due to factors such as only one person being able to speak at a time, bottlenecks in one person writing down/recording ideas, or any challenges faced by the facilitator in stewarding the group through to a positive outcome. In this context the level of facilitation skill in general, and familiarity with brainstorming skills in particular, is critically important. If one is going to assess the performance of a "nominal group" in relation to that of a brainstorming group, the level of experience of the group and in particular the level of facilitation skills is not unimportant and needs to be addressed in the study design.

Were the groups doing brainstorming as Osborn had defined it?

Secondly, in Taylor's "nominal group" research both the brainstorming groups and the individuals in the nominal groups were given the same training in "brainstorming" techniques.

This training consisted of a one hour class or lecture on brainstorming, a description of the research agenda, provision of a one-page article on brainstorming reproduced from Time, and reviewing four key "rules" of brainstorming, that
  • criticism is ruled out
  • free-wheeling is welcomed
  • quantity is wanted
  • combination and improvement is sought
Third, in Osborn's articulation of brainstorming, it is vitally important to provide a clear and simple statement of the problem to the group participants before they attend the session, in order to give them time to mull over the problem and start generating ideas. However, in Taylor's research, the problem statements were not sent out in advance but rather read out at the start of the session.

Fourth, Osborn suggests that "experience has . . . indicated that the ideal [brainstorming] panel should consist of a leader, an associate leader, about five regular or "core" members and about five guests." Taylor's study made no attempt to include an associate leader or experienced "core" members.

It is clear, therefore, that in Taylor et al's study, there are question marks regarding whether Osborn's brainstorming processes were followed, including whether the facilitators had sufficient training and experience to achieve substantial group synergies, whether participants had sufficient advance notice of the problems to be considered, or whether the structure of the brainstorming group reflected Osborn's ideal. If Osborn's brainstorming processes were not clearly followed, question marks arise over the validity of Taylor et al's research regarding the output of nominal groups in relation to brainstorming groups.

Did the success of nominal groups disprove or prove the value of brainstorming?

Putting aside whether or not Taylor had addressed Osborn's methodology fully and accurately, the notion of nominal groups merits further methodological consideration in itself.

Very early in the history of brainstorming, Alex Osborne's ideas were taken up by Sidney Parnes and others, who were concerned with the question of whether brainstorming techniques could benefit individuals. Parnes et al packaged brainstorming principles into an approach or method for individual creativity, which they called the Creative Problem Solving process. The Creative Problem Solving process was researched and supported by this group and a wider community of scholars, who sought to demonstrate the Creative Problems Solving techniques could be taught and that the technique improved the quantity and quality of individual creative output.

Creative Problem Solving was based on similar principles to the four points of brainstorming articulated by Taylor.

The fact that the individuals in
Taylor's "nominal groups" were given the same training in the principles of brainstorming as the brainstorming groups suggests that, in effect, individuals in nominal groups were being taught the creativity principles underlying brainstorming, in a form similar to that developed in early versions of Creative Problem Solving - and if the principles of Creative Problem Solving are valid, then individual's creative output should have been elevated to some extent as a consequence.

In other words, if both the actual brainstorming groups and individuals in the nominal groups were taught essentially similar creativity principles underlying brainstorming aimed at increasing the quantity and quality of creative ideas, it is unsurprising that both groups performed well at generating ideas - in fact the success of the individual participants in the nominal groups in generating ideas could arguably be viewed as a validation of the principles underlying brainstorming.

This is not a significant issue for Taylor's study design, as Taylor's objective was to try to isolate whether or not there were any synergistic benefits to group ideation in brainstorming compared to the same number of individuals doing individual brainstorming. Indeed,
as Taylor himself put it, his :
". . . experiment includes no evaluation of the basic rules of brain-storming - only an examination of the effects of group participation when using brainstorming. "
The fact that individuals also utilised creativity training and principles underlying brainstorming principles, is however, a pertinent point to bear in mind when interpreting Taylor's results.


Taking stock, the discussion to this point has highlighted:
  1. Both the individuals in the nominal groups and the brainstorming group sessions were given training in similar creativity techniques - Taylor's objective was to isolate if group dynamics make a positive difference when brainstorming
  2. Taylor found that group dynamics made a negative difference, but he was using as facilitators a couple of graduate students with no stated extensive training or history in either group facilitation or brainstorming techniques in particular leading a group of undergraduate students as participants. The study therefore leaves open the important question of whether issues of process loss in group facilitation loss were adequately addressed in the study design. If the facilitation process did not facilitate effectively, then it leaves open the possibility that the negative results for brainstorming sessions that Taylor identified were a direct result of a lack of facilitation skills or brainstorming expertise on the part of the facilitators
Mongeau and Morr's 1999 paper reinforced the above observations. Mongeau and Morr concluded that:
"There are two important conclusions that should be drawn from this review. First and foremost, considerable research clearly and consistently shows that untrained, ad-hoc, face-to-face brainstorming groups are inferior to nominal groups in the production of the quantity and quality of ideas. Their inferiority increases with group size. Furthermore, electronic brainstorming groups tend to produce more ideas than either face-to-face brainstorming groups or nominal groups (whether electronic or manual). The superiority of electronic brainstorming groups increases with group size.
Second, the research performed on both face-to-face and electronic brainstorming differs in important ways from what Osborn (1957) called suspended judgment and/or brainstorming. There are many important differences between Osborn's description of the spirit, structure, and functioning of brainstorming groups and the way in which groups were formed, trained, and expected to generate ideas. In short, although considerable research has been performed on brainstorming, little of this research is a valid test of Osborn's ideas."
In order to obtain the strongest and most convincing possible results Taylor - or subsequent researchers - would have needed to demonstrate that Osborn's recommendations for brainstorming were closely followed by experienced group brainstorming facilitators (who minimise process loss) who believe they are obtaining superior results (with demonstrated commercially valued outcomes) and that still even in this context the performance of the nominal group outperforms the brainstorming groups. It is not clear that this has been established, and therefore the research results need to be examined in context.
It is however interesting that the electronic brainstorming sessions are identified in the literature as producing much better outcomes (in terms of more ideas). Perhaps this could be attributed to less process loss during a session. It would be interesting to see more research about the quality of the ideas generated during electronic brainstorming sessions.

There is one other point to note. The above studies have focused on the idea-generation activity. Brainstorming however is a process that consists of idea-generation together with deferred evaluation - subsequent activity to evaluate the ideas and select the best ones for further use. It is not clear to what extent the studies mentioned above address idea evaluation as well as idea generation.

How Does Idea Evaluation and Selection Work in Idea Management Software?

I recently received an email asking me "what methods and processes are used in Idea Management software to filter, group and select the ideas?"

The answer is that different software applications may use different methods.

Some of the methods that may be included in given idea management software may include:
  • initial peer assessment of ideas (e.g. other employees or participants rate ideas on a scale of 1 to 5)
  • expert review of ideas (designated experts provide an expert opinion of the viability of the ideas)
  • an evaluator or team of evaluators assesses each idea or set of ideas to determine and select the strong ones
  • the participants use formal voting methods to determine the strongest ideas (e.g. each participant can select only what they see as the 3 strongest ideas from all the ideas submitted)
Typically some combination of the above is offered, with the client organisation being given the option to configure which methods are used either in general or specifically for each Idea Campaign.

All Idea Management software assists with the implementation of a 'Stage-Gate' process to progressively filter and qualify ideas and select the strongest ones for commercialisation or implementation. However, there is some variation within Idea Management software applications regarding the ability to define and implement custom Stage-Gate stages and checkpoints flexibly within the software. Some software focuses predominantly on the idea generation and capture stages of the Idea Management or Stage Gate process - and covers this activity extremely well - while other software allows organisations to extend the idea evaluation and selection process a little further by defining or providing stages for activities such as market research and testing, prototyping or technical feasibility testing, SWOT analysis, or developing and approving an initial business case.

Innovation Metrics - Which Ones Are Best To Use?

Paul Sloane recently posted on his BQF blog an article detailing the outcomes of a recent Global Business Partnership Alliance meeting where members reflected on which innovation metrics have been found to be most useful by members.

Some of the key points raised by the group were that:
  • Many of the common innovation metrics such as "% of revenue from products released in the last two years" are "lag" indicators after the event, and "lead" indicators are needed
  • It is useful to draw flow-chart diagrams of the innovation approval and pipeline processes and ask some searching questions about this - and relate metrics to the stages
The group suggested the following metrics may be particularly useful:

Input metrics:

  • Number of ideas generated
  • Resources allocated to innovation - people and budget

Process metrics:

  • Average time from idea approval to implementation
  • Number of ideas approved and number implemented
  • Stage-gate pass rates
  • Value of the innovation pipeline

Output metrics:

  • Number of new products or services launched
  • Revenue from new products or services
  • ROI on innovation spend
  • Market Perception
  • Number of new customers