Wednesday, June 18, 2008

Brainstorming - Is It Worth The Effort, And Does Online Software Do It Better?

In Adrian Furnham's article "The Brainstorming Myth", Furnham reviewed literature on Brainstorming over the last 50 years. Furnham argued that:
  1. The experimental evidence for brainstorming does not support the notion that face to face brainstorming sessions are as effective as they are claimed to be
  2. Online brainstorming tools (such as those found in Idea Management Systems) facilitate more effective brainstorming sessions than face-to-face group brainstorming sessions
  3. Face-to-face brainstorming sessions can be improved for better outcomes
Furnham's theme is explored and elaborated at greater length in Mongeau and Morr's 1999 article "Reconsidering Brainstorming".

These articles make interesting reading.

A key piece of research cited by both Furnham and by Mongeau and Morr and underlying a stream of subsequent research is some (1958) research from Donald Taylor (et al).

Taylor's research team argued that it is inappropriate to compare the brainstorming output of a group of people with with that of an individual, as Alex Osborn had done in support of his brainstorming methodology. In order to determine if group synergies had added value, the brainstorming output of a group of X people should instead be compared with the aggregated brainstorming output from X individuals. Taylor termed the latter aggregation a "nominal group", and proceeded to compare the brainstorming performance of actual brainstorming sessions with that of nominal groups. The expectation was that if Alex Osborn's theories regarding Brainstorming were correct, the actual brainstorming group sessions should outperform the nominal group.

Taylor's team found that the converse was the case - it was found that nominal groups outperformed actual brainstorming groups.

As detailed in the above mentioned articles, Taylor's research has been supported by later research. This body of research has suggested that one key exception is when the group members are interacting or brainstorming together online through online brainstorming software (such as Idea Management Software).

But these results need to be understood in context. Did Taylor (and later researchers) really evaluate the "brainstorming" methodology as Osborne had introduced it? And are the findings in relation to the value of face-to-face brainstorming as damning as they might first appear?

Were the sessions run by skilled group facilitators?

It is pertinent to observe firstly that in Taylor's research, the participants in the study were 96 undergraduate "juniors and seniors" at Yale, and the facilitators for both the group and individual sessions were two "advanced graduate students in psychology." The total prior experience of the facilitators in brainstorming was obtained during "pre-testing" of the problems to be posed to the study groups by the research team, just prior to running the study. In addition, the facilitators were not focused on one discussion session, but apparently divided their time between two brainstorming groups simultaneously. The length of the brainstorming sessions was limited to only12 minutes, this length being chosen on the basis of being around the estimated time people could keep suggesting ideas without a significant pause.

The level of experience in group facilitation in general and brainstorming facilitation in particular during the study is important.
The level of facilitation skills demonstrated has a potential impact on possible "process loss" in the group. "Process loss" is the loss of productivity that can arise in group sessions due to factors such as only one person being able to speak at a time, bottlenecks in one person writing down/recording ideas, or any challenges faced by the facilitator in stewarding the group through to a positive outcome. In this context the level of facilitation skill in general, and familiarity with brainstorming skills in particular, is critically important. If one is going to assess the performance of a "nominal group" in relation to that of a brainstorming group, the level of experience of the group and in particular the level of facilitation skills is not unimportant and needs to be addressed in the study design.

Were the groups doing brainstorming as Osborn had defined it?

Secondly, in Taylor's "nominal group" research both the brainstorming groups and the individuals in the nominal groups were given the same training in "brainstorming" techniques.

This training consisted of a one hour class or lecture on brainstorming, a description of the research agenda, provision of a one-page article on brainstorming reproduced from Time, and reviewing four key "rules" of brainstorming, that
  • criticism is ruled out
  • free-wheeling is welcomed
  • quantity is wanted
  • combination and improvement is sought
Third, in Osborn's articulation of brainstorming, it is vitally important to provide a clear and simple statement of the problem to the group participants before they attend the session, in order to give them time to mull over the problem and start generating ideas. However, in Taylor's research, the problem statements were not sent out in advance but rather read out at the start of the session.

Fourth, Osborn suggests that "experience has . . . indicated that the ideal [brainstorming] panel should consist of a leader, an associate leader, about five regular or "core" members and about five guests." Taylor's study made no attempt to include an associate leader or experienced "core" members.

It is clear, therefore, that in Taylor et al's study, there are question marks regarding whether Osborn's brainstorming processes were followed, including whether the facilitators had sufficient training and experience to achieve substantial group synergies, whether participants had sufficient advance notice of the problems to be considered, or whether the structure of the brainstorming group reflected Osborn's ideal. If Osborn's brainstorming processes were not clearly followed, question marks arise over the validity of Taylor et al's research regarding the output of nominal groups in relation to brainstorming groups.

Did the success of nominal groups disprove or prove the value of brainstorming?

Putting aside whether or not Taylor had addressed Osborn's methodology fully and accurately, the notion of nominal groups merits further methodological consideration in itself.

Very early in the history of brainstorming, Alex Osborne's ideas were taken up by Sidney Parnes and others, who were concerned with the question of whether brainstorming techniques could benefit individuals. Parnes et al packaged brainstorming principles into an approach or method for individual creativity, which they called the Creative Problem Solving process. The Creative Problem Solving process was researched and supported by this group and a wider community of scholars, who sought to demonstrate the Creative Problems Solving techniques could be taught and that the technique improved the quantity and quality of individual creative output.

Creative Problem Solving was based on similar principles to the four points of brainstorming articulated by Taylor.

The fact that the individuals in
Taylor's "nominal groups" were given the same training in the principles of brainstorming as the brainstorming groups suggests that, in effect, individuals in nominal groups were being taught the creativity principles underlying brainstorming, in a form similar to that developed in early versions of Creative Problem Solving - and if the principles of Creative Problem Solving are valid, then individual's creative output should have been elevated to some extent as a consequence.

In other words, if both the actual brainstorming groups and individuals in the nominal groups were taught essentially similar creativity principles underlying brainstorming aimed at increasing the quantity and quality of creative ideas, it is unsurprising that both groups performed well at generating ideas - in fact the success of the individual participants in the nominal groups in generating ideas could arguably be viewed as a validation of the principles underlying brainstorming.

This is not a significant issue for Taylor's study design, as Taylor's objective was to try to isolate whether or not there were any synergistic benefits to group ideation in brainstorming compared to the same number of individuals doing individual brainstorming. Indeed,
as Taylor himself put it, his :
". . . experiment includes no evaluation of the basic rules of brain-storming - only an examination of the effects of group participation when using brainstorming. "
The fact that individuals also utilised creativity training and principles underlying brainstorming principles, is however, a pertinent point to bear in mind when interpreting Taylor's results.


Taking stock, the discussion to this point has highlighted:
  1. Both the individuals in the nominal groups and the brainstorming group sessions were given training in similar creativity techniques - Taylor's objective was to isolate if group dynamics make a positive difference when brainstorming
  2. Taylor found that group dynamics made a negative difference, but he was using as facilitators a couple of graduate students with no stated extensive training or history in either group facilitation or brainstorming techniques in particular leading a group of undergraduate students as participants. The study therefore leaves open the important question of whether issues of process loss in group facilitation loss were adequately addressed in the study design. If the facilitation process did not facilitate effectively, then it leaves open the possibility that the negative results for brainstorming sessions that Taylor identified were a direct result of a lack of facilitation skills or brainstorming expertise on the part of the facilitators
Mongeau and Morr's 1999 paper reinforced the above observations. Mongeau and Morr concluded that:
"There are two important conclusions that should be drawn from this review. First and foremost, considerable research clearly and consistently shows that untrained, ad-hoc, face-to-face brainstorming groups are inferior to nominal groups in the production of the quantity and quality of ideas. Their inferiority increases with group size. Furthermore, electronic brainstorming groups tend to produce more ideas than either face-to-face brainstorming groups or nominal groups (whether electronic or manual). The superiority of electronic brainstorming groups increases with group size.
Second, the research performed on both face-to-face and electronic brainstorming differs in important ways from what Osborn (1957) called suspended judgment and/or brainstorming. There are many important differences between Osborn's description of the spirit, structure, and functioning of brainstorming groups and the way in which groups were formed, trained, and expected to generate ideas. In short, although considerable research has been performed on brainstorming, little of this research is a valid test of Osborn's ideas."
In order to obtain the strongest and most convincing possible results Taylor - or subsequent researchers - would have needed to demonstrate that Osborn's recommendations for brainstorming were closely followed by experienced group brainstorming facilitators (who minimise process loss) who believe they are obtaining superior results (with demonstrated commercially valued outcomes) and that still even in this context the performance of the nominal group outperforms the brainstorming groups. It is not clear that this has been established, and therefore the research results need to be examined in context.
It is however interesting that the electronic brainstorming sessions are identified in the literature as producing much better outcomes (in terms of more ideas). Perhaps this could be attributed to less process loss during a session. It would be interesting to see more research about the quality of the ideas generated during electronic brainstorming sessions.

There is one other point to note. The above studies have focused on the idea-generation activity. Brainstorming however is a process that consists of idea-generation together with deferred evaluation - subsequent activity to evaluate the ideas and select the best ones for further use. It is not clear to what extent the studies mentioned above address idea evaluation as well as idea generation.

No comments: