Farris, J., Jack, H., “Towards a More Rigorous Approach to Concept Generation and Selection”, ASEE Annual Meeting, Vancouver, BC, June 2011.

Abstract

Free-form problem solving is an important part of undergraduate studies. Open-ended design is a particularly poignant activity for engineering students.

The Pugh concept selection matrix is widely taught for design concept selection. The power of this method is that students may rapidly select design criteria and compare concepts to identify the best options. During the application of the method the students must i) have a clear set of concepts, ii) have a clear set of evaluation criteria, iii) declare the relative importance of each criteria, iv) rate the concepts against the criteria, and v) then develop numerical scores/rankings for each design concept. When done objectively the results of this process can be very good, however when subjective bias is introduced the method falters. Typical procedural problems that students encounter are i) eliminate viable design concepts before using the matrix, ii) evaluation criteria are under- or over-stated, and iii) the relative weight is not appropriate. These problems arise from a number of factors including personal bias, a rush to finish, and misunderstanding. While it is sometimes useful for students to experience constructive failure, it can be avoided by adding a few steps to the process to reduce the arbitrary nature of the decisions.

The authors will describe enhancements for the method that guide students during the concept selection process. This will include i) the identification of constraints as opposed to objectives, ii) recognition of the concept risks , iii) methods to reduce concept risks, iv) the use of estimates and calculations for design objectives, and iv) guided criteria for selection criteria weights. Examples of applications projects will be used to illustrate the method.

Introduction

In the typical engineering design text books, concept selection is taught after concept generation but before detailed design 1, 2, 3. Most texts present a variation of the Pugh Concept Selection Process. However, the Pugh concept selection method has many drawbacks that are aggravated when inexperienced designers use the process. Inexperienced designers tend to use the method to justify the selection of a low risk solution without the benefit of a technical analysis or even an estimate of the capabilities of the alternatives.

Inexperienced designers tend to select familiar concepts without properly considering alternative designs that may meet the customer requirements better. Premature selection of a concept often precludes the selection the best alternative. Therefore an ideal selection method must encourage students to consider a wide variety of technologies and methods for solving the problem. An ideal selection system would prompt students to use their engineering analysis skills to create math models to predict the performance of the different alternatives. Then the results of their analysis could be used to justify the selection of the “best” alternative. In addition the ideal method must help students to define the key advantages and disadvantages of each approach with respect to the specifications for the design. The advantages and disadvantages must also be considered with respect to the importance the customer places on each specification. In real world design problems, customers and users place a higher value on some product attributes.

Pugh Selection Method

The Pugh concept selection method was developed by Stuart Pugh in 1990 4. The method is designed to be used by design teams to develop a consensus on the optimal conceptual design given a set of competing or alternative conceptual design solutions. Since the method is easy to document, an alternative was selected that can be critically reviewed by management and peers not on the design team. The steps in the Pugh concept selection are shown below.

1) Form consensus on criteria

2) Form consensus on alternatives conceptual designs

3) Select a baseline alternative

4) Compare the alternatives

5) “Attack the negatives”

6) Assess and repeat

The criteria are used to compare the different concepts and are often generated from the list of specifications for the project or the interpreted needs contained in the house of quality 5. Although a design team may be able to generate many potential criteria, experienced practitioners try to identify the critical criterion that will highlight the important differences between the alternatives concepts. Next, the design team must reach a consensus on the conceptual designs to consider. For this method, a conceptual design is a complete solution to the problem. The visual representations of each concept must be at the same level of refinement. If one concept is represented by a ‘sketch on a napkin’ and another concept is represented by solid models from a CAD system, an unbiased comparison is difficult to achieve. A simple chart, like the one shown in table 1, is used to document the comparisons.

Table 1. A Typical Pugh Selection Chart.

One concept, usually the alternative considered the best by the team, is designated as the baseline concept. Then each alternative is compared to the baseline relative to each criterion. A “+” indicates that the team believes that the concept is superior to the baseline at fulfilling the criteria. A “-” indicates that the team believes that the concept is markedly worse at fulfilling the criteria compared to the baseline and an empty cell indicates that the baseline concept and the concept under consideration are roughly equally capable of meeting the criteria.

After the team has filled out the chart, they attempt to “Attack the Negatives” of each concept. In other words the team tries to improve each concept to eliminate the negatives assigned to each concept. Often features are borrowed from other concepts. In addition the process of attacking the negatives may lead to the generation of new concepts. Any new or modified concepts are documented and the process is repeated until the team reaches a consensus on the best concept.

Variations of this method include replacing the “+” and “-” scoring with a numerical score. In this variation the concept with the highest total numerical score is considered the optimal concept and brought forward. Some practitioners assign weights to each criterion and multiply the weights by the each score before summing the scores for each concept.

Inexperienced designers often assign the ratings without a rigorous justification. The creation of the justification requires the use of engineering modeling and estimating skills that students find very difficult to apply to concrete problems. Thus, the concept selection method taught often undermines the attempts to show students how theory that they have labored to learn can be applied to design problems.

Proposed Method

The proposed method for concept generation and selection consists of the following steps:

1. Develop a clear set of specifications for the design problem.

2. Generate ideas to satisfy the specifications.

3. Map the ideas onto the addressed specifications.

4. Classify each idea as either a macro idea or a micro idea.

4. Create a decision precedence chart.

6. Check feasibility of each macro idea.

7. Generate set of feasible concepts from feasible macro ideas.

8. Create quantifiable criteria from the specifications.

9. Use the decision matrix method to rank concepts.

The proposed method assumes that a complete set of specifications for the design have been generated and reviewed and a large number of ideas have been generated. Detailed specifications are defined at the beginning of any project to provide direction for the work and determine when the project is complete. In this context, ideas are different from concepts. Concepts are a complete solution to the design problem and ideas are partial solutions to the problem.

Idea-Specification Mapping

The next step requires the team to identify the specifications that each idea addresses. As a designer generates concepts for a design it will often result in a collection of notes and sketches. A matrix, like the one shown in table 2, succinctly shows the relationship between ideas and the specifications. It is expected that no concept would be able to satisfy all of the specifications, but the some ideas will satisfy or support more of the specifications. To impose order all ideas should be classified as either a micro or a macro idea. A macro idea will change the overall structure of the design and apply to many specifications while the micro ideas will solve more focused design issues and apply to fewer specifications. For example, macro ideas for a student team designing a human powered transport may be a two wheel conventional bike arrangement and a three wheel tadpole recumbent bike configuration. The choice of configuration will influence many specifications. However the design of the handle bars may influence only one specification. Therefore the two wheel conventional configuration may be represented by idea one in the table two and the idea for the handle bars may be represented by idea two in table two. The mapping of specification to ideas is used to distinguish between macro ideas and micro ideas.

Table 2. Mapping Specifications to Ideas.

Once the matrix is complete it can be critically reviewed using criteria such as those below.

i) Each specification should be addressed by multiple concepts. If any specification only has a single concept a critical justification is required.

ii) For each specification find the conceptual alternatives. These will probably require a decision matrix.

iii) Capture the alternatives as a decision matrix. This will include a few of the specifications, and the concepts related to these.

If the pool of ideas is rich with alternatives, the designer can move forward confidently to the next step. If there are no viable alternatives the process can be stopped, more ideas can be generated or the specifications can be revised.

Decision Precedence Chart

The ideas can also be assembled into a hierarchy that shows the precedence of choices required to form a concept. A concept is a complete solution to the design problem. A sample decision precedence chart is shown in figure 1. Once a macro idea is chosen at level one then the set decisions must be made on a subset of other macro and micro decisions that are associated with the first choice. For instance one concept may consist of ideas 1, 3, 6 and 9. Another concept may be formed with the ideas 2, 8, 11 and 12.

Figure 1. A Decision Precedence Chart.

Inexperienced designers can learn much about the design problem by creating and studying a decision precedence chart. By creating the chart, the designer identifies the decisions that are driving the design. In the example chart, the choice between idea 1 and idea 2 must be made before any other choices can be made. In this way the decision precedence chart reveals the structure of the design problem. The chart can also be analyzed for completeness. In the example chart idea 5 seems to be a dead end. The designer or a reviewer might ask some of the following questions based on the example chart:

i) Can a concept based on ideas 2 and 5 fulfill all of the specifications?

ii) Why are no other ideas compatible with 5?

iii) Are there alternatives to ideas 1 and 2 that should be considered?

Checking Feasibility

The chart can also be used to guide the work of checking feasibility. If idea 1 in figure 1 is not feasible then there is no sense in checking any of the ideas in Levels 2, 3 and 4 that are only compatible with idea 1.

Obviously each of the macro designs will need to be feasible, and where there is doubt, micro concepts and/or more research is needed. In a strategic sense the macro concepts organize the micro concepts and will be the most important decisions. Analysis of the micro and macro concepts is an iterative process that is driven by the questions

i) Is the macro concept feasible?

ii) Can it be implemented with known approaches?

iii) Do suitable micro ideas exist to satisfy all of the specifications?

iv) Where are the holes and unknowns that should be addressed immediately?

The designer can rule the macro concept valid or invalid and move forward or return to step I to gather more information. This process is repeated for each of the macro concepts until there are a good set of viable ideas.

Generate set of feasible concepts

Next the macro and micro ideas are assembled into concepts that address all specifications. These concepts will be compared and ranked in the next step. An alternative method is to use the decision precedence chart to optimize the choices made at each level. To apply this approach to the design problem represented in figure 1, the designer would first make a choice between macro ideas 1 and 2 at level one. Depending on the outcome, the designer would next move to choose the best idea in level two that is compatible with the previous choice. For instance if idea 2 is judged superior to idea 1 at level one then the choice at level 2 requires the designer to choose between ideas 5 and 8. Although this approach is quicker than generating complete concepts and comparing the concepts, there is a possibility that the best design will be overlooked because the component decisions were optimized not the entire design.

Create quantifiable criteria from the specifications

A set of evaluation criteria are created from the specifications. Scores for each of the evaluation criteria are then selected using standard analysis techniques. Ideally the scoring technique is quantitative to preserve objectivity.

Use the Decision Matrix Method to Rank Concepts

If the value of all design features are similar we can use the simple +/- system. However if the relative importance of the factors are not the same a weighted decision matrix is a well known technique to develop combined scores. The steps to use the technique are listed below. Essentially the approach is used when selecting between two or more equivalent concepts, for example two or more macro concepts. A set of evaluation criteria are selected, hopefully using the specifications. Scores for each of the evaluation criteria are then selected using standard analysis techniques, hopefully quantitative. The weights and the scores are then multiplied for each of the criteria, and then summed for the design. The result is an overall score for the design. These can then be used to rank the choices from best to worst.

1. List the conceptual design decisions as columns.

2. List the criteria as rows.

3. A weight is given to each criterion.

4. A score is given to the concept for each criterion. The ranking is done relative to one of the design concepts, with the middle of the scale being the first concept. A scale of -3 to +3 is reasonable.

5. Using the criteria weights, the column values are multiplied and added to get a score for the design.

6. The design with the highest score is often judged the best candidate for detailed design (although other designs may be chosen).

Figure 3 Weighted Decision Matrix

The weaknesses of this technique are;

i) all of the parameter weights assume a linear relationship,

ii) the weights and scores are easy to misjudge or manipulate, and

iii) it permits the inclusion of unreasonable designs in comparison.

Nonlinear Criteria Scoring

Consider a design factor, such as cost. In a simple analysis it is multiplied by a weighting factor to determine overall utility, or value to the customer. This does simplify the process, but when factors are dramatically different, such as cost these linear weights become ineffective. When the design features are more varied it may be necessary to consider a non-linear scale. This approach, or at least the concept, is valuable for students doing more advanced design work. A formal basis for this approach can be found in systems optimization methods.

Consider three design concepts for a car with upper speeds of 80 kph, 100 kph, or 140kph. If dealing with an urban commuting vehicle that has a maximum speed specification of 60kph, all of these are acceptable and should receive the same maximum score. If the car is to be used on surface streets and highways the 80kph would be unacceptable and should receive a score of 0, 100kph is marginally acceptable and should receive a poor or mediocre score, while the 140kph speed should receive the highest score. And of course other scoring is possible.

Figure 3 shows a few alternatives for design scores. The figure at the bottom right shows a simple non-linear weight where a cost below $40 is considered good, and anything less is still good. But, when the cost rises above $40 the value drops off quickly, and above $60 there is little or no value to the design. If the design in the example cost $60 then there would need to be other very compelling reasons to select the concept.

Figure 3 Decision matrix value examples (assume bad is 0, good is 1)

Conclusion

The proposed method has been is designed to reduce the prevalence of the following issues.

Students present invalid solutions to justify a preferred design.

Designs with well reasoned upper level design concepts, but without viable components or sub-systems.

The next steps include applying the method to past design projects to determine the effectiveness of the method and then devising an assessment strategy for the proposed method.

References

1. Eggert, Rudolph, J. Engineering Design. Upper Saddle, NJ, Pearson Prentice Hall: 2005.

2. Ulich Karl, T. and Eppinger, Steven, D. Product Design and Development. Boston: McGraw Irwin, 2004.

3. Ullman, David, G. The Mechanical Design Process. Boston: McGraw Hill, 2003.

4. Pugh, Stuart. Total Design, Integrated Methods for Successful Product Engineering. Wokingham, England: Addison-Welsley Publishing Company, 1991.

4. Pugh, Stuart. Total Design, Integrated Methods for Successful Product Design. Reading Massachusetts, Addison-Wesley Publishing Company, 1990, 70-85.

5. Hauser, John R., and Don Clausing, The House of Quality, Harvard Business Review, May-June 1988.