Because an experiment involves examining the relationship between independent variables and changes in one or more dependent variables, defining what is measured—dependent variables—is crucial. The dependent variables are what can be measured and relate to the outcomes described in the research questions.
The research questions are often stated in terms of theoretical constructs, where constructs describe abstract entities that cannot be measured directly. Common constructs in human factors studies include: workload, situation awareness, fatigue, safety, acceptance, trust, and comfort. These constructs cannot be measured directly and the human factors researcher must select variables that can be measured, such as subjective ratings and performance data that are strongly related to these constructs. To assess how smartphones affect driving, the underlying construct might be safety and the measure that relates to safety might be error in lane keeping where the car’s tire crosses a lane boundary. Safety might also be measured by ratings from the drivers indicating how safe they felt. Subjective ratings are often contrasted with objective performance data, such as error rates or response times. The difference between these two classes of measures is important, given that subjective measures are often easier and less expensive to obtain, with a larger sample size. Both objective and subjective measures are useful. For example, in a study of factors that lead to stress disorders in soldiers, objective and subjective indicators of event
stressfulness and social support were predictive of combat stress reaction and later posttraumatic stress disorder. The subjective measure was a stronger predictor than the objective measure [68].
In considering subjective measures, however, what people rate as “preferred” is not always the system feature that supports best performance [69]. For example, people almost always prefer a color display to a monochrome one, even when color undermines performance.
Furthermore, people cannot always predict how they would respond to surprising events in different conditions, like during system failures. Human factors is much more than intuitive judgment (of either the designer OR the participant). It is for this reason that objective data from controlled experiments are needed to go beyond the expert judgments in heuristic evaluations and subjective data.
Subjective and objective dependent variables provide important and complementary information. We often want to measure how causal variables affect several dependent variables at once.
For example, we might want to measure how use of a smartphone affects a number of driving performance variables, including deviations from the lane, reaction time to cars or other objects in front of the vehicle, time to recognize objects in the driver’s peripheral vision, speed, acceleration, and so forth. Using several dependent variables helps triangulate on the truth—if all the variables indicate the same outcome then one can have much greater confidence in that outcome.

P3.28 For an evaluation of a vehicle entertainment system, identify possible dependent variables.

P3.29 What are the benefits of subjective measures?

P3.30 What are the limitations of subjective measures?

P3.31 What is the relationship between a construct and a measure?

be familiar with the range of methods that are available and know which methods are best for specific types of design questions. It is equally important for researchers to understand how practitioners ultimately use their findings. Ideally, this enables a human factors specialist to work in a way that will be useful to design, thus making the results applicable. Selecting an evaluation method that will provide useful information requires that the method be matched to its intended purpose.

3.1 Purpose of Evaluation In Chapter 2 we saw how human factors design occurs in the understand-create-evaluate cycle. Chapter 2 focused on understanding peoples’ needs and characteristics and using that understanding to create prototypes that are refined into the final system through iteration. Central to this iterative process is evaluation. Evaluation identifies opportunities to improve a design so that it serves the needs of people more effectively. Evaluation is both the final step in assessing a design and the first step of the next iteration of the design, where it provides a deeper understanding of what people need and want. Evaluation methods that serve as the first step of the next iteration of the design are termed formative evaluations. Formative evaluations help understand how people use a system and how the system might be improved. Consequently, formative evaluations tend to rely on qualitative measures—general aspects of the interaction that need improvement. Evaluation methods that serve as the final step in assessing a design are termed summative evaluations. Summative evaluations are used to assess whether the system performance meets design requirements and benchmarks. Consequently, summative evaluations tend to rely on quantitative measures—numeric indicators of performance. The distinctions between summative and formative evaluations can be described in terms of three main purposes of evaluation: • Understand how to improve (Formative evaluation): Does the existing product address the real needs of people? Is it used as expected? • Diagnose problems with prototypes (Formative evaluation): How can it be improved? Why did it fail? Why isn’t it good enough? • Verify (Summative evaluation): Does the expected performance meet design requirements? Which system is better? How good is it? Each of these questions might be asked in terms of safety, performance, and satisfaction. For Erika’s analysis, predicting the effect of smartphones on driving safety is most important: how dangerous is talking on a phone while driving?

Table 3.1

Table 3.1 shows the example evaluation techniques for three evaluation purposes. The first rows of this table show methods for understanding and diagnosing problems with qualitative data. Qualitative data are not numerical and include responses to openended questions, such as “what features on the device would you like to see?” or “what were the main problems in operating the device?” Qualitative data also include observations and interviews. These data are particularly useful for diagnosing problems and identifying opportunities for improvement. These opportunities for improvement make qualitative data particularly important in the iterative design process, where the results of a usability test might guide the next iteration of the design. The third row of the table shows methods associated with verifying the performance of the system with quantitative data. Quantitative data include measures of response time, frequency of use, as well as subjective assessments of workload. Quantitative data include any data that can be represented numerically. The table shows that quantitative data are essential for assessing whether a system has met its objectives and if it is ready to be deployed. Quantitative data offer a numeric prediction of whether a system will succeed. In evaluating whether there should be a ban of smartphones, quantitative data might include a prediction of the number of lives saved if a ban were to be adopted. The last two rows show how both quantitative and qualitative data can support understanding people’s needs and characteristics relative to the design. Although methods for understanding (Chapter 2) and methods for evaluation (Chapter 3) are presented in separate chapters, there is substantial overlap between them. In this chapter, we focus on diagnosing design problems and verifying its performance, but evaluations often produce data that can also enhance understanding and guide future designs. Beyond evaluating specific systems or products, human factors specialists also evaluate more general design concepts and develop design principles. Such concept evaluations include assessing the relative strengths of keyboard versus mouse or touchscreen or rotating versus fixed maps. Concept evaluation reflects the basic science that supports the design principles and heuristics that make it possible to guide design without conducting a study for every design decision.

P3.1 How is evaluation related to understanding in the human factors design cycle?

P3.2 What are the three general purposes of evaluation?

P3.3 Would qualitative or quantitative data be more useful in diagnosing why a design is not performing as expected?

P3.4 Would qualitative or quantitative data be more useful in assessing whether a design meets safety and performance requirements?

P3.5 What is the role of quantitative and qualitative data in system design?

P3.6 Why is qualitative data an important part of usability testing?

P3.7 Give examples of qualitative data in evaluating a vehicle entertainment system.

P3.8 Give examples of quantitative data in evaluating a vehicle entertainment system.

P3.9 Describe the role of formative and summative evaluations in design.

P3.10 Identify a method suited to formative evaluation and another more suited to summative evaluation

Three recent interventions are presented briefly
to contrast the project and strategic approaches. First it
should be noted that all three companies and projects
had a high degree of similarity:

1

All were threatening closure of the plant
due to competitive pressures

2

All were in the process of talking about
change in manufacturing, but none had
proceeded as far down this path.

3

In all plants, the active cooperation of top
management and union leadership was a
prerequisite of the project.

4

All were plants in mature industries, i.e.
where technological changes in the product
would be less important to the company
survival than manufacturing prowess.

5

All were large plants, with several hundred
operators

In company. A, an ergonomics program was
started as a project. The aim was to teach operators,
foremen and technical staff how to do ergonomics, both
by classroom training and by undertaking a series of
demonstration projects throughout the plant. Each
project was to be assessed by before-and-after measures
to demonstrate how ergonomics increases both system
performance and operator well being. Over a two-year
period, the project was technically successful in that it
did train many people to become users of task analytic
and job redesign techniques. Success could also be
measured by workplace changes successfully
implemented. However, the project’s non-strategic
aspects were constantly in evidence: operators could not always get released for team meetings, promised
changes were rarely completed on time, direct
intervention by the plant manager was often needed to
insure implementation. To date the ergonomics
program has had minimal impact on the plant's goal of
achieving company-wide top status in quality before the
announced deadline of 199 1.

Company B was tackled in a more strategic
manner. First a team of university personnel worked
with the company to establish strategic needs of the
business. This assessment concerned general
management, strategic planning, sales 8z marketing,
financials, labor relations and manufacturing. Human
factors was a small part of the "manufacturing" area.
From this assessment came a series of immediate needs,
represented by projects which, if completed
successfully, would have a major impact of the business.
One of these projects was operator training, using
human factors techniques of knowledge elicitation to
form the basis of a knowledge and skill training
program. The first part of this system has now been
implemented. Company B cites the University's
intervention as a major reason for deciding to stay in the
region, and to locate its new manufacturing facility here

The case of Company C was in many ways an
intermediate case between the two levels. The company
manufactured precision aircraft parts, using small batch
production by skilled machinists. The brief was to
improve manufacturing quality, again using human
factors techniques of process control analysis.
However, the team included specialists to work with
manufacturing management and labor unions at a high
level as well as at shop-floor level to bring some order
to the a11 too typical chaos of high scrap rates, missed
deadlines and the end-of-the-month shipping crisis.
Ergonomics intervention was centered around a
manufacturing cell, and included a complete system for
floor-level process control based on operator input and
human factors techniques. The presence of this working
cell was a major factor in the decision of another
company to but the plant, expand it, and make it their world headquarters for aviation components. Here the
intervention achieved strategic-level results despite a
lack of prior strategic analysis, compens2ted for to some
extent by the non-ergonomic interventions

CONCLUSIONS

In only one company (A) was the intervention
labeled as "ergonomics" and that had the least impact
upon the company's future. In the other two, it is
doubtful whether either management would classify our
interventions as "ergonomics" or "human factors"
despite the role of these disciplines in helping the
company achieve its strategic goals

Case studies in small numbers can never provide
statistical proof of the superiority of one approach over
another, but they can indicate that manufacturing
success may be achieved by allowing our discipline to
be part of a relatively homogeneous team. We may
have to change the way in which we operate, and lose
some of our separate identity, if we truly wish to impact
manufacturing

Abstract

Now that manufacturing has become a respectable topic in industry, an obvious question is how
human factors/ergonomics can contribute to the improvement of manufacturing. The traditional route
for ergonomics intervention has been a Project route, with a set of objectives agreed between the human
factors engineer and people within the company. Projects, however, do not ask the question of whether
human factors intervention is likely to have an impact on the company's strategic objectives, for
example, remaining in the manufacturing of a particular product

Case studies in a variety of industries are used to conrrast the project approach with a more
strategic approach. It is concluded that the project may represent sub-optimization in that a successful
outcome of the project may have no impact upon company survival without a careful examination of the
strategic plans of the company

MANUFACTURING IS CHANGING

The past ten years have seen the realization that
if the USA is to compete successfully in the world,
manufacturing cannot be neglected. Lessons from more
successful manufacturing nations have been learned by
companies of different sizes, and the results are
beginning to be seen in manufacturing excellence

Perhaps the most fundamental change in the way
leading companies treat manufacturing is to focus more
clearly on the ultimate objectives of the company.
Almost every company of any size now has a "Mission
Statement" with fine words about customer satisfaction
and product quality which, if heeded, would radically
change the way the company thinks. This change is
truly radical as it is forces employees at all levels to
evaluate their decisions against very specific outcomes.
These outcomes rarely include the traditional measures
of monthly output, labor cost variances, minimum first
cost of investment, or machine utilization; measures
which most company employees have known to be the
overriding criteria in day-to-day operations

At higher levels, companies are now undertaking
strategic planning to aid the long-term shaping of the
company. Not every opportunity should be pursued,
only those which fit long-term goals, goals which are
themselves based on an honest assessment of the
strengths and weaknesses of the company. Thus a
company may abandon a traditional market for large
scale mass-production of identical components to
concentrate on manufacturing a customized family of components at lower production volumes. In this way it
can be extremely responsive to customer needs, a key
component of customer satisfaction

Such responsiveness cannot be achieved without
changes in traditional production systems. Quality is
essential: any defect will have an immediate impact
upon output, an impact which cannot be hidden from the
customer by large inventories in a highly responsive
manufacturing environment. Responsiveness also forces
decreased reliance on a multi-level decision-making
hierarchy. Hence the modern emphasis on well-trained
self-organizing small groups (or cells) which are
responsive directly to customer needs. When response
time, quality, and training come to the fore in
manufacturing industry, so must human factors
engineering.

HUMAN FACTORS IN MANUFACTURING

One arena in which the USA is a major power is
in human factors/ergonomics so that it is natural to
consider how we can use this power to improve
manufacturing. Results to date have been less than
spectacular, with major human factors involvement
mainly in nuclear power production (for cognitive and
behavioral interventions) and in a variety of
manufacturing industries at the level of musculo-skeletal
injury reduction. Results in this context means having
an impact on the company's major decisions of which
markets to pursue, where to manufacture goods and how
manufacturing contributes to overall company goals

On a strategic basis, if the USA has a lead in
human factors, that lead should be exploited in
manufacturing. We have had numerous examples from
around the world of how human factors can be a key
element in manufacturing. Harris & Chaney’s book
(1969) was based on a major implementation of
ergonomics within the quality control function of an
aerospace company. On a smaller scale Hasselquist
(1981) showed how redesigning a line on ergonomics
principles had a productivity payback period of less than
half a year, with the added bonuses of a halving of the
error rate and elimination of musculo-skeletal injuries.
The authors suspect that most ergonomists could
produce similar examples from their files.

But the direct impact of human factors on the
manufacturing system which has been changed is really
not the whole question. For example, a program of
ergonomic changes in a shoe manufacturing company
(Drury & Wick, 1984) was particularly successful in
reducing muscular skeletal injuries in the plant, but it
did not prevent eventual closure of the plant as the
parent company responded to foreign competition.
Even a four-year program of ergonomics throughout the
company (which was estimated to have saved over $6
million in productivity increases and injury reduction)
was not enough to prevent closure of the ergonomics
department with the rest of the engineering functions
after a hostile take-over.

Part of the reason for human factors still not
being of central impact on manufacturing is the way in
which we have traditionally intervened. Whether the
ergonomics expertise comes from a group inside the
company or external to the company, the typical
intervention consists of a project. This project is
defined by both the customer and ergonomists in such a
way that both groups are satisfied with the potential
outcome and the intervention methodology.
Unfortunately, the customers and human factors
engineers may have reached a satisfactory
understanding at the wrong level.

Human factors engineers are trained to ask
technical questions (What is your workhest schedule?
How do you extract information from that display?) but
only have a rudimentary idea of the system functioning
beyond this level in manufacturing industry. Thus, ideas
of customer satisfaction with cost, on-time delivery and
quality are addressed, if at all, in terms of cycle times
and error rates. Our customers are often little better. A
safety manager may wish to reduce lost-time injuries, an
R & D manager may wish to estimate performance of a
prototype system or a quality control manager many
wish to reduce human-caused errors. At these levels,
the two parties can talk comfortably, as the connections
between their variables are reasonably obvious. But
what if the customer’s problem has no impact on the
company’s fortunes: Or what if the ergonomist’s time
should be better spent with another project or another
company? We all have a duty to ensure that what may
be a scarce national resource (human factors talent) is
used to maximum advantage.

In contrast the military typically has learned
many years ago that human factors expertise needs to be
continuously available throughout system development
(e.g. Meister, 1971) where it can impact on the broader
aspects of systems effectiveness. Can we make use of
this model in a manufacturing environment where the
higher level customers are (at best) skeptical about
human factors, and the human factors engineers are
untrained in such areas as strategic planning, cost
accounting or employmenflabor policies?

One way in which human factors can maximize
its strategic impact on manufacturing is to be part of a
team which operates at the strategic level. This means
that the team interacts with, and reports to, the highest
levels in management and union so that the strategic
aspects of the intervention are explicitly part of the
project. It does mean however that the intervention will
not be seen as an ergonomics intervention, but as an
indistinguishable part of the overall systems
intervention.

the Training Needs Detector, a tool dedicated to the human
resources management that reasons in the long-run by ponder-
ing the gaps between required skills in the production and those
provided in the actual job allocations in order to identify
persistent skill shortages towards the definition of personalized
training paths (Table 2: O2).

For the validation of the new approach and developed tools, two
lines,oneformicrowaveovensandanotherforfridgeproduction,were
modelled relying on the representation capabilities offered by the
KNOW Platform. The single jobs at each workstation were designed
trying at the same time: (i) to balance the workload by shifting the
tasks among workstations to cope with the upper limit set at 90% of
takt time; (ii) to distribute the tasks with critical requirements, to
balance the cognitive burden and increase the possibility that a pool of
workers can effectivelyand completely match the skill demand; (iii) to
reduce the risk of musculoskeletal disorders due to ergonomically
impactingtasksinsomeworkstations.Joballocationtestswerecarried
outoff-lineinvolvingteamleaderstocomparetheirchoiceswiththose
offered by the linear optimizer

The improved workplace adaptability results in the reduction of
cumulative trauma disorders and psychological stress. The
availability of an all-encompassing digital solution capable to
characterize, from a human-centric perspective, both workers and
production lines allows to integrate all necessary human-related
information and permits to support production facilities improve-
ment in terms of skill-matching, ergonomics and safety. Further-
more, the integration of multi-perspective tools provides
environments that promote awareness of all the human-related
aspects at a glance, thus fostering
first-time-right solutions and
reducing the several design iterations that were necessary before.
Finally, the long-term analysis of mismatch at the boundaries of
human-automation interaction allows to redesign a more humanaware
automation and to promote training activities to
fit workers’
profiles to automation needs.

5. Concluding remarks

In the context of emerging smart factories, the reported analysis
examines the potential cooperation that can bring to a synergic
human-automation solution when the roles are modulated on the
basis of the automation level. The described industrial cases elicit
the current issues experienced within an automation dominated
environment that impacts production systems productivity and
workers’ well-being.

The proposed model for worker-aware adaptive shows how
human well-being drivers are harmonised in the automation
design. According to this model, well-being is achievable only by
implementing adaptability and
flexibility within a sociotechnical
system; automation becomes the keystone of this human-in-theloop
adaptive approach to production. The new factory automation
integrates seamlessly human and digital decision-making by monitoring production performances and workers’ physiological
parameters; this scheme permits the reintroduction of the man-inthe-
loop within factory automation.

This new framework has been validated in the experiments
carried out in two different domains showing respectively how:

a dedicated tool can relief mental workload while operating in
the context of adaptive automation;

an integrated toolset, covering different phases of factory design
and operation, can support human-centric automation.

The proposed automation model can address the identified
gaps. The obtained results prove, qualitatively and quantitatively,
that the integration of the human factors analysis, within
automation design, is a compelling condition for a synergic
improvement of manufacturing performance and human wellbeing
at once. Specifically, the experimental assessment presented
in this paper shows the effectiveness of the proposed model that
permits to overcome the trade-offs of automated manufacturing
environment and humans well-being through a synthesis of the
technical and psychological tools

Future work must employ methods and tools presented in this
paper to face the gaps outlined in Table 2, in different
manufacturing contexts and at any phase of the production
process.

Nowadays the human-automation synergy is fundamental for a
sound development of the innovative production scenarios
according to new emerging paradigms, like Industry 4.0 or Society
5.0. Particularly, the acceleration of scientific and technological
innovations will make the automation role more and more
important and pervasive. Consequently, the future evolution of
manufacturing industry should tackle the challenge to properly
balance the industrial targets with the workers well-being, by
means of human-in-the-loop adaptive automation systems.

3. A new systemic model for man-in-the-loop automation

In order to represent operators’ safety and well-being according
to the recent
findings and theories [10], a systemic model is
required; the model includes the cognitive and physical well-being
aspects of operators interacting with automation [11]. Basically,
the main aspects are:

the human operator: actions, errors, violations, mental models,
expectations, skills, culture;

the team: team cooperation dynamics;

the organization: managerial decisions, policies, culture, vision;

the physical environment: temperature, noise, layout;

the social environment: external pressures;

the tools: technology and automation level;

the rules: procedures, guidelines, checklists, laws;

the task: unexpected, habitual, repetitive, etc.

These features can be regarded as interacting, like fragments of
a bowl that dynamically move in order to
fill in the gaps that may
arise at their borders

The water in the bowl represents the operators’ well-being,
whose optimal level depends on the dynamic interaction among the main aspects of a system; well-being level is determined by the
gaps that the fragments create at their borders

Fig. 1. The well-being bowl

Sometimes an action aimed at increasing well-being could take
into account only one fragment; but changing just one part could
lead to breaking the bowl if the other elements do not adapt to it

Automation design should be guided by this model to promote
harmonization of the fragments’ behaviour by changing its role
according to the degree of control (Table 1). Adaptive automation
could provide a
flexible fragment that copes with the inherent
variability of the production system

coming from human and
organizational factors, process changes and productivity needs.
The constant adaptation enables the system to
fill the gaps
ensuring operators’ well-being

This vision can be translated into a framework that supports a
seamless adoption of a man-in-the-loop automation approach
reducing the risk of negative gaps. Basically, only by making the
most out of the capabilities residing in the human dimension of the
factory, it is possible to unleash automation’s full potential and to
enhance productivity and well-being

Fig. 2 shows the proposed man-in-the-loop automation system
within the factory; the focus is not only on the optimization of
production performances but it includes also the human operator
as a full-fledged part of the whole process. However, it is not
enough to consider the human dimension only as one of the
controlled variables of the automation system; human dimension
must be integrated into the management of the control loop to
couple automation’s efficiency with the
flexible human mind set.
This approach presents a twofold interaction:

the human acts as a decision maker who works in synergy with
the control system at a decisional level;

the human acts also at operational level taking part to the
controlled production process where the interaction is played on
a more operative
field

Fig. 2. Framework for human-in-the-loop factory adaptive automation

The two different cooperation roles could trigger the risks listed
in paragraph 2.1 and cause suboptimal results for the operators’
well-being and the production performance. To reduce the risks
and smooth their negative influences, each role requires a careful
management taking into account its specificity

The proposed man-in-the-loop automation framework requires
to set goals specifically aimed at enhancing the working conditions
by constantly monitoring a stream of physiological measures to detect in real time any deviations from personalized safe patterns
and propose mitigation actions aimed at mediating or mitigating
the cognitive demand that the worker is experiencing. It also
requires reconfigurable automation policies that apply in the
distributed automation structure. to explore and possibly
eliminate the sources of cognitive gaps such as skill mismatching
and alienating duties.

4. Model implementation in demonstration cases

4.1. Adaptive automation in air traffic control

Mental workload in air traffic management is a crucial factor for
safety. Human errors typically occur both in underload and
overload conditions, the former because operators are disengaged
by the task, the latter because they are overwhelmed. In the
interaction with automation, changing the number, quality, rate,
dynamics of stimuli and data presented on the display is crucial to
maintain a proper level of workload

A passive Brain-Computer Interface (pBCI) was developed [14]
in order to track operators’ brain activity (EEG), which is
considered a reliable, sensitive, real-time, and continuous measure
of mental workload. The pBCI was integrated in an air traffic
control simulator and was tested for its capacity to produce
adaptive solutions in real-time, according to the mental workload
measured by means of operators’ brain activity. When operators’
under- or overload is detected, the system automatically triggers
adaptive solutions (e.g. displaying only critical alarms, highlighting
the aircraft currently speaking, animating the icons related to a
short term collision, and displaying only the aircrafts that are
relevant for the task at hand).

The pBCI is able to activate adaptive automation solutions
during high-workload scenarios, while it does not activate them
during periods of normal workload, in order to avoid underload. In
addition, the pBCI induces a decrease in the perception of mental
workload by the operators when adaptive solutions are activated.
Behavioural performance analysis demonstrated that the task
performance significantly increased when adaptive automation
solutions were triggered (Table 2: C3).

4.2. Automation in the white-goods industry

The white-goods industry is characterised by work-intensive
production environments where humans are mainly employed to
assemble a highly diversified range of products manufactured in
continuously changing and relatively small lots. Such production
and high production pace, due to the automation component of the
line, poses serious cognitive demands to the workers.

The involved white-goods industry uses continuous
flow
production lines with a takt time
fluctuating next to one minute
and a number of product variants exceeding one hundred. In the
past, the company tried to map the workforce capabilities
(experience on the job, management of safety and quality aspects)
in order to establish a personal skill matrix for each worker.
However, the lack of a systemic approach to the humanautomation
interaction prevented any substantial improvement
in adapting the workplaces, and the production system at large, to
the characteristics and condition of the individual workers
 

Adaptive automation and human factors in manufacturing: An
experimental assessment for a cognitive approach

A B S T R A C T

Despite increasing automation levels and digital solutions, production systems still very much rely on the
inescapable contribution of the human factor. The changing relationship between man, the technological
system and the organization framework together with the increased complexity result in high risks for
workers’ safety and their psychophysical health. Adaptive factory automation and management solutions
integrating the man in the loop are proposed in order to achieve production performance, workers safety
and well being in a balanced way in varying boundary and exogenous conditions. Particularly, this paper
presents new methods and the related recent case studies in different sectors

1. Introduction

While automation in manufacturing dates back to the
first
industrial revolution, the increasing manufacturing systems’
complexity and technology advancements require nowadays
reliable tools to plan, assess, and drive the production chain in
order to get the planned goals. Many methodologies have been
developed to face the organisational aspects by considering the
production plant capacity, customer demand and products
characteristics

At the level of production shop-floor, the
growing
flexibility of production means calls for automation
systems whose behaviour is driven by dynamically adapting
management policies

In the last
fifty years automation evolved
dramatically along several generations: from the direct involve-
ment of workers in the manufacturing process, to intelligent
automation systems where workers play a supervisor role

Indeed, the introduction of new technologies impacts on the
complexity of manufacturing system management and requires to
promote harmonisation between automation and the human
factors

especially considering the cognitive workload related to
manufacturing operations at different decisional levels

This paper proposes a methodology, validated in two selected
industrial cases, to integrate cognitive workload into the design of
workplaces to match the human safety and well-being necessities
and tasks’ cognitive requirements. Specifically, a new framework to classify the fabrication tasks of production processes according to
their cognitive complexity and the required capacity enables an
anthropocentric optimisation of the manufacturing activities

2. Automation and human factors

2.1. Interaction challenges

Human factors and human performance limitations, resulting
in errors and violations, are the main contributors to accidents and
injuries in complex systems

A common approach aimed at
reducing this incidence has been to transfer to automation a
variable portion of the tasks that were previously performed by the
human operator

The shared responsibility between human and automation
could be broadly located along a continuum, as represented in
Table 1

Notwithstanding the successful integration, many issues
occur when considering the relationship between humans and
automation; the problems are essentially:

Out-of-the-loop condition: the difficulty of operators to have a
clear and complete picture of the automation states and
processes, lead to a diminished ability to detect possible
automation failures and to regain manual control

Surprising mode transitions: operators may become unaware of
changes in the operating mode performed by automation

Skill loss: pervasive automation will decrease the opportunity for
training manual skills, which will be ineffective in case of an
urgent manual control of the system

Table 1
The continuum of shared responsibility between human and automation, adapted
from

Automation-induced errors: while automation may compensate
or reduce some typical human errors (counting, remembering,
monitoring, etc.), more automation could lead to new, unex-
pected forms of human errors

Behavioural adaptation: automation may grow the perception of
safety and operators could adapt their behaviour taking higher
risks

Inappropriate trust: trust in automation could change according to
the perception of its reliability. When this perception is biased,
inappropriate trust will result as misuse, disuse and complacency

Job satisfaction: automation could be perceived as a threat to
workers professional profile, especially when it is introduced
without a proper transition and a management care for reskilling
their workers

In order to face the described issues, a comprehensive approach
to map them and an effective strategy at different and relevant
levels, is required. The identification of the intervention areas and
gaps is a crucial step to build a coherent system capable to
harmonize the automated components with their human counterpart.
In current efficiency-driven contexts the cognitive level,
that acts as the interface between human and automation, is
responsible for the workload impacting the worker. Much of this
impact is determined by the decisions taken in the design of the
automation processes as well as by the organizational strategies;
therefore, the organization level must be included in the analysis.
Table 2 shows the gaps and the involved processes for the
cognitive, organizational and technological levels

Table 2
Cognitive, organizational, technological gaps and processes

2.2. Design guidelines

Traditional approaches to enhance the human-automation
interaction are based on the Fitts list

reported in Table 3 Such a
list allocates functions considering the information processing
stage and static conditions. A more effective approach should
assign the functions according to the task, the operator, and the
situation considering the system dynamics: adaptable automation
allows the operator to decide the level of control according to the list reported in Table 1. Adaptive automation can change the
control level by automatically adjusting itself to the operator’s
performance, operator’s state and the system status

Table 3
Relative strengths of humans and automation

RESEARCH METHOD AND RESULTS

This study utilized a survey instrument on the U.S. manufacturing firms who were most likely to adopt AMS. The firms were selected from the following three publications: Moody's Industrial Manual, American Association of Manufacturing Technology (AAMT), and A.MS trade journals. Within each firm, a plant manager was identified as target for the questionnaire. To ensure an appropriate level of respondent knowledge, only participants meeting the following criteria were included in the study: (1) at least six months of AMS use by the organization, and (2) at least six months AMS experience by the individual participant. Those respondents that did not meet these conditions were asked to so indicate and return the questionnaire. This action ensured that each participant was familiar with and able to objectively evaluate the AMS adopted in his/her organization. The respondents were urged to take part in the study only if the requirements stated above were met.

Of the 400 questionnaires mailed out, we received 117 responses (29.3 percent). Nineteen were discarded, eight because they were not completely filled out and eleven because the respondents indicated insufficient experience with AMS. There remained 92 (23 percent) usable responses that were included in the study.

Human factors were investigated by asking respondents to indicate the extent to which they felt the eight human factors discussed earlier were present during and after AMS implementation. Over 37% of the participating firms had a process manufacturing environment while about 30% operated in a repetitive manufacturing environment. The respondents reported that A_MS projects were initiated mostly (over 80%) by management rather than workers or vendors. In about 80% of the cases, the AMS projects were directed by management rather than a steering committee or appointed individuals. Over 38% of the respondents were top management, 38.5% were middle management, and about 19% belonged to other ranks.

The Partial correlation coefficients of the human factors and AMS benefit measures were computed. According to the results, the benefit measures did not indicate any multi-collinearity problem. The Partial Coefficients (r) of 0.1900 or higher are significant at p < .06. The correlation analysis reveals that all human factors positively correlate the benefit measures. The strongest correlation were found between morale and reduced throughput time (r = .4318; p < .000); satisfaction and return on equity (r = .2026, p < .060); reward system and reduced throughput time (r = .3274, p < .002); belief in AMS and improved work conditions (r = .3651, p < .001); top management commitment and enhanced competitiveness (r = .3714, p < .000); response to workers' concerns and better control (r = .3453, p < .001); effective facilitator and better control (r = .2483, p ~ .020); training and improved quality (r = .2305, p < .011).

Table 1 is a record of the Chi-Square test obtained by cross-tabulating human factors and AMS benefit measures. The association between any two variables was significant if p _< .001 (*) or p< .05 (**). In order to test the hypotheses of the study, Chi Square testes were conducted to determine if there exist any associations between human factors and the benefits of AMS. As shown on Table 1, every human factor considered in this study has significant association with some of the AMS benefits measures.

Of the 400 questionnaires mailed out, we received 117 responses (29.3 percent). Nineteen were discarded, eight because they were not completely filled out and eleven because the respondents indicated insufficient experience with AMS. There remained 92 (23 percent) usable responses that were included in the study.

Human factors were investigated by asking respondents to indicate the extent to which they felt the eight human factors discussed earlier were present during and after AMS implementation. Over 37% of the participating firms had a process manufacturing environment while about 30% operated in a repetitive manufacturing environment. The respondents reported that A_MS projects were initiated mostly (over 80%) by management rather than workers or vendors. In about 80% of the cases, the AMS projects were directed by management rather than a steering committee or appointed individuals. Over 38% of the respondents were top management, 38.5% were middle management, and about 19% belonged to other ranks.

The Partial correlation coefficients of the human factors and AMS benefit measures were computed. According to the results, the benefit measures did not indicate any multi-collinearity problem. The Partial Coefficients (r) of 0.1900 or higher are significant at p < .06. The correlation analysis reveals that all human factors positively correlate the benefit measures. The strongest correlation were found between morale and reduced throughput time (r = .4318; p < .000); satisfaction and return on equity (r = .2026, p < .060); reward system and reduced throughput time (r = .3274, p < .002); belief in AMS and improved work conditions (r = .3651, p < .001); top management commitment and enhanced competitiveness (r = .3714, p < .000); response to workers' concerns and better control (r = .3453, p < .001); effective facilitator and better control (r = .2483, p ~ .020); training and improved quality (r = .2305, p < .011).

Table 1 is a record of the Chi-Square test obtained by cross-tabulating human factors and AMS benefit measures. The association between any two variables was significant if p _< .001 (*) or p< .05 (**). In order to test the hypotheses of the study, Chi Square testes were conducted to determine if there exist any associations between human factors and the benefits of AMS. As shown on Table 1, every human factor considered in this study has significant association with some of the AMS benefits measures.

CONCLUSIONS

The dramatic advancement and the adoption of advanced manufacturing systems (AMS) in organizations can be attributed to its numerous benefits that can improve the competitive position of the manufacturing firms. The results of this study show that the eight human factors considered in this study have positive associations with all the eight AMS benefits. While other factors may play major roles in realization of AMS benefits, sociotechnical or human factors have been shown to be essential ingredients for AMS success. For a manufacturing firm to actualize the full benefits of AMS, steps need to be taken to ensure effective human resource management. The implication is that when a firm takes care of the human needs, the AMS implementation will be well received and understood and as such, the desired effects of AMS will be brought to bear in the firm's production cost. In essence, the manufacturing firm that pays attention to human factors may realize the benefit of shorter throughput time because AMS is properly implemented and efficiently operated. If human factors are ignored in a firm during AMS implementation, there is a good chance that the workers will be discouraged and reluctant to apply themselves and as such, delays may occur in the production schedules. If that is the case, components of the manufacturing system will not be interacting in the most efficient way thereby resulting in a long throughput time.

ABSTRACT

This paper analyzes the data collected from 98 manufacturing companies to investigate the associations between human factors and the success of advanced manufacturing systems (AMS).

The AMS measures and human factors were cross tabulated and the Chi Square values were used to test the hypotheses of the study. The results show that statistically significant, positive associations exit between human factors and the success of AMS implementation. The implications of the findings to the practitioners and researchers are discussed.

INTRODUCTION

The drive to lower operating costs and improve manufacturing efficiency has led many manufacturing companies to implement different forms of advanced manufacturing systems (AMS). The dramatic developments in advanced manufacturing technologies at various organizational levels can be attributed to numerous benefits that improve the competitive position of the company. AMS affects not just manufacturing, but the whole company operations, giving new challenges to a firm's ability to manage both manufacturing and information systems. AMS can be defined as a group of integrated hardware-based and software based technologies which, when properly implemented, monitored, and evaluated, can improve the operating efficiency and effectiveness of the adopting firm. It encompasses a broad range of computer-based technological innovations which are integrated using communication links made possible through advanced computing technologies and are referred to as computer-integrated manufacturing (CIM)

AMS has the potential to dramatically improve production performance and create vital business opportunities for companies that are capable of successfully implementing and managing it. (King and Ramamurthy, 1992). AMS can also provide distinctive competitive advantages in cost and process leadership. Practitioners and researchers have since developed strong interest on how AMS

can be used to combat global competition. A growing number of organizations are now adopting AMS to cope with fragmented mass markets, shorter product lifecycle, and increased consumer demand for customization (Zummuto, et al., 1992). Although AMS can help manufacturers compete under these circumstances, they often serve as a double-edged sword, imposing organizational challenges and, at the same time, providing competitive benefits

The benefits of AMS have been widely reported in the literature and classified as being tangible and intangible (Udo and Ehie, 1996). Although the benefits of AMS are numerous and have been found to have direct links with the firm's operating performance, only a handful of companies have been able to realize the full benefit of AMS. The rate at which these benefits are derived varies to a large extent from one company to another. Beatty (1993) concludes that only half of those companies adopting AMS ever achieve the benefits they sought. Success in AMS implementation becomes a reality when the set goals and objectives stipulated by the adoption strategy are fully realized.

The potential offered by AMS to deal with the emerging realities of the twenty-first century competitive environment is widely recognized, but concerns have also been expressed about the ability of firms to exploit this to their advantage. The literature is replete with arguments in support of the presence of one or more of the critical success factors as requirements for successful AMS implementation. Sociotechnical or human factors axe among the critical factors believed to have some impact on the success of AMS implementation. The purpose of this study is to investigate the extent to which the identified human factors affect the benefits of AMS. This agrees with the observation made by Huber and Brown (1991), who suggest that some empirical research is needed to investigate the impact of sociological variables on the implementation of manufacturing processes.

The eight most cited benefits of AMS considered in this study are return on equity, reduced manufacturing cost, reduced throughput, enhanced competitiveness, better control, quick response, improved working conditions, and improved quality.

The null hypothesis of this study is that there is no association between the AMS benefits and human factors. That is: human factors do not affect the success of AMS implementation

HUMAN FACTORS

Several technical and social changes often take place when a company adopts A_MS. As Hopkins (1989) points out, if an organization focuses solely on the technical issues from the outset of a manufacturing project implementation and at the expense of the human issues, its performance will be less favorable than if it pays attention to both sets of issues. The adoption of AMS certainly changes the social relationship and interactions among employees and their supervisors.

Given the potential impact on employee attitudes, motivation, and retention, these social changes call for an effective management (Huber and Brown, 1991). When employees are affected, the success of A.MS is likely to be affected as well. In this study, human factors comprise of three components namely: self-interest (four factors), top management (three factors), and preparation (one factor)

(a) Self Interest. It is known that human beings are self-interested in that we tend to strive to succeed on those tasks we believe to be of a personal interest to us. King and Ramamurthy (1992) maintain that no matter how attractive the benefits or the sophistication of technology, if personnel-related aspects (such as motivation, participation, reward schemes, etc) are not planned for, the end result is bound to be a frustrating failure. They discovered in their study that people problems could prove to be more difficult to solve than technical problems and could have serious consequences on AMS implementation. The self-interest factors considered in this study include: general employees' morale, satisfaction levels, personal belief that AMS can lead to personal reward or benefits to the individual, and equitable reward structures

(b) Top Management. Top management support provided in the form of creating project mission; allocation of sufficient resources; establishment of a reward system that fits the project; maintaining project accountability; personnel recruitment, selection, and training; monitoring and feedback functions.

Research and experience support the fact that the degree of management support of a project will lead to significant variations in the degree of acceptance or resistance to the project, and also to the degree of success (Udo and Ehie, 1996). The top management factors included in this study are commitment by top management, effective facilitator, and quick response to workers concerns by management

Preparation. The main preparation needed by the workers in the AMS environment is training. The need for training has been heavily emphasized by Beatty (1993)

Collaborative Work

Agility emphasizes a collaborative design process whereby engineering disciplines affected
by design decisions are integral participants in making those decisions (Forsythe
et al., 1995). Collaboration permeates all aspects of an agile product realization process,
with specific steps necessary to assure concurrence is obtained early in design, before
precious time and resource commitments have been made. Such extensive collaboration requires an awareness and appreciation of the interests and contributions of each discipline.
However, this is often difficult to attain due to “engineering arrogance” or the belief
that “what I do is difficult and what you do is easy,” and organizational dynamics that
often allot considerable power, influence, and respect to designers, and substantially less
to supporting disciplines.

Many technical innovations may be applied to support collaborative work. For example,
X applications sharing software allows designers, working at their desktops, with
only a moment’s notice, to open a shared CAD representation of a design that they and
other team members may view, and freely manipulate from within the CAD application.
In this way, X applications software enables collaborative design and decisions, making
codesigners of team members who otherwise would have only been reviewers. In addition,
solid models and animated illustrations of machining and robot assembly processes
allow Manufacturing and Assembly engineers to more readily and clearly communicate
their concerns to designers

5.3.

Enterprise Integration of Information Technologies

Agility requires the removal of information bottlenecks and improvement in the continuity
of information flow through the enterprise. It is unacceptable to have work delayed
because information available at one point in the process has not or cannot be transferred
and used at another. In striving for this objective, there is the need to open channels for
information flow and to remove resistance (e.g., cross-platform, cross-application incompatibility)
to information flow

Information may be transmitted via multiple channels depending on urgency, content,
and distribution (e.g., phone, voice-mail, fax, e-mail, ftp, PDM, http). Product Data Management
(PDM) is of particular significance in that it provides team members a central
information repository that offers automatic notification of file and design changes. Thus,
notification and updating of team members does not require a conscious effort, but is
integrated into the day-to-day interactions with the PDM

Agility is enhanced by a seamless flow of information between software applications,
and between software and production hardware. Burdensome file conversions create intolerable
delays for the production process and waste valuable human resources. Agility
requires that the cognitive resources of project personnel be directed toward the challenges
of design, analysis, and decision, and not spent on mundane activities such as data
entry or recoding. Through development of software routines that translate between software
applications and some standardization to compatible software applications, a production
process may be developed that is seamless, from beginning to end

6

CONCLUSION

Agile manufacturing will proceed, with or without the contributions of human factors.
For the field of human factors, agile manufacturing is, more than anything else, an opportunity.
By raising human factors issues, and applying the knowledge and skills gained
from other domains, there is an opportunity for human factors to assume an important
role, positively influencing the future of agile manufacturing. As of 1997, agile manufacturing
is still immature and not yet fully defined. Agile manufacturing poses many
questions best answered by human factors. Our willingness, as a profession, to address these questions will determine whether our role is in the definition of a paradigm or in the
limited role of after-the-fact, fixing what is broken

5

COMMUNICATIONS AND INFORMATION INFRASTRUCTURE

As shown in Table 1, the communications and information infrastructure of an agile enterprise
differs greatly from that of traditional enterprises. “Virtual Corporations” (Nagel
and Dove, 1992) and availability of electronic data communications lead to coworkers
collaborating from geographically dispersed locations. Concurrent engineering within a
fast-paced product development environment favors collaborative work between engineering
disciplines over “meet and disperse” work patterns (Forsythe and Grose, in press).
Emphasis is placed on seamless and direct information flows. With agility, many of the
traditional bottlenecks that provided time buffers to the product development process are
no longer present, due to automation and streamlining. Consequently, changes to product
and related artifacts occur much faster, leading to increased demands for information and
product data management. Finally, as enterprises focus on core competencies and opportunistically
enter Virtual Corporations, it is no longer practical to maintain and rely on
internal sources of information. Instead, the need arises for mechanisms that enable the
accessibility and efficient utilization of diverse, external sources of information.

TABLE 1.

General Differences Between Traditional and Agile Enterprises

Traditional Enterprise

Agile Enterprise

• Geographical colocation • Geographical separation
• Solitary work • Collaborative work
• Sequential information flow • Parallel information flow
• Time is negotiable • Time is critical
• Standardization of technology • Opportunistic technology use
• Artifacts relatively static • Artifacts change rapidly
• Information flow correlated with
organizational structure
• Information flow correlated with
project structure
• Extensive use of hard media • Extensive use of electronic media
• Constant, known, internal sources
of information
• Diverse, often unknown, external
sources of information
• Many indirect lines of communication • Mostly direct lines of communication

5.1

Electronic Data Communications

Agility introduces a dynamic, fast-paced environment that requires extensive collaboration
between widely dispersed team members who must work together with an efficiency
comparable to their being located in the same building, if not the same room (Virtual
Colocation). In the development of an agile product realization process for custom electromechanical
devices, information flow requirements for an agile enterprise have been
analyzed (Forsythe and Ashby, 1994). This analysis provided an understanding of the
roles filled by each participant in the enterprise, their information needs and sources, the
timing of information needs and information availability, and restrictions on the ability of
participants to use information once it had been received.
The information flow analysis followed the sequence illustrated in Figure 1. In the first step, team members were surveyed to determine their information needs. Before this survey
could take place, measures were necessary to heighten team member’s awareness of
their information needs. This was accomplished through a series of team meetings during
which the team jointly developed the project plan, including objectives, strategies for
meeting objectives, a detailed task network, schedule, and resource and funding projections.
Initially, team members completed paper surveys describing their roles and activities,
followed by in-depth interviews utilizing techniques from information requirements
analysis. Based on the information needs survey, the seven categories of information exchange
shown in Table 2 were developed. Subsequently, a matrix was developed showing
the information exchange categories relevant to each pair of team members.

To allocate technologies to various modes of information exchange, it was necessary to
develop functional requirements for each category of information exchange. The nature
of these requirements is illustrated by the following example of functional requirements
identified for the General Knowledge or Requests category:

TABLE 2.

Categories of Information Exchange

General Knowledge or Requests Transfer of Work/General Level of Skill
Meetings and events Documents
Memos and letters Figures and tables
Schedules Presentation graphics
Agendas Spreadsheets
Forms (surveys, progress reports) Project management materials
News, reports and announcements (e.g., PERT charts, Gantt charts)
Policies
Requests for information Collaborative Work
General Information (e.g., phone lists) Design and design analysis
Brainstorming
Person-to-Person Discourse Group planning
Communications requiring unequivocal Process documentation
and immediate confirmation or response Computer code development
Communications where a need exists to Document and presentation development
assure understanding of information content Project management
Communications in which a spontaneous
dialogue is essential Urgent Communications
Communications where concern exists Changes in schedules or availabilities
for emotional connotations or responses Meetings
Communications where socialization is Demonstrations
part of the implicit or explicit intent Important visitors
Demonstrations and tours Events, cccurrences, or other situations
Team building Requests for information
Transfer of Work/Specialized Level of Skill Casual or Informal Communications
CAD, CAM and solid model files Lunch or hallway discussions
Computer code Casual exchanges before and after meetings
Numerical control programs Discussions to break-up work sessions
FYI conversations
Authorized Gateway Customer Bull sessions

• It should be relatively easy (a few simple steps in addition to creating the message)
to transmit the message to every intended recipient.
• After receiving the message, each recipient should have a clear understanding of
what action is expected on their part and any needed details regarding how to accomplish
this action (e.g., meeting place and time.
• Given the occasional urgency of this type of communication, messages should be
available to recipients almost immediately following transmission and some mechanism
should be provided to alert recipients of the presence of the message.
• Most often the message content may be readily captured verbally or with alphanumeric
characters; however, there may occasionally be a need to include limited tables
or figures (e.g., maps, calendars, charts, etc.).

Alternative technologies were identified and assessed relative to the functional requirements
of each category. Based on these assessments, technology solutions for each information
exchange category were developed. These solutions were then incorporated into a
communications infrastructure design, which allocated technologies to meet the needs of
each team member. This infrastructure included e-mail, voice-mail, file sharing, product
data management, and collaborative work tools.Aproject-wide infrastructure was implemented
through these technologies, the success of which was demonstrated by the design
and production of custom, precision, electromechanical devices in 24 days or less (Forsythe
et al., 1995).

Chris Forsythe
Sandia National Laboratories, MS 0829, Albuquerque, NM 87185-0829

ABSTRACT
Agile manufacturing has been promoted as a national strategy for improving industrial competitiveness.
Agility refers to the capability to very rapidly go from a set of unique customer requirements
to a quality, finished product. An appreciation of the human factors inherent to agile product
development is pivotal to the successful integration of agility-enabling technologies, as well as the
coordination of personnel working within a concurrent engineering environment. This article briefly
summarizes human factors contributions to: (1) development of agile business practices; (2) design
of enabling technologies; and (3) management of the introduction and fielding of new technologies
and business practices. More detailed discussion is offered for human factors related to the communications
and information infrastructure essential to an organization making the transition from
traditional to agile product development. © 1997 John Wiley & Sons, Inc.

1. INTRODUCTION
As industries position themselves for the competitive markets of today, and the increasingly
competitive global markets of the 21st century, agility, or the ability to rapidly develop
and produce new products, represents a common trend (Kovac, 1993; Levary, 1992;
Nagel and Dove 1992). Agility manifests itself in many different forms, with the agile
manufacturing paradigm proposed by the Iacocca Institute offering a generally accepted,
long-term vision (Nagel and Dove, 1992). In its many forms, common elements of agility
or agile manufacturing include: (1) changes in business, engineering, and production practices;
(2) seamless information flow from design through production; (3) integration of
computer and information technologies into all facets of product development and production
processes; (4) application of communications technologies to enable collaborative
work between geographically dispersed product development team members; and (5)
introduction of flexible automation of production processes.

Industry has rarely experienced as dramatic an infusion of new technologies or as extensive
a change in culture and work practices. In recognition of these emerging trends, a
panel session entitled Human Factors in Agile Manufacturing was held at the 1995 Human
Factors and Ergonomics Society Meetings in San Diego, CA, USA to discuss human
factors issues relevant to agile manufacturing and to present different perspectives on
those issues. The articles appearing in this special issue represent the perspectives presented
at that panel session and reflect the continuing evolution of thought concerning
these matters. This article briefly summarizes human factors related to the development of agile business practices, design of enabling technologies, and management of the introduction
of new technologies and business practices. Afterwards, more thorough discussion
is offered concerning human factors related to the communications and information
infrastructure essential to an enterprise fully realizing agility.

2

DEVELOPMENT OF BUSINESS PRACTICES

Implementation of agile manufacturing requires, at a minimum, extensive modification
of existing business practices, but often complete overhaul of existing practices (Greiss,
1993). In a market environment where corporate success hinges on rapid turnaround of
quality products, each new product development effort cannot begin from a blank slate.
Instead, corporations must maximize their ability to capture and utilize corporate history
and lessons learned (Goldman and Priess, 1992). Likewise, manufacturing processes fitted
to the demands of a given product must be replaced by flexible systems composed of
different pieces of equipment that readily accommodate a range of product parameters
(Brost et al., 1992; Staffend, 1992). Serial progression of designs through the product
development cycle is unacceptable. Concurrent engineering of designs is desirable, but
collaborative design is preferred for fast-paced design decisions in an environment that
offers little or no tolerance for error (Forsythe and Ashby, 1994). For the development of
agile business practices, there needs to be consideration of human factors affecting decision
making within fast-paced, dynamic environments (Eisenhardt, 1989). Likewise, knowledge
of team dynamics, individual information requirements and information flow,
information management and utilization, and monitoring and assessment of the status of
complex, dynamic systems is needed (Forsythe and Ashby, 1996). These areas have received
considerable attention from human factors within military, space, aviation, and
traffic domains, but little attention within business contexts.

Of comparable importance to accomplishing the goals of agile manufacturing as product
design and manufacture is the corporate administrative and infrastructure support structure.
Support for the communications and information infrastructure is of particular concern.
System and software compatibility is essential to the seamless flow of product data through
the agile enterprise (Forsythe and Ashby, 1996).With major vendors on update cycles of
6 months or less, this compatibility cannot be maintained without the coordination and
empowerment of administrative and support staff. With agile manufacturing, integration
and networking of information technologies occurs at all levels of the enterprise. As a
consequence, the enterprise must address the support needs of a complex infrastructure
and the numerous human points of failure in supporting such an infrastructure (Haney
et al., 1994). When information does not flow, due to technical or human causes, agility
is lost. For this reason, elimination of human points of failure in infrastructure support is
essential (Forsythe and Ashby, 1996).

3

DESIGN OF ENABLING TECHNOLOGIES

Agile manufacturing is possible primarily as a result of recent and projected technical
innovations. Human factors has an important role to play: first in technology development,
and second in defining technology systems and their usage (Karwowski, 1994).
Use of computer-aided design and manufacturing (CAD and CAM) systems to electronically
represent product design is fundamental to agile manufacturing (Bertoline et al., 1995). Currently, CAD and CAM technologies are advancing at a phenomenal pace, with
alternative vendors in a race to keep up with each other. Unfortunately, users express
frequent discontent with the usability of these systems. Furthermore, capabilities of CAD
and CAM systems are underutilized, partially because they have not been fully integrated
into existing work practices (Forsythe and Ashby, 1996). As CAD and CAM systems, as
well as related product data managers, are implemented at the enterprise level, human
factors should ideally be incorporated into improved user interface designs. At a minimum,
human factors and usability should play a large part in benchmarking and similar
assessments of alternative commercial products.

4

MANAGEMENT OF THE INTRODUCTION AND FIELDING OF NEW
TECHNOLOGIES AND BUSINESS PRACTICES

The above issues are significant, but the most significant challenges posed by agile manufacturing
are sociotechnical (Forsythe and Ashby, 1996). If users are unwilling or reluctant
to accept agile business practices and enabling technologies, agile manufacturing
will fail from the inability to overcome the inertia of traditional, often deeply ingrained
practices. With little exception, businesses that will adopt agile manufacturing over the
next 5 to 10 years are currently designing and manufacturing products, with some success.
Agile manufacturing poses threats to the comfort of managers and line workers
alike. For management, there is a substantial relinquishment of power by the empowerment
of product development teams and the increased openness of information. For the
designer, much control is lost in the collaborative development of designs. At the level of
the line worker, there is increased, often undesired, responsibility in being brought into
the product development decision-making process, not to mention the threats posed by
computerization and automation of fabrication and assembly tasks. All of these threats
occur within an often stressful, fast-paced environment in which cognitively demanding
decision-making tasks replace most mundane, largely undemanding tasks.

P2.1 What is the difference between user interface design and user experience design?

P2.2 Why is designing a beautiful interface often insufficient in creating a useful system?

P2.3 What type of product or system might be best suited to Vee, plan-do-check-act, and Scrum
development cycles?

P2.4 What is the difference between the plan-do-check-act, and Scrumdevelopment cycles?

P2.5 Why are observations and interviews of users important for designers and engineers?

P2.6 Describe how the speed-accuracy tradeoff would lead you to apply different human factors
methods to a smart phone app versus a commercial airliner.

P2.7 Describe the difference between user-centered system design and the user designing the
system.

P2.8 Why are observations generally preferred over focus groups for front-end analysis?

P2.9 Give an example of how the lack of holistic, systems thinking could lead technology development
to have unintended consequences.

P2.10 What alternatives to evaluation would be preferable to a comprehensive test and evaluation
because they are less resource intensive?

P2.11 Explain why understand, create, and evaluate activities are described as a cycle.

P2.12 Discuss how post-release surveillance would be used differently for consumer products and
high-risk systems.

Questions for 2.2 Understanding Users, Context, and Tasks

P2.13 Describe the role of the Five Whys in understanding the role of human error in system
performance.

P2.14 How does the master-apprentice mindset influence how onemight observe and interview
people as part of a contextual inquiry?

P2.15 How does a task analysis help designers develop a deeper empathy for those they design
for?

P2.16 What are the three basic elements of a task analysis?

P2.17 How does the iterative nature of task analysis affect how you would organize your data
collection and interpretation?

Questions for 2.3 How to Perform a Task Analysis

P2.18 Why is it important to identify the focus and purpose of a task analysis before you begin?

P2.19 What four types of data are typically collected and how might they be more or less relevant
to designing the layout of machines in a factory? Compare that to designing news feed on a
smartphone.

P2.20 What data recording technique or techniques would you use to design a route planning and
navigation app for pizza delivery?

P2.21 Why is the critical incident technique particularly useful for understanding the tasks people
performin high-risk environments?

P2.22 What general limitation affects all task analysis data collection techniques?

P2.23 If the focus of your task analysis is on coordinating the communication of baristas for a local
coffee shop, what task summary approach would you use and why: Task hierarchy, task flow,
or task sequence?

P2.24 If the focus of your task analysis is on supporting decisions with a checklist, which task
summary approach would you use and why: Task hierarchy, task flow, or task sequence?

P2.25 If the focus of your task analysis is on training operators on the concepts needed to control a
nuclear power plant, which task summary approach would you use and why: Task hierarchy,
task flow, or task sequence?

P2.25 If the focus of your task analysis is on training operators on the concepts needed to control a
nuclear power plant, which task summary approach would you use and why: Task hierarchy,
task flow, or task sequence?

P2.27 What is a benefit of defining a persona compared to simply listing user characteristics?

P2.28 How do scenarios differ from use cases in the context of moving from task analysis to system
design?

Questions for 2.4 Iterative Design and Refinement

P2.29 Is it possible to create prototypes based on use cases and scenarios? Explain.

P2.30 Describe a specific analysis that goes beyond the development of personas and use cases to
address a particular issue.

P2.31 Describe the difference in the two types of understanding that guides prototype design: user
tasks and general human capabilities.

P2.32 Apply two of the design heuristics to the re-design of the instrument cluster of a car.

P2.33 How might a design pattern speed the design of the payment system for the website of car
rental company?

P2.34 How does a decision matrix help justify the selection of particular product features for
inclusion in a product?

P2.35 You are designing a website for a local real estate agent, describe how you would use wireframes,
mockups, and prototypes. Describe the primary audience and purpose of each.

P2.36 Identify two elements of a system beyond the obvious focus of a prototype that often merit
attention in design.

Y Informal evaluation guides
the iterative design, but formal
evaluation is needed
to ensure design objectives
have been met

2.6 Summary

In this chapter we described some of the techniques used to understand
user needs and to create systems to meet those needs.
Designers who skip the front-end analysis techniques that identify
the users, their needs, and their tasks risk creating technologycentered
designs that tend to fail. The techniques described in
this chapter provide the basic outline for creating human-centered
systems: develop an understanding of people’s needs through observation
and then test that understanding with prototypes that
can be quickly adjusted to better meet people’s needs. A critical
step in designing human-centered systems is to define the human
factors requirements. Many of these requirements depend on cognitive,
physical, and social considerations.

“Indifference towards people
and the reality in which they
live is actually the one and
only cardinal sin in design.”
(D. Rams) [46]

Additional Resources

One of the best resources for task analysis is Guidebook to Task
Analysis [53], which describes 41 differentmethods for task analysis
with detailed examples. For a more general set of designmethods
an excellent source is Universal Methods of Design: 100 Ways to
Research Complex Problems, Develop Innovative Ideas, and Design
Effective Solutions [54], as is Guide toMethodology in Ergonomics:
Designing for human use [41].

TaskArchitect is a computer-based tool for implementing some
of these tasks analysis methods (http://www.taskarchitect.com).
Human factors specialists usually rely on many sources of information
to guide their involvement in the design process, including
previous published research, data compendiums, human factors
standards, and more general principles and guidelines.

Data compendiums provide detailed information concerning
human factors aspects of system design. One example is the fourvolume
publication by Boff and Lincoln [55], Engineering Data
Compendium: Human Perception and Performance.

Human Factors design standards are another formof information
to support design. Standards are precise recommendations
that relate to very specific areas or topics. One of the commonly
used standards in human factors is the Human Engineering Department
of Defense Design Criteria Standard MIL-STD-1472G [56].
This standard provides requirements for areas such as controls,
visual and audio displays, labeling, anthropometry, workspace design,
environmental factors, and designing for maintenance, hazards,
and safety. Other standards include the ANSI/HFES-100 VDT
standard and the ANSI/HFES-200 ANSI/HFES 200 Human Factors
Engineering of Software User Interfaces [57].

Human Factors principles and guidelines provide more general
information than standards. Standards do not provide solutions
for all design problems. For example, there is no current
standard to tell a designer where to place the controls on a camera.
The designer must look to more abstract principles and guidelines
for this information. Human factors principles and guidelines
cover a wide range of topics, some more general than others.
Rams, Nielsen, and Tognazzini provide general principles for
design [46, 47, 48], and Van Cott and Kinkade provide human factors
guidelines for equipment design [58]. The following chapters
reference specific guidelines related to physical facilities medical
devices, and vehicle design.

Y Paper prototypes are more
of a tool to understand user
needs than an initial design
solution.

Early prototypes for software development are created by drawing
dialog boxes and other interface elements to create a paper prototype
as shown in the sidebar (Table 2.4). Paper prototypes of software
systems are useful because screen designs can be sketched,
thenmodified with little effort, making it possible to try outmany
design alternatives. For this reason, they are useful early in the
design process. Because paper prototypes are sketchy versions
of the system, users feel more open to identifying flaws. Paper
prototypes can even be created during the interviews and used as
props to clarify conversations with users. The main purpose of
paper prototypes is to guide interaction design and ensure that the
structure of the systemmeets the users’ needs.

After paper prototypes, wireframes are created, which are simple
layouts that show grouping and location of content, but which
omit graphics and detailed functionality. Wireframes are primarily
used to communicate with the design team (see Chapter 10 for
more details), and are helpful in documenting decisions and communicating
the essential interactions people might have with the
product. Wireframes lack details for the look and feel of the interface
or product; these are elements that are the focus ofmockups.
Mockups focus on the look and feel, and include color, font, layout,
and choices of the final product. Wireframes are limited to software
systems, but mockups are often created for hardware systems.
Wireframes communicate the system’s functional characteristics
to the design team, and mockups are used to communicate the system’s
physical features to the design teamand other stakeholders,
such as users and managers.

Building on wireframes andmockups, we create high-fidelity
prototypes so that users can experience elements of the final design.
Collecting information from these experiences leads to redesigning
the prototype. One analysis showed that user performance
improved 12% with each redesign iteration and that the average
time to performsoftware-based tasks decreased 35% from the first
to the final design iteration [51], while another analysis showed 20-
40% improvement per iteration [52]. This redesign and evaluation
continues for many iterations, sometimes as many as 10 or 20, or
more for complex products.

Rough, easily created and easily
changed paper prototypes invite
changes.
A refined, high-fidelity prototype
provides an experience that more
precisely matches that of the final
product.
Source: X. Lu and X.Mei. 4

Table 2.4 Paper prototype and highfidelity
prototype

To summarize, using paper prototypes, wireframes, mockups,
and prototypes in the design process has a number of advantages:

• Paper prototypes help understand user needs and if the early
design concepts meet those needs

• Wireframes communicate and document ideas for the design
team

• Mockups make ideas concrete to stakeholders and sponsors

• Prototypes support heuristic evaluation

• Prototypes support evaluation by giving users something to
react to and use

Beyond these specific uses, prototypes help build empathy
for the user by allowing designers to directly experience the use
of their system. However, simply using the prototype is often insufficient
for the designer to have the same experience as the actual
user because designers are often very different from the users.
One method for designers to have an experience thatmore closely
matches that of actual users is to use empathy suits. Empathy suits
can help a 30-year old feel what it might be like to be an 85-year old
to get into car, or what it is like to get into a car when nine months
pregnant.

Although this discussion has focused on software prototypes,
prototypes of hardware are equally important, as are prototypes of
new work processes. Important elements of the overall system design
that the prototyping process might neglect include the support
systems, such as instruction manuals, and the broader organizational
design. Human factors professionals can help design these
with a prototyping mindset, where they develop initial designs
based on a task analysis and understanding of human capabilities
and then evaluate and improve the designs in an iterativemanner
before a final version is delivered. Prototypes of support material,
such as manuals and help systems, and of the team and organizational
design are sometimes neglected if the team is too focused
on the physical and software elements of the system.

Dieter Rams (1932–) Highly inuential designer
who asked and answered the question,
What is good design [10]?
His answers to this question remains an important
touchstone for design thinking.
Source: Vitsoe, CC BY-SA 3.0. 5
Good design is innovative: The possibilities
for innovation are not, by any means, exhausted.
Technological development is always
oering new opportunities for innovative design.
But innovative design always develops in
tandem with innovative technology, and can
never be an end in itself.
Good design makes a product useful: A
product is bought to be used. It has to satisfy
certain criteria, not only functional, but also
psychological and aesthetic. Good design
emphasizes the usefulness of a product whilst
disregarding anything that could possibly
detract from it.
Good design is aesthetic: The aesthetic
quality of a product is integral to its usefulness
because products we use every day aect
our person and our well-being. But only wellexecuted
objects can be beautiful.
Good design makes a product understandable:
It claries the product’s structure. Better
still, it can make the product talk. At best, it is
self-explanatory.
Good design is unobtrusive: Products fullling
a purpose are like tools. They are neither
decorative objects nor works of art. Their
design should therefore be both neutral and
restrained, to leave room for the userâ€s
self-expression.
Good design is honest: It does not make a
product more innovative, powerful or valuable
than it really is. It does not attempt to manipulate
the consumer with promises that cannot
be kept.
Good design is long-lasting: It avoids being
fashionable and therefore never appears
antiquated. Unlike fashionable design, it lasts
many years—even in today’s throwaway
society.
Good design is down to the last detail:
Nothing must be arbitrary or le€ to chance.
Care and accuracy in the design process show
respect towards the user.
Good design is environmentally-friendly:
Design makes an important contribution
to the preservation of the environment. It
conserves resources and minimizes physical
and visual pollution throughout the lifecycle of
the product.
Good design is as little as possible: Less,
but better—because it concentrates on the
essential aspects, and the products are not
burdened with non-essentials.
Back to purity, back to simplicity.

Cost/benefit analysis builds on the QFD analysis, which calculates
the importance of features that best serve the user needs.
This importance serves as the input to cost/benefit analysis, which
compares different designs according to their costs relative to their
benefits. Costs and benefits can be defined monetarily or by the
9/3/1 rating scale. A decision matrix similar to Figure 2.6 can support
the cost/benefit analysis. The features are listed as rows on
the left side of a matrix, and the different design alternatives are
listed as columns. Each feature is given a weight representing importance
of the feature—the result of the QFD analysis. For the
features in Figure 2.6 this would be the total importance shown
in the bottom row of the decision matrix. Then, each design alternative
is assigned a rating representing how well it addresses
each feature. This rating is multiplied by the weighting of each
feature and added to determine the total benefit of a design. The
cost for each design is divided by this number to determine the
cost/benefit ratio. The design with the lowest cost/benefit ratio
represents the greatest value.

Tradeoff analysis identifies the most promising way to implement
a design. If multiple factors are considered (e.g., effort, speed,
and accuracy), design tradeoffs might be based on the design that
has the largest number of advantages and the smallest number of
disadvantages. Alternatively, a decision matrix can be constructed.
The matrix would assess how well systems, represented as columns,
compare according to the performance criteria, represented as
rows. For example, for the design of a new car, the performance criteria
of a key-less entry system could be represented in one row and
an existing key entry system could be another row. The columns
would be time to enter the car, likelihood of errors, and ease of use.

Although the decision matrix analyses can be very useful, they
tend to consider each product’s features independently. Focusing
on individual featuresmay fail to consider global issues concerning
the interactions of each feature on the overall use of the product.
People use a product, not a set of features—a product is more than
the sumof its features. Because of this, matrix analyses should be
complemented with other approaches, such as scenario specification
and user journeys, so that the product is a coherent whole
that supports the user rather than simply a set of highly important
but disconnected features. The overall objective of these analyses
is to identify a small set of the most promising alternatives
for implementing in prototypes for further evaluation. Chapter 7
on decision making provides a more detailed discussion for the
strengths and weaknesses of decisionmatrix analysis.

Once the front-end analysis has been performed, the designers
have an understanding of the user’s needs. This understanding
can be used to identify initial system specifications and create
prototypes. Creating prototypes depends on two types of understanding:
understanding tasks and understanding general human
capabilities. Prototypes must support user tasks in a way that is
consistent with how people see, hear, feel, comprehend and act on
the world. This understanding is often distilled into principles or
design heuristics and Chapters 4 through 18 describe these in detail.
As initial prototypes are developed, the design team begins to
characterize the product in more detail, and evaluates how people
respond to the evolving product. The human factors specialist usually
works to ensure that the tasks people will performfall within
the limits of human capability. In other words, can people perform
the tasks safely and easily with the proposed system?

Design Heuristics
Create useful innovation: Address a
need, solve a problem (Chapter 2).
Attend to details: Small changes to
the design can have a big effect on
people.
Simplify: Remove irrelevant information,
but do notmask essential
indicators and feedback (Chapter 8).
Honest and understandable: Functions
should be reflected in forms
that make their states visible,
changes predictable, and interactions
intuitive (Chapter 4, 9, and 11).
Provide flexibility: People should be
able to adjust, navigate, undo and
redo, adopt shortcuts (Chapter 10,
12).
Consistency: The same label or action
should mean the same thing
in different situations—don’t deviate
from well-defined conventions
(Chapter 6, 10 and 11).
Anticipate needs: Provide options
rather than require people to recall
them. Choose thoughtful defaults
because people often adopt initial
settings (Chapter 6 and 10).
Minimizememory demands: Interactions
with technology should not
disrupt the flow of activities unless
necessary (Chapter 6).
Consider adaptation: Adopt a systems
perspective to identify otherwise
unanticipated outcomes,
particularly as people adapt to the
changes in the system (Chapter 18).
Fit the task to the person rather
than the person to the task

Table 2.3 General design heuristics

2.4.1 Providing Input for System Specifcations

Design heuristics help human factors professionals provide design
teams with quick input on whether the design alternatives are
consistent with human capabilities. Design heuristics or principles
provide a response that is grounded in years of design practice
and much research on human behavior. Table 2.3 shows 10
design heuristics derived from those of Rams, Nielsen, and Tognazzini
[46, 47, 48]. The table also shows the chapters that contain
the specific information on cognitive, physical and organizational
characteristics that underlies these heuristics.

These heuristics suggest promising design alternatives, but
are not simple rules that can be applied without thought. In fact,
in many instances, the heuristics conflict. As an example, providing
flexibility is needed to accommodate differences between
people and environments, such as hearing impairment making it
impossible to define an optimal volume setting. At the same time,
providing too much flexibility burdens the user with finishing the
design, and might lead to the user designing a poor system. As
a consequence, the 11 heuristics are not a set of strict rules, but
instead should be thought of as a checklist to provoke conversation.
Chapter 3 describes how these and other principles can be
used as part of heuristic evaluations and cognitive walkthroughs.
In a heuristic evaluation, you assess whether the system design
is consistent with the heuristics. To apply any of these principles
effectively requires an understanding of the underlying human characteristics described in forthcoming chapters, as indicated for
each heuristic.

Design patterns are solutions to commonly occurring design
problems and are most typically associated with software, but also
apply to physical systems. A number pad for a phone is an example
of a design pattern. Using design patterns, such as a conventional
number pad for entering phone numbers, has the benefit of eliminating
many design decisions. It is also likely to present people
with a familiar interaction that is consistent with other systems they
might use. Design patterns for user interfaces include common
ways of navigating multi-page applications, getting user input, and
browsing data (http://ui-patterns.com). Design patterns provide
a ready-made shortcut to the final design, but should be carefully
assessed to determine whether previously developed patterns fit
the current situation.

Human Factors requirements and system specifications need
to be defined where design patterns don’t fit the current situation.
These requirements include the system characteristics that are
needed to achieve the desired levels of safety, performance, and satisfaction.
For software design, human factors requirements might
include error recovery, or the ability to support people performing
more than one task at a time. As an example, for an ergonomic
keyboard design,McAlindon [49] specified that the new keyboard
must eliminate excessive wrist deviation, eliminate excessive key
forces, and reduce fingermovement. The design that resulted from
these requirements was a “keybowl” that is drastically different
from the traditional QWERTY keyboard currently in use, but a design
that satisfied the ergonomic criteria.

Identifying system requirements is a logical extension of the
task analysis that draws on the task data to specify (1) the overall
objectives of the system, (2) performance requirements and
features, and (3) design constraints. The challenge is to generate
system specifications that identify possible features and engineering
performance requirements that best satisfy user objectives and
goals. The objectives should be written to avoid premature design
decisions. They should describe what must be done to achieve the
user’s goals, but not how to do it. The system objectives should
reflect the user’s goals and not the technology used to build the
system.

Detailed human factors requirements
are most critical
to the Vee design approach

After the objectives, designers determine the means by which
the system will help the user achieve the goals. These are termed
performance requirements and features. The features define what
the system will be able to do and under what conditions. The
performance requirements and system features provide a design
space in which the team develops various solutions.

Finally, in addition to the objectives and system features, the
specifications document identifies design constraints, such as
weight, speed, cost, abilities of users, and so forth. More generally,
design constraints include cost,manufacturing, development time, and environmental considerations. The constraints limit possible
design alternatives.

Translating the user needs and goals into system specifications
requires the human factors specialist to take a systems thinking
approach, analyzing the entire systemto determine the best configuration
of features. The focus should not be on the technology
or the person, but on the person-technology system as a unit. The
systems design approach draws upon several tools and analyses,
that we highlight in the following discussion.

Quality Function Deployment can help prioritize systemfeatures.
How does the human factors specialist contribute to writing
the system specifications? He or she compares the system features
with user characteristics, activities, environmental conditions, and
especially the users’ preferences or requirements. This ensures
that the design specifications meet the needs of users and avoids
adding features that people do not want. Human factors designers
often use a simple yet effective method for this process known as
the QFD (quality function deployment), which uses the “house of
quality” analysis tool [50]. This tool uses a decision matrix to relate
user needs to system features, allowing designers to see which

features will satisfy user needs.

specifications.
User identification and persona development describes the
most important user populations of the product or system. For
example, designers of amore accessible ATMmight characterize
the user population as people ranging from teenagers to senior
citizens, having at least a third-grade English reading level, and
a range of possible physical disabilities. After identifying characteristics
of the user population, designers should also specify the
people who will be installing or maintaining the systems.

It is important to completely describe the potential user population.
This usually includes characteristics such as age, gender,
education level or reading ability, physical size, physical abilities (or
disabilities), familiarity with the type of product, and task-relevant
skills. For situations where products or systems already exist, one
way that designers can determine the characteristics of users is to sample the existing population of users. For example, the ATM designer
might measure the types of people who currently use ATMs.
However, this will result in a description of users who are capable
of using, and do use, the existing ATMs. This is not an appropriate
analysis if the goal is to attract, or design for, a wider range of users.

A simple list of user characteristics often fails to influence design.
Disembodied user characteristics may result in an “elastic
user” whose characteristics shift as various features are developed.
Designing for an elastic user may create a product that fails to satisfy
any real user. Cooper [28] developed the concept of personas to
represent the user characteristics in a concrete and understandable
manner. A persona is a hypothetical person developed through
interviews and observations of real people. Personas are not real
people, but they represent key characteristics of the user population
in the design process. The description of the persona includes
not only physical characteristics and abilities, but also the persona’s
goals, work environment, typical activities, past experience,
and precisely what he or she wishes to accomplish. The persona
should be specific to the point of having a name.

Sequence diagrams help define personae by identifying roles,
tasks, and communications. The task hierarchy specifies goals
and motivations. For most applications, three or four personae
can represent the characteristics of the user population. Separate
personae may be needed to describe people with other roles in
the system, such as maintenance personnel. The personae exist
to define the goals that the system must support and describe
the capabilities and limits of users in concrete terms. Personae
describe who the design is for and act as the voice of the user,
preventing the natural tendency of the design team to assume
users are like themselves.

Because personae define several “typical” users, this method
runs the risk of neglecting the extremes, such as the 5th and 95th
percentiles of a population. Techniques to systematically accommodate
these extremes are discussed in the context of fitting designs
to the physical dimensions of people in Chapter 12.

Design Exercise:
Stanford wallet project
This exercise provides a brief exposure
to design thinking. The goal is to
practice designing and recognizing a
user’s need and then translating that
need into a protype product that is
then evaluated.
This exercise is an abbreviated
version of the StanfordWallet
Project [45]. Depending on time,
each step can be completed in less
than five minutes.
Step 1 Everyone. Design/draw the
ideal wallet.
(Recognize that your perfect wallet is
not perfect for others.)
Step 2. Pair up in teams of two. Person
1 acts as the designer. Person 2
acts as the user.
Step 3. Show NOT tell. Designer asks
user about the ideal wallet. Some
questions to consider: What should
it look like? How should it feel? What
should it be able to hold? How do you
want to carry it? What functions do
you want?
Step 4. Switch roles. Person 1 is now
the user. Person 2 is now the designer.
Step 5. Continue Show NOT tell.
Repeat Step 3.
Step 6. Self reflection Designer explains
the users needs in one sentence:
“[user] needs a way to [user’s
needs] because... (or “but”... or “surprisingly”...)

Table 2.2 Stanford design exercise

Scenarios, user journeys, and use cases complement personas.
Personas are detailed descriptions of typical users and scenarios
are stories about these personas in a particular context. Scenarios,
also termed user journeys, describe situations and tasks relevant
to the use of the system or product being developed. Scenarios
are a first step in creating the sequence of screens in software development,
and they also define the tasks users might be asked to
complete in usability tests. In creating a scenario, tasks are examined,
and only those that directly serve users’ goals are retained.
Two types of scenarios are useful for focusing scenario specification
on the design. The first is daily use scenarios, which describe
the common sets of tasks that occur daily. In the car example, this
might be the sequence of activities associated with entering the car
when parked in the owners’ garage. The second is necessary use
scenarios, which describe infrequent but critical sets of tasks that must be performed. In the car example, thismight be the sequence
of activities associated with entering the car during a snowstorm.
Scenarios can be thought of as the script that the personae follow
in using the system[28].

Scenarios typically support conceptual design, where the general
activities of people are described independent of technology.
Use cases help move from conceptual design to prototypes. Use
cases are a user-centered description of what the technology is
meant to do. At the simplest level, a use case is a sequence of tasks
that produce ameaningful outcome, such as entering and starting
a car. These tasks can be described in a more formal way in a flow
diagram and implemented in a software or hardware prototype.

Observations organized in task hierarchies, flows, and sequences
help define personae and scenarios. Personae and scenarios, in
turn, help define new task hierarchies, flows and sequences that
the new design will make possible. As an example, a flow diagram
associated with the new system, such as a keyless entry system
for a car, would document the intended interactions between the
person and new system. Often it is possible to use scenarios, use
cases and personae to create prototypes. Personae and scenarios
also provide a starting point for more specific task analysis, such
as those that focus on the environment, workload, safety, and automation.
The type of analyses needed depends on the scope of
the design and the particulars of the system.

Environment and context analysis describes where the tasks,
scenarios, and personae live. For example, if ATMs are to be placed
indoors, environmental analysis would include a somewhat limited
set of factors, such as type of access (e.g., will the locations be
wheelchair accessible?), weather conditions (e.g., will it exist in a
lobby with outdoor temperatures?), and type of clothing people will
be wearing (e.g., will they be wearing gloves?), issues considered
in more detail in Chapter 14 where we discuss the physiology of
work. Beyond the physical environment, the culture and norms of
workplace should be considered as discussed in Chapter 18.

Once task-related information has been collected, it must be organized,
summarized, and analyzed. At the simplest level, the task
analysis might be summarized as a list of challenges faced by people.
As an example, observing drivers trying to unlock their cars
during aWisconsin winter could show how fumbling for keys might
threaten drivers with frostbite. Sometimes these challenges can
inspire important innovations, but often a more detailed analysis
of tasks is needed to identify solutions and to avoid unintended
consequences. Some of the most common ways to organize task
data include:

1. Task hierarchy: Goal, task, subtask decomposition

2. Task flow: Control, decisions regarding the flow from one
task to another

3. Task sequence: Task duration and sequence, as well as communication
between system components

Task hierarchy data can be shown as an arrangement of tasks
where tasks are broken into more specific subtasks. Goals are at the
top of the hierarchy and the tasks at the bottom represent detailed
actions needed to accomplish those goals. The tasks higher in the
hierarchy are why the ones below are performed, and the tasks
lower in the hierarchy describe how the tasks above are achieved.
A task hierarchy makes it possible to organize a complex array of
many actions into a few general tasks, linking a detailed description
to a more general description.

Figure 2.3 shows a task hierarchy for unlocking a car prior to
driving. General tasks, such as “Car entry” are broken into more
specific tasks, such as “Unlock car” and “Open driver side door.”
These are further broken into very specific actions. For “Unlock car”
thismight include “Find key and key fob”, “Press Open on key fob,”
and so on. The level of task hierarchy should be aligned with the
purpose of the analysis and should avoid unnecessary detail. Figure
2.3 shows how the purpose of the analysis focuses attention on
describing unlocking the car and so the “Buckle seatbelt” activity
is not broken into subtasks.

A task hierarchy prompts innovation by identifying different
ways of achieving the same overall task with different subtasks. For
example, considering different ways you can perform“Car entry”
may lead to the use of a smart phone rather than keys. A task hierarchy
also makes it easy for the analyst to develop spreadsheets to
record information for each task or subtask. The spreadsheet contains
a row for each task, and columns for information describing
the tasks. This information might include task duration, conditions
thatmust be met to performthe tasks, why the task is difficult, common
errors, strategies, skills, or knowledge. One simple analysis
that might be part of a time-motion study is to use a spreadsheet
to calculate total task time and compare it with a new design with
different tasks

Figure 2.3 Task hierarchy for driving a car with a focus on unlocking the
door.

Task flow data from one task to another is captured by a flow
chart. Activity diagrams build on flow charts and also show tasks
that are performed concurrently (Figure 2.4). This diagramshows
the flow from one task to another and the decision points that
determine which task should follow. The flow from one task to
another can be sequential, shown as an arrow connecting tasks,
which are shown as rounded rectangles. The flow can also be
branched, where the decision to flow to one task or another is
indicated by a diamond. The flow can also be concurrent where
tasks can occur in parallel, which is indicated by a set of tasks
bounded on the top and bottom by horizontal bars.

Figure 2.4 An activity diagram shows task flow for entering and starting
a car.

Figure 2.4 shows an activity diagram for the entering and starting
the car. It begins with unlocking the door and finishes when
the settings are adjusted and the engine is started. The diamond
after the unlock door indicates that the door can be opened only
if the correct key is turned and loop from the diamond back to
unlock door indicates that keys are tried until the correct one is
inserted. After opening the door, the horizontal bar indicates that
the settings can be adjusted and the engine can be started in any
order.

Activity diagrams highlight decisions and the information required
to make them. Sometimes this information is trivial and
built into the interaction, such as the resistance experienced with
the wrong key is used to open a car door. In other situations, identifying
the cues that guide decisions can specify critical information
for an interface. Activity diagrams also indicate mandatory
ordering of tasks that need to be conveyed through the physical
configuration of the device, the interface, or through instructions.
In the case of a car, the physical configuration of the door and
key makes it impossible to start the engine before opening the
door. Positive locking of the door by a keyfob only outside the car
makes it impossible to lock ones keys in the car. Beyond inspecting
these diagrams, it is possible to “run” these diagrams as a computer
simulation and estimate the time it takes to performthese tasks.

Y Activity diagrams capture task
flow and information.
Sequence diagrams capture
task sequence and timing.

Task sequence data are shown in sequence diagrams that show
the order and duration tasks for each object and person in the
system. The activity of each person and object is represented as a
timeline that runs from the top of the diagram to the bottomand
rectangles on this timeline indicate when the person or object is
active in responding to other elements of the system. Horizontal
arrows indicate communication between people and objects. Solid
lines and arrows indicate synchronous messages, where a response
is required before other activities can proceed. Dashed lines with
open arrows indicate asynchronous messages, where activities can
proceed without a response. Responses are indicated by dashed
lines with open arrows at the of a rectangle.

Figure 2.5 Sequence diagram for unlocking and entering a car.

Figure 2.5 shows a sequence diagram for entering and turning
on a car. The left-most timeline begins with the driver inserting
a key into the door. The next timeline indicates the feedback provided
by the door that the door is unlocked. The driver then inserts
the key into the ignition and tries to start the car, which is indicated
by the engine being on. This figure shows a sequence diagram,
which focuses on one of many particular sequences of tasks. It
shows the sequence associated with inserting the correct key and
not what would have happened with the incorrect key or using
a key fob. Fault tree analysis (see Chapter 16 on system safety)
directly addresses how the probability of failure associated with
many tasks combine to influence safety. Sets of tasks with many
decision points are better represented by activity diagrams than by
sequence diagrams.

Sequence diagrams highlight communication, particularly the
responses that provide people with feedback regarding the success
or failure of their actions. Interaction design should ensure clear
and timely feedback. Diagrams that have manymessages that cross
several timelines indicate a need to reorganize and simplify the
communication so that each person or object communicates with
neighbors and that messages only occasionally cross timelines.
One way to avoid messages crossing multiple timelines is to ensure
that each object and person has a clear role that describes how it is
responds to messages. High activity for one person and low activity
for another might indicate workload that could be balanced by
adjusting the roles of each.

1. Observations and questions of people as they use an existing
system

2. Retrospective and prospective verbal protocol analysis

3. Unstructured and structured interviews including focus groups

4. Surveys and questionnaires

5. Automatic data recording

Observation involves watching users as they interact with existing
versions of the product or system. This is one of themost useful
data collection methods. If we were interested in car design, we
would find drivers who represent the different types of people the
car is to be designed for and then observe how they use their cars.
People are asked to performthe activities under a variety of typical
scenarios, and the analyst observes the work, asking questions as
needed. Observation should be performed in the environment that
the person normally accomplishes the task (See Table 2.1).

Source: Wikimedia Commons. 3
Probe questions for theMaster/Apprentice
observation approach:
Who and what is needed to perform
the task?
What happens before, what after?
What does the task change, how is
this detected?
What has to be remembered?
What is the consequence of failure to
complete the task?
When in the day, and when relative to
other events, is the task performed?
How do people communicate and
coordinate their activity?

Table 2.1 Observing people to guide
design.

The meaning behind users’ tasks is often revealed in their
thoughts, goals, and intentions, and so observations of physical
activity may not be sufficient to understand the tasks. This is particularly
true with primarily cognitive tasks that may generate little
observable activity. In such cases, it can be useful for people to
think out loud as they performvarious tasks. One approach is to
adopt a master-apprentice relationship, where the observer acts
as an apprentice trying to learn how the user performs tasks [39].
Adopting this relationship makes it easy for observers to ask questions that help users to describe their underlying goals, strategies,
decisions

Retrospective and prospective protocol analysis address important
limits of direct observations. Direct observations disrupt
ongoing activity or they can fail to capture rarely occurring situations.
For example, trying to understand how people deal with
being lost as they drive would be difficult to observe because talking
to the driver could be distracting and the analyst would have to
ride with the driver for many trips to observe the rare case that they
get lost. Talking about tasks is termed verbal protocol, and retrospective
verbal protocols require that people describe past events
and prospective verbal protocols require that people imagine how
they would act in future situations

Video recordings of users’ activity are an effective way to prompt
retrospective protocols. The human factors specialist and user can
watch the video together and the users can describe what they
were thinking as they performed the tasks. The human factors specialist
can pause the playback and ask probe questions. Because
users do not have to performthe task and talk about it at the same
time retrospective protocols can be easier on the user. Retrospective
protocols can even yield more information than concurrent
protocols.

Structured and unstructured interviews involve the human
factors specialist asking the user to describe their tasks. Structured
interviews use a standard set of questions that ensure the interview
captures specific information for all interviewees. Unstructured
interviews use questions that are adjusted during the interview
according to the situations. The analyst might ask about how the
users go about the activities and also about their preferences and
strategies. Analysts should also note points where users fail to
achieve their goals,make errors, show lack of understanding, and
seem frustrated or uncomfortable.

Unstructured interviews use probe questions similar to those
used for direct observation. These questions address when, how,
and why a particular task is performed, as well as the consequences
of not performing the task. Critical incident technique is a particularly
useful approach for understanding how people respond to
accident and near accident situations in high-risk systems. Because
accidents are rare, direct observation is not feasible. With
the critical incident technique, the analyst asks users to recall the
details of specific situations and relive the event. By reliving the
event with the user, the analyst can get insights similar to those
from direct observation [42].

Critical incident technique
makes it possible to “observe”
past events.

Focus groups are interviews with small groups of users, rather
than individuals [43, 44]. Focus groups typically consist of between
six and ten users led by a facilitator familiar with the task and
system. The facilitator should be neutral with respect to the outcome
of the discussion and not lead the discussion to a particular
outcome. One benefit of focus groups is that they cost less than individual interviews because they require less time for the analyst.
Also discussion among users often draws out more information
because the conversation reminds them of things they would not
otherwise remember.

Observations are typically more valuable than interviews or
focus groups because what people say does not alwaysmatch what
they do. In addition, peoplemay omit critical details of their work,
they may find it difficult to imagine new technology, and theymay
distort their description to avoid appearing incompetent. It is often
difficult for people to describe how they would perform a given
task without actually doing it—describe how you tie your shoes
without being able to touch and see your shoes.

Surveys and questionnaires are typically used after designers
have obtained preliminary descriptions of activities or basic tasks.
Questionnaires are often used to affirmthe accuracy of the information,
determine the frequency with which various groups of
users performthe tasks, and identify any user preferences or biases.
These data help designers prioritize different design functions or
features. See Chapter 3 for a more complete discussion of surveys
and their limits.

Automatic data recording uses smartphones and activity monitors
to record people’s activities unobtrusively. An example of such
a system is a data logging device that records the position of the
car, its speed, and any hard braking or steering events. Such a
device can show where drivers go, when they choose to travel, and
whether they travel safely. Such data has the benefit of providing a
detailed and objective record, but it lacks information regarding the
purpose behind the activity. This limitation might be avoided by
using automatically recorded data to prompt retrospective verbal
protocol.

Limitations of data collection techniques. All methods have
limits, but combinations of the methods can compensate. A more
general limit is that all of these methods document existing behavior.
Designing to support existing behavior means that new
controls, displays, or other performance aids might simply enable
people to do the same tasks better, butmight not produce dramatic
innovation. Innovation requires the analysis to focus on underlying
goals and needs, and identify different ways of accomplishing
these goals.

1. Define the purpose and identify the required data

2. Collect task data

3. Interpret task data

4. Innovate from task data

We describe this process as sequential, but in practice it is often
iterative. As an example a hierarchical task diagrammight be drawn
during an interview andmight be revised and adjusted as part of
the interview process. This diagram might be further refined based
on observations of work being performed. The observations and
analysis that make up a task analysis help focus on the activity
details that matter for the user. A deep, empathetic, and obsessive
understanding of these details is what makes designs succeed.

2.3.1 Step 1: Dene Purpose and Required Data

The first step of task analysis is to define the design considerations
that the task analysis will address. Because a task analysis can be
quite time consuming, it is critical to clearly define the purpose of
the analysis. Typical reasons for performing a task analysis include:

• Redesigning processes

• Identifying software and hardware design requirements

• Identifying content of the human-machine interface

• Defining procedures, manuals, and training

• Allocating functions across teammates and automation

• Estimating system reliability

• Evaluating staffing requirements and estimating workload

As an example, a task analysis for entering a car, adjusting settings,
and starting the engine could provide important information
to re-imagine the car key and create a new system for entering and
selecting vehicle settings. The task analysis could also be used to
define features of the interface to adjust settings, and could even
be used to define the content of the owner’s manual.

Both the purpose of the analysis and the type of task will influence
the information gathered. Tasks can be physical tasks, such as
opening the car door, or they can be cognitive tasks, such as selectingmusic
to listen to while driving after entering the car. Because
an increasing number of jobs have a large proportion of cognitive
tasks, the traditional task analysis is being increasingly augmented
to describe the cognitive processes, skills, strategies, and use of
information required for task performance. There are methods
specifically developed for cognitive task analysis [40, 41], but we
will treat these as extensions of standard task analyses, referring
to all as task analysis. However, designers should pay particular
attention to the cognitive components in conducting the analysis
if any of the following characteristics are present:

• Complex decision making, problem solving, diagnosis, or
reasoning

• Conceptual knowledge is needed to performtasks

• Large and complex rule structures that are highly dependent
on the situation

• Performance depends on memory of information that needs
to be retained for seconds or minutes

Whether the task analysis is focused on the physical or cognitive
aspects of the activity, four categories of information are typically
collected:

• Hierarchical relationships: What, why, and how tasks are
performed

• Information flow: Who performs the task, with what indications,
and with what feedback

• Sequence and timing: When, in what order, and how long it
takes to performtasks

• Location and environmental context: Where and under what
physical and social conditions tasks are performed

Hierarchical relationships

describe how subtasks combine
into tasks, and how tasks combine to achieve people’s goals. With
the car example, a goal is to keep the car secure, a task is to lock
the door, and subtasks include inserting the key, turning the key, or
press the lock icon on a keyfob. Moving up the hierarchy describes
why a task is performed—secure the car—and moving down the
hierarchy describes how a task is performed—turn the key/click
lock on keyfob. Describing the hierarchical relationships between
goals, tasks, and subtasks makes the reason for the many subtasks understandable. These hierarchical relationships help us focus
on the underlying goals of people and can prompt innovations by
identifying new and more efficient ways of achieving people’s goals,
such as securing the car using a keyfob rather than a key.

Y Hierarchical relationships
identify new ways of doing
things.

Information flow

specifies the communication between people
and the interactions between people and technology. This
information flow can also include stored information needed to
complete the task, such as knowledge and skills or information
on a display. With the car example, the flow of information might
include a signal to identify that the doors are unlocked. For some
systems, a complex network of people and automation must be
coordinated. In other systems, it may be only a single person and
the technology. However,most systems involve coordination with
multiple people. Defining their roles and their information needs
often identifies important design considerations that might otherwise
go unnoticed, such as how to get passenger preferred music
into the car.

Y Information flow helps specify
interface content.

Sequence and timing

describe the order and duration of tasks.
In the car example, the driver must first turn the key, then lift the
door handle, and finally pull the door open. Performed in a different
order, these tasks would not achieve the goal of opening the
door. In other situations, tasks could be performed in parallel. Task
sequence often determines howmuch time a set of tasks will take
to complete. Specific task sequence information includes the goal
or intent of task, sequential relationship (what tasks must precede
or follow), trigger or event that starts a task sequence, results or
outcome of performing the tasks, duration of task, number and
type of people required, and the tasks that will be performed concurrently.
Eliminating tasks, reducing their duration, or assigning
them to different people can make systems easier to use and more
efficient.

Sequence and timing specifies
efficient interactions.

Location and environmental context

describe the physical
and social world in which tasks occur. In the car example, important
location information might be the physical layout of the
vehicle interior that can make it difficult to insert the key to start
the car. Specific location information might include places people
work and the paths people take from one place to another, as
well as the location of equipment and the physical barriers such as
walls and desks. Location of equipment can greatly influence the
effectiveness of people in production-line settings. The physical
space can also have a surprisingly large effect on computer-based
work, as anyone who has had to walk down the hall to a printer
knows.

Physical layout can strongly
affect task difficulty.

1. Who are the users? This includes not only users in the traditional
sense, but also the people who will install, maintain,
monitor, repair, and dispose of the system.

2. Why do users need the product and what are their preferences?

3. What are the environmental conditions under which the
system or product will be used?

4. What is the physical and organizational context of the users’
activity?

5. What major functionsmust be fulfilled by a person, team, or
machine?

6. Whenmust tasks occur, in what order, and how long do they
take?

Y “If you want to improve a
piece of software all you have
to do is watch people using it
and see when they grimace,
and then you can fix that.”
D. Kelley [38]

These questions are answered with various analyses that begin
by collecting data, often by observing and talking with people.
These data are then analyzed to support design decisions. We use
the term task analysis to describe this process, which can vary
substantially in its level of detail. In general, the more complex and
critical the system, such as air traffic control, the more detailed the
task analysis. It is not unusual to spend several months performing
this analysis for a complex product or system.

Although direct observation is the primary technique for collecting
information about tasks and activities, it is not always the
most effective. Accident prevention is a major goal of the human
factors profession, especially as humans are increasingly called
upon to operate large and complex systems. Although human
factors experts rarely observe accidents directly these critical incidents
can be analyzed to determine the underlying causes. In
Chapter 1 we discussed how Fitts and Jones interviewed pilots after
crashes and near crashes to identify opportunities to improve aircraft
design. Accident analysis has pointed to several cases where
poor system design has resulted in human error. As an example, in
1987 Air Florida Flight 90 crashed into the 14th Street Bridge on the
Potomac River shortly after taking off fromWashington National
Airport, killing 74 of the 79 passengers and crew. Careful analysis of
the events leading up to the crash identified training and decision
errors that led the pilots to take off even though snow and ice had accumulated on the wings. Accidents usually result from several
coinciding breakdowns, and so identifying human error is the first
and not the last step in understanding accidents.

The “Five Whys” help identify
the multiple causes of
accidents.

Practicing the “Five Whys” by tracing back the causes of an
event by asking “why” at least five times can be useful in going beyond
human error as the cause of an accident. For the Air Florida
Flight 90, this might mean asking why the aircraft had inadequate
lift? (ice accumulated on the wings). This leads to the question:
why did the aircraft take off with ice on its wings? (aircraft accumulated
ice as it waited in taxi line for 49 minutes before takeoff),
which then leads to the questions: why did the pilots decided not to
return for de-icing? (production pressure and lack of experience),
why did the pilots did not notice the severity of the icing problem as
they began taxied for takeoff? (captain failed to address concerns
of first officer). These series of questions typically show multiple
unsafe elements associated with training, procedures, controls and
displays, that should be considered before rather than after an accident.
This requires a proactive approach to system safety analysis
rather than a reactive one such as accident analysis or accident
reconstruction. This safety analysis is one particular example of
understanding users, their operating environment, and the tasks
they must performand is addressed in greater detail in Chapter 16,
where we discuss system safety.

In contrast to accident analysis, which typically focuses on systemsafety,
time-motion studies developed by Taylor (Chapter 1), focuses
on improving performance of manual work. Taylor’s detailed
observations dramatically improved steelworker productivity by
precisely recording the movements and timing of actions of workers
on assembly lines. These time-motion studies continue to be an
important technique to improve efficiency and avoid injury associated
with manualmaterials handling, which we discuss in more
detail in Chapter 13 (biomechanics). Human factors engineers can
incorporate knowledge gained in time-motion studies to understand
user needs, the context that the work is to be conducted, and
the sequences of tasks.

Time-motion studies identify
ways to improve worker
efficiency

Understanding users’ needs for computer systems and consumer
products often benefits from an approach termed contextual
inquiry [39]. Contextual inquiry provides an understanding
of users’ needs by going to users’ workplace or wherever the systemwould
be used, and adoptingmaster-apprentice relationship.
The interviewer acts as an apprentice learning from the master
regarding how to perform a particular activity. As an apprentice,
the interviewer asks the master to verify his or her understanding
by commenting on task descriptions and prototypes, as simple as
a series of sketches to show screens of a potential application.

Y Contextual inquiry reveals
user needs through careful
observation.

Understanding tasks is essential, whether the goal is to prevent
accidents, make assembly lines more efficient, or create delightful
products. This understanding goes beyond simply documenting
activity, but involves establishing a deep empathy for the user.
Without some formof front-end analysis, designers and engineers often find it hard to create systems that serve peoples’ needs effectively.
Depending on how the data are collected and analyzed,
the methods take on many names, but they all aimto understand
users, their environment, and the tasks they must perform. Here
we describe them under the general termof task analysis.

The overriding principle in human factors engineering is to center
the design process around people, thus making it a humancentered
design process [32]. In other words “honor thy user.” The
human-centered design of a system or product revolves around
the users. It mustmeet their needs and be compatible with their
abilities [33].

We put this principle into practice involving users in all stages
of the design process. That is, the human factors specialist will
study the users’ job or tasks, elicit their needs and preferences,
ask for their insights and design ideas, and collect data on their
response to design solutions. User-centered design does not mean
that the user designs the product. The goal of the human factors
specialist is to find a systemdesign that supports the user’s needs
rather than designing a system to which users must adapt.

A holistic perspective or systems thinking is an important part of user-centered design. Rather than considering elements of the
design as independent, unrelated parts, systems thinking focuses
on the interaction and relationships of parts—a focus on the whole
rather than just the parts. Such holistic thinking can identify important
benefits for integrating elements of a device, such as making it
possible to place a call from a smartphone by touching (rather than
dialing) a number on a webpage. Systems thinking can also avoid
unintended consequences. For example, shortening the shifts of
healthcare workers to reduce fatigue and improve healthcare quality
can have the unintended consequence of increasing the need
to handoff a patient from one healthcare worker to another. More
handoffs can undermine healthcare quality [34].

major phases: understand users, create a prototype, and evaluate
the prototype [35]. Understanding users involves careful observations
of people and the tasks they perform to the point of
establishing empathy for their situation. Creating a prototype involves
designers combining this understanding with a knowledge
of human characteristics, interface guidelines, and principles of
human behavior, which we will discuss in later chapters, to produce
initial design concepts. Soon after these initial concepts are
developed, designers evaluate these prototypes. Evaluating can
include heuristic evaluations and usability tests with low-fidelity
mock-ups or prototypes. Usability tests are particularly useful because
they often help designers better understand the users and
their needs. This enhanced understanding provides input to the
next cycle of creating an improved prototype.

Figure 2.2

The Understand, Create, and Evaluate cycle describes design
as an iterative cycle that repeatsmany times at multiple time scales.

Figure 2.2 shows the cycles of the design process, gravitating
from inside to out, as the prototype evolves into a final product.
The cycles vary in how long they take to complete, with the outer
cycles taking months or years and inner cycles taking minutes. In
the extreme, onemight complete a cycle during an interview with
a user where the designer creates a simple paper prototype of a possible solution, and the user provides immediate feedback. Taking
hours rather than seconds, a heuristic evaluation, where the
design principles and guidelines are applied to the prototype, can
quickly assess how design might violate human capabilities. Usability
tests typically take days or weeks to collect data from how
end users respond to the system, and so provide amore detailed
and precise understanding of how people will react to a design.
The inner elements of the design cycle provide rapid, but approximate
information about how a particular design might succeed
in meeting people’s needs, and the outer elements of the cycle
are more time consuming, butmore precise. This speed-accuracy
tradeoff means that the time and resources needed to understand,
create, and evaluate should be matched to the system being developed.
Rapidly changingmarkets place a premiumon fast and
approximate methods.

Y Human factors experts conduct
heuristic evaluations
and involve no end users; usability
tests collect data from
end users.

Usability tests are conducted multiple times as the interface
design goes through modifications. Each repetition of the testing
and modification cycle can produce significant improvements, and
many iterations of design should be expected. At the beginning,
it is not necessary to worry about the details of screen design or
making the screens look elegant. Rather, the emphasis should
be on identifying useful functions, and how the user responds
to those functions. This iterative process has been shown to be
incredibly valuable in refining software, hardware, and even work
process designs. Although each usability test typically includes
only five people (see Chapter 3 for more detail), as many as 60
cycles of testing can provide benefits that outweigh the costs [36].
At a minimum, three to five iterations should be considered and
one can expect improvements of 25–40% for each iteration [37].

Y Iterative design is central to
understanding and meeting
people’s needs.

When the system design nears completion, it may be placed in
an operational environment for comprehensive testing and evaluation
(see Chapter 3). This evaluation can be considered the final
step of product development. It can also be considered as the first
step in developing a better understanding of the user for the next
version of the product. The outermost cycle in Figure 2.2 indicates
that even after the product is released, the cycle continues with
data being collected to understand how people use the system.
For many consumer products, early beta versions of products are
released for this purpose, but this is not the case for high-risk systems.
For high-risk systems post-release surveillance to detect
design flaws is important. In the automotive industry, post-release
surveillance occasionally results in recalls to fix design flaws that
were not detected during the design process. The remainder of this
chapter describes critical elements of each of these three phases—
Understand, Create, and Evaluate—with a focus on understanding
the user.

Many products and systems are designed without adequate consideration
of human factors. Designers tend to focus on the technology
without fully considering its use from the human point of
view. In a book every engineer should read, Norman [23] writes:
Why do we put up with the frustrations of everyday
objects, with objects thatwe can’t figure out howto use,
with those neat plastic-wrapped packages that seem
impossible to open, with doors that trap people, with
washing machines and dryers that have become too
confusing to use, with audio-stereo-television-videocassette-
recorders that claim in their advertisements
to do everything, but that make it almost impossible
to do anything?

Even when designers attempt to consider human factors, they
often complete the product design first and only then hand off the
blueprint or prototype to a human factors expert to evaluate. This
expert is then placed in the unenviable position of having to come
back with criticisms of a design that took several months to develop.
It is not hard to understand why the design team would be less than
thrilled to receive the results of a human factors analysis. Designers
clearly believe in the design, and so are often reluctant to accept human
factors recommendations. Bringing human factors analysis at
the end of the design process places everyone involved at odds with
one another. Because of the initial investment and the designer’s
resistance to change, the result is often a product that is not particularly
successful in supporting human safety, performance, and
satisfaction. Effectively integrating human factors considerations
depends on understanding the system design process.

Y Considering human factors
at the start of the design
smooths the design process.

2.1.1 System Design Processes
Systematic design processes specify a sequence of steps for product
analysis, design, and production. Even though there are many
different design processes, they generally include stages that reflect
understanding the users needs (pre-design or front-end analysis activities),
creating a product or system (prototypes, pre-production
models), evaluating how well the design meets user’s needs; all of
which is an iterative process that cycles back to understanding the
user’s needs. Product lifecycle models, are design processes that include
product implementation, utilization andmaintenance, and
dismantling or disposal. Design processes differ to the degree that
they are defined by sequential steps or by iteration, flexibility, and
adaption to uncertainty.

Vee process

Figure 2.1 shows three common design processes,
the first is the Vee process, which is often used in the design of
large, high-risk systems, such as the design of a new aircraft, where
sequential development is possible and verification, validation,
and documentation are critical. The Vee shape starts with a broad
system description and design requirements, which are decomposed
into detailed requirements. For the dashboard of a car, these
detailed requirementsmight include information elements, such
as speed and level of the gas tank. Design of these components
are then integrated and verified by comparing them to the original
system requirements. In the Vee process, the general specifications
are well-defined at the start and emphasis is given to documenting
a successful implementation of those specifications.

Plan-Do-Check-Act cycle.

A second design model is the Plan-
Do-Check-Act cycle (PDCA), which is commonly used to enhance
workplace efficiency and production quality [30]. The cycle begins
with the target improvement. The Plan stage describes objectives
and specifies the targeted improvement. The Plan is then implemented
in the Do stage where a product, prototype or process
is created. The Check stage involves assessing the intervention
defined by the Do stage to understand what effect it had. Act completes
the cycle by implementing the intervention or developing a
new Plan based on the outcomes. This cycle reflects the scientific
management approach of Taylor in that each plan represents a
hypothesis of how the system or product might be improved.

Scrum process

A third design model is the Scrum approach,
which is more typical of consumer software products, such as
smartphone and web applications, where an iterative and incremental
approach is needed to resolve uncertainty in design requirements.
The Scrum approach focuses on creating products
and using those products to discover requirements [31]. Early prototypes
reveal design opportunities that are visible only after the
technology has been implemented. Central to the Scrumapproach
is delivering systemcomponents quickly and accommodating requirements
discovered during development. “Sprints,” which are
short duration efforts, typically 24 hours to 30 days, focus effort

Figure 2.1

Three system design processes that correspond roughly to
design of high-risk systems, the work-place, and consumer products.

on quickly producing new iterations of the product. The Scrum
approach is well-suited to situations that demand high degree
of innovation, such as those where technology changes rapidly
and potential applications emerge abruptly. This flexibility is why

such techniques are sometimes termed agile design The Scrum
approach relies on close interaction between co-located workers
to develop solutions in an ad-hoc manner and therefore, the approach
tends to place less emphasis on standardized work processes,
documentation, and testing.

As noted in the introduction, cars are increasingly becoming
highly computerized consumer products. Consequently, one might
think a Scrumapproachmight be appropriate for designing a car
given the rapidly changing technology and the associated need
for innovation to stay ahead of competitors. Rapid technology
change makes it difficult to specify detailed requirements in advance.
Cars also have elements of high-risk systems that intensify
the demands to verify and validate critical safety features,making
the “Vee” model more appropriate. Such design situations demonstrate
the need for a hybrid approach that combines elements of
the Vee, Plan-Do-Check-Act, and Scrum.

Vee process focuses on methodical
implementation,
PDCA guides incremental
improvement, and Scrum
focuses on fast iteration.

Integrating Human Factors into design processes.

Effectively
integrating human factors considerations depends on matching
the methods to the demands and opportunities of the particular
design process. For example, with a short development timeline
there may be no opportunities for time consuming human factors
methods. Some of themethods described in this chapter, such as a
comprehensive task analysis, provide an accurate description, but
require weeks to months to complete. Such comprehensive methods
best fit the Vee model. Other methods that provide a less accurate
description, such as an informal observations or an Internetbased
survey, might be completed in days. These rapidmethods
best fit the Scrum model. Human factors methods trade accuracy
for speed. Understanding how to make this speed-accuracy tradeoff
is critical for inserting human factors considerations into design.

Y Select human factors methods
that fit the demands of
the design process.

1. identify appropriate design process for high-risk systems, the work place, and consumer products

2. apply human-centered design using the understand, create, and evaluate iterative cycle

3. identify the role of human factors in system design processes

4. identify design opportunities using focus groups, observations, and accident investigation

5. define design requirements using task analysis 6. create prototypes using iterative design and refinement

Thomas Edison was a great inventor but a poor businessman. Consider the phonograph. Edison invented it, he had better technology than his competitors, but he built a technology-centered device that failed to consider his customers’ needs, and his phonograph business failed. One of Edison’s failings was to neglect the practical advantages of the disc over the cylinder in terms of ease of use, storage, and shipping. Edison scoffed at the scratchy sound of the disc compared to the superior sound of his cylinders. Edison thought phonographs could lead to a paperless office in which dictated letters could be recorded and the cylinders mailed without the need for transcription. The real use of the phonograph, discovered by a variety of other manufacturers, was for prerecorded music. Once again, he failed to understand the real desires of his customers. Edison decided that big-name, expensive artists did not sound that different from the lesser-known professionals. He is probably correct. Edison thought he could save considerable money at no sacrifice to quality by recording those lesser-known artists. He was right; he saved a lot of money. The problem was, the public wanted to hear the well-known artists, not the unknown ones. Edison bet on a technology-centered analysis and lost. The moral of this story is to know your customer. Being first, being best, and even being right do not matter; what matters is understanding what your customers want and need. Many technology-oriented companies are in a similar muddle. They develop technology-driven products without understanding their customers (Adapted from Norman [23]). The goal of a human factors specialist is to make systems successful by enhancing safety, performance, and satisfaction. This is achieved by applying human factors principles, methods, and data to the design of products or systems. The concept of “design” is very broad and can include activities such as: • Creating new products, systems, and experiences • Improving existing products to address human factors problems • Ensuring safety in the workplace, car, and home • Implementing safety-related activities, such as hazard analyses, industrial safety programs, and safety-related training • Developing performance support materials, such as checklists and instruction manuals • Developing methods to train and assess groups and teams • Guiding team and organizational design In this chapter, we review some of the methods that human factors specialists use to support design, with particular emphasis on the early stages of design. Human factors methods and principles are applied in all product design phases: front-end analysis, prototyping, technical design, and final test and evaluation.

Although interface design may be the most visible design element, human factors specialists go beyond interface to design the tasks, interaction, overall experience, and even the organization of people and technology. Cooper [28] argues that focusing solely on interface design is ineffective and calls it “painting the corpse.” Making a pretty, 3-D graphical interface cannot save a system that does not consider the job or organization it supports. Reflecting this need to go beyond user interface (UI), is the increasing prominence of user experience (UX) design, which extends beyond the interface to include all aspects of users’ interaction with a system [29]. This chapter provides an overview of the process needed to address these broad considerations, and later chapters provide the basic content necessary to carry out those processes. Later chapters also provide specialized processes needed to address considerations beyond user experience design, such as organizational design.

 7.8 Metacognition

Throughout this chapter we have cited the importance of metacognition: thinking about ones’ own thinking and cognitive processes. Metacognition influences the decision-making process by guiding how people adapt to the particular decision situation. Here we highlight five of the most critical elements of metacognition for macrocognition.

1. Knowing what you don’t know. That is, being aware that your decision processes or those necessary to maintain adequate situation awareness are inadequate because of important cues that are missing, and, if obtained, could substantially improve situation awareness and assessment.

2. The decision to “purchase” further information. This can be seen as a decision within the decision. Purchasing may involve a financial cost, such as the cost of an additional medical test required to reduce uncertainty on a diagnosis. It also may involve a time cost, such as the added time required before declaring a hurricane evacuation, to obtain more reliable information regarding the forecast hurricane track. In these cases, metacognition is revealed in the ability to balance the costs of purchase against the value of the added information [476]. The metacognitive skills here also clearly involve keeping track of the passage of time in dynamic environments, to know when a decision may need to be executed even without full information.

3. Calibrating confidence in what you know. As we have described above, the phenomenon of overconfidence is frequently manifest in human cognition [351], and when one is overconfident in ones’ knowledge, there will be both a failure to seek additional information to reduce uncertainty, and also a failure to plan for contingencies if the decision maker is wrong in his/her situation assessment. 4. Choosing the decision strategy adaptively. As we have seen above, there are a variety of different decision strategies that can be chosen; using heuristics, holistic processing, System 1, recognition primed decisions, or deploying the more elaborate effort-demanding algorithms, analytic decision strategies using System 2. The expert has many of these in her toolkit, but metacognitive skills are necessary to decide which to employ when, as Amy did in our earlier example, by deciding to switch from an RPD pattern match, to a more time analytical strategy when the former failed. 5. Processing feedback to improve the toolkit. Element 4 relates to a single instance of a decision—in Amy’s case, the diagnosis and choice of treatment for one patient. However metacognition can and should also be employed to process the outcome of a series of decisions, realize from their negative outcomes that they may be wanting, and learning to change the rules by which different strategies are deployed, just as the student, performing poorly in a series of tests, may decide to alter his/her study habits. To deploy such metacognitive skills here obviously requires some effort to obtain and process the feedback of decision outcomes, something we saw was relatively challenging to do with decision making.

7.8.1 Principles for Improving Metacognition

As with other elements of macrocognition, metacognition can be improved by some combination of changing the person (through training or experience) or changing the task (through task and technology).

1. Ease information retrieval. Requiring people to manually retrieve or select information is more effortful than simply requiring them to scan to a different part of the visual field [138, 477], a characteristic that penalizes the concepts of multilevel menus and decluttering tools that require people to select the level of decluttering they want. Pop-up messages and other automation features that infer and satisfy a person’s information needs and relieve the effort of accessing information [478].

2. Highlight benefits and minimize effort of engaging decision aids. Designers must understand the effort costs generated by potentially powerful features in interfaces. Such costs may be expressed in terms of the cognitive effort required to learn the feature or the mental and physical effort and time cost required to load or program the feature. Many people are disinclined to invest such effort even if the anticipated gains in productivity are high, and so the feature will go unused.

3. Manage cognitive depletion. An extended series of demanding decisions can incline people towards an intuitive approach to decisions, even when an analytic one would be more effective. Coaching people on this tendency might help them take rest breaks, plan complicated decisions early rather than late in the day, and avoid systems that introduce unnecessary decisions. People tend to make the easy or default decision as they become fatigued. As an example, Figure 7.9 shows how cognitive depletion changes the ruling of Israeli judges making parole decisions [479]. The timeline starts at the beginning of the day and each open circle represents the first decision after a break. The pattern cannot be explained by obvious confounding factors such as the gravity of the offense or time served. Similar effects are seen in other domains such as physicians choosing to prescribe more antibiotics as they become cognitively depleted over the day [480].

4. Training metacognition. Training can improve metacognition by teaching people to: (1) consider cues needed to develop situation awareness, (2) check situation assessments or explanations for completeness and consistency with cues, (3) analyze data that conflict with the situation assessment, and (4) recognize when too much conflict exists between the assessment and the cues. Training metacognition also needs to consider when it is appropriate to rely on the automation and when it is not [435].

Figure 7.9 Effect of cognitive depletion on rulings in favor of prisoners (Adapted from Proceedings of National Academy of Sciences, Dantziger, Levav, and Pesso (2011), Extraneous factors in judicial decisions. PNAS, 108, 17, Figure 1, p. 6890. [479].)

7.9 Summary

We discussed decision making and the factors that make it more and less effective. Normative mathematical models of utility theory describe how people should compare alternatives and make the “best” decision. However, limited cognitive resources, time pressure, and unpredictable changes often make this approach unworkable, and people use simplifying heuristics, which make decisions easier but also lead to systematic biases. In many situations people often have years of experience that enables them to refine their decision heuristics and avoid many biases. Decision makers also adapt their decision making by moving from skill- and rule-based decisions to knowledge-based decisions according to the degree of risk, time pressure, and experience. This adaptive process must be considered when improving decision making through task redesign, choice architecture, decision-support systems, or training.

Techniques to shape decision making discussed in this chapter offer surprisingly powerful ways to affect decisions and so the ethical dimensions of these choices should be carefully considered. As an example, should the default setting be designed to provide people with the option that aligns with their preference, what is best for them, what is likely to maximize profits, or what might be best for society [18]? The concepts in this chapter have important implications for safety and human error, discussed in Chapter 16. In many ways the decision-support systems described in this chapter can be considered as displays or automation—Chapter 11 addresses automation, and we turn to displays in the next chapter.

7.6.1 Principles for Improving Problem Solving and Troubleshooting

The systematic errors associated with troubleshooting suggest several design principles. 1. Present alternate hypotheses. An important bias in troubleshooting, resulting from top-down or expectancy-driven processing, is often referred to as cognitive tunneling, or confirmation bias [464, 407]. In troubleshooting, this is the tendency to stay fixated on a particular hypothesis (that chosen for testing), look for cues to confirm it (top-down expectancy guiding attention allocation), and interpret ambiguous evidence as supportive (top-down expectancy guiding perception). In problem solving, the corresponding phenomenon is to become fixated on a particular solution and stay with it even when it appears not to be working. Decision aids can challenge the persons’ hypothesis and highlight disconfirming evidence. 2. Create displays that can act as an external mental model. These cognitive biases are more likely to manifest when two features characterize the system under investigation. First, high system complexity (the number of system components and their degree of coupling or links) makes troubleshooting more difficult [465]. Complex systems are more likely to produce incorrect or “buggy” mental models [466], which can hinder the selection of appropriate tests or correct interpretation of test outcomes. Second, intermittent failures of a given system component turn out to be particularly difficult to troubleshoot [462]. A display that shows the underlying system structure, such as flow through the network of pipes in a refinery, can remove the burden of remembering that information. 3. Create systems that encourage alternate hypotheses. People generate a limited number of hypotheses because of working memory limitations [390]. Thus, people will bring in somewhere between one and four hypotheses for evaluation. Because of this people often fail to consider all relevant hypotheses [351]. Under time stress, decision makers often consider only a single hypothesis [467]. This process degrades the quality of novice decision makers far more than expert decision makers. The first option considered by experts is likely to be reasonable, but not for novices. Systems that make it easy for people to suggest many alternate hypothesis make it more likely a complete set of hypotheses will be considered.

The diagnosis error made by the medical specialist, Amy in our vignette can be examined more thoroughly using the concept of situation awareness (SA). Situation awareness, or SA, characterizes people’s awareness and understanding of dynamic changes in their environment [447, 448, 449]. A pilot loses SA whenever he or she suffers a catastrophic controlled-flight into terrain [450, 229], and as we shall see later in Chapter 16, control room operators at the Three Mile Island nuclear power plant lost SA when they believed the water level in the plant to be too high rather than too low, a misdiagnosis that led to a catastrophic release of radioactive material [395]. SA is “the perception of the elements in the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future” [383](p. 36). These three levels, perception (and selective attention), understanding, and prediction, must be applied to a specific situation. Thus, a person cannot be said to have SA without specifying what that awareness is (or should be) about. A car driver might have good awareness of navigational information and time (where I am and how much time it will take me to drive to my destination), but poor awareness of the vehicle ahead that is merging onto the highway. Improving situation awareness for navigation and for the merging vehicle would require very different designs. Note that SA does not define nor incorporate action. That concerns the decisions made from one’s awareness or assessment of the situation. Many elements of microcognition support SA and were covered in the previous chapter. Selective attention is necessary for the first level, while the second level of understanding depends very much upon both working memory and long-term memory. The third level, projection and prediction, has not yet been discussed but will be considered in more detail in the planning and scheduling section. In addition, mental models guide SA development by defining what information people pursue and the interpretation of that information. For example, Amy’s mental model of the operating room procedures might guide her to ask a nurse for estimated completion time for the perforated viscus procedure. She only asks about this procedure because her mental model of the other procedures gives her a good sense of when they would be done and so she only needs information about the procedure with an uncertain completion time. As noted above, situation awareness is not the same as performance. One can have good performance (a lucky decision outcome that was correct) without good awareness. Correspondingly, the pilot of an out-of-control aircraft may have very good situation awareness of the loss of stability; but be unable to perform the necessary actions to recover.

7.5.1 Measuring Situation Awareness

The importance of SA can often be realized after an accident by inferring that the loss of SA was partially responsible. In controlledflight-into-terrain accidents it is almost always assumed that the pilot lost awareness of the aircraft’s altitude over the terrain [450]. However, “measuring” SA after the fact by assuming its absence is not the same as measuring how well a particular system or operator maintains SA in the absence of an unexpected event [451]. A popular technique for SA measurement is the SA global assessment technique (SAGAT) [452]; in which the operator is briefly interrupted in the performance of a dynamic task and asked questions about it; for example, asking a driver to identify the location of other road traffic [453] or asking an anesthesiologist about the patient’s state [454] or asking the pilot to identify the direction to the nearest hazardous terrain [455]. Sometimes the display is blanked after the question, to assure that the information is stored in memory. One can then assess the accuracy of answering such questions. Alternatively, one can assess the time required to retrieve the correct answer off of a display that remains visible, in a technique called SPAM (Situation Present Assessment Method) [456]. SA can sometimes be measured by a subjective evaluation (“rate your SA on a scale of 1 to 10” [457]), which has been embodied in a well-used measurement tool called SART (situation awareness rating technique) [458]. However, a concern about the validity of such self-rating techniques is that people are not always aware of what they are not aware. This issue of metacognition is addressed at the end of this chapter. SA can be an important tool for accident analysis, understanding when its loss was a contributing factor [450]. To the extent that accidents may be caused by SA loss, an added implication is that systems should be designed and, when appropriate, certified to support SA. This becomes important when federal regulators are responsible for certification, such as the case with new aircraft or nuclear power plants. Although situation awareness is most commonly applied to individuals, distributed situation awareness merits consideration when multiple people work together [459]. Distributed situation awareness refers to the SA that the members of a team jointly hold. Distributed SA, like concepts of team mental model, can guide design when the focus shifts from individual to team performance. We cover these issues in more depth in Chapter 18.

7.5.2 Principles for Improving Situation Awareness

 Specific principles that follow from these considerations and from a recent review include [449]: 1. Create displays that help people notice changes (level 1 SA). Particularly in multitasking situations with dynamic systems, displays should highlight changes to make them easy for people to notice. Chapter 8 addresses issues of display layout to support SA. 2. Make the situation easy to understand (level 2 SA). Present information about the state of the system relative to the person’s goals rather than require that they interpret and mentally combine and transform information. This might also mean bringing together there are several display elements that might otherwise be placed in different locations. 3. Keep the operator somewhat “in the loop”. This issue will be addressed in more detail in Chapter 11 (Automation). The critical concept introduced here is related to the generation effect. People are more likely to remember actions, and the consequence of actions, if they themselves have generated the action, than if they were watching another agent generate the same action. Automobile manufacturers of self driving cars are struggling to find ways of keeping the driver somewhat in the loop (e.g., hands on the wheel), even as automation is steering the car, in order to preserve SA, should automation fail. 4. Help people project the state of the system into the future (level 3 SA). This is particularly important when the system responds slowly, like a supertanker, industrial oven, or air traffic system. Here create a display that shows the future state, such as the predictive displays we discuss in Chapter 8. This relieves the person of mentally simulating and projecting future states. 5. Organize information around goals. Rather than arbitrary or technology oriented placement of information, displays should cluster information according to the goals the person is trying to achieve. 6. Display to broaden attention. Recognizing that SA may be most critical for dealing with unexpected situations, displays should avoid narrowing people’s attention to a limited array of information that is specific to a particular task or limited to routine situations. Supporting SA when unexpected things happen typically means adding information to the display. This information must be carefully integrated to avoid issues of clutter. 7. Train for SA. When training for SA, it is important to realize that training for routine performance may conflict with training to maintain SA [460]. The former will focus on the information needed for the task as it was intended to be performed. The latter should focus on what is often a broader scope of selective attention, to be aware of the state of the world should the system fail.) Many of the biases relevant to diagnosis, discussed above, are paralleled by biases in situation awareness: for example the confirmation bias or anchoring. Hence debiasing training, can be effective here.

In contrast suppose you are late for a job interview across town. You can speed, with a high chance of getting to the appointment on time, but also incurring the risk of getting caught by the police, fined, and be very late for the appointment. Alternatively you can choose to drive the speed limit, and certainly be slightly late. Here the choice is between two negatives, a risky one and a sure thing. You are “caught between a rock and a hard place”, and under such circumstances people tend to be risk-seeking. [413, 410]. The second of these contexts, the negative frame of choice, is often characteristic of real life decisions. For example, in addition to the speeding choice above, consider a company with major safety violations in its plant. Management can choose to invest heavy funding into addressing them through new equipment, hiring safety consultants, and pulling workers off the line for safety training, thus incurring the sure loss of time and money. Alternatively they can chose to take the risk that there will be neither a serious injury nor a surprise inspection from federal safety inspectors. All too-often, the framing bias will lead to an inclination toward the second option, at the expense of worker safety. A direct expression of this form of the framing bias is known as the sunk cost bias [414, 415]. This bias affects individual investors who hesitate to sell losing stocks (a certain loss), but tend to sell winning stocks to lock in a gain. Likewise, when you have invested a lot of money in a project that has “gone sour”, there is a tendency to keep it in the hopes that it will turn around. Similarly, managers and engineers tend to avoid admitting a certain cost when replacing obsolete equipment. The sunk cost bias describes the tendency to choose the risky loss over the sure one, even when the rational, expected value choice should be to abandon the project. Because people tend to incur greater risk in situations involving losses, decisions should be framed in terms of gains to counteract this tendency. Y Sunk cost bias makes it difficult for you to make money in the stock market. 7. Default heuristic. Faced with uncertainty regarding what choice to make people often adopt the default alternative [416]. Most countries use their drivers’ licenses to allow people to specify whether to donate their organs or not in the event of a fatal crash. Countries differ according to whether people need to opt in and decide to donate, or opt out and decide not to donate. Over 70% people follow the default and let the designers of the form decide for them. A similarly large effect is seen for people choosing to enroll in a retirement savings plan or having to opt out. Defaulting people into a retirement plan increased participation from about 50% to about 90% [417, 418].

7.4.2 Benets of Heuristics and the Cost of Biases The long list of decision-making biases and heuristics above may suggest that people are not very effective decision makers in everyday situations, and might suggest that human contributions to decision making are a problem that should be fixed. However, this perspective neglect the fact that most people do make good decisions most of the time, and have the flexibility to deal with situations that can’t be reduced to an equation. The list of biases accounts for the infrequent circumstances, like the decision makers in the Three Mile Island nuclear plant, when decisions produce bad outcomes. One reason that most decisions are good, is that heuristics are accurate most of the time. A second reason is that people have a profile of resources: information-processing capabilities, experiences, and decision aids (e.g., a decision matrix) that they can adapt to the situations they face. Experts are proficient in adjusting their decision strategies. To the extent that people have sufficient resources and can adapt to them, they make good decisions. When people are not able to adapt, such as where people have little experience with the situations, poor decisions can result [357]. The focus can be either on the general high quality of most decisions, or on the errors due to biases associated with heuristics. Both of these approaches are equally valid, but focusing on the errors supports the search for human factors solutions to eliminate, or at least mitigate those biases that do show. It is to this that we now turn. 7.4.3 Principles for Improving Decision Making Decision making is often an iterative cycle in which decision makers are often adaptive, adjusting their response according to their experience, the task situation, cognitive ability, and the available decision-making aids. It is important to understand this adaptive decision process because system design, training, and decision aids need to support it. Attempts to improve decision making without understanding this process tend to fail. In this section, we briefly discuss some possibilities for improving human decision making: task redesign, including choice architecture and procedures; training; displays; and automated decision support systems. Task redesign. We often jump to the conclusion that poor performance in decision making means we must do something “to the person” to make him or her a better decision maker. However, sometimes a change in the system can support better decision making, eliminating the need for the person to change. As described in Chapter 1, decision making may be improved by task design. Changing the system should be considered before changing the person through training or even providing a computer-based decision aid. For example, consider the situation in which the removal of a few control rods led to a runaway nuclear reaction, which resulted in 3 deaths and 23 cases of exposure to high levels of radioactivity. Learning from this experience, reactor designers now create reactors that remain stable even when several control rods are removed [227]. Creating systems with greater stability leaves a greater margin for error in decisions and can also make it easier to develop accurate mental models. Choice architecture. The structure of the interaction influences choice in much the same way architecture of a building influences the movement of people through buildings [18]. Choice architects influence decisions by recognizing the natural cognitive tendencies we have discussed and presenting people with information and options that will take advantage of these tendencies to generate good decisions. The following principles show how choice architecture can nudge people towards decisions [419]. 1. Limit the number of options. Because too many options place a high burden on the decision maker, the number of options should be limited to the fewest number that will encourage exploration of options. Although the appropriate number depends on the specific elements of the decision maker and situation, four to five options where none is better on all dimensions. Fewer options should be offered if decision makers are less capable, such as older people, those in a time pressured situation, or less numerate decision makers faced with numerical options [420, 419]. 2. Select useful defaults. The effect of defaults on organ donation rates demonstrates the power of defaults: People often choose default options. Options for designing defaults include random, uniform choice for all users, forced choice, persistent default where the system remembers previous settings, and predictive default where the system picks based on user characteristics. If there is no time pressure and the choice is important then active choice should be used. If there is an obvious benefit to a particular choice then a uniform default for all users should be used, such when organizations select double-sided printing as the default [421]. As laptops, tablet and desktop computers, as well as phones, TVs and cars become more integrated predictive defaults become more feasible and valuable.

3. Make choices concrete. People focus on concrete immediate outcomes and tend to be overly optimistic about future regarding available time and money. To counteract people’s tendency to neglect the abstract future situation a limited window on opportunity can focus their attention like: “offer ends midnight tonight.” Another approach is to translate the abstract future value choices into immediate, salient consequence. For example, show people their future self so they can invest for that future self [422]. People who saw realistic computer renderings of older version of themselves invested more.

4. Create linear, comparable relationships. People tend to struggle to consider complex transformations and non-linear relationships. Transforming variables to their concrete linear equivalent promotes better decisions. For example, describing interest rates in terms of the number of payments to eliminate debt in three years is more effective than expecting people to calculate the non-linear, compounding effect of interest. Likewise, presenting fuel economy data in terms of gallons per 100 miles rather than miles per gallon, eliminates the mental transformation that is needed to compare vehicles [423]. The units presented should be those directly relevant to the decision.

7.4.1 Vulnerabilties of Heuristics: Biases Cognitive heuristics are rules-of-thumb that are easy ways of making decisions. Heuristics are usually very powerful and efficient [378], but they do not always guarantee the best solution [354, 379]. Unfortunately, because they represent simplifications, heuristics occasionally lead to systematic flaws and errors. The systematic flaws represent deviations from the normative model and are sometimes referred to as biases. Experts tend to avoid these biases because they draw from a large set of experiences and they are vigilant to small changes in the pattern of cues that might suggest the heuristic is inappropriate. To the extent a situation departs from these experiences, even experts will fall prey to the biases associated with various heuristics. Although the list of heuristics is large (as many as 37 [380]), the following presents some of the most notorious ones. Acquire and Integrate Cues: Heuristics and Biases. The first stage of the decision process begins with attending to information and integrating it to understand the situation or form a situation assessment (e.g., to support stage 2).

1. Attention to a limited number of cues. Due to working memory limitations, people can use only a relatively small number of cues to develop a picture of the world or system. This is one reason why configural displays that visually integrate several variables or factors into one display are useful (see Chapter 8 for a description). 2. Anchoring and cue primacy. When people receive cues over a period of time, there are certain trends or biases in the use of that information. The first few cues receive greater weight than subsequent information–cue primacy [381]. It often leads people to “anchor" on initial evidence and is therefore sometimes called the anchoring heuristic [354], characterizing the familiar phenomenon that first impressions are lasting. Amy anchored on the cues supporting her initial diagnosis, and gave little processing to additional information available in the phone call by the patient 24 hours later. Importantly, when assessing a dynamic changing situation, the anchoring bias can be truly detrimental because older information becomes progressively less reliable, even as the older information was, by definition, the first encountered and hence served as the anchor. The order of information has an effect because people use the information to construct plausible stories or mental models of the world or system. These models differ depending on which information is used first [382]. The key point is that, information processed early is often most influential. 3. Cue salience. Perceptually salient cues are more likely to capture attention and be given more weight [383, 11]; see also Chapter 6. As you would expect, salient cues in displays are things such as information at the top of a display, the loudest alarm, the largest display, the loudest most confident sounding voice in the room, and so forth. Unfortunately, the most salient cue is not necessarily the most diagnostic, and sometimes very subtle ones, such as the faint discoloration observed by Amy are not given much weight. 4. Overweighting of unreliable cues. Not all cues are equally reliable. In a trial, some witnesses, for example, will always tell the truth. Others might have faulty memories, and still others might intentionally lie. However, when integrating cues, people often simplify the process by treating all cues as if they are all equally valid and reliable. The result is that people tend to give too much weight to unreliable information [384, 385]. Interpret and Assess: Heuristics and Biases. After a limited set of cues is processed in working memory, the decision maker generates and interprets the information, often by retrieving similar situations from long-term memory. These similar situations represent hypotheses about how the current situation relates to past situations. There are a number of heuristics and biases that affect this process:

1. Availability. The availability heuristic reflects people’s tendency to make certain types of judgments or assessments, for example, estimates of frequency, by assessing how easily the state or event is brought to mind [386, 387, 388]. People more easily retrieve hypotheses that have been considered recently and hence more available to memory. The implication is that although people try to generate the most likely hypotheses, the reality is that if something comes to mind relatively easily, they assume it is common and therefore a good hypothesis. As an example, if a physician readily thinks of a hypothesis, such as acute appendicitis, he or she will assume it is relatively common, leading to the judgment that it is a likely cause of the current set of symptoms. Unusual illnesses tend not to be the first things that come to mind to a physician. Amy did not think of the less likely condition. In actuality, availability to memory may not be a reliable basis for estimating frequency. 2. Representativeness. Sometimes people diagnose a situation because the pattern of cues “looks like” or is representative of the prototypical example of this situation. This is the representativeness heuristic [353, 389], and usually works well; however, the heuristic can bias decisions when a perceived situation is slightly different from the prototypical example even though the pattern of cues is similar or representative. 3. Overconfidence. People are often biased in their confidence with respect to the hypotheses they have brought into working memory [390, 351], believing that they are correct more often than they actually are and reflecting the more general tendency for overconfidence in metacognitive processes, as described in Chapter 6 [391]. Such overconfidence appears to grow when judgments are more predictive about the future (than of the current state) and when predictions become more difficult [11]. As a consequence, people are less likely to seek out evidence for alternative hypotheses or to prepare for the circumstances that they may be wrong. Less skilled people are more likely to overestimate their ability, even when they understand their relative ability [392]. 4. Cognitive tunneling. As we have noted earlier in the context of anchoring, once a hypothesis has been generated or chosen, people tend to underutilize subsequent cues. We remain stuck on our initial hypothesis, a process introduced in the previous chapter as cognitive tunneling [393]. Examples of cognitive tunneling abound in the complex systems [394]. Consider the example of the Three Mile Island disaster in which a relief valve failed and caused some of the displays to indicate a rise in the level of coolant [395]. Operators mistakenly thought that that emergency coolant flow should be reduced and persisted to hold this hypothesis for over two hours. Only when a supervisor arrived with a fresh perspective did the course of action get reversed. Notice that cognitive tunneling is different than the primacy, which occurs when the decision maker is first generating hypotheses.

Figure 7.5 shows the analysis of four different options, where the options are different cars that a student might purchase. Each car is described by five attributes. These attributes might include the sound quality of the stereo, fuel economy, insurance costs, and maintenance costs. The utility of each attribute reflects its importance to the student. For example, the student cannot afford frequent and expensive repairs, so the utility or importance of the fifth attribute (maintenance costs) is quite high (8), whereas the student does not care as much about the sound quality of the stereo (4) or the fourth attribute (color), which is quite low (1). The cells in the decision table show the magnitude of each attribute for each option. For this example, higher values reflect a more desirable situation. For example, the third car has a poor stereo, but low maintenance costs. In contrast, the first car has a slightly better stereo, but high maintenance costs. Combining the magnitude of all the attributes shows that third car (option 3) is most appealing or “optimal” choice and that the first car (option 1) is least appealing.

Figure 7.5 Multi-attribute utility analysis combines information from multiple attributes of each of several options to identify the optimal decision.

Multi-attribute utility theory, shown in Figure 7.5, assumes that all outcomes are certain. However, life is uncertain, and probabilities often define the likelihood of various outcomes (e.g., you cannot predict maintenance costs precisely). Another example of a normative model is expected value theory, which addresses uncertainty. This theory replaces the concept of utility in the previous context with that of expected value. The theory applies to any decision that involves a “gamble” type of decision, where each choice has one or more outcomes and each outcome has a worth and a probability. For example, a person might be offered a choice between: 1. Winning $50 with a probability of 1.0 (a guaranteed win), or 2. Winning $200 with a probability of 0.30. Expected value theory assumes that the overall value of a choice (Equation 7.1) is the sum of the worth of each outcome multiplied by its probability where E(v) is the expected value of the choice, p(i) is the probability of the ith outcome, and v(i) is the value of the i th outcome.

E(v)= به کتاب مراجعه شود

The expected value of the first choice for the example is $50×1.0, or $50, meaning a certain win of $50. The expected value of the second choice is $200 × 0.30, or $60, meaning that if the choice were selected many times, one would expect an average gain of $60, which is a higher expected value than $50. Therefore, the normative decision maker should always choose the second gamble. In reality, people tend to avoid risk and go with the sure thing [372].

Figure 7.6 shows two states of the world, 1 and 2, which are generated from situation assessment. Each has a probability, P1 and P2, respectively. The two choice options, A and B, may have four different outcomes as shown in the four cells to the right. Each option may also have different Utilities (U), which could be positive or negative, contingent upon the existing state of the world. The normative view of decision making dictates that the chosen option should be the one with the highest (most positive) sum of the products within the two different states. Descriptive decision making accounts for how people actually make decisions. People can depart from the optimum, normative, expected utility model. First, people do not always try to maximize, EV nor should they because other decision criteria beyond expected value can be more important. Second, people often shortcut the time and effort-consuming steps of the normative approach. They do this because time and resources are not adequate to “do things right” according to the normative model, or because they have expertise that directly points them to the right decision. Third, these shortcuts sometimes result in errors and poor decisions. Each of these represents an increasingly large departure from normative decision making.

As an example of using a decision criterion different from maximizing expected utility, people may choose instead to minimize the possibility of suffering the maximum loss. This certainly could be considered as rational, particularly if one’s resources to deal with the loss were limited. This explains why people purchase insurance; even though such a purchase decision does not maximize their expected gain. If it did, the insurance companies would soon be out of business! The importance of using different decision criteria reflects the mismatch between the simplifying assumptions of expected utility and the reality of actual situations. Not many people have the ability to absorb a $100,000 medical bill that might accompany a severe health problem. Most decisions involve shortcuts relative to the normative approach. Simon [373] argued that people do not usually follow a goal of making the absolutely best or optimal decision. Instead, they opt for a choice that is “good enough” for their purposes, something satisfactory. This shortcut method of decision making is termed satisficing. In satisficing, the decision maker generates and evaluates choices only until one is found that is acceptable rather thanone that is optimal. Going beyond this choice to identify something that is better is not worth the effort. Satisficing is a very reasonable approach given that people have limited cognitive capacities and limited time. Indeed, if minimizing the time (or effort) to make a decision is itself considered to be an attribute of the decision process, then satisficing or other shortcutting heuristics can sometimes be said to be optimal—for example, when a decision must be made before a deadline, or all is lost. In the case of our car choice example, a satisficing would be to take the first car that gets the job done rather than doing the laborious comparisons to find the best. Satisficing and other shortcuts are often quite effective [366], but they can also lead to biases and poor decisions as we will discuss below. Our third characteristic of descriptive decision making concerns human limits that contribute to decision errors. A general source of errors concerns the failure of people to recognize when shortcuts are inappropriate for the situation and adopt the more laborious decision processed. Because this area is so important, and its analysis generates a number of design solutions, we dedicate the next section to this topic.

What is decision making? Generally, it is a task in which (a) a person must select one option from several alternatives, (b) a person must interpret information for the alternatives, (c) the timeframe is relatively long (longer than a second), (d) the choice includes uncertainty; that is, it is not necessarily clear which is the best alternative. By definition, decision making involves risk—there is a consequence to picking the wrong alternative—and so a good decision maker effectively assesses risks associated with each alternative. The decisions discussed in this chapter range from those involving a slow deliberative process, involving how to allocate resources to those which are quite rapid, with few alternatives, like the decision to speed up, or apply the brakes, when seeing a yellow traffic light, or whether to open a suspicious e-mail [362]. Decision making can generally be represented by four stages as depicted in Figure 7.4: (1) acquiring and integrating information relevant for the decision, (2) interpreting and assessing the meaning of this information, (3) planning and choosing the best course of action after considering the costs and values of different outcomes, and (4) monitoring and correcting the chosen course of action. People typically cycle through the four stages in a single decision.

Figure 7.4 The four basic stages of decision making that draw upon limited attention resources and metacognition.

1. Acquire and integrate a number of cues, or pieces of information, which are received from the environment and go into working memory. For example, an engineer trying to identify the problem in a manufacturing process might receive a number of cues, including unusual vibrations, particularly rapid tool wear, and strange noises. The cues must be selectively attended, interpreted and somehow integrated with respect to one another. The cues may also be incomplete, fuzzy, or erroneous; that is, they may be associated with some amount of uncertainty. 2. Interpret and assess cues and then use this interpretation to generate one or more situation assessments, diagnoses, or inferences as to what the cues mean. This is accomplished by retrieving information from long-term memory. For example, an engineer might hypothesize that the set of cues described previously is caused by a worn bearing. Situation assessment is supported by maintaining good situation awareness, a topic we discuss later in the chapter. The difference is that while maintaining SA refers to a continuous process, making a situation assessment involves a one time discrete action with the goal of supporting a particular decision. 3. Plan and choose one of alternative actions generated by retrieving possibilities from long-term memory. Depending on the time available, one or more of the alternatives are generated and considered. To choose an action, the decision maker might evaluate information such as possible outcomes of each action (where there may be multiple possible outcomes for each action), the likelihood of each outcome, and the negative and positive factors associated with each outcome. This can be formally done in the context of a decision matrix in which actions are crossed against the diagnosed possible states of the world that could occur, and which could have different consequences depending on the action selected. 4. Monitor and correct the effects of decisions. The monitoring process is a particularly critical part of decision making and can serve two general purposes. First, one can revise the current decision as needed. For example, if the outcomes of a decision to prescribe a particular treatment are not as expected, as was the case with Amy’s patient is getting worse, not better, then the treatment can be adjusted, halted or changed. Second, one can revise the general decision process if that process is found wanting and ineffective, as Amy also did. For example, if heuristics are producing errors, one can learn to abandon them in a particular situation and instead adopt the more analytical approach shown to the left of Figure 7.2. In this way, monitoring serves as an input for the troubleshooting element of macrocognition. Monitoring, of course, provides feedback on the decision process. Unfortunately, in decision making that feedback is often poor, degraded, delayed or non-existent, all features that undermine effective learning [11]. It is for this reason that consistent experience in decision making does not necessarily lead to improved performance [363, 357]. Figure 7.4 also depicts the two influences of attentional resources and metacognition. Many of the processes used to make ideal or “optimal” decisions impose intensive demands on perception and selective attention (for stage 1), particularly on the working memory used to entertain hypotheses in stage 2, and to evaluate outcomes in stage 4. If these resources are scarce, as in a multitasking environment, decision making can suffer. Furthermore, because humans are effort conserving, we often tend to adopt mental shortcuts or heuristics that can make decision making easier and faster, but may sacrifice its accuracy. Metacognition describes our monitoring of all of the processes by which we make decisions, and hence is closely related to stage 4. We use such processes for example to assess whether we are confident enough in a diagnosis (stage 2) to launch an action (stage 3) without seeking more information. We describe metacognition in more detail near the end of the chapter.7.3.1 Normative and Descriptive Decision Making Decision making has, for centuries, been studied in terms of how people should make optimal decisions: those likely to produce the best outcomes in the long run [364, 365]. This is called normative decision making. Within the last half century however, decision scientists have highlighted that humans often do not, in practice, adhere to such optimal norms for a variety of reasons, and so their decisions can be described in ways classified as descriptive decision making. We now discuss both the normative and descriptive frameworks. Normative decision making considers the four stages of decision making in terms of an idealized situation in which the correct decision can be made by calculating the mathematical optimal choice. This mathematical approach is often termed normative decision making. Normative decision making specifies what people should do; they do not necessarily describe how people actually perform decision-making tasks. Importantly, these normative models make many assumptions that incorrectly simplifies and limits their application to the decisions people actually face [366]. Normative models are important because they form the basis for many computer-based decision aids, and justify (often wrongly) that humans’ fallible judgment should be removed from the decision process [367]. Although such normative models often outperform people in situations where their assumptions hold, many real-life decision cannot be reduced to a simple formula [368]. Normative decision making revolves around the central concept of utility, the overall value of a choice, or how much eachoutcome is “worth” to the decision maker. This model has application in engineering decisions as well as decisions in personal life. Choosing between different corporate investments, materials for product, jobs, or even cars are all examples of choices that can be modeled using multiattribute utility theory. The decision matrix described in Chapter 2 is an example of how multiattribute utility theory can be used to guide engineering design decisions. Similarly, it has been used to resolve conflicting objectives, to guide environmental cleanup of contaminated sites [369], to support operators of flexible manufacturing systems [370], and even to select a marriage partner [371]. The number of potential options, the number of attributes or features that describe each option, and the challenge in comparing alternatives on very different dimensions make decisions complicated. Multiattribute utility theory addresses this complexity, using a utility function to translate the multidimensional space of attributes into a single dimension that reflects the overall utility or value of each option. In theory, this makes it possible to compare apples and oranges and pick the best one. Multiattribute utility theory assumes that the overall value of a decision option is the sum of the magnitude of each attribute multiplied by the utility of each attribute (Equation 7.1), where U(v) is the overall utility of an option, a(i) is the magnitude of the option on the i th attribute, u(i) is the utility (goodness or importance) of the i th attribute, and n is the number of attributes.

U(v) = به کتاب رجوع شود 

In understanding decision making over the last 50 years, there have been a variety of approaches to analyzing the skill or proficiency in reasoning that develops as the decision maker gains expertise. These are shown in Figure 7.2. To some degree, all of these approaches are related, but represent facets of decision making and macrocognition. These approaches provide a framework for many of the sections to follow. In the first row of Figure 7.2, Rasmussen [347] has proposed a three-level categorization of behavior. These levels evolve as the person develops progressively more skill or as the problems become progressively less complex. The progression from knowledgebased to rule-based to the more automatic skill-based behavior parallels the development of automaticity described in the previous chapter. Closely paralleling this, in the second row, is the distinction between careful analytic processing (describing all the options and factors that should enter into a choice), and the more “gut level” intuitive processing, often less accessible to conscious awareness [348]. Here, as with Rasmussen’s levels of behavior, more intuitive decisions are more likely to emerge with greater skill and simpler problems.

The third row shows different cognitive systems that underly how people make decisions [349, 350, 351]. System 2, like analytical judgments and knowledge-based reasoning, is considered to serve a deliberative function that involves resource-intensive effortful processes. In contrast, System 1 like intuitive judgments and the skill-based reasoning, engages relatively automatic “gut-feel” snap judgments. System 1 is guided by what is easy, effort-free and feels good or bad; that is, the emotional component of decision making. In partial contrast with skill-based behavior and intuitive judgments however, engaging System 1 does not necessarily represent greater expertise than engaging System 2. Instead, the two systems operate in parallel in any given decision, with System 1 offering a snap decisions of what to do, but then System 2, if time and cognitive resources or effort are available, overseeing and checking the result of System 1 to assure its correctness. System 1 also aids System 2 by focusing attention and filtering options—without it we would struggle to make a decision [352]. In the fourth row, we show two different “schools” of decision research that will be the focus of much of our discussion below. The “heuristics and biases” approach, developed by Kahneman and Tversky [353, 354] has focused on the kinds of decision shortcuts made because of the limits of reasoning, and hence the kinds of biases that often lead to decision errors. These biases identify “what’s wrong” with decision making and what requires human factors interventions. In contrast, the naturalistic decision making school, proposed by Klein [355, 356] examines decision making of the expert, many of whose choices share features of skill-based behavior, intuitive decision making that are strongly influenced by System 1. That is, such decisions are often quick, relatively effortfree, and typically correct. While these two approaches are often set in contrast, it is certainly plausible to see both as being correct,but applicable in different circumstances, and hence more complementary than competitive [357]. Heuristics and intuitive decision making work well for experienced people in familiar circumstances, but biases undermine performance of novices or experts in unfamiliar circumstances. In the final row, we describe a characteristic of metacognition that appears, generally to emerge with greater skill. That is, it becomes increasingly adaptive, with the human better able to select the appropriate tools, styles, types, and systems, given the circumstances. That is, with expertise, people develop a larger cognitive toolkit, as does the wisdom regarding which tools to apply when. The first row of Figure 7.2, shows skill-, rule-, and knowledgebased (SRK) behavior depends on people’s expertise and the situation [358, 347, 359]. High levels of experience with analog representations promote relatively effortless skill-based behavior (e.g., riding a bicycle), whereas little experience with numeric and textual information will lead to knowledge-based behavior (e.g., selecting an apartment using a spreadsheet). In between, like the decision to bring a raincoat on a bike ride, follows rule-based behavior: “if the forecast chance of rain is greater than 30%, then bring it.” These SRK distinctions also describe types of human errors [360], which we discuss in Chapter 16). These distinctions are particularly important because we can improve decision making and reduce errors by supporting skill-, rule-, and knowledge-based behavior. Figure 7.3 shows the SRK process for responding to sensory input that enters at the lower left. This input can be interpreted at one of three levels, depending on the operator’s degree of experience with the particular situation and how information is represented [358, 348]. The right side shows an example of sensory input: a meter that an operator has to monitor. The figure shows that the same meter is interpreted differently depending on the level of behavior engaged: as a signal for skill-based behavior, as a sign for rule-based behavior, and as a symbol for knowledge-based behavior. Signals and skill-based behavior. People who are extremely experienced with a task tend to process the input at the skill-based level, reacting to the perceptual elements at an automatic, subconscious level. They do not have to interpret and integrate the cues or think of possible actions, but only respond to cues as signals that guide responses. Because the behavior is automatic, the demand on attentional resources described in Chapter 6 is minimal. For example, an operator might turn a valve in a continuous manner to counteract changes in flow shown on a meter (see bottom left of Figure 7.3). Y Designs that enable skillbased behavior are “intuitive”. Signs and rule-based behavior. When people are familiar with the task but do not have extensive experience, they process input and perform at the rule-based level. The input is recognized in relation to typical system states, termed signs, which trigger rules for accumulated knowledge. This accumulated knowledge can be inthe person’s head or written down in formal procedures. Following a recipe to bake bread is an example of rule-based behavior. The rules are “if-then” associations between cue sets and the appropriate actions. For example, Figure 7.3 shows how the operator might interpret the meter reading as a sign. Given that the procedure is to reduce the flow if the meter is above a set point, the operator then reduces the flow. Symbols and knowledge-based behavior. When the situation is new, people do not have any rules stored from previous experience to call upon, and do not have a written procedure to follow. They have to operate at the knowledge-based level, which is essentially analytical processing using conceptual information. After the person assigns meaning to the cues and integrates them to identify what is happening, he or she processes the cues as symbols that relate to the goals and decides on an action plan. Figure 7.3 shows how the operator might reason about the low meter reading and think about what might be the reason for the low flow, such as a leak. It is important to note that the same sensory input, the meter in Figure 7.3, for example, can be interpreted as a signal, sign, or symbol. The relative role of skill-, rule-, and knowledge-based behavior depends on characteristics of the person, the technology, and the situation [354, 361]. Characteristics of the person include experience and training. As we will see people can be trained to perform better in all elements of macrocognition; however, as with most human factors interventions, changing the task and tools is more effective.In the following sections, we first discuss the cognitive processes in decision making: how it too can be described by stages, the normative approach to decision making (how it “should” be done to produce the best outcomes), and the reasons why people often do not follow the normative decision making processes. Two important departures from normative decision making, receive detailed treatment: naturalistic decision making and heuristics and biases. Because decision errors produced by the heuristics and biases can be considered to represent human factors challenges, we complete our treatment of decision making by describing several human factors solutions to mitigate decision errors. Finally, our chapter concludes by describing four “close cousins” of decision making within the family of macrocognitive processes: situation awareness, troubleshooting, planning and metacognition.

The cognitive environment governs how characteristics of microcognition, such as the limits of working memory, influence human performance. In a similar way, the environment governs how the characteristics of macrocognition influence performance, and when it is important to consider macrocognition.

Features of situations where macrocognition matters :

III-structured problems with ambiguous goals , example : There is no single “best” way of responding to a set of a patient’s symptoms.

Uncertain, dynamic environments. example : The situation at Amy’s hospital is continually changing, presenting new decisions and considerations.

Information-rich environments. example: There is information on status boards, electronic patient records, and through talking with others.

Iterative perception-action feedback loops. example: Any decision to regarding treatment, particularly after an initial misdiagnosis, is monitored and used decide what to do next.

Time pressure. example: Decisions often need to be made quickly because delays can jeopardize the outcome of a procedure.

High-risk situations. example: Loss of life can result from a poor decision..

Multiple shifting and competing individual and organizational goals. example: As the day evolves, the goals may shift from minimizing delays for routine procedures to responding to a major emergency. Also, what might be the top priority physician might not be the same for a nurse or patient.

Interactions with multiple people. example: Many people contribute information and perspectives to decisions: patients and nurses negotiate with Amy

People often make decisions in dynamic, changing environments, like those confronting the internal medicine specialist, Amy, described at the outset of the chapter [344, 345, 346]. Amy faced incomplete, complex, and dynamically changing information; time stress; interactions with others; high risk; uncertain outcomes, each with different costs and benefits. Not every situation is so complicated, but those that include these elements indicate a need to consider the processes of macrocognition discussed in this chapter. Table 7.1 summarizes features of the cognitive environment that makes it important to consider macrocognition. These features cause us to adopt different decision processes. Sometimes, particularly in high-risk situations, we carefully calculate and evaluate alternatives, but in many cases, we just interpret it to the best of our ability and make educated guesses about what to do. Some decisions are so routine that we might not even consider them to be decisions. Unlike the situations that influence microcognition, critical features associated with macrocognition include poorly defined goals that might not be shared by all involved. As in Amy’s situation, concepts of macrocognition are particularly important in situations that have multiple people interacting in an evolving situation where decisions and plans are made and then revisedover time. In many cases, these features make decision making and problem solving difficult and error-prone. This makes macrocognition a central concern to human factors specialists working in complex systems, such as military operations, hospitals, aircraft cockpits, and process control plants. We begin this chapter by describing the overall nature of skill and expertise in macrocognition, and how they change with practice and experience. We present three types of behavior that have implications for all elements of macrocognition, and then consider these behaviors with respect to decision making. Decision making highlights the challenges of engaging analytic thinking, the power of heuristics and the pitfalls of the associated biases. Principles to improve decision making are described in terms of task design, decision support systems, displays, and training. The final sections of the chapter addresses four closely related areas of macrocognition: situation awareness, troubleshooting, planning, and metacognition.

Figure 7.1 highlights five elements of macrocognition, with the elements arrayed in a circle roughly in the order they might occur, but in reality, the process is more complex with all processes being linked to all other processes and occurring in a repeated cycle. At the center is metacognition—thinking about one’s own thinking—which guides the individual macrocognitive processes. Microcognition and macrocognition offer complementary perspectives that suggest different ways to enhance safety, performance, and satisfaction.