Psychological and Physiological Stressors and Factors

Level 2/3 Plan (DRAFT 2)

Outline of Contents

0. Overall Program Description

0.1 Background
0.2 Objectives
0.3 Approach

1. Task 1. Perceptual Models and Metrics

Level 3 Program Lead - Dr. Al Ahumada (ARC/AFH)

1.2.1 Visibility Models
1.2.2 Eye-Movement Metrics of Human Perception
1.2.3 Image Processing for Improved Displays
1.2.4 Metrics and Models of Range and Closure Perception
1.2.5 Metric and Models for the Perceptual Design of Virtual Transparency
1.2.6 Spatial Auditory Displays

2. Task 2. Cognitive Models and Metrics (548-50-42-xx)

Level 3 Program Lead - Dr. Roger Remington (ARC/AFI)

2.2.1 Eye-Movement Metrics of Human Cognition
2.2.2 Models and Metrics of Human Executive Control
2.2.3 Models and Metrics of Human Spatial Attention and Memory

3. Task 3. Physiological Factors (548-50-22/32)

Level 3 Program Lead - Dr. Mark Rosekind (ARC/AFS)

3.2.1 Fatigue Countermeasures (POC: M. Rosekind, ARC)
3.2.2 Hazardous States of Awareness (POC: A. Pope, LaRC)

0. Overall Program Description

0.1 Background


The coming increase in air space utilization and the corresponding need to reduce the incidence of human and system errors is expected to place increasing stress on the human operators and decision makers who directly interact with the air transportation system. It is expected that existing technologies and procedures will be inadequate to alleviate this stress at minimum cost and to reduce the incidence of human error and potentially catastrophic system failure. Consequently, research into novel displays, controls, and procedures is required to explore innovative techniques for safe and efficient management of the increasingly dense air traffic system. This element is intended to spark this innovation. It is divided into three sub-elements: human perception, cognitive models and metrics, and physiological research.

0.2 Objectives

The Psychological/ Physiological Stressors and Factors research project goal is to develop new technologies and procedures to measure and reduce this increased stress within the air traffic system. Techniques will be developed to quantify the specific errors that the stress may cause. Knowledge will be developed that will enable innovative technologies and procedures that may be integrated into the national air transportation system to preserve its integrity. Stress is not simply considered to be the psychological stress of operators who have to deal with increasingly frequent takeoffs and landings, but it also includes the increasing visual clutter of the electronic displays they use and the increasing aural clutter of the audio channels. Research conducted on the enumerated elements below will 1) develop computational tools that will better allow the aeronautical community to analyze the perceptual fidelity of the human machine interfaces that they use, 2) assist analysis of perceptual problems with existing displays, and 3) explore the utility of revolutionary new perceptual display technology that may be adopted by the aeronautical community in the next century.

0.3 Approach

The Psychological/ Physiological Stressors and Factors research project uses analytical, experimental and actuarial methods to measure and predict human performance within all sectors of the nation's air transportation system. The human perception sub-element focuses on development of new methods, computational models, and metrics that will enable optimization of operator sensory-motor interaction with the displays and controls of the national air space system. The cognitive model sub- element focuses on models of the human operator information processing during interaction with the air transportation system with the goal of understanding how operator attention may focused or misfocused by the system. The physiological sub- element will consider the role of physiologically based variation in alertness and develop novel work rules to manage disturbances in operators' circadian rhythms while working within the air transportation system. This sub-element will also assess the impact of these innovative work rules. In addition to the usual publication of technical reports and scientific journal articles, the results of the research and development conducted under the Elements below will be disseminated to the aeronautical community through workshops and site visits organized and conducted by the principal investigators.

1. Task 1. Perceptual Models and Metrics (548-50-12-xx)

Level 3 Program Lead - Dr. Al Ahumada (ARC/AFH)

1.1 Overall Task Description

1.1.1 Background

The human perceptual system carries information from the operator's environment to his cognitive and action systems. Although the perceptual system gives us the illusion of being in direct contact with "the real world," it is a complex system that uses heuristics to construct a world representation from sensory data that can be sparse and unreliable without any such notification to the user. Understanding the perceptual system is necessary to predict the performance of a human operator and improve display interfaces.

1.1.2 Objectives

The goal of this program is to provide the aviation community with research and technology in perceptual systems that allows scientific assessment of perceptual system capabilities and fosters improved aviation display technology.

1.1.3 Approach

1) Develop improved methods for measuring perceptual system performance. 2) Collect experimental data on perceptual system performance. 3) Develop computational models and metrics that predict perceptual system performance. 4) Develop display technologies that exploit understanding of perceptual systems.

1.2 Sub-Tasks

1.2.1. Sub-Task 1-1: Visibility Models

(POCs - Drs A. Watson and A. Ahumada)

1.2.1.1 Background

A frequently asked perceptual question is how visible a target, a display element, or an image compression artifact will be in certain conditions. Sometimes psychophysical measurements could be taken to answer this question, but often a computational model is what the designer or system engineer really needs, because the target system is not yet realized or the situations are too numerous.

1.2.1.2 Objectives

The goal of this program is to develop computational models which take as input computer images or video sequences of such images, and give as output the probability that an observer can see the difference between a pair of such images or sequences or the probability that specific targets are detectable in such images.

1.2.1.3 Approach

The basic approach to the discrimination problem is obtain additional basic discrimination data and then develop and calibrate the models so they account for the data. The model development is guided by biological vision science as well as psychophysical measurements. While the discrimination models are intended to reflect the limitations of sensory processing, target detection involves the selection of target features as a function of the image content. Techniques will be further developed to examine the learning of target features, to identify observer target templates, and to predict detection performance in noisy or cluttered images when these processes become important.

1.2.1.4 Level 3 Milestones

FY98- Contrast-gain control visibility model FY99-Time domain extension of the contrast-gain control visibility model FY00-Color extension of the contrast-gain control visibility model FY01-Template learning extension of the contrast-gain control visibility model FY02-Public domain distribution of the contrast-gain control visibility model FY03-Shape perception module

1.2.2 Sub-Task 1-2: Eye-Movement Metrics of Human Perception

(POCs: L. Stone, J. Mulligan, and B. Sweet)

1.2.2.1 Background

Although our visual space appears to be a wide-field detailed representation, it is ac- quired by active motion of the eyes, whose total spatial resolution is comparable to standard TV resolution. Research in relating the motion of the eyes to perception and in smarter eye movement measuring devices allows both the possibility of veri- fying display feature perception and the possibility of adjusting the information dis- played as a function of involuntary and voluntary eye movements.

1.2.2.2 Objectives

Establish the relationships between eye movements, perception and construct predictive models of the interaction of the eye movement control and displayed information.

1.2.2.3 Approach

Psychophysical experiments correlating eye movement measurements with observer judgments and control responses will be used to develop and validate models. They will be used to develop guidelines for eye movement controlled information display.

1.2.2.4 Level 3 Milestones

FY98-Quantitative validation of eye-movement metrics of motion perception FY98-Quantitative validation of eye-movement metrics of spatial localization FY99-Calibration algorithm for combined eye/head tracking FY99-Model of display-induced errors in manual control FY00-Measure display effects on temporal dynamics of eye movements FY00-Model of human eye movements during search FY01-Guidelines for using eye-movement metrics for display design FY02-Guidelines for using eye-movement feedback during training FY03-Guidelines for hands-free eye-movement interfaces

1.2.3 Sub-Task 1-3. Image Processing for Improved Displays

(POCs Drs. A. Watson, A. Ahumada, J. Mulligan)

1.2.3.1 Background

Models of the visual system can be used to improve image processing technologies. Although this application can be done by the developers of such technologies, often such application is greatly facilitated by simplifications of the models for the specific application and the simplification needs to be tested psychophysically in its application.

1.2.3.2 Objectives

Use the vision models to develop image quality metrics for specific image processing applications and then develop metric-based image processing technologies for improved display interfaces.

1.2.3.3 Approach

To keep the models computationally manageable, simplified versions of the models are developed for specific applications. Three kinds of simplification are being used:

1.2.3.4 Level 3 Milestones

1.2.4 Sub-Program 4. Metrics and Models of Range and Closure Perception

(POCs M. Kaiser and B. Sweet)

1.2.4.1 Background

The ability of humans to reliably and accurately estimate target range and closure rates is relied upon by many vehicular control tasks. In modern aircraft, pilotage is often based on synthetic displays rather than contact displays; further, there is are strong incentives to maintain airline operations even when visual conditions become degraded. Thus, it is critical to understand how pilots extract range and closure rate information from contact and perspective displays, and how these processes are impacted when information sources are degraded, absent, or in conflict with other visual cues.

1.2.4.2 Objectives

The program goal is to provide guidelines for perspective displays for vehicular control, and evaluation tools to determine the likelihood of pilot error/disorientation under various display and visibility conditions.

1.2.4.3 Approach

Develop models for human performance and evaluate their ability to predict human performance (especially errors) in low, mid, and high-fidelity vehicle control simulations.

1.2.4.4 Level 3 Milestones

1.2.5. Sub-Program 5. Metric and Models for the Perceptual Design of Virtual Transparency

(POC S. Ellis and M. Kaiser)

1.2.5.1 Background

Commercial aviation operations are often constrained by poor visibility due to weather conditions. Modern sensors, however, make it possible to accurately detect the position and orientation of a sufficient number of relevant aircraft for ATC system planners to contemplate new "electronic" flight rules. Under these rules flight operations could safely continue despite instrument meteorological conditions that would otherwise shut down or restrict safe aircraft operation. These new sensors, such as ground radar or infrared vision systems, provide spatial data that must be processed by information systems to schedule and space individual aircraft. But results of the scheduling and spacing algorithms need to be monitored by human operators who ultimately have responsibility for safe operation of the air transportation system. This need is particularly salient for air traffic control tower operation which may become highly restricted during low visibility conditions, but corresponding needs for cockpit operation in modern commercial aircraft with restricted visibility also exist.

1.2.5.2 Objectives

The goal of this sub-element is to determine new psychophysical knowledge and devise new wide field of regard (FOR), i.e. > 180 degrees, visual display technology that will allow perceptually accurate, presentation of spatially conformal aircraft position information so that VFR-like operations can be extended into low visibility conditions and to condition in which line of sight contact with aircraft is blocked by obstructions. This knowledge will enable the design of virtual objects displays that will allow operators accurately to see aircraft through fog and physical obstructions. Thus, the fog and obstructions will be made to appear transparent but the range and direction of the aircraft of interest will be accurately displayed.

1.2.5.3 Approach

Initially, a functional testbed for presenting spatially conformal, virtual image information in an unlimited field of regard will be constructed. It will be used for psychophysical testing of the speed and accuracy with which direction and distance information may be presented via virtual objects made to appear visible though fog and physical obstruction. The psychophysical and oculomotor parameters of the displayed virtual objects will be investigated to determine those settings that allow most accurate spatial perception with acceptable visual fatigue.

1.2.5.4 Level 3 Milestones

1.2.6. Sub-Program 6. Spatial Auditory Displays

(POC: E. Wenzel)

1.2.6.1 Background

The flight deck environment contains multiple channels of auditory and visual information that must be accessed under high-stress, high-workload conditions. The difficulty of segregating monaural audio channels in high level noise can necessitate repeated commands, cause mistakes in communication, and thereby compromise safety as well as audiological health. Visual head-down displays such as those for TCAS may be improved by assigning or co-assigning some situational awareness and alerting functions to a spatial auditory display that allows the pilot's eyes to be out the window during the information acquisition.

1.2.6.2 Objectives

Develop auditory displays that prioritize and spatially segregate auditory information for improved intelligibility and for possible eyes-out-the-window advantages.

1.2.6.3 Approach

Combining 3-D audio technologies with active noise cancellation, the auditory display system controllers can be improved by examining prototype systems in part- task and model simulations. Separate channels of auditory information will be placed at different virtual locations to provide situational awareness (e.g., airborne or ground traffic collision avoidance alerts; taxiway navigation aids and announcements);increase intelligibility (through the use of binaural delivery systems);and reduce auditory fatigue. The simulations studies will be supplemented by basic research in human sound localization and acoustic modeling of the cockpit environment.

1.2.6.4 Level 3 Milestones

1.2.7 Easy Access to Human Factors Databases (POC: ????)

1.2.7.1 Background

Human Factors knowledge is often not incorporated into the design of systems with which human operators closely interact at an early stage of the overall design process. As a consequence, human machine interaction with these systems is often suboptimal and sometimes intolerable. An example of a severe difficulty of this type is found in the human factors problems encountered in the first attempt to duplicate the function of the ATC controllers flight strips on a computer display. One reason for the difficulty of incorporating up-to-date human factors knowledge has been its relative inaccessibility and the fact that it is often expressed in terms that obscure its application to specific purposes.

1.2.7.2 Objectives

The objectives of this subelement is to improve ATC systems designers ease of access to relevant, new human factors knowledge and expertise so that it may be efficiently and effectively incorporated into the design of new ATC technology.

1.2.7.3 Approach

The improvement in access to human factors expertise and data will occur through the development of WEB-based pages corresponding to all of the subelements in this project and which will be maintained by each subelement's PI. These pages will contain the essential reporting information required for the program and will provide pointers to the technical details of the activities on the WEB and elsewhere.

1.2.7.4 Level 3 Milestones

2.Task 2. Cognitive Models and Metrics (548-50-42-xx)

Level 3 Program Lead - Dr. Roger Remington (ARC/AFI)

2.1 Overall Task Description

2.1.1 Background

Human error is the causal or contributing factor in the majority of aviation accidents. It is now well understood that these errors are the product of both the propensity of human to make errors and specific design features that may induce errors. Attempts to reduce human error by improved design have met with only partial success. A significant obstacle to error tolerant design is the complexity of human behavior itself. Simple tests of system interfaces are insufficient to uncover the wide range of errors human operators will make in using them. The nature of human error in aircraft accidents is often puzzling since the crew typically will have performed the same sequence of tasks many times. Thus the accident data shows a seemingly capricious tendency for error, making the measurement of the benefit of new error tolerant systems problematic. For the same reasons, simulations often fail to provide adequate estimates of human error, yet simulations remain the most useful means of examining human behavior in circumstances approximating operational condition. Complexity affects simulations in an additional way: it is difficult to measure and quantify important aspects of human behavior because of the variability inherent in complex performance. Improved understanding of how errors are generated in performing tasks would facilitate the design of error tolerant (or error resistant) systems. Improvements in measuring complex performance will be necessary to adequately test new designs.

2.1.2 Objectives

The proposed work seeks to

2.1.3 Approach

We approach the task of understanding human error by noting that many instances of error reflect failures of executive control. In some cases this failure leads to certain classes of memory errors, such as a failure to remember intended actions (prospective memory failures), as well as habit capture error. A review of ASRS incidents revealed a significant number of such memory failures. In other cases, routine behavior is carried out but without active executive control. This class is most clearly evidenced by monitoring failures, where the observer may actually look at an information source but not apply the executive control needed to process the information. Examination of NTSB accident reports shows that in almost all cases failure to note obvious discrepancies or the failure of the pilot not flying to perform cross-checks are cited as causal or contributing factors in the accident.

In current theories of human cognition, executive control is associated with limited- capacity attention-demanding aspects of information processing and responding. Some aspects of processing can proceed independent of executive control, but except in very special cases cannot control behavior without the intervention of executive control (attention). Common cognitive acts such as fetching items from memory, reading text, solving problems, etc., all require some involvement of executive control, while some early perceptual processes and low-level motor behaviors can often proceed without executive control. The dissociation between executive control and motor programs leads to particular kinds of error, such as the familiar situation of going through the motions of reading a passage without comprehension. A better understanding of the relationship between executive control and the more autonomous information gathering and motor behaviors would lead to significant advances in our understanding of human error. One pressing question is the identification of specific task actions which require attention. The abstract stage models developed through empirical experimentation can be elaborated through imaging techniques that record brain activity directly.

Many important cognitive variables are not directly observable. This has impeded progress in cognitive theory development. Recent advances in brain imaging techniques promise new and largely unobtrusive techniques for identifying underlying processing, even in complex tasks such as piloting. The feasibility of these techniques should be examined. In order to examine the relationship between executive control and information gathering it will be necessary to determine the relationship between eye fixations and attention, since important questions involve the nature of the information acquired outside of controlled processing. This will require the application of eye movement recording. Finally, some of the variability in complex behavior may be due to the lack of sequential constraints in the tasks themselves. While it may seem that behavior is varying, experts may be uniform in the sequencing and timing of specific actions. Indeed, the degree to which tasks impose sequential restrictions may interact with memory or cognitive agenda management to influence errors. It is important to develop measures that assess the variability of human performance against the variability permitted by the task.

We propose to address the question of executive control and of measurement through a research program that includes both empirical research and modeling. Major areas of research are described below with their objectives, and resource requirements.

2.2 Sub-Tasks

2.2.1 Sub-Task 2-1: Eye-Movement Metrics of Human Cognition

(POCs: R. Remington, J. Johston, L. Stone, J. Mulligan)

2.2.1.1 Background

Human error often arises from mistaken beliefs about the state of the world. These beliefs, like other aspects of cognition, are not directly observable, posing a problem for a cognitive analysis that could reveal the source of error. The pattern of information acquisition from visual displays, as revealed by patterns of eye fixation, promises to shed light on the cognitive processes that shape operator behavior, and lead to the generation of human error.

2.2.1.2 Objectives

The goal of this element is to develop direct tests of the usefulness of eye movements for inferring behaviors of interest. If successful, further efforts will be directed at developing a system for using eye fixation patterns to infer cognitive state that can be applied to man-in-the-loop simulations. The ultimate goal is a validated technique for finer analysis of operator information gathering strategies.

2.2.1.3 Approach

We first explore tasks for which the information requirements are well understood. For example, by having controllers make judgments that entail the use of altitude information we can test whether eye movements directly reflect information gathering. Since altitude is available only in the data block, fixations necessary to read the information will be necessary. If successful in using eye fixations to infer cognitive goals, we will use a similar paradigm to examine conditions under which controllers actively seek information as opposed to retrieving information from memory. Models of human behavior have difficulty in deciding when information will be gathered anew or existing information retrieved from memory. By varying the relative difficulties and length of time since a particular piece of information was used, it will be possible to better understand how mistaken impressions of aircraft state can occur in the face of clearly disconfirmatory evidence.

2.1.3.4 Level 3 Milestones

2.2.2 Sub-Task 2-2: Models and Metrics of Human Executive Control

(POCs: R. Remington, J. Johston)

2.2.2.1 Background

The allocation of attention and executive control is at the center of our understanding of how information is selected from the world. This in turn determines the knowledge of the user with respect to the state of the world at any point in time. Here we address the issue of how external stressors influence the allocation of executive control. In particular we focus on the role of time pressure in determining how operators sequence between tasks and information sources.

2.2.2.2 Objectives

Extend and validate the APEX model of executive control by specifying the role of executive control in cognitive processing and how control failure leads to human error in complex task environments.

2.2.2.3 Approach

Our hypothesis is that time pressure leads to a truncating of normal processing, leading to a first-come-first-served task allocation strategy. We pursue this by both empirical testing, by modeling, and by direct observation of brain activity. The existing APEX model of executive control developed in the Cognition Laboratory at NASA Ames will be refined and used as the basis of modeling.

2.2.2.4 Level 3 Milestones

2.2.3 Sub-Task 2-3: Models and Metrics of Human Spatial Attention and Memory

(POC: W. Johnson)

2.2.3.1 Background

Two and three dimensional spatial displays of traffic, weather, and terrain are expected to become common on flight decks in the next 10 years. These displays are also expected to provide information such as heading, altitude (for 2-D displays), speed, expected flight path, threat/conflict level, etc., and ATC airspace restrictions. A significant requirement of such cockpit displays is that they provide the necessary information and quot;situational awarenessquot; without forcing the pilots into a quot;heads- downquot; mode. That is, these displays must not require prolonged withdrawal of attention from primary flight tasks. Optimal displays need to be designed such that they enhance attention to critical elements, and thus allow greater efficiency during periodic monitoring. Two of the primary determinants of attention on displays are simple salience and display organization, or structure. Determining how these two properties influence attention individually, and in combination, will aid in the design of more optimal displays.

2.2.3.2 Objectives

The goal of this project is to measure attentional distribution over a spatial display as a function of salience (various forms of highlighting), and as a function of display structure (frames of reference and grouping properties), and then to generate a corresponding predictive model of attentional distribution.

2.2.3.3 Approach

Experiments will be run correlating highlighting characteristics (intensity, motion, color) and display structure (grouping, absolute vs. relative reference frame) with measurements of information detection accuracy /latency, information retention, and also with eye-movements. This data will then be used to develop predictive models of attentional allocation for display design.

2.2.3.4 Level 3 Milestones

3. Task 3. Physiological Factors (548-50-22/32)

Level 3 Program Lead - Dr. Mark Rosekind (ARC/AFO)

3.1. Overall Task Description

3.1.1 Background

Flight operations create fatigue, sleep loss, and circadian disruption leading to significant decrements in alertness and performance. Furthermore, the performance problems can occur before the degraded alertness is even detected and reversed. By increasing the chances of an accident or incident, these alertness and performance decrements reduce the margin of safety.

3.1.2 Objectives

The goal of this program is to understand the physiological underpinnings of fatigue, circadian rhythm disruption, and faltering alertness, to determine their operational effects, and to provide the aviation community with a range of products from research findings to countermeasure strategies and policy input to minimize adverse effects, maximize performance and alertness during operations, and maintain or improve the safety margin.

3.1.3 Approach

3.2. Subtasks

3.2.1. Fatigue Countermeasures (POC: M. Rosekind)

3.2.1.1 Background

Flight operations create fatigue, sleep loss, and circadian disruption that lead to reduced performance, alertness and safety. The Fatigue Countermeasures Program directly supports safety goals through the development of fatigue countermeasures, educational tools, incident/accident investigation methods, and providing technical input to national policy considerations.

3.2.1.2 Objectives

The objective of the Fatigue Countermeasures Program is to minimize the adverse effects of fatigue and maximize performance and alertness during flight operations, thereby maintaining, and where possible, improving the safety margin. This objective is accomplished through the following activities:

3.2.1.3 Approach

The Fatigue Countermeasures Program conducts a complementary research approach that capitalizes on laboratory-based experimental research, flight simulations, and field research during regular operations. Ground-based, controlled laboratory investigations allow safe examination of operational issues (e.g., total and cumulative sleep loss) that could not be conducted in field studies. Operational/flight research involves intensive investigation of fatigue factors during actual flight and field operations. Flight simulations provide a setting for collecting operational flight variables in study manipulations that would not be possible or safe during regular operations. A variety of physiological, performance, behavioral, self-report, environmental, and survey measures are used in all of these research approaches.

3.2.1.4 Level Milestones

3.2.2. Hazardous States of Awareness (POC: A. Pope)

3.2.2.1 Background

The Hazardous States of Awareness subelement develops and validates human response measurement technologies for assessing human attention and awareness hazards. The effort develops methods for identifying task design factors associated with task underload, and identifies and tests crew procedural and system design factors which contribute to effective and hazardous states of awareness. These activities are in support of the goal of the Critical Technologies Program of the NASA Implementation Plan for the National Plan for Civil Aviation Human Factors (p. 34), and Recommendations SA-8 and SA-9 of the FAA Human Factors Team Report on The Interfaces between Flightcrews and Modern Flight Deck Systems.

3.2.2.2 Objectives

The objective of this subelement include 1) Development and validatation of methods and techniques for identifying hazardous states of awareness, such as complacency, boredom, and preoccupation, in automated-system design and 2) the exploitation of opportunities to demonstrate dual-use applications of methods, techniques and principles in fields within aeronautics as well as beyond, such as process control and medicine

3.2.2.3 Approach

The approach taken includes establishing basic concepts and theories, developing and validating new concepts in collaboration with universities, proving innovative techniques through analysis, simulation, and laboratory testing, and, ultimately, demonstration of the most promising concepts in operational environment tests. Technology transfer mechanisms include demonstrations of methodological innovations to industry and media at LaRC and at technology transfer expositions, MOA as described below, and contributions to transfer publications and databases. Metrics include number of requests from customers and partners in aerospace and non-aerospace industries for our technology, and number of actual uses by customers.

3.2.2.4 Level Milestones