Health-Related Warning Message Processing
Summary and Keywords
Warnings are risk communication messages that can appear in a variety of situations within the healthcare context. Potential target audiences for warnings can be very diverse and may include health professionals such as physicians or nurses as well as members of the public. In general, warnings serve three distinct purposes. First, warnings are used to improve health and safety by reducing the likelihood of events that might result in personal injury, disease, death, or property damage. Second, they are used to communicate important safety-related information. In general, warnings likely to be effective should include a description of the hazard, instructions on how to avoid the hazard, and an indication of the severity of consequences that might occur as a result of not complying with the warning. Third, warnings are used to promote safe behavior and reduce unsafe behavior. Various regulatory agencies within the United States and around the globe may take an active role in determining the content and formatting of warnings.
The Communication-Human Information Processing (C-HIP) model was developed to describe the processes involved in how people interact with warnings and other information. This framework employs the basic stages of a simple communication model such that a warning message is sent from one entity (source) through some channel(s) to another (receiver). Once warning information is delivered to the receiver, processing may be initiated, and if not impeded, will continue through several stages including attention switch, attention maintenance, comprehension and memory, beliefs and attitudes, and motivation, possibly ending in compliance behavior. Examples of health-related warnings are presented to illustrate concepts. Methods for developing and evaluating warnings such as heuristic evaluation, iterative design and testing, comprehension, and response times are described.
Promoting Health and Safety Is a Fundamental Goal of the Human Factors/Ergonomics (HF/E) Discipline
Human Factors/Ergonomics (HF/E) is an academic and professional discipline that deals with the design of various kinds of systems based on people’s abilities and limitations to promote productivity, satisfaction, and safety (Salvendy, 2012; Sanders & McCormick, 1993). Within the context of healthcare, the term system refers to the physical and cognitive aspects of people doing tasks in environments with tools often involving technology. These components constituting the system interact together to accomplish some goal (Carayon, 2012). Consider a surgeon’s goal of repairing a patient’s hernia. If the surgeon uses a new medical device (technology) that was recently purchased to replace an older device, she (or he) may have minimal familiarity (cognitive) with this particular system and erroneously use previous experience with the earlier device to direct her (or his) efforts, potentially resulting in poor motor coordination (physical). Threats to safety frequently involve some form of human error that may result from improper interaction with technology or ambiguous work processes (Bogner, 2004). In fact, some cases of human error can result from poor equipment or system design rather than operator shortcomings. Since about the mid-1980s, a substantial amount of HF/E research has focused on improving safety and reducing error through the communication of hazard information through warnings (see Wogalter, 2006 for a comprehensive review).
Warnings in the Health Context
Warnings are a type of risk communication message intended to inform users of potential hazards and to provide instructions to minimize the likelihood of adverse events. Given the complexity and diversity of the health context, warnings can appear in a variety of applications. For instance, a physician may encounter “black box” warnings (i.e., a specially formatted warning message mandated by the U.S. Food and Drug Administration) in labeling for an implantable medical device. Likewise, other health professionals such as pharmacists or registered nurses may encounter prescription product inserts (PPI) associated with prescription medications that they disseminate to patients. Outside these clinical, managed environments, everyday consumers may encounter health-related warnings on over-the-counter (OTC) medication labels or on more mundane items such as exercise equipment or a heating pad.
Although warnings might be encountered in an assortment of situations, warnings generally serve three distinct purposes (Conzola & Wogalter, 2001). First, warnings are used to improve safety by reducing the likelihood of events that might result in death, personal injury, or property damage. Second, they are used to communicate important safety-related information to a target audience such as healthcare professionals (e.g., physicians, nurses, etc.) or consumers in the public domain. In general, warnings should include a description of the hazard, instructions on how to avoid the hazard, and a mention of the severity of consequences that might occur as a result of not complying with the warning (see e.g., Rogers, Lamson, & Rousseau, 2000; Wogalter et al., 1987). Lastly, warnings are used to promote safe behavior and reduce unsafe behavior. Warnings can also serve as reminders (or cues) about information that people already know about but that is not in conscious awareness (Leonard, Otani, & Wogalter, 1999). For example, hospital employees need to be periodically reminded of information provided in previous safety training where they were instructed to assume that all bodily fluids are potential sources of infection and that prior to interacting with these substances they must don personal protective equipment such as face shields and protective gloves.
Because warnings are only one approach to promoting safety, it is important to understand a fundamental principle of safety. Entities such as manufacturers, distributors, and providers should analyze a product to determine whether there are any potential hazards associated with its use and foreseeable misuse before end users encounter them. There are several formal methods of hazard analysis such as failure modes and effects analysis (FMEA), Fault Tree Analysis (FTA), and Critical Incident Analysis (CIT) to assist in identifying potential hazards that could be dangerous to users (e.g., Israelski & Muto, 2012). When hazards are discovered as a result of such analyses, these entities are responsible for controlling the hazards in an effort to reduce the likelihood and severity of an adverse event.
Hazard Control Hierarchy
When product-related hazards have been identified, procedures should be employed to control those hazards. A well-accepted method of hazard control is to use the prioritized strategies of a hazard control hierarchy to guide hazard reduction (Laughery & Wogalter, 2006; Lenorovitz, Karnes, & Leonard, 2014). Manufacturers (and others) can use this guidance to search for and employ safety methods to limit the risks to users. The first and best strategy for controlling hazards is to design them out or eliminate or reduce them. For example, a pharmaceutical manufacturer should search for chemical compounds that are clinically effective and entirely or relatively safe, with no or minimal harmful side effects. Finding a way to eliminate or at least reduce the dangerous aspects of products is the best way to control hazards.
If the first strategy is not feasible or practical, then a second best strategy is to guard against the hazard. There are many ways to separate people from the hazard. One example is child-resistant caps on drug containers. There can be certain areas of a hospital or clinic in which only qualified persons can enter, for example, a control room for equipment used in radiation therapy. Also, the U.S. Food and Drug Administration (FDA) sometimes limit distribution of certain prescription drugs to hospital pharmacies where only specialists can prescribe them in an attempt to closely monitor their usage and effects in controlled environments.
Only when these first two strategies, designing out and guarding against, are not possible or impractical should warnings be used as the main method of controlling hazards. Warnings are used to “guard” against residual hazards that remain following the implementation of design and guarding considerations. Warnings, the third level of the hazard control hierarchy, are the last-resort approach, and they should not be used as a substitute for proper design and guarding (Lehto & Salvendy, 1995). Of course, there is nothing preventing the use of warnings in conjunction with efforts to design out or guard against a hazard. Warnings function via cognitive and behavioral mechanisms because they are meant to influence people as the method of controlling the hazard. For example, a physician may make a decision to prescribe a prescription drug based on the labeling approved by the FDA. Therefore, if the goal is to protect patients from harm in using a product where the hazard has not been controlled by design or guarding, then the third strategy in the hierarchy, warning about the hazard, should be used. Warnings ought to be designed to maximize their effectiveness to appropriately influence people’s cognitions and behavior. Unfortunately, warnings are usually considered the least reliable of the three basic hazard control strategies. If the warning system is poorly designed, physicians (and other healthcare professionals) and patients may not see the warning, they may not read the warning, they may not understand the warning, they may not believe the warning, and the message may not be sufficiently strong to influence behavior—all of which are stages of processing that may lead to a failure of the warnings to do their intended purpose and therefore could potentially result in consequential injury or death. Given the recognized critical nature of warnings, several government regulatory agencies within the United States (and around the world) are tasked with providing oversight for many kinds of health communications.
The Role of Regulatory Agencies
Due to the variety of products that might contain warnings, a number of government agencies in the United States provide regulatory guidance on the formatting and content of warnings. For instance, the FDA oversees the labeling of prescription and over-the-counter (OTC) drugs (see Ostrove, 2006) as well as medical devices. Pharmaceutical manufacturers must comply with various 21 CFR (Code of Federal Regulations) 201 labeling laws. Black box warnings are a special device used by the FDA to direct healthcare providers’ attention to particularly serious side effects, contraindications, and precautions for the purpose of avoiding serious injury or death. For example, consider a black box warning that prescribers of a powerful analgesic (fentanyl patch) would encounter in its labeling to assist in making decisions to provide this drug to patients.
While the FDA regulations address labeling for drugs and medical devices, the U.S. Consumer Product Safety Commission (CPSC) regulates warnings on consumer products, and the Occupational Safety and Health Administration (OSHA) is concerned with work-related hazards usually involving equipment and chemicals. The CPSC monitors the safety of consumer products such as exercise equipment and heating pads by tracking and analyzing adverse events such as injuries and deaths. If necessary, the CPSC has the regulatory power to induce manufacturers to recall products that are reported as dangerous. Figure 1 demonstrates an example of a warning that appears on the container for an exercise band that can be purchased by consumers.
The CPSC also interacts with manufacturers and other industry entities to create voluntary or consensus standards for warning labels such as the American National Standards Institute’s Product Safety Signs and Labels Z535.4 standard of the American National Standards Institute (ANSI) (American National Standards Institute, 2002).
Because hospitals or medical practices are places of work, OSHA is involved in hazard prevention. OSHA may collect and disseminate information about hazards regarding ergonomic injuries in moving patients. OSHA provides mandatory standards for the design of accident prevention signs and tags that are published in the CFR, Volume 29, section 1910. Consider for example, a physical therapist or nurse tasked with cleaning a hydrotherapy pool. Such an individual, as part of his or her job, might encounter a chemical warning such as one designed to inform users about the corrosive hazards associated with a pool cleaner product.
Many of the OSHA regulations are based on ANSI standards (Monroe & Orr, 2006). The Hazard Communication Standard is used by OSHA to evaluate how employees are informed of hazards and protective measures in the workplace. Labels on containers as well as more detailed technical bulletins known as safety data sheets (SDSs) are channels for manufacturers and employers to identify potential hazards in the workplace environment.
As we live in a global economy, it is important to understand that many countries have similar agencies. For example, the Health and Safety Executive (HSE) in the United Kingdom serves to regulate warning labeling and safety signage in the workplace in the same vein as OSHA within the United States. Many agencies outside the United States utilize warning standards generated by the International Organization for Standardization (ISO 3864-2011). Likewise, many agencies around the world have collaborated to create the Global Harmonization System (GHS) of Chemical Hazard Communication so that laws, standards, and hazard information regarding chemicals are standardized in workplaces (United Nations, 2015).
While these regulatory standards are important to protecting the safety and health of all parties involved, they are often predicated on understanding how and why people (regardless of their roles) interact with warnings.
Assessing Warning Quality with the Communication-Human Information Processing (C-HIP) Model
A number of models (Edworthy & Adams, 1996; Lehto & Miller, 1986; Lindell & Perry, 2004; Rogers, Lamson, & Rousseau, 2000; Wogalter, DeJoy, & Laughery, 1999) have been proposed to describe the warning process. The Communication-Human Information Processing (C-HIP) model described by Wogalter (2006) is consistent with other frameworks in that it describes how characteristics of the user interact with the physical attributes of the warning to influence behavior.
The C-HIP model has two major sections, each with several component stages. A representation of the model can be seen in Figure 2. The first section of the framework employs the basic stages of a simple communication model. Here the model focuses on a warning message being sent from one entity (source) through some channel(s) to another (receiver).
The second major section of the model focuses on the receiver and how people cognitively process information. For this to occur, effective delivery of the warning information to members of the target audience is necessary. Once a warning is delivered to the receiver, processing may be initiated and, if not impeded, will continue through several stages including attention switch, attention maintenance, comprehension and memory, beliefs and attitudes, and motivation, possibly ending in compliance behavior.
The C-HIP model serves as both a stage and a process model. The model is useful in describing a general sequencing of stages and the effects warning information might have as it is processed. If information is successfully processed at a given stage, the information diffuses to the next stage. If processing at a stage is unsuccessful, the flow of information will be obstructed and will not reach the next stage. If a person does not initially notice or attend to a warning, then processing of the warning goes no further. However, even if a warning is noticed and attended to, the individual may not understand it, and, consequently, no additional processing occurs beyond that point. Should further processing occur following incomplete or erroneous understanding of the message, it is likely that safety will be impacted. But even if the message is understood, it still might not be believed to be credible, thereby causing a blockage to occur at this point. Even with believing the safety message, low motivation (to carry out the warning’s instructed behavior) can be an impediment to further processing. If all of the stages are successful, the warning process could result in safety behavior (compliance) attributable to the warning information.
Although the model appears to emphasize a linear sequence from source to behavior, there are feedback loops from later stages in the process to earlier stages of processing as depicted by the arrows on the right side of Figure 2. For example, if a warning stimulus were to be seen or heard repeatedly over time and exposure resulting in habituation, then less attention will be given to it on subsequent occasions. A more specific example could be given in terms of prescription pharmaceuticals (Guchelaar, Colen, Kalmeijer, Hudson, & Teepe-Twiss, 2005). If a new hazard is added to a warning, a pharmacist may not notice it if she (or he) had previously read and was familiar with the earlier version. Here, the later stages of comprehension (i.e., memory) affect the earlier stages, attention switch and maintenance. A second example of processing feedback from latter stages of the process to earlier stages concerns the influence of beliefs on attention. Products believed to possess low hazard (i.e., relatively safe) may not entice users to look for or attend to a warning. Thus if a healthcare professional believes that a commonly prescribed drug has few or limited side effects or other adverse consequences, he or she will be less likely to read a new warning about significant drug interactions (Russ, Zillich, McManus, Doebbeling, & Saleem, 2012). Thus, a later stage concerning beliefs and attitudes affects the earlier stages of attention to a warning. Factors affecting each stage of the C-HIP model are described.
The source (for example, a person or manufacturer that has a responsibility for warning) is the initial transmitter of the warning information. The source assumes the critical role of determining if there are hazards present or potentially present that necessitate a warning through some form of hazard analysis such as failure modes and effects analysis (FMEA) as previously discussed. Consistent with the aforementioned hazard control hierarchy, a source should initially consider if there are better ways of controlling the hazard than warnings (Laughery & Wogalter, 2006). This role is important because if the source fails to adequately identify and control hazards, people can be injured or killed.
The channel includes the medium and modality for information being transmitted from the source to receivers. Warnings can be transmitted in many ways: in labels directly on products (e.g., pill bottles, exercise equipment, etc.), on shipping containers, in user manuals, in package inserts, in brochures, in advertising in the print media and in broadcasting, by face-to-face meetings (e.g., pharmaceutical sales), or via the Internet (e.g., direct-to-consumer [DTC] advertising from the manufacturer or third-party sites).
There are two dimensions of the channel: the medium where the information is embedded (e.g., label, prescription product insert [PPI], Dear Doctor letter, emailed advertisement), and the sensory modalities involved (visual, auditory). Some media involve a single modality (e.g., a printed drug label involves visual perception); others may involve two modalities (e.g., video usually has both visual and auditory information). Moreover, visually presented information can be in the form of text or graphics, such as symbols. One example of a symbol is the traditional skull and crossbones image that appears on labels to connote poison.
Some evidence suggests that multi-modal warnings are more effective than single modality warnings because they provide redundancy (Baldwin et al., 2012). Also Dual Code Theory from the cognitive psychology literature suggests that multi-modal presentation enhances learning because the information is richer and can be further enhanced by links to internal representations within long-term memory (Clark & Paivio, 1991).
While warnings may be disseminated through several channels, sometimes they might not reach all of the targets at risk. Delivery refers to the point of reception where a warning arrives at the receiver. To emphasize its importance as the literal gateway to the receiver, it is shown as a separate stage in the current Communication-Human Information Processing (C-HIP) model (see Figure 2). A warning that a person sees or hears is a warning that has been delivered. A warning in a pamphlet that is produced by a manufacturer but never leaves corporate headquarters or sits in boxes in a warehouse is ineffective with respect to end users because they never get it. Whatever utility these warnings might have had has not been realized. Because warnings may miss being delivered to individuals, manufacturers need to consider using multiple channels so as to increase the likelihood that the parties at risk will receive the warning.
Other stimuli are almost always simultaneously or concurrently present with warnings. These other stimuli can be other warnings or a wide assortment of non-warning stimuli in the environment. These stimuli compete with the warning for attention and could interfere with warning processing. Interference is more likely if the other stimuli in the environment are highly salient (conspicuous or prominent). Consider the role of an anesthesiologist working as part of a surgical team within a hospital operating room environment. Previous research from Shapiro and Berland (1972) suggested that the noise levels in this context can often exceed 90 decibels (i.e., similar to the sound of a nearby accelerating motorcycle or loud truck). Loud ambient sound of certain frequency spectra can mask the sounds of an important alarm causing a health professional to miss noticing it (Weinger, 2004).
The receiver is the person to whom the warning is directed. Before a warning can effectively influence the receiver, it must first be delivered. Once delivered, the receiver must switch his or her attention to the warning and maintain attention long enough to extract the necessary safety information. Next, the warning must be understood and also coincide with the receiver’s existing beliefs and attitudes. Finally, a receiver must be motivated to perform the directed behavior.
Typically, warnings compete for people’s attention in relation to other stimuli in the environment. For this reason, warnings should be salient (prominent and distinctive) from their surrounding environments so that users will more likely notice them and direct their attention to them. There are several warning-design factors that influence how well warnings can draw attention (Wogalter & Vigilante, 2006). Some of those attributes are described.
Color is a frequently used attribute to facilitate the likelihood that attention will switch to a warning (Wogalter, Mayhorn, & Zielinska, 2015). The ANSI Z535 (American National Standards Institute, 2002) warning standard uses color as one of several components of the signal word panel to attract attention. For instance, a warning meant to convey the hazards of a hot surface might be colored red to denote heat.
Graphical components such as symbols and icons can also be useful for capturing attention. One commonly used method to do this uses an alert symbol (a triangle enclosing an exclamation point) shown on the upper left corner, next to the signal word. It is used in the ANSI Z535 warning label and sign standard for nonchemical products. This icon is a general alert, but others serve more explicit purposes. For instance, Bzostek and Wogalter (1999) found people were faster at locating a warning on a medicine label when it was accompanied by an icon or symbol, such as a skull and crossbones, described earlier. The skull and crossbones also carries with it some information about a poison-related hazard.
The top portion of an ANSI Z535 warning, sometimes called the signal-word panel, has several features designed to attract attention. ANSI-style signs use relatively large print, color, and an alert symbol. It is more likely to capture attention than simple running text.
One of the potential downsides of consistently using a recommended configuration, as warning design standards (such as ANSI or International Organization for Standardization [ISO]) generally advocate, is that it could negatively affect attention, through habituation processes. In other words if all warnings were constructed the same way with similar features, then seeing the same kinds of warnings over and over could result in inadequate attention being given to them in the future. Repetitive exposure can reduce a warning’s attention attraction capability (Thorley, Hellier, & Edworthy, 2001). Because a fundamental purpose of warnings is to attract attention, designers must be sensitive to this issue when effectiveness is critical. It may be important to change the warnings from a standard configuration and appearance if it is known that habituation is likely. The process of habituation can eventually occur even with well-designed but repeatedly exposed warnings; however, features such as distinctive shapes and color can likely slow the habituation process.
Warning salience can be determined empirically in a number of ways. The simplest way is to ask participants to rate on a numbered scale how well a warning attracted their attention when features (such as color, presence of symbols, etc.) are manipulated experimentally (Zielinska, Wogalter, & Mayhorn, 2014). Measuring reaction time or speed of responses provides a more objective measure of salience (e.g., Bzostek & Wogalter, 1999; Laughery, Young, Vaubel, & Brelsford, 1993). Additionally, eye movement studies can measure where people make initial glances and saccadic movements to various parts of visual materials (Laughery, Young, Vaubel, & Brelsford, 1993).
Even if a warning is noticed, target audiences may not stop to examine it. Attention must be maintained long enough for its content to be understood. Attention maintenance requires some length of time for information to be encoded into memory and assimilated with existing knowledge stored in long-term memory.
With brief text or symbols, the warning message may be grasped very quickly, sometimes as fast as at a glance. For longer, more complex warnings, attention must be held for a longer duration to acquire the information. To maintain attention in these cases, the warning needs to have qualities that evoke interest, so that the person is willing to maintain and focus his or her attention on it. The effort necessary to acquire the information should be as little as feasible. Or in other words the information should be provided in ways so that it can be grasped as easily as possible. Some of the same design features that facilitate the switch of attention (mentioned earlier) also help to maintain attention. For example, large print not only attracts attention, it also tends to increase legibility, which makes the print easier to read.
Print legibility can be affected by numerous factors, including choice of font, stroke width, letter compression and distance between them, resolution, and justification (see Frascara, 2006). Although there is not much research to support an unequivocal preference for particular fonts, the general recommendation is to use relatively plain, familiar alphanumeric lettering. It is sometimes suggested that sans serif font like Helvetica, Futura, and Univers be used for large text sizes and a serif font like Times, Times Roman, and New Century Schoolbook be used for smaller sized text, but in reality there is not much difference as long as they are not fancy, elaborate, unfamiliar fonts. Recent work with the fonts of pharmaceutical labels notes that particular font characteristics aid in discriminating between drug names that look or sound similar to one another (Dehenau, Becker, Bello, Liu, & Bix, 2015).
Mixed case, with both upper and lowercase, type is more legible than type set in all uppercase type (e.g., Tinker, 1963; Poulton, 1967). Moreover, formatting can assist in information acquisition by creating distinct categories that can assist in parsing ideas such that information is easier to search and assimilate into memory (Hartley, 1994). A list or outline format is preferred over continuous prose text (Desaulniers, 1987). Also, structured formatting reduces perceived difficulty and mental workload (Mendat, Watson, Mayhorn, & Wogalter, 2005). For instance, current over-the-counter (OTC) pharmaceutical product labels display the “Drug Facts” format as required by the U.S. Food and Drug Administration (FDA). Previous research suggests that consumers are quicker at extracting warning-related information from this standardized, well-formatted label than from labels without a standard format (e.g., Kalsher, Wogalter, & Racicot, 1996; Wogalter, Shaver, & Chan, 2002).
Comprehension and Memory
Comprehension, in this context, concerns understanding the intended message of a warning. Comprehension may derive from several components: inferred understanding such as hazard connotation, interpretation of language and symbols, and an individual’s background knowledge. Background knowledge refers to relatively permanent long-term memory structures that may have resulted from previous exposure to safety information such as job-related training, safety meetings, or from reading manuals or pamphlets associated with particular equipment. Some major conceptual research areas with respect to comprehension and warnings are briefly reviewed.
According to Wogalter et al. (1987), the content of warning messages should generally include three main informational components: information about the hazard, instructions on how to avoid the hazard, and the potential consequences if the instructions are not followed and the hazard is not avoided. Additional information may be required beyond these general categories. For instance, explicit descriptions of hazards are more likely to encourage users to act cautiously than general information (Laughery, Vaubel, Young, Brelsford, & Rowe, 1993). Consider the warning depicted in Figure 1 for an exercise band/door anchor (attachment). It has some of the components discussed in this chapter, but it lacks a description of how an injury might occur during use. Many users might not realize that the door anchor might separate from the door frame while force is exerted and snap back with great force into the face and eyes, and thus being sure that the attachment never comes undone during exercise is critical. It is likely that users would benefit from explicit information regarding the hazard, instructions, and consequences associated with this product, as the potential for it to forcefully snap back toward the user and strike the body (possibly an eye) is not expressed. An explicit warning would likely be more useful in terms of avoiding injury than generic advice to see a doctor before adopting a new exercise regimen. It would elicit knowledge and awareness that might not be apparent to users without the explicit cue.
Safety symbols can provide information about one or all of the main categories of message content. They may be used in lieu of or in conjunction with text statements (e.g., Mayhorn & Goldsworthy, 2007, 2009; Mayhorn, Wogalter, & Bell, 2004; Wolff & Wogalter, 1998). They can sometimes be used as a means to communicate to people who do not understand the text components, possibly due to illiteracy or for those who do not speak the language used in the warning.
Hazard comprehension of symbols can have a direct influence on consumer health as illustrated by the FDA’s identification of medications used to treat a variety of clinical conditions such as recalcitrant nodular acne (Accutane), male pattern baldness (Propecia), and cancer (Thalidomide). These particular substances are recognized as teratogens because they cause varying degrees of birth defects. In some cases, even brief exposure to these medications during pregnancy or prior to conception can cause significant harm to the fetus (Meadows, 2001; Perlman, Leach, Dominguez, Ruszkowski, & Rudy, 2001). As a result, symbols were developed to communicate this teratogenic hazard to the public, but these were not always effective. A symbol comprehension study investigated how well people understood the existing symbol when compared to experimental symbols designed by the researchers to convey the likelihood of birth defects when pregnant women were exposed to prescription medications with teratogenic properties (Mayhorn & Goldsworthy, 2007).
To gauge effectiveness, symbols should be designed to have the highest level of comprehension possible. The ANSI Z535 (2012) standard establishes a method of assessment and further suggests a goal of at least 85% comprehension using a sample of 50 participants deemed representative of the target audience for a symbol that will be used without accompanying text. If 85% comprehension cannot be achieved, the symbol may still have utility by aiding attention switch, yet it must be recognized that some kinds of errors are worse than others. For instance, symbols should not be misinterpreted in ways that could increase the potential for injury. According to the ANSI Z535 standard, an acceptable symbol must produce fewer than 5% critical confusions (opposite or wrong answers that might lead to unsafe behavior) within the suggested sample of 50 participants. ISO has similar comprehension criteria (Peckham, 2006).
According to these comprehension criteria, the results from Mayhorn and Goldsworthy (2007) demonstrate that the rate of correct interpretation (68%) elicited by the baseline symbol failed to meet the ANSI criterion for acceptability of 85% correct comprehension in the absence of text. In fact, although none of the alternate symbols in development surpassed the 85% ANSI criteria, five exceeded the levels of correct interpretation elicited by the baseline symbol, with two approaching the ANSI criteria. Likewise, none of the symbols exceeded the ANSI limit of 5% critical confusion. Given these values, Mayhorn and Goldsworthy (2007) concluded that two of the alternate symbols might be more effective than the baseline symbol because they were more likely to elicit correct interpretations of the consequences of medication usage. A review of symbol comprehension literature can be found in Wogalter, Silver, Leonard, and Zaikina (2006). Additional research on pictorial symbols can be found in Sojourner and Wogalter (1997, 1998) and Wogalter, Sojourner, and Brelsford (1997).
People possess vast stores of knowledge in long-term memory collected from previous experience yet only a small portion of that knowledge is consciously available at any given time. As people are performing their daily tasks, their minds may be focused on the activities at hand (or something else) and not much attention, if any, may be given to processing risk- or safety-related information. Thus, hazard-related knowledge may not be actively brought to bear unless there is a cue to activate it. The cue can come from the presence of a warning. In effect, the warning creates an event-based prospective memory task where external cues are used as a memory aid to perform some safety-related action such as taking medications on time (Park, Hertzog, Kidder, Morrell, & Mayhorn, 1997). Previous research has demonstrated that the use of handheld computers for medication reminding facilitates adherence to medication regimens (Mayhorn, Lanzolla, Wogalter, & Watson, 2005); thus, health-related warnings can be delivered electronically, which is a very promising method to aid adherence.
Level of Knowledge
The knowledge level of warning recipients encompasses their language skill as well as their technical knowledge. Language skill can be assessed based on a person’s ability to read text. Reading levels for warning text that targets the general public should be as low as reasonably possible. Readability formulae such as that popularized by Flesch (1948) and others can also be used to assess passages of text (e.g., by taking a sample of 100 words), but while convenient, these formulae should be used with some caution because they can give wrong assessments based on the specific criteria they use (e.g., word length and frequency of use) to calculate scores. As mentioned earlier, comprehension of a proposed warning can be assessed through an open-ended test of whether people understand the hazard and the consequences and instructions statements. The open-ended test and cognitive interview (see e.g., Brantley & Wogalter, 1999; Wolff & Wogalter, 1998) are considered “gold-standard” methods of assessing comprehension.
Warnings for medical products intended for highly trained individuals such as health professionals do not need to be written at the reading levels that the general reading public would need, but should, nevertheless, provide appropriate instructions and verbiage, consistent with recipients’ training, to promote understanding of the material. Technical experts have a more complete understanding of domain-specific hazards and can benefit from appropriate technical language and data that are capable of cuing their pre-existing extensive knowledge on the topic. Less expert laypersons cannot be expected to have extensive knowledge, and so the material needs to be understandable given their lower skill and knowledge level. Consider the black box warning previously described for fentanyl patches. When deciding whether to prescribe this particular analgesic patch, a physician may understand that patients must have certain characteristics such as not being opioid intolerant, not managing short-term pain, not taking cytochrome P450 3A4 inhibitors, and not managing intermittent post-operative pain (among other considerations). However, for patients who are “ordinary,” who do not have extensive medical training or knowledge in this domain, the content of the black box warning would not likely be fully understood.
Beliefs and Attitudes
Beliefs refer to an individual’s knowledge base that is subjectively (and perhaps erroneously) accepted as true based on knowledge stored in long-term memory. Attitudes are similar to beliefs except they involve emotion.
There are two design aspects of warning that can be assigned to beliefs although are commonly ascribed to comprehension or attention stages. These are attributes built into the warning itself that can connect to general feelings (attitudes or affective) conveyed in a message.
The American National Standards Institute (ANSI) Z535 standard, and other standards, designate three, specifically defined signal words (DANGER, WARNING, and CAUTION) as intended to convey different levels of hazard probability and severity. DANGER connotes more significant injury (i.e., risk or likelihood of injury as well as the magnitude of injury) than either WARNING or CAUTION (Wogalter, Kalsher, Frederick, Magurno, & Brewster, 1998; Wogalter & Silver, 1990). Research indicates that people do not readily distinguish between CAUTION and WARNING.
According to the ANSI Z535 standard, the three signal words are assigned specific colors: red for DANGER, orange for WARNING, and yellow for CAUTION. Most research shows that red is rated higher than other colors, yet people do not distinguish between orange and yellow in hazard-level connotation (Chapanis, 1994; Mayhorn, Wogalter, & Shaver, 2004).
Other design aspects reflect the verbal and symbolic information in the main message text of a warning. Generally effective warnings concur (or at least are not discrepant) with the receiver’s current beliefs and attitudes. Warning information that is similar to existing beliefs and attitudes is easily assimilated into memory, has knowledge tied to it, and does not require much effort to process. However, if the warning information does not concur with existing beliefs and attitudes, then those beliefs and attitudes may need to be altered by a warning message (or other information). It needs to be strong, strident, and persuasive so as to override erroneous existing beliefs and attitudes. Persuasion by a warning is particularly important when a product is actually more hazardous than people believe it to be. This imbalance between perceived safety and actual hazard can be caused by design and subjective interpretations of warning information. Repeated use of a product without injury leads to a build-up of positive and benign memories associated with the product. For example, an individual may have used over-the-counter pain relief drugs containing paracetamol or acetaminophen for years, with no adverse effects, and the positive experience of pain relief may in turn reduce his or her receptivity to new warning messages about adverse effects added to the label at a later time. Moreover, incorrect or incomplete beliefs about safety can come from a variety of inputs including direct-to-consumer (DTC) advertising campaigns where consumers can sometimes receive an unbalanced presentation emphasizing benefits but with risks understated (Goldsworthy & Mayhorn, 2010; Vigilante, Wogalter, & Mayhorn, 2007). Several relevant and interrelated factors associated with the beliefs and attitudes stage include hazard perception, familiarity, prior experience, and relevance (see DeJoy, 1999; Riley, 2006).
In general, the greater the perceived hazard, the more responsive people will be to warnings, as in looking for, reading, and complying with them. The converse is also true. People are less likely to read warnings for products they believe are relatively safe. Perceived hazard and willingness to act with caution are closely tied to beliefs about injury-severity consequences (Wogalter, Young, Brelsford, & Barlow, 1999).
Familiarity, formed from past, similar experience, is a belief that can influence warning-related processing. If an individual believes that he or she is adequately familiar with a product it will reduce the likelihood of looking for or reading warnings (Godfrey & Laughery, 1984). For example, the bag of “practi-saline” solution in Figure 3 looks nearly identical to a sterilized bag of saline solution yet it is dangerous if used in any medical procedures involving humans or animals. It should have been foreseeable that a nurse could easily glance at this practi-product and not notice in tiny print (smallest on the label) that it states “for clinical simulation only.” The similarity to a standard sterilized bag of saline influences the likelihood that it will be mistaken for the real thing. It should have been considered by the manufacturer that it might end up being placed in the operating theater where the surgical team might erroneously infuse it into a patient without realizing that it would be dangerous to do so. According to an FDA report, there were more than 40 such incidents, some of which resulted in serious consequences (Food and Drug Administration, 2015). This product needed a conspicuous, strong, and persuasive warning. A better warning was put on the bags after the recall.
Familiarity is important, but hazard perception and awareness is stronger (Wogalter, Brelsford, Desaulniers, & Laughery, 1991). The problem with familiarity beliefs is that one might believe that hazard knowledge is adequate when it is insufficient (and thus warnings are not read). Uneventful prior experience with a product (or related products) or situation might be associated (not completely) with unrealistically low levels of hazard perception. Having experienced some form of injury or having personal knowledge of someone else being injured increases hazard perception (Mayhorn, Nichols, Rogers, & Fisk, 2004). Thus, revisiting the practi-product in Figure 3, a nurse might be cued to look for warnings on all saline IV bags after learning about life-threatening conditions in patients mistakenly receiving a similar looking, nonsterile simulation or training product. Similarly, a lack of such experiences may lead to underestimation of dangers, or simply not thinking about them at all. Warnings that are salient and give cues with vivid explicit consequences may provide some of the persuasion needed to change beliefs when users have inappropriately low levels of perceived hazard.
For a warning to succeed, the recipient must believe that it is relevant. The individual may instead attribute the warning as being directed to other persons and not to himself or herself. One way to counter this is to personalize the warning so that it is specifically directed to relevant users and conveys facts that are relevant to them (Wogalter, Racicot, Kalsher, & Simpson, 1994). There are many kinds of sensors or detectors, signal generators, and displays that are available to do this (see Wogalter & Mayhorn, 2005).
Motivation is the drive that an individual will use to carry out an activity, thereby serving to link beliefs and attitudes to actual behavior (Ajzen, 1991). Several relevant motivational factors (cost of compliance, severity of injury, social influence, and stress) are described.
Compliance generally requires that people take some action, and usually there are costs associated with doing so, including the time and effort to carry out the safety behavior or behaviors (Wogalter et al., 1987). When people perceive the costs of compliance to be too high, they are less likely to perform the safety behavior, a problem commonly encountered in warnings containing instructions that are inconvenient, difficult, uncomfortable, or occasionally impossible to carry out. A warning’s perceived cost of compliance of using protective gloves can be reduced by including gloves with the product (Dingus, Hathaway, & Hunn, 1991). For example, surgical kits often contain sterilized gloves so that the healthcare professional does not have to break the sterile field (once established) to search for gloves. Additionally people report higher willingness to comply with warnings when they understand there is high probability and severity of injury (Wogalter, Brelsford, Desaulniers, & Laughery, 1991). Warning messages accompanied by explicit wording and images depicting severe consequences may serve to motivate compliance.
Another motivational factor is social influence (Edworthy & Dale, 2000). When people see others comply with a warning, they are more likely to comply themselves. The reverse is also true. Other factors affecting motivation are time stress (Wogalter, Magurno, Rashid, & Klein, 1998) and workload (Wogalter & Usher, 1999). Under high stress and workload, competing activities can take mental resources away from processing warning information, thus reducing the likelihood of compliance behavior.
Behavioral compliance to a warning is the end goal of the warning process and can be used as a measure of warning effectiveness (Kalsher & Williams, 2006; Silver & Braun, 1999). Some researchers have used “intentions to comply” as the method of measurement because it is usually quite difficult to conduct actual behavioral tests. The difficulties include the following: (a) researchers cannot expose participants to real risks because of ethical and safety concerns; (b) events that could lead to injury are relatively rare; (c) the stimulus scenario must appear to have a believable risk, yet at the same time must be safe; and (d) running such research is costly in terms of time and effort. Nevertheless, compliance is an important criterion for determining which factors work better than others to boost warning effectiveness and consequently, safe behavior.
Virtual reality may play a role in allowing research to be conducted in simulated conditions that avoid some of these problems (Duarte, Rebelo, Teles, & Wogalter, 2014; Vilar, Rebelo, Noriega, Duarte, & Mayhorn, 2014). In the Vilar et al. (2014) experiment, the design of evacuation signage inside buildings was experimentally manipulated during a simulated emergency situation. Because it would be inappropriate to expose people to a fire hazard, the use of a simulated fire hazard to study evacuation signage was deemed a useful tool in assessing various signage design characteristics. Compliance can also be measured in another way—indirectly. For example, determining whether protective gloves have been worn can be gleaned from whether they appear to be used or stretched in appearance (Wogalter & Dingus, 1999).
Techniques to Evaluate the Effectiveness of Warnings
Methods of assessing warnings involve many techniques. One of the main methods is to evaluate warning efficacy using a checklist of characteristics or features that have been found useful in previous research (Lenorovitz, Karnes, & Leonard, 2014; Wogalter, Laughery, & Mayhorn, 2012). Within the healthcare domain, the use of checklists has been firmly established to reduce errors in the surgical process and within intensive care units, so it stands to reason that such an approach could be effective with warnings as well (Gawande, 2010). If a warning has very few of the attributes noted in an established checklist then it probably will not be as effective as it might be if it had additional features (Lenorovitz, Leonard, & Karnes, 2012). Another way to do an assessment of a warning is to do a heuristic evaluation. This is similar to the checklist evaluation except that it is done by persons who are experts in the warnings domain.
Conducting testing with participants is yet another means for evaluating warning effectiveness. Focus group procedures allow a facilitator to discuss the perceived attributes of warnings with small groups of people (usually 6–12) to learn their reactions. While focus groups can sometimes be beneficial in collecting ideas, it should also be recognized that this qualitative method has limitations, such as the group being influenced too much by one or more participants. Likewise, the quality of the resulting data analysis is reliant on the quality of the coding scheme, which might not be at the relevant level of analysis (Krueger, 1994).
Perhaps a better method to assess warning efficacy might be to conduct usability research where individuals are tested one-at-a-time (Nielsen, 1993). Usually a small number of people participate in each round and the information they provide is used to aid in re-designing the warning based on their feedback. The revised warning is then shown to another small group of participants who again give feedback, and then the warning is altered again and so on as needed. This is called iterative design-test prototyping (see Wogalter, Conzola, & Vigilante, 2006). The process continues until the warning has been changed to the point that it appears satisfactory. Even at this point the process is not fully complete until some larger pool of participants is tested such that warning message content meets some criteria of acceptability (e.g., American National Standards Institute [ANSI] Z535’s 85% comprehension of key content). However, it should be noted that once a warning is placed as a sign or label on a product, it does not mean that the process is complete. For example, further warning design revisions may be necessary if reports of adverse events are noted by the manufacturer or regulatory agencies such as the Food and Drug Administration or and Consumer Product Safety Commission. Moreover, it should be noted that much of the focus here is on visual warnings rather than auditory warnings, yet these evaluation techniques should be effective across modalities (see Edworthy & Hellier, 2006; Haas & Edworthy, 2006 for a comprehensive review of auditory warnings).
Using the C-HIP Model as an Investigative Tool to Promote Effective Warning Design
This description of health-related warnings used the Communication-Human Information Processing (C-HIP) model to organize discussion of the information processing steps that occur when safety information is encountered. Given the large amount of previous research on warnings described here, the C-HIP model can be a useful tool in systematizing the assessment process to determine why a warning is or is not effective. Use of C-HIP can aid in pinpointing the location of processing bottlenecks and can suggest potential solutions to allow processing to continue to subsequent stages. Moreover, tests of warning effectiveness can be performed using methods similar to those used in research to obtain objective measures of performance. Evaluations of the processing can be directed to any of the stages described in the C-HIP model: source, channel, environment, delivery, attention, comprehension, attitudes and beliefs, motivation, behavior, and receiver variables. Some of the methods for doing this evaluation are briefly described.
Sources of products such as manufacturers (and other relevant entities) are responsible (both morally and oftentimes legally) for analyzing their products and services to determine whether there are foreseeable risks associated with their use and misuse. Product manufacturers (and other relevant entities) should determine and document the residual hazards not controlled by design and guarding. When risks or hazards become known, these entities have an obligation to employ methods to try to control the hazards to reduce personal injury and property damage, and one of those methods is to effectively warn people. Whether the source does these things can influence the quality (form and content) of the warnings.
Efforts to evaluate the channel of warning delivery mainly assess how safety information is sent to end users. Evaluation at this stage might be done in light of the following questions. What media and modalities are being used and are those adequate? What other ways might be better? Some methods of warning communication might affect some (a percentage of) relevant users appropriately, but other (additional) methods may be necessary to reach users not otherwise accessed by other methods. Similarly, assessment regarding delivery determines whether end users receive the warnings. If they are not getting the warnings then other channels of distribution for warning materials may need to be considered. The concept of “cascading responsibility” requires that the product manufacturers, intermediaries (e.g., distributors and retailers), and employers within occupational settings share a responsibility to ensure that persons at risk “downstream” are provided with needed safety information (Williams, Kalsher, & Laughery, 2006). The concept of “product stewardship,” which means that manufacturers should take steps to ensure that their products are used in appropriate safe ways by entities downstream is having increasing acceptance in the chemical industry and is a useful concept here as well. Also, in cases where the end user is a healthcare professional such as a physician or nurse, these individuals must act to assure the safety of their patients.
The process of attention switch is assessed by whether people notice the warning and draw their attention to it. Measurement of a warning’s performance in this regard could be determined by placing a warning in expected environments or locations and having people carry out a relevant task then asking them later whether they saw it (McGrath, 2011) or by recording eye and head movement (Just & Carpenter, 1976). Response-time measurement is another way to assess the warning’s attention attraction. Likewise, a proxy for attention maintenance could be assessed via “dwell time” or eye fixation on a warning component such as an American National Standards Institute (ANSI) signal word.
Well-established methodologies involving memory tests, open-ended response tests, and structured interviews can be used to assess warning comprehension such that it can be determined what information was or was not understood. Results can be valuable for suggesting revisions to warning text or symbols. To assess beliefs and attitudes, surveys could be developed to quantify people’s preexisting beliefs on topics such as perceived hazard and familiarity with the tool, task, or environment. For example, if users’ perceived hazard is too low for a potentially dangerous product or situation, more salient warnings with substantial persuasiveness may be in order.
To assess motivation, measures of behavioral intentions can be used. An observed low intention to comply may suggest that consequence information should be enhanced (e.g., by being more explicit with regard to potential hazard severity) or that cost of compliance should be reduced. In situations where behavioral compliance can be directly assessed, systematic observation can be used in both laboratory and field settings. Unfortunately, direct assessment of behavioral compliance can be problematic because ethical issues such as participants’ exposure to risk must be considered. As a result, behavioral intentions are often used as a proxy for overt behavioral compliance—but some caution should be exercised. Future behavioral compliance research may be conducted in virtual reality environments to provide hazard realism and extant safety (e.g., Duarte, Rebelo, Teles, & Wogalter, 2014).
In summary, the Communication-Human Information Processing (C-HIP) model can be a way of organizing warning processes and assessing a warning’s efficacy. Using the model as a guide to investigate the specific causes of a warning’s failure may result in the generation of potential safety solutions. Resources can then be allocated more appropriately in taking action to improve the warning to eliminate specific shortcomings that limit its effectiveness.
Why should such a high level of care be taken in the design and presentation of health-related warning information? One reason is that health problems are a universal domain that all people will encounter at one time or another. Thus it is important that hazard control be effective in limiting injury and disease in healthcare contexts. Warnings are one approach to hazard control that are used when product designers or employers or public communities cannot (or for other reasons do not) design out or guard against all of the hazards. Although warnings are supposedly a known method to protect people from harm, their success is dependent on their ability to facilitate attention capture and understanding of message content, among other processes noted in the C-HIP model. Yet many warnings are quite poor. Warning development should not be a lackadaisical and uneducated affair. Warnings should be constructed to be effective, and the techniques and methodology to accomplish this goal have been well-documented. It is our hope that those tasked with generating health-related warnings will find the C-HIP model useful in organizing and focusing discussion for effective ways to protect health and enhance safety.
Ajzen, I. (1991). Theory of planned behavior. Organizational Behavior and Human Decision Processes, 50(2), 179–211.Find this resource:
American National Standards Institute. (2002). Accredited Standards Committee on Safety Signs and Colors. Z535.1–5. Arlington, VA: National Electrical Manufacturers Association.Find this resource:
Baldwin, C., Spence, C., Bliss, J., Brill, C. J., Wogalter, M. S., Mayhorn, C. B., & Ferris, T. (2012). Multimodal cueing: The relative benefits of the auditory, visual, and tactile channels in complex environments. Proceedings of the Human Factors and Ergonomics Society 56th Annual Meeting. Santa Monica, CA: Human Factors and Ergonomics Society.Find this resource:
Bogner, M. S. (2004). Misadventures in health care: Inside stories. Mahwah, NJ: Lawrence Erlbaum Associates.Find this resource:
Brantley, K. A., & Wogalter, M. S. (1999). Oral and written symbol comprehension testing: The benefit of cognitive interview probing. Proceedings of the Human Factors and Ergonomics Society, 43, 1060–1064.Find this resource:
Bzostek, J. A., & Wogalter, M. S. (1999). Measuring visual search time for a product warning label as a function of icon, color, column, and vertical placement. Proceedings of the Human Factors and Ergonomics Society, 43, 888–892.Find this resource:
Carayon, P. (2012). Human factors and ergonomics in health care and patient safety (2d ed.). Boca Raton, FL: CRC.Find this resource:
Chapanis, A. (1994). Hazards associated with three signal words and four colours on warning signs. Ergonomics, 37, 265–275.Find this resource:
Clark, J. M., & Paivio, A. (1991). Dual coding theory and education. Educational Psychology Review, 3(3), 149–210.Find this resource:
Conzola, V. C., & Wogalter, M. S. (2001). A communication-human information processing (C-HIP) approach to warning effectiveness in the workplace. Journal of Risk Research, 4(4), 309–322.Find this resource:
Dehenau, C., Becker, M. W., Bello, N. M., Liu, S., & Bix, L. (2015). Tallman lettering as a strategy for differentiation in look-alike, sound-alike drug names: The role of familiarity in differentiating drug dopplegangers. Journal of Applied Ergonomics, 52, 77–84.Find this resource:
DeJoy, D. M. (1999). Beliefs and attitudes. In M. S. Wogalter, D. M. DeJoy, and K. R. Laughery (Eds.), Warnings and risk communication (pp. 183–219). London: Taylor & Francis.Find this resource:
Desaulniers, D. R. (1987). Layout, organization, and the effectiveness of consumer product warnings. Proceedings of the Human Factors Society, 31, 56–60.Find this resource:
Dingus, T. A., Hathaway, J. A., & Hunn, B. P. (1991). A most critical warning variable: Two demonstrations of the powerful effects of cost on warning compliance. Proceedings of the Human Factors Society, 35, 1034–1038.Find this resource:
Duarte, E., Rebelo, F., Teles, J., & Wogalter, M. S. (2014). Behavioral compliance for dynamic versus static signs in an immersive virtual environment. Applied Ergonomics, 45, 1367–1375.Find this resource:
Edworthy, J., & Adams, A. (1996). Warning design: A research prospective. London: Taylor & Francis.Find this resource:
Edworthy, J., & Dale, S. (2000). Extending knowledge of the effects of social influence in warning compliance. In Proceedings of the XIVth Triennial Congress of the International Ergonomics Association and 44th Annual Meeting of the Human Factors and Ergonomics Society, July 29–August 4, 2000, San Diego, CA (vol. 4, pp. 107–110). Santa Monica, CA: Human Factors and Ergonomics Society.Find this resource:
Edworthy, J., & Hellier, E. (2006). Complex nonverbal auditory signals and speech warnings. In M. S. Wogalter (Ed.) Handbook of warnings (pp. 199–220). Mahwah, NJ: Lawrence Erlbaum Associates.Find this resource:
Flesch, R. (1948). A new readability yardstick. Journal of Applied Psychology, 32, 2211.Find this resource:
Food and Drug Administration. (2015, April 8). FDA’s investigation into patients being injected with simulated IV fluids continues. Washington, DC: Food and Drug Administration.Find this resource:
Frascara, J. (2006). Typography and the visual design of warnings. In M. S. Wogalter (Ed.), Handbook of warnings (pp. 385–406). Mahwah, NJ: Lawrence Erlbaum Associates.Find this resource:
Gawande, A. (2010). The checklist manifesto: How to get things right. New York: Picador.Find this resource:
Godfrey, S. S., & Laughery, K. R. (1984). The biasing effect of familiarity on consumer’s awareness of hazard. Proceedings of the Human Factors Society, 28, 483–486.Find this resource:
Goldsworthy, R. C., & Mayhorn, C. B. (2010). Direct to consumer (DTC) prescription drug advertising: Exploring self-reports of media exposure and associated behaviors. In Proceedings of the Third Applied Human Factors and Ergonomics International Conference (pp. 810–817). Miami, FL: CRC.Find this resource:
Guchelaar, H., Colen, H. B., Kalmeijer, M. D., Hudson, P. T., and Teepe-Twiss, I. M. (2005). Medication errors: Hospital pharmacist perspective. Drugs, 65(13), 1735–1746.Find this resource:
Haas, E., & Edworthy, J. (2006). An introduction to auditory warnings and alarms. In M. S. Wogalter (Ed.), Handbook of warnings (pp. 189–198). Mahwah, NJ: Lawrence Erlbaum Associates.Find this resource:
Hartley, J. (1994). Designing instructional text (3d ed.). London: Kogan Page.Find this resource:
International Standards Organization. (2011). International standard for safety colors and safety signs: ISO 3864. Geneva, Switzerland: International Standards Organization.Find this resource:
Israelski, E. W., & Muto, W. H. (2012). Human factors risk management for medical products. In P. Carayon (Ed.), Human factors and ergonomics in health care and patient safety (2d ed.). (pp. 475–505). Boca Raton, FL: CRC.Find this resource:
Just, M. A., & Carpenter, P. A. (1976). Eye fixations and cognitive processes. Cognitive Psychology, 8, 441–480.Find this resource:
Kalsher, M. J., & Williams, K. J. (2006). Behavioral compliance: Theory, methodology, and results. In M. S. Wogalter (Ed.), Handbook of warnings (pp. 289–300). Mahwah, NJ: Lawrence Erlbaum Associates.Find this resource:
Kalsher, M. J., Wogalter, M. S., & Racicot, B. M. (1996). Pharmaceutical container labels and warnings: Preference and perceived readability of alternative designs and pictorials. International Journal of Industrial Ergonomics, 18, 83–90.Find this resource:
Krueger, R. A. (1994). Focus groups: A practical guide for applied research (2d ed.). London: SAGE.Find this resource:
Laughery, K. R., Vaubel, K. P., Young, S. L., Brelsford, J. W., & Rowe, A. L. (1993). Explicitness of consequence information in warning. Safety Science, 16, 597–613.Find this resource:
Laughery, K. R., & Wogalter, M. S. (2006). Designing effective warnings. In R. Williges (Ed.), Reviews of human factors and ergonomics. Santa Monica, CA: Human Factors and Ergonomics Society.Find this resource:
Laughery, K. R., Young, S. L., Vaubel, K., & Brelsford, J. B. (1993). The noticeability of warnings on alcoholic beverage warnings. Journal of Public Policy & Marketing, 12(1), 38–56.Find this resource:
Lehto, M. R., & Miller, J. M. (1986). Warnings, volume 1: Fundamentals, design, and evaluation methodologies. Ann Arbor, MI: Fuller Technical.Find this resource:
Lehto, M. R., & Salvendy, G. (1995). Warnings: A supplement not a substitute for other approaches to safety. Ergonomics, 38, 2155–2163.Find this resource:
Lenorovitz, D. R., Karnes, E. W., & Leonard, S. D. (2014). Mitigating certain hazards with just warnings: When and why such warnings are likely to fail. Human Factors and Ergonomics in Manufacturing & Service Industries, 24(3), 275–297.Find this resource:
Lenorovitz, D. R., Leonard, S. D., & Karnes, E. W. (2012). Ratings checklist for warnings: A prototype tool to aid experts in the adequacy evaluation of proposed or existing warnings. Work (Reading, MA), 41, Suppl. 1 (CD ROM).Find this resource:
Leonard, S. D., Otani, H., & Wogalter, M. S. (1999). Comprehension and memory. In M. S. Wogalter, D. M. DeJoy, & K. R. Laughery (Eds.), Warnings and risk communication (pp. 149–188). London: Taylor & Francis.Find this resource:
Lindell, M. K., & Perry, R. W. (2004). Communicating environmental risk in multiethnic communities. Thousand Oaks, CA: SAGE.Find this resource:
Magurno, A., Wogalter, M. S., Kohake, J., & Wolff, J. S. (1994). Iterative test and development of pharmaceutical pictorials. In Proceedings of the 12th Triennial Congress of the International Ergonomics Association, August 15–19, 1994, Toronto (vol. 4, pp. 360–362). Ontario: Human Factors Association of Canada.Find this resource:
Mayhorn, C. B., & Goldsworthy, R. C. (2007). Refining teratogen warning symbols for diverse populations. Birth Defects Research Part A: Clinical and Molecular Teratology, 79 (6), 494–506.Find this resource:
Mayhorn, C. B., & Goldsworthy, R. C. (2009). “New and improved”: The role of text augmentation and the application of responses interpretation standards (coding schemes) in a final iteration of birth defects warnings development. Birth Defects Research Part A: Clinical and Molecular Teratology, 85(10), 864–871.Find this resource:
Mayhorn, C. B., Lanzolla, V. R., Wogalter, M. S., & Watson, A. M. (2005). Personal digital assistants (PDAs) as medication reminding tools: Exploring age differences in usability. Gerontechnology, 4(3), 128–140.Find this resource:
Mayhorn, C. B., Nichols, T. A., Rogers, W. A., & Fisk, A. D. (2004). Hazards in the home: Using older adults’ perceptions to inform warning design. Journal of Injury Control and Safety Promotion, 11(4), 211–218.Find this resource:
Mayhorn, C. B., & Podany, K. I. (2006). Warnings and aging: Describing the receiver characteristics of older adults. In M. S. Wogalter (Ed.), Handbook of Warnings (pp. 355–362). Mahwah, NJ: Lawrence Erlbaum Associates.Find this resource:
Mayhorn, C. B., Wogalter, M. S., & Bell, J. L. (2004). Are we ready? Misunderstanding homeland security safety symbols. Ergonomics in Design, 12(4), 6–14.Find this resource:
Mayhorn, C. B., Wogalter, M. S., & Shaver, E. F. (2004). What does Code Red mean? Ergonomics in Design, 2(4), 12.Find this resource:
McGrath, J. M. (2011). The role of equipment warning labels in the industrial workplace. International Journal of Occupational Safety and Ergonomics, 17(1), 49–60.Find this resource:
Meadows, M. (2001). The power of Accutane. The benefits and risks of a breakthrough acne drug. FDA Consumer Magazine, 35(2), 18–23.Find this resource:
Mendat, C. C., Watson, A. M., Mayhorn, C. B., & Wogalter, M. S. (2005). Age differences in search time for two over-the-counter (OTC) drug label formats. Proceedings of the Human Factors and Ergonomics Society, 49, 200–203.Find this resource:
Monroe, K. A., & Orr, G. (2006). Warnings and the U.S. Occupational Safety and Health Administration. In M. S. Wogalter (Ed.), Handbook of warning (pp. 537–551). Mahwah, NJ: Lawrence Erlbaum Associates.Find this resource:
Nielsen, J. (1993). Usability engineering. San Diego, CA: Academic Press.Find this resource:
Ostrove, N. M. (2006). Warning information in the labeling and advertising of pharmaceuticals. In M. S. Wogalter (Ed.), Handbook of warning (pp. 515–528). Mahwah, NJ: Lawrence Erlbaum Associates.Find this resource:
Park, D. C., Hertzog, C., Kidder, D. P., Morrell, R. W., & Mayhorn, C. B. (1997). The effect of age on event-based and time-based prospective memory. Psychology and Aging, 12(2), 314–327.Find this resource:
Peckham, G. M. (2006). ISO design standards for safety signs and labels. In M. S. Wogalter (Ed.), Handbook of Warnings (pp. 455–462). Mahwah, NJ: Lawrence Erlbaum Associates.Find this resource:
Perlman, S. E., Leach, E. E., Dominguez, L., Ruszkowski, A. M., & Rudy, S. J. (2001). “Be smart, be safe, be sure.” The revised Pregnancy Prevention Program for women on isotretinoin. Journal of Reproductive Medicine, 46(2 Suppl.), 179–185.Find this resource:
Poulton, E. (1967). Searching for newspaper headlines printed in capitals or lower-case letters. Journal of Applied Psychology, 51, 417–425.Find this resource:
Riley, D. M. (2006). Beliefs, attitudes, and motivation. In M. S. Wogalter (Ed.), Handbook of warnings (pp. 289–300). Mahwah, NJ: Lawrence Erlbaum Associates.Find this resource:
Rogers, W. A., Lamson, N., and Rousseau, G. K. (2000). Warning research: An integrative perspective. Human Factors, 42, 102–139.Find this resource:
Russ, A. L., Zillich, A. J., McManus, M. S., Doebbeling, B. N., & Saleem, J. J. (2012). Prescribers’ interactions with medication alerts at the point of prescribing: A multi-method, in situ investigation of the human-computer interaction. International Journal of Medical Informatics, 81(4), 232–243.Find this resource:
Salvendy, G. (2012). Handbook of human factors and ergonomics (4th ed.). New York: Wiley Interscience.Find this resource:
Sanders, M. S., & McCormick, E. J. (1993). Human factors in engineering and design (7th ed.). New York: McGraw-Hill.Find this resource:
Shapiro, R., & Berland, T. (1972). Noise in the operating room. New England Journal of Medicine, 287, 1236–1238.Find this resource:
Silver, N. C., & Braun, C. C. (1999). Behavior. In M. S. Wogalter, D. M. DeJoy, & K. R. Laughery (Eds.), Warnings and risk communication (pp. 245–262). London: Taylor & Francis.Find this resource:
Sojourner, R. J., & Wogalter, M. S. (1997). The influence of pictorials on evaluations of prescription medication instructions. Drug Information Journal, 31, 963–972.Find this resource:
Sojourner, R. J., & Wogalter, M. S. (1998). The influence of pictorials on the comprehension and recall of pharmaceutical safety and warning information. International Journal of Cognitive Ergonomics, 2, 93–106.Find this resource:
Thorley, P., Hellier, E., & Edworthy, J. (2001). Habituation effects in visual warnings. In M. A. Hanson (Ed.), Contemporary ergonomics 2001 (pp. 223–228). London: Taylor & Francis.Find this resource:
Tinker, M. A. (1963). Legibility of print. Ames: Iowa State University Press.Find this resource:
United Nations. (2015). Globally harmonized system of classification and labelling of chemicals (6th ed.). Geneva, Switzerland: United Nations.Find this resource:
Vigilante, W. J., Jr., Wogalter, M. S., & Mayhorn, C. B. (2007). Direct-to-consumer (DTC) prescription drug advertising on television and online purchases of medications. In Proceedings of the Human Factors and Ergonomics Society 51st Annual Meeting, October 1–5, 2007, Baltimore, MD. Santa Monica, CA: Human Factors and Ergonomics Society.Find this resource:
Vilar, E., Rebelo, F., Noriega, P., Duarte, E., & Mayhorn, C. B. (2014). Effects of competing environmental variables and signage on route choices in simulated everyday and emergency wayfinding situations. Ergonomics, 57(4), 511–524.Find this resource:
Weinger, M. B. (2004). Anesthesia incidents and accidents. In M. S. Bogner (Ed.), Misadventures in health care: Inside stories (pp. 89–103). Mahwah, NJ: Lawrence Erlbaum Associates.Find this resource:
Williams, K. J., Kalsher, M. J., & Laughery, K. R. (2006). Allocation of responsibility for injuries. In M. S. Wogalter (Ed.), Handbook of warnings (pp. 617–628). Mahwah, NJ: Lawrence Erlbaum Associates.Find this resource:
Wogalter, M. S. (2006). Handbook of warning. Mahwah, NJ: Lawrence Erlbaum Associates.Find this resource:
Wogalter, M. S., Brelsford, J. W., Desaulniers, D. R., & Laughery, K. R. (1991). Consumer product warnings: The role of hazard perception. Journal of Safety Research, 22, 71–82.Find this resource:
Wogalter, M. S., Conzola, V. C., & Vigilante, W. J., Jr. (2006). Applying usability engineering principles to the design and testing of warning text. In M. S. Wogalter (Ed.), Handbook of warnings (pp. 487–498). Mahwah, NJ: Lawrence Erlbaum Associates.Find this resource:
Wogalter, M. S., DeJoy, D. M., & Laughery, K. R. (Eds.). (1999). Warnings and risk communication. London: Taylor & Francis.Find this resource:
Wogalter, M. S., & Dingus, T. A. (1999). Methodological techniques for evaluating behavioral intentions and compliance. In M. S. Wogalter, D. M. DeJoy, & K. R. Laughery (Eds.), Warnings and risk communication (pp. 53–82). London: Taylor & Francis.Find this resource:
Wogalter, M. S., Godfrey, S. S., Fontenelle, G. A., Desaulniers, D. R., Rothstein, P. R., & Laughery, K. R. (1987). Effectiveness of warnings. Human Factors, 29, 599–612.Find this resource:
Wogalter, M. S., Kalsher, M. J., Frederick, L. J., Magurno, A. B., & Brewster, B. M. (1998). Hazard level perceptions of warning components and configurations. International Journal of Cognitive Ergonomics, 2, 123–143.Find this resource:
Wogalter, M. S., Laughery, K. R., & Mayhorn, C. B. (2012). Warnings and hazard communications. In G. Salvendy (Ed.), Handbook of human factors and ergonomics (4th ed.) (pp. 868–894). New York: Wiley Interscience.Find this resource:
Wogalter, M. S., Magurno, A. B., Rashid, R., & Klein, K. W. (1998). The influence of time stress and location on behavioral compliance. Safety Science, 29, 143–158.Find this resource:
Wogalter, M. S., & Mayhorn, C. B. (2005). Providing cognitive support with technology-based warning systems. Ergonomics, 48(5), 522–533.Find this resource:
Wogalter, M. S., Mayhorn, C. B., & Zielinska, O. (2015). Use of color in warnings. In A. J. Elliot, M. Fairchild, & A. Franklin (Eds.), Handbook of color psychology (pp. 377–400). Cambridge, U.K.: Cambridge University Press.Find this resource:
Wogalter, M. S., Racicot, B. M., Kalsher, M. J., & Simpson, S. N. (1994). The role of perceived relevance in behavioral compliance in personalized warning signs. International Journal of Industrial Ergonomics, 14, 233–242.Find this resource:
Wogalter, M. S., Shaver, E. F., & Chan, L. S. (2002). List vs. paragraph formats on time to compare nutrition labels. In P. T. McCabe (Ed.), Advances in ergonomics 2002 (pp. 458–462). London: Taylor & Francis.Find this resource:
Wogalter, M. S., & Silver, N. C. (1990). Arousal strength of signal words. Forensic Reports, 3, 407–420.Find this resource:
Wogalter, M. S., Silver, N. C., Leonard, S. D., & Zaikina, H. (2006). Warning symbols. In M. S. Wogalter (Ed.), Handbook of warnings (pp. 159–176). Mahwah, NJ: Lawrence Erlbaum Associates.Find this resource:
Wogalter, M. S., Sojourner, R. J., & Brelsford, J. W. (1997). Comprehension and retention of safety pictorials. Ergonomics, 40, 531–542.Find this resource:
Wogalter, M. S., & Usher, M. (1999). Effects of concurrent cognitive task loading on warning compliance behavior. Proceedings of the Human Factors and Ergonomics Society, 43, 106–110.Find this resource:
Wogalter, M. S., & Vigilante, W. J., Jr. (2006). Attention switch and maintenance. In M. S. Wogalter (Ed.), Handbook of warnings (pp. 245–266). Mahwah, NJ: Lawrence Erlbaum Associates.Find this resource:
Wogalter, M. S., Young, S. L., Brelsford, J. W., & Barlow, T. (1999). The relative contribution of injury severity and likelihood information on hazard-risk judgments and warning compliance. Journal of Safety Research, 30, 151–162.Find this resource:
Wolff, J. S., & Wogalter, M. S. (1998). Comprehension of pictorial symbols: Effects of context and test method. Human Factors, 40, 173–186.Find this resource:
Zielinska, O. A., Wogalter, M. S., & Mayhorn, C. B. (2014). A perceptual analysis of standard safety colors and their fluorescent counterparts. Proceedings of the Human Factors and Ergonomics Society, 58, 1879–1883.Find this resource: