BY DARRELL MENDENHALL, MS; STEVEN MOFFATT, MD; TISCH WILLIAMS, MA; MIKE REEVES; JAMES GREESON; C. SHAWN SHELTON, BA; HOWARD STAHL; TERRELL W. ZOLLINGER, DRHP; ROBERT M. SAYWELL JR., PH.D., MPH; and CAROLYN MUEGGE, MS, MPH
Intense physical exertion is an occupational hazard inherent to firefighting. Since the National Fire Protection Association (NFPA) began reporting line-of-duty injury and fatality data in 1977, heart attack has been the leading cause of death, accounting for approximately half of all firefighter fatalities. In nearly all cases, these fatal heart attacks have been directly attributed to stress and overexertion. These data indicate that firefighters are regularly confronted with situations demanding levels of sustained physical effort often exceeding their physical working capacities, thus resulting in stress, overexertion, and heart attack.
As supported by the mortality statistics, cardiorespiratory fitness is well accepted as a critical requisite to the health, safety, and performance of firefighters in executing the duties of their job. In response to this, fire departments across the United States have implemented physical fitness programs, which include, among other things, a cardiorespiratory fitness assessment. While this type of an assessment provides information regarding a firefighter’s cardiorespiratory fitness, it may not be a perfect indicator of whether a firefighter can perform firefighter-specific tasks. However, while a physical work performance evaluation can determine whether a firefighter can perform a determined set of critical essential firefighter tasks, it provides little insight regarding the cardiorespiratory capacity needed to perform them with reasonable safety. To better address this question effectively, it is important that when possible firefighters be physiologically evaluated while performing actual firefighter tasks or closely simulated tasks.
The Indianapolis (IN) Fire Department (IFD) joined forces with its International Association of Fire Fighters Local 416, Public Safety Medical Services (PSMS), and the Indiana University School of Medicine’s Department of Family Medicine, Bowen Research Center to develop, validate, and physiologically evaluate a work performance evaluation for incumbent firefighters. In conjunction with its existing medical and fitness program, administered by PSMS, it is believed that a properly validated, job-relevant work performance evaluation for incumbent firefighters will further help protect the health and safety of incumbent firefighters while preserving the quality of protection it provides to the community.
PURPOSE
The purpose of this study was twofold: to validate a job-specific physical work performance evaluation that represents the critical essential functions firefighters must be able to perform and to estimate the strength of the physiologic correlates associated with the successful completion of the evaluation. Results of the first phase of this study (the validation process) are presented in this article. Results of the second phase of the study (physiologic correlates) will be presented at a later date.
METHODS
The methods used to validate this work performance evaluation have been modeled after the methods used by the International Association of Fire Fighters/International Association of Fire Chiefs (IAFF/IAFC) Joint Labor Management Wellness Fitness Initiative in validating their Candidate Physical Ability Test. The following is a brief outline of the study methods, processes, and timelines.
Recruitment of Subjects
The following eligibility criteria were established for participation in the validation of the work performance evaluation:
1. The firefighter must have been on full active duty status at the time of the study.
2. The firefighter must have been post-probationary employment status at the time of the study.
3. The firefighter must have completed a personal fitness evaluation (PFE) and medical examination within 12 months of study.
4. The firefighter must have passed test day resting biometrics (heart rate <100 bts/min. & blood pressure <160/100mm/hg).
The IFD Human Resources Department determined a pool of subjects who met the eligibility criteria. From this subject pool, 75 subjects were randomly selected from across all three shifts (24-A, 24-B, 27-C). IFD administration invited the subjects to participate; however, participation in the study was voluntary. The subject pool of 75 firefighters was evaluated for conformity with the demographic characteristics of the department. The subject pool closely resembled the demographic characteristics of the department as defined by age distribution, gender, and race.
Explanation of Study and Obtained Consent
Subjects were informed of the purpose of the study in an initial invitation letter sent by the IFD Human Resources Department. Furthermore, all subjects were provided a participant consent form to read and sign, which detailed the purpose of the study, the procedures of the study, the inherent risks of participating in the study, and the confidentiality of subject data.
Pretest Measurements
Height, weight, resting heart rate, resting blood pressure, body mass index, percent body fat, and lean body mass were measured prior to participation in the validation study of the work performance evaluation.
Description and Demonstration of Physical Work Performance Evaluation
Subjects were asked to participate in a physical work performance evaluation that consisted of eight firefighter-specific tasks. On the day of their scheduled participation, each subject received a verbal explanation of the work performance evaluation while an IFD peer fitness trainer walked them through each station of the evaluation. These tasks were performed while wearing full personal protective equipment (PPE) and an SCBA tank. However, for the purposes of the study, in lieu of wearing an SCBA mask, a VMAX ST portable metabolic analyzer was worn on the chest and connected to a facemask worn by the subject, which enabled the collection and analysis of cardiorespiratory data on each subject throughout the evaluation. The eight firefighter-specific tasks were as follows.
 1 |
1 Personal Protective Equipment. This event allows the firefighter to safely don his complete issue of PPE (hood optional). The firefighter is to don a complete set of turnout gear, pants, boots, and coat; SCBA with face piece; and helmet (hood optional). The firefighter will move to a position that places him at the proper lifting point of his SCBA, lift and secure the SCBA, secure the face piece, and engage the second-stage regulator. Then, he signals ready for testing by raising either arm. (If the firefighter runs out of air during any event, he removes and stores the second-stage regulator and continues testing with ambient air; time will continue to run during this procedure.)
 2 |
2 Ladder Carry, Raise, Extension. This event simulates the firefighter’s ability to handle a 16-foot ground ladder and place and extend a 24-foot extension ladder. The firefighter performs a one-person lift and carries a 16-foot ground ladder from a marked starting position 75 feet to a marked ending position, and sets the ladder down. The firefighter then moves to the top rung of a 24-foot extension ladder, lifts the unhinged end from the ground, and walks it up until it is stationary against the wall.
 3 |
3 Forcible Entry Simulation. This event simulates the critical task of using force to open a locked door or to breach a wall. The firefighter walks 75 feet to the forcible entry machine, picks up a 10-pound sledgehammer, and strikes the measuring device in the target area until the buzzer is activated.
 4 |
4 High-Rise Pack Carry. This event simulates the firefighter’s ability to carry a high-rise pack to an upper-story location. The firefighter walks 75 feet to this event to the marked area. The proctor places the high-rise pack over the top of the SCBA bottle. The firefighter walks with the high-rise pack 30 feet to the simulated stair platform. The firefighter then simulates climbing three floors by completely stepping up and down on the stair platform 30 times. The firefighter then carries the high-rise pack 30 feet back to the marked area; the proctor removes the high-rise pack.
 5 |
5 Handline Advancement & Pull. This event simulates the critical task of advancing an uncharged hoseline and repositioning an uncharged handline. The firefighter walks 75 feet to the uncharged hoseline, picks up the nozzle, and advances the handline 75 feet to the marked position. He places the handline down and crawls back 50 feet, keeping one hand in constant contact with the hoseline. He then assumes a kneeling or sitting position and pulls back 50 feet of the hose.
 6 |
6 Pike Pole Simulation. This event is designed to simulate the critical task of breaching and pulling down a ceiling to check for fire extension. The firefighter picks up a pike pole and walks 30 feet to the marked area and raises the pike pole 30 times. Both hands must be below the mark on the pike pole handle. A complete repetition consists of fully extending both arms upward and returning them to the starting position. After 30 complete repetitions, the firefighter returns to the starting point and places the pike pole in the marked area.
 7 |
7 Equipment Carry. This event is designed to simulate the critical task of carrying equipment from a fire apparatus scene and returning the equipment to the apparatus. The firefighter walks to the marked area and carries the designated equipment 75 feet to the next marked area, sets the equipment down, lifts and carries the second item 75 feet back to the starting position, and sets the equipment down in the marked area. One-hand and two-hand carries with designated equipment are alternated. Tools are not to be dropped or thrown at any time.
 8 |
8 Victim Rescue and Drag. This event simulates the critical task of removing a victim or an injured partner from a fire scene. The firefighter walks to the 165-lb. rescue dummy, lifts and moves the rescue dummy a distance of 75 feet, moves around the cone, returns to the starting position 75 feet away, and places the dummy in the marked area.
These events were performed in the above order in a continuous manner. Event 1 (PPE) was not a timed event. Timing of the evaluation began with the onset of event 2 (ladder carry, raise, extension) and concluded at the completion of event 8 (victim rescue and drag). Firefighters were instructed to perform tasks 2 through 8 at a rate of work (pace) they felt would reasonably be expected during an actual live fireground operation.
Physiologic Monitoring and Data Collection
Cardiorespiratory data were collected on each subject throughout the evaluation using a Sensormedics V-Max ST portable metabolic analyzer. Data were “marked” on the telemetry system of the metabolic analyzer at the conclusion of each task to identify individual time requirements for performing each task and to perform a metabolic analysis of each individual task as well as of the entire evaluation.
Validation of a Physical Work Performance Evaluation for Incumbent Firefighters
 All participants “agreed” or “strongly agreed” that they understood how to complete the WPE. Figure 1 |
In addition, within two weeks of participating in the work performance evaluation, a maximal cardiorespiratory assessment with gas analysis and 12-lead ECG was performed on all participants who met clinical eligibility criteria.
 94.7% of participants completed the evaluation in a manner consistent with how they would work at a fire scene. Figure 2 |
null
 96.0% of participants rated the overall criticality of the tasks of the work performance evaluation as “important” to “extremely important.” Figure 3 |
null
 94.7% of participants rated the overall physicality of the tasks of the work performance evaluation as requiring “moderate” to “extreme effort.” Figure 4 |
null
 93.3% of participants rated overall how well the tasks of the WPE represented what occurs at a fire scene as “somewhat” to “very well.” Figure 5 |
null
 Average Time to Completion of Task, 28 seconds Figure 6 |
null
Videotaping
Each subject’s performance was videotaped for the entirety of events 2 through 8 to be reviewed by a panel of experts at a later date.
Post-Test Measurements and Survey Completion
Immediately following completion of the evaluation, firefighters went to a rehabilitation station, where post-test heart rate and blood pressure were monitored by a paramedic. While in rehabilitation, each firefighter completed a Work Performance Validation Survey. Subjects responded to questions regarding their understanding of how to complete the specific tasks of the evaluation and whether they performed the tasks in a manner consistent with live fireground operations. Each individual task was rated on a five-point Likert scale for criticality, physicality, prevalence, and similarity. The following definitions and descriptions were provided in the survey evaluation.
Criticality is the degree to which the task or set of tasks in question is considered critical to the performance of firefighting.
Criticality Rating
• 0=Not critical (failure to perform results in no negative consequences)
• 1=Least Critical (failure to perform results in minimal negative consequences)
• 2=Important (beneficial for the successful performance of the job)
• 3=Critical (essential for the successful performance of the job)
• 4=Extremely Critical (failure to perform results in extreme negative consequences)
Physicality is the level of physical effort required to perform a specific task or set of tasks.
Physicality Rating
• 0=No effort required (elicits no changes in heart rate, breathing rate, or overall feelings of fatigue; could continue indefinitely)
• 1=Minimal effort required (elicits minimal changes in heart rate, breathing rate, or overall feelings of fatigue; could continue for 1 hour or more)
• 2=Moderate effort required (elicits moderate changes in heart rate, breathing rate, and overall feelings of fatigue; could continue for 30 minutes or more)
• 3=Excessive effort required (elicits excessive changes in heart rate, breathing rate, and overall feelings of fatigue; could continue for 15 minutes or less)
• 4=Extreme effort required (elicits extreme changes in heart rate, breathing rate, and overall feelings of fatigue; could continue for 5 minutes or less)
Prevalence is the frequency with which the task or set of tasks is routinely performed at a fire scene.
Prevalence Rating
• 0=Never performed as a routine task at a fire scene
• 1=Seldom performed as a routine task at a fire scene
• 2=Occasionally performed as a routine task at a fire scene
• 3=Frequently performed as a routine task at a fire scene
• 4=Always performed as a routine task at a fire scene
Similarity is the degree to which the task or set of tasks performed in the physical work performance evaluation is the same as a task or set of tasks performed at a fire scene.
Similarity Rating
• 0=No similarity at all to what happens at a fire scene
• 1=Slightly similar to what happens at a fire scene
• 2=Somewhat similar to what happens at a fire scene
• 3=Highly similar to what happens at a fire scene
• 4=Exactly the same as what happens at a fire scene
The surveys were collected and summarized.
 Average Time to Completion of Task, 37 seconds Figure 7 |
null
 Average Time to Completion of Task, 1:46 seconds Figure 8 |
null
 Average Time to Completion of Task, 1:22 seconds Figure 9 |
null
 Average Time to Completion of Task, 1:03 seconds Figure 10 |
null
 Average Time to Completion of Task, 1:11 seconds Figure 11 |
null
 Average Time to Completion of Task, 59 seconds Figure 12 |
null
Videotape Selection
The review panel selected videotapes for viewing based on the following criteria. Beginning with the median videotape, videotapes were selected in descending order of time approximately 30 seconds apart to and including the slowest tape of the group. Seven tapes were selected for viewing.
Review Panel
A panel of IFD training officers served as content specialists and reviewed selected videotapes of firefighters performing the work performance evaluation. The officers were separated into three shift panels of approximately nine firefighters, each representing their respective shifts. The panels viewed and evaluated selected videotapings of IFD firefighters participating in the evaluation.
The panel members were instructed to provide their expert recommendation on determining a maximal acceptable time in which an incumbent firefighter should reasonably be able to complete the evaluation. After viewing each videotape, panel members rated each tape as “acceptable,” “marginally acceptable,” “marginally unacceptable,” or “unacceptable” based on the pace at which the firefighter completed the evaluation.
After all three shift panels viewed and rated the videotapes, their responses were analyzed to identify the first tape in the sequence of longer completion times for which a minority of the panel members indicated the rate of work, or pace, was still acceptable to marginally acceptable. As a result, a time of 8:44 was established as the maximum time acceptable for successfully completing the work performance evaluation.
RESULTS
The graphs above detail the results of the validation survey that all participating firefighters completed. Figures 1 through 5 provide an assessment of the work performance evaluation as a whole; figures 6 through 12 focus on each individual task of the evaluation.
The average time to complete the evaluation was 7:23 minutes. The maximal acceptable time established for successful completion of the evaluation based on the videotape ratings of the review panel was 8:44 minutes. Of the 75 participants in the validation phase, only seven (9.3%) completed the evaluation in more than 8:44 minutes.
CONCLUSION
Firefighters determined this work performance evaluation to be a valid tool in evaluating an incumbent firefighter’s ability to successfully perform the physically demanding essential functions of firefighting. The IFD and the IAFF Local 416 have agreed to incorporate this evaluation into their existing health and safety program. ■
■ DARRELL MENDENHALL, MS, is director of fitness and health promotion for Public Safety Medical Services, Indianapolis, Indiana.
■ STEVEN MOFFATT, MD, is the medical director for Public Safety Medical Services.
■ TISCH WILLIAMS, MA, is the manager of fitness testing for Public Safety Medical Services.
■ MIKE REEVES is president of International Association of Fire Fighters Local 416.
■ JAMES GREESON is the chief of the Indianapolis (IN) Fire Department (IFD).
■ SHAWN SHELTON, BA, is a lieutenant in the IFD.
■ HOWARD STAHL is a captain in the IFD.
■ TERRELL W. ZOLLINGER, DRHP, is a professor of family medicine and associate director of the Bowen Research Center, Indiana Department of Family Medicine, Indiana University School of Medicine, Indianapolis.
■ ROBERT M. SAYWELL JR., Ph.D., MPH, is a professor of family medicine and a senior investigator in the Bowen Research Center, Indiana Department of Family Medicine, Indiana University School of Medicine.
■ CAROLYN MUEGGE, MS, MPH, is a project manager in the Bowen Research Center, Indiana Department of Family Medicine, Indiana University School of Medicine.
