Immediate Knowledge Gain from Agricultural Transportation Safety Workshops

Cynthia Houlden, Frank Tenkorang, Stephen King, Bruce Schanbacher, Bruce Elder
University of Nebraska-Kearney

INTRODUCTION
     Transportation is an integral part of any society. It ensures that goods produced in one geographic location get to all parts of the country. According to the United States Department of Agriculture (USDA), agriculture accounts for half of all freight transportation in the United States (USDOT). Food transportation is important in the state of Nebraska because of its agrarian economy and location along the Central North American Trade Corridor. In 2006, Nebraska ranked fourth in the nation in terms of cash receipts ($12 billion) from farm marketing (Nebraska Agriculture Fact Card, 2008). Cattle, grains, and dairy accounted for 95 percent of these total receipts. The inherently geographic nature of food production coupled with the bulky and highly perishable nature of food commodities makes efficient and safe transportation important. Therefore, efforts are needed to improve safety awareness in the agricultural transportation arena.
     In August of 2005, the Nebraska Safety Center received a $496,000 grant from the U.S. Department of Transportation (USDOT) for research and development work. The grant was intended to answer the question:  “Is there a need for agricultural transportation safety education?” The initial project was divided into three phases:  needs assessment, program development, and pilot testing of programs. 
     The Social Sciences Research Center at Wayne State College conducted the needs assessment survey for the Nebraska Safety Center in 2006. The survey results indicated that large vehicle safety training was a topic of interest for participants as was developing regional training events targeted toward end users (Nitzke, Hallgren, Nelson, & Mancastroppa, 2006). The survey results also indicated that workshops or seminars located within 60 miles of the respondent’s home were the preferred training format.
     Two workshops where developed on Truck Safety to gauge regional interest and to determine the direction to proceed with program development. As a result, the Nebraska Safety Center acquired a mobile training lab and developed training modules for delivery at various locations throughout the state. The training modules address All Terrain Vehicle (ATV) and truck safety as well as cattle handling procedures. 
     The objective of this paper is twofold. First, to evaluate the level of safety transportation knowledge among program participants, and second, determine the knowledge gained through the program delivery phase of the Nebraska Safety Center’s Agricultural Transportation Training program workshops.
BACKGROUND
     When developing the workshops for the Agricultural Safety Training program, the Nebraska Safety Center reviewed the transportation methods currently used on Nebraska farms and roads. The Wayne State Survey indicated that 75.9 percent of respondents (Nitzke et al., 2006) used pick-up trucks. Heavier trucks followed with (62.9 percent), farm machinery (55.2 percent), and ATVs (38.4 percent).
     According to the Nebraska Department of Roads (NDOR), in 2006, more than 800 semi-trucks and 1,000 other heavy trucks were involved in accidents on Nebraska roads resulting in 28 deaths and 600 injuries (Table 1). The data also indicates that fatigue, usually associated with long distances, accounted for only 0.57 percent of the 53,065 motor vehicle accidents recorded in 2006. This leaves other factors such as maintenance issues, negligent operations, and defective equipment responsible for most of the accidents. Therefore, the Nebraska Safety Center determined a need for training on appropriate maintenance of equipment and operation of trucks in agricultural situations.
     ATVs have also become a popular form of on-farm transportation. According to the ATV Safety Institute (Info Sheet: Industry At A Glance, 2002), about 15 million Americans ride seven million ATVs, including over 70,000 ATVs in Nebraska. The sale of ATV’s has increased by over 200 percent between 1995 and 2006 (Speciality Vehicle Institute of America, Summer 2007). The increasing numbers of ATVs and ATV operators raise safety concerns, which are the primary reason for their inclusion in the study. ATVs, often mistaken for toys, can cause serious injuries, or deaths in accidents. Between 1982 and 2006, 73 ATV related fatalities were recorded in Nebraska (Streeter, 2008). Over 92 percent of ATV, accidents are due to improper user behaviors including not wearing a helmet, overloading, riding on public roads, and excessive speed (ATV Safety Institutute, 2004).
Although farm related ATV accident statistics are not available for Nebraska, they are available in other states. The Integrated Plant Protection Center of Oregon State University reports that about 19 percent of fatal injuries to youth on U.S. farms involve motor vehicles, including ATVs (Oregon State University). In 1993, at least 50 injuries from farm-related ATV accidents were recorded in Iowa (Schwab, Miller, & Satre, 1994). Data from Kansas Farm Bureau shows that in 2005, 11 of the 19 ATV farm-related accidents in the state were fatal, while three of the six similar accidents in 2007 and 2008 were fatal. Since a majority of ATV-related accidents are associated with rider behaviors, the ATV industry advocates for state regulation to address the problem (Speciality Vehicle Institute of America, Summer 2007).
     The large increase in ATV sales data indicates that the number of ATV operators is expected to continue increasing. The long-term forecast for drivers of heavy trucks in long-haul freight transportation shows a 3.4 percent average annual growth between 2004 and 2014 (Global Insight, Inc., 2005). Therefore, accidents rates are likely to increase as well, especially when coupled with an expected increase in agricultural production. This scenario requires the development and adherence to transportation safety practices by truck and farm vehicle operators.
Literature Review
     The approach used to measure knowledge gain could influence the results. Comparing the results of pre/post course assessment, Miller, Coughlin, and Yenger (2007) found that number-right scoring underestimates knowledge gain because it promotes guessing. Negative marking, on the other hand, improved measurement reliability of the knowledge gain since it penalizes for incorrect answers. In addition to these scoring techniques, the National Knowledge Service found that purposefully designed and targeted information leaflets effectively impart knowledge (Roy, et al., 2008).
     Trainers often assume that the knowledge gained through training will lead to changes in behavior and more focused safety practices. However, this is not always the case. Several factors have been found to influence the relationship between knowledge gain and behavioral change. These factors include intention, environmental constraints, skills, attitudes, norms, self-standards, emotion, and self-efficacy (Fishbein, Triandis, Kanfer, Becker, Middlestadt, & Eichler, 2001; Pollock, Coffman, & Lopez, 2002) factors hold true in many cases. For instance, McDivitt (1985) found that the standard of living in Gambian villages significantly affected the use of oral rehydration solution (ORS) although all villagers were aware of ORS.
Galavotti, O'Reilly, Sheridan, & Cohn (1989) found that a belief about normative behavior has influence on the likelihood of gay/bisexual men changing unsafe sexual behavior.
      Hale (2002) indicates three factors that need attention in measuring behavioral change. These include 1) the expected behaviors, choices, or output, 2) the opportunities where people can exhibit the behavior, and 3) when and how to best measure the presence or absence of these behaviors. Kirkpatrick (1960; 1996; 1998; Hayford, 2002) provides four levels of evaluation. The first level, reaction, concerns participant’s feelings about the training program. A pleasant environment enhances willingness and reception (Kirkpatrick, 1996). The second level, learning, measures knowledge acquired, skills improved, or attitudes changed due to the training (Kirkpatrick, 1998). This level involves the use of pre- and post-assessment. The third level, behavior, determines whether acquired knowledge leads to behavior change. The last level is a measure of results, which includes improved productivity to higher quality or reduced costs (Kirkpatrick, 1960)
METHODOLOGY
     The factors influencing the relationship between knowledge gain and behavior change discussed above are applicable to transportation safety knowledge. The challenge is the measurement of this knowledge gain in relation to behavior change. The best approach is to follow Kirkpatrick’s four levels of evaluation. However, this could be time consuming, too complex, and expensive to monitor (especially the last two levels) in this study. In addition, the negative-marking approach presented by Miller et al. (2007) is also a good measure of knowledge gain, however; the informal nature of the Nebraska Safety Center workshop and survey administration makes this method impracticable. Therefore, researchers used number-right scoring in this study although it underestimates knowledge gain.
     Using levels I and II of Kirkpatrick’s evaluation process, researchers developed unique activities. The activities discussed in this section include the selection of participants, training exercise, and the survey.
Selection of Participants
     Participants included event attendees at trade shows, training workshops, county/regional fairs, and agribusiness meetings. Researchers positioned the mobile training lab to provide one entrance and one exit. Research team members were stationed at the entrance to explain the research goal to attendees. Because lack of interest undermines results (based on Kirkpatrick’s level I) unwilling attendees were not persuaded to participate. Anyone willing to participate did so and participants had the freedom to end their participation at anytime without any consequences. Because of human research protocols, participants under 18 years of age were required to have parental consent to participate in the study.
Training Exercise
     The training exercise began in November 2007 and ended in June 2008. At the entrance of the mobile training lab, willing participants completed a pre-survey. The training exercise itself was informal in nature but well organized. Participants observed displays of pictures and videos of various truck and ATV accidents. The causes of each accident as well as preventive measures were provided. In addition, a dummy operator on an ATV demonstrated rollover and passenger safety concerns. This part of the exercise generated interesting discussions. The discussion then led to posters displaying information on basic safety practices with a research team member available to answer questions. At some workshops, a Nebraska State Trooper with the Motor Carrier Division was on hand to explain Nebraska truck regulations as well as a variety often-ignored truck safety issues. The overall exercise took 10 to 20 minutes depending on the individual participant’s enthusiasm. Once the training exercise was complete, willing participants completed a post-survey.
Survey
      The pre- and post-surveys (see Appendix I) contained the same set of questions: five truck safety related questions, and five ATV safety related questions. An 80 percent correct response rate was considered sufficient knowledge in safety practices.
     An 80 percent correct response rate is considered the minimum threshold for sufficient knowledge in safety practices. This 80 percent standard is dual based: it is the average of the correct responses to actual driving test questions used by GMAC (GMAC Insurance National Drivers Test, 2008), and 80 percent is the passing score for the written driving test in all Nebraska Department of Motor Vehicle testing stations (Department of Motor Vehicles, 2008). The average score on the GMAC survey was 78.1 percent for the U.S., and 83.4 percent for Nebraska (MSN Money Staff, 2008).
     The purpose of the pre- and post-surveys was to demonstrate any positive gain in immediate knowledge from the training workshop. Paired T-test was used to determine whether the difference in the pre- and post-responses was statistically significant. The test was done for the overall means of the two surveys as well as by categories such as age, gender, and questions.
Limitations
     Two significant limitations were encountered. A performance objective could not be assessed because the study did not address behavioral change. In addition, consent issues disqualified participants less than 18 years-old from the study even though this group forms a significant proportion of ATV operators.
RESULTS
     First, this section presents the overall survey results, followed by the response analysis, which presents the knowledge base of participants and the knowledge transfer resulting from the training. Finally, the analysis of the results is presented by the following categories: CDL holders, ATV operators, and other participants.
Overall Results
     Over 500 participants completed the surveys but only 352 participants completed either the pre- or both pre- and post- surveys appropriately. Of the 352, only 264 completed both surveys appropriately. Therefore, the analysis of objective one, determining knowledge base, was done with the 352 pre- responses, while the analysis of the second objective, measuring knowledge gain, was based on the 264 pre- and post- responses.
     Demographic questions were asked to determine the makeup of the participant group. Participants were categorized by, age, occupation, gender, commercial driver’s license (CDL) holder, and ATV operators. Table 2 presents demographic statistics of the participants.
     Males dominated the survey, making up about 79 percent of the 352 participants. About 38.1 percent of participants were between 19 and 24 years of age, the majority of whom described themselves as students. Approximately 34 percent had CDLs, while about 70 percent operated ATVs.
Response Analysis                                              
     The response analysis is presented in two sections. First, the agricultural transportation knowledge base before the safety training workshops is presented. Then pre- and post- responses are compared to determine knowledge gained through the safety workshop. The analysis of the results is based on correct responses.
Knowledge Base
     Correct responses with less than 80 percent were considered inadequate. Consequently, participants were considered knowledgeable in only two safety areas (Table 3). These include when a helmet should be worn when riding an ATV (Q8), and ATV carrying capacity (Q9). The “helmet” question was a control question while the “capacity” question was more or less based on common sense. Therefore, the high level of correct responses was expected.
     Questions on tread depth of tires, pre-trip inspection, safety equipment, and ATV rollover angle, had less than 50 percent correct responses. These poor results alone underscore the need for safety training workshops. 
Knowledge transfer
     Comparison of the pre- and post- results shows significant gain in knowledge. The average correct response is 60.7 percent and 89.4 percent for pre- and post-, respectively. The difference of 28.7 percentage points is statistically significant at any conventional test level (1 to 10 percent). Table 4 presents the breakdown of the difference by question. All the post- questions had correct response of over 80 percent except the question on tread depth, which recorded 62 percent. In hindsight, some participants may have misunderstood the question “What is the minimum tread depth for ‘drive tires’ on a truck?” Participants may have incorrectly understood “drive tires” to be the front tires, which respond to the steering wheel. Technically, “drive tires” are the rear tires on the tractor that push the tractor, which in turn pulls the trailer. The improvement in the correct response was statistically significant at any conventional test level for all questions except question 8 concerning when a helmet should be worn when riding an ATV. The lack of significance attests to its control status. It should also be noted that, although the drive tire question had a low post- correct response, it there was an improvement over the pre- response.
Results by Category
     For future educational purposes, it is important that the analysis be categorized to reveal the strengths and weaknesses in the various groups. The core of the analysis is the comparison of the results of CDL holders to non-CDL holders on the truck questions, and ATV operators to non-ATV operators on the ATV questions. Results were also analyzed based upon the demographic profile of the participants.
CDL Holders
     Table 5 compares knowledge gained by CDL status, and by ATV status. Out of the 264 participants, 76 had CDLs. CDL holders fared better than non-CDL holders on all the truck pre- and post- questions. Non-CDL holders scored less than 80 percent on all the pre- questions while CDL holders scored above 80 percent on two questions. The gain in knowledge was significantly high to put both participants above 80 percent on the post- questions except question 2, which participants may have misunderstood as discussed in the previous section. The CDL post- response was especially encouraging. The average knowledge gain was significantly higher for non-CDL holders because of their initial low score.
ATV operators 
     Similar to the CDL holders, ATV operators performed better than non-ATV operators on the ATV questions. (Table 5). Interestingly, however, non-ATV operators performed better than ATV operators on the rollover pre-question (Q7) although both participants had very low scores (low 30s). Such low percentages could be one of the reasons for the high ATV rollovers, which are often blamed on poor vehicle models. The significant improvement in the overall correct responses in the post- survey is encouraging. ATV operators had an average score of 93.6 percent which is about 20 percentage points above the pre- results (p-value = 0.05). Correct responses were well above 80 percent. There was no significant difference between responses by riders and non-riders about the question on ATV tiredness remedy question (Q10). A positive outcome is that both groups scored above 95 percent on question 10 in the post- survey.
Other Categories
     Using the difference between pre- and post- correct responses, teachers gained the most knowledge (32 percentage points) among the occupation categories. They were followed by farmers/ranchers, and students and agricultural professionals, in that order. The elderly (over 65 years) gained the most within the age category, while females gained more than men in the gender category did.
CONCLUSION
     Safe transportation practices are essential in the movement of commodities from the farm to the market. The research team used pre- and post- surveys to determine the level of knowledge in safety practices and knowledge gain through workshops organized by the Nebraska Safety Center. Comparison of pre- and post-survey responses shows significant improvement in knowledge in truck and ATV transportation safety practices. Overall, the safety workshops led to a 28.7 percent increase in knowledge for workshop participants. This figure reflects the effectiveness of the safety workshop. The seriousness of ATV and the high cost of truck accidents, coupled with the significant knowledge gain make the expansion of this program imperative to many communities in Nebraska and other states.
     It is expected that the improvement in safety knowledge will transform into fewer agricultural transportation related accidents. Future safety workshops will focus on questions that received low correct responses. When sufficient time has elapsed, another study could be conducted to investigate accident rates before and after the agricultural transportation safety program began.

REFERENCES

ATV Industry Continues to Put Safety First – Advocates Training Programs, State ATV Safety Legislation, and Parental Supervision as Keys to Improving ATV Safety. (2004, November 22). Retrieved 2008, from ATV Safety Institutute:
http://www.atvsafety.org/asi.cfm?pagename=Media%20Information&content=6305BEEE-3048-280F-E87835BCDD331DBA&referer=ASI%20Press%20Releases

Department of Motor Vehicles. (2008). State of Nebraska Department of Motor Vehicles Driver Licensing Services Policies and Procedures Manual.

Fishbein, M., Triandis, H., Kanfer, F., Becker, M., Middlestadt, S. E., & Eichler, A. (2001). Factors Influencing Behavior and Behavior Change. Retrieved December 2008, from The Communication Initiative Network:
http://www.comminit.com/en/print/265656

Galavotti, C., O'Reilly, K. R., Sheridan, J., & Cohn, D. L. (1989, June). Behavioral intentions and other factors influencing behavior change in a cohort of gay men: a prospective study. Retrieved December 2008, from NLM Gateway:
http://gateway.nlm.nih.gov/MeetingAbstracts/ma?f=102179756.html

Global Insight, Inc. (2005). The US Truck Driver Shortage: Analysis and Forecasts. Global Insight Inc.

GMAC Insurance National Drivers Test. (2008). Retrieved December 2008, from GMAC Insurance: http://www.gmacinsurance.com/SafeDriving/

Hale, J. (2002). Performance Based Evaluations: Tools and Techniques to Measure the Impact of Training. San Francisco: Jossey Bass Publishers.

Hayford, P. J. (2002, December). Measurement of Immediate Gain in Knowledge and Long Term Change of Behavior After Attending an Enhanced Case Management Workshop. MS Thesis . University of Wisconsin Stout.
Info Sheet: Industry At A Glance. (2002). Retrieved 2008, from ATV Safety Institute:
http://www.atvsafety.org/infosheets/industry_at_a_glance.GIF

Kirkpatrick, D. L. (1998). Evaluating Training Programs: The Four Levels (2nd Edition ed.). San Francisco: Koehler Publishers, Inc.

Kirkpatrick, D. L. (1996). Great Ideas Revisited: Techniques for Evaluating Traing Programs. Training and Development , 50 (1), 54-60.

Kirkpatrick, D. L. (1960). Techniques for Evaluating Training Programs. Journal for the American Society of Training Directors , 14, 28-32.

Lakshmanan, T., Anderson, W. P., & Chatterjee, L. (2002). Techno-Economic Innovations in Transport and Trade Expansion: The Case of NAFTA. Working Paper, Boston University, Center for Transportation Studies, Boston.

Lockwood, D. (1998). From Knowledge Gain to Behavior Change: Shifting the Focus of HIV Prevention Efforts. VOICES , 4 (2).

McDivitt, J. A. (1985). Constraints of Knowledge Gain and Behavior Change in Response to a Multi-Media Health Education Project in Gambia, West Africa. PhD Dissertation . University of Pennsylvania.

Miller, M. J., Coughlin, E. M., & Yenger, R. D. (2007, July 14). Assessing Knowledge Gain Using Number-Right and Negative-Marking Conditions in a Literature Evaluation Methods Course. Retrieved December 2008, from Paper presented at the annual meeting of the American Association of Colleges of Pharmacy, Disney’s Yacht & Beach Club Resort, Lake Buena Vista, Florida:
http://www.allacademic.com/meta/p_mla_apa_research_citation/1/9/4/1/6/p194165_index.html

Montufar, J., & Clayton, A. (2002). Seasonal weight limits on prairie region highways: opportunities for rationalization and harmonization. Canadian Journal of Civil Engineering , 29, 8-16.

MSN Money Staff. (2008, May 28). Could you pass a driving test? Retrieved December 2008, from MSN Money:
http://articles.moneycentral.msn.com/Insurance/InsureYourCar/CouldYouPassADrivingTest.aspx

Nebraska Agriculture Fact Card. (2008, February). Retrieved April 24, 2008, from Nebraska Department of Agriculture:
http://www.agr.state.ne.us/facts.pdf

Nitzke, J., Hallgren, K., Nelson, J., & Mancastroppa, R. (2006). Analysis and Report: Nebraska Agricultural Transportation Safety Training. A survey for the Nebraska Safety Center at the University of Nebraska at Kearney. Wayne State College, Wayne.

Oregon State University. (n.d.). Safety Tip - Children on Farms. Retrieved December 2008, from Integrated Plant Protection Center:
http://www.ipmnet.org/Tim/Farm_Safety/Farm_Safety_Tip_-_Children_On_Farms.pdf

Pollock, A., Coffman, J., & Lopez, M. E. (2002, Winter). Using Behavior Change Theory to Communicate Effectively: The Case of Latino Parent Involvement. Retrieved December 21, 2008, from Harvard Family Research Project - Evaluation Exchange:
http://www.hfrp.org/var/hfrp/storage/original/application/10a8ef764e6fa21c9823b8e6cfc9e3b2.pdf

Roy, A., Abubakar, I., Yates, S., Chapman, A., Lipman, M., Monk, P., et al. (2008, October 23). Evaluating knowledge gain from TB leaflets for prison and homeless sector staff: the National Knowledge Service TB pilot. Retrieved December 2008, from The European Journal of Public Health: http://eurpub.oxfordjournals.org/cgi/content/abstract/18/6/600

Schwab, C. V., Miller, L., & Satre, S. (1994, September). ATV Safety for Farm Work, Recreation. Retrieved December 2008, from National Ag Safety Database, Iowa State University: http://www.cdc.gov/nasd/docs/d001001-d001100/d001072/d001072.html

Speciality Vehicle Institute of America. (Summer 2007). Specialty Vehicle Institute of America, Special Report. Retrieved April 2008, from ATV Safety Institute: http://www.atvsafety.org/sviapressreleases/SVIA_Special_Report.pdf

Standard Summary of Nebraska Motor Vehicle Accidents. (2006). Retrieved April 24, 2008, from Nebraska Department of Roads:
http://www.nebraskatransportation.org/highway-safety/docs/acc-summ-repts/2007reports.pdf

Streeter, R. A. (2008, February). 2006 Annual Report of ATV-Rleated Deaths and Injuries. Retrieved December 2008, from Consumer Products Safety Commission: http://www.cpsc.gov/LIBRARY/atv2006.pdf

US Census Bureau. (2006, December 20). 2002 Commodity Flow Survey. Retrieved December 2008, from US Census Bureau:
http://factfinder.census.gov/servlet/IBQTable?_bm=y&-geo_id=&-fds_name=EC0200A1&-_skip=0&-ds_
name=CF0200A05&-_lang=en

USDOT. (n.d.). Freight Analysis Framework - FHWA Freight Management and Operations. Retrieved January 28, 2008, from USDOT:
http://www.ops.fhwa.dot.gov/freight/freight_analysis/faf/index.htm

Appendix I

Pre/Post Survey
Agricultural Transportation Safety Training Survey
Nebraska Safety Center
University of Nebraska at Kearney
We appreciate your participation in this brief survey. Please fill out and return to your instructor.
The purpose of the survey is to assess knowledge acquired by participants of the Agricultural Transportation Safety Training held at the University of Nebraska, Kearney. All individual responses will be treated confidentially. If you have any questions, please contact Frank Tenkorang, Department of Economics, University of Nebraska, Kearney, NE, (308) 865 8573.
Thank you for your cooperation

1. Which of the following would best describe you?

2. What is your age?

3. What is your gender?

4. Do you have a commercial driver’s license (CDL)? 

5. Do you drive ATV’s?

1. You can use re-treaded tire on a steering axle of a truck if it is for farm use only.
   1. True
   2. False
2. What is the minimum tread depth for drive tires on a truck?
   1. 2/32”
   2. 4/32”
   3. 6/32”
   4. 8/32”
3. A pre-trip vehicle inspection should be done on a truck at least:
   1. Twice per day
   2. Before each load
   3. Once per day
   4. Once per week
4. Every truck must have the following safety equipment in it except:
   1. Three red triangle reflectors
   2. A properly rated fire extinguisher
   3. Spare fuses (unless the truck is equipped with circuit breakers)
   4. A spare tire in case of a flat in a remote area
5. What is the standard maximum weight that a five axle tractor trailer can weigh when it travels the interstate highway system?
   1. 40,000 lbs
   2. 60,000 lbs
   3. 80,000 lbs
   4. 100,000 lbs 

1. You should always lean away from a turn with an ATV in order to gain more leverage on the handlebars during the turn.
   1. True
   2. False
2. What angle is a stationary four wheeler (with 150 lb rider and 10 gal spray tank) in danger of rolling over backwards?
   1. 40-45 degrees
   2. 50-55 degrees
   3. 60-65 degrees
   4. 70-75 degrees
3. When should one wear a helmet when riding an ATV?
   1. At all times
   2. When an accident is likely
   3. During recreational riding
   4. When performing farm duties
4. How many people can an ATV safely carry?
   1. 1
   2. 2
   3. 3
   4. 4
5. Riding an ATV can be tiring therefore it is advised to:
   1. Ride with a friend
   2. drink extra liquid and rest before riding
   3. Take frequent rest breaks
   4. Take long break