The Impact of a Teaching American History Opportunities
for Educators Grant on Student Achievement 

Riza D. Marjadi
Martin I. Milkman
MurrayStateUniversity

     This paper examines the impact of a Teaching American History Opportunities for Educators (Project TAHOE) grant on student achievement. The grant was a federal grant that served a consortium of 15 school districts in Far Western Kentucky. All of the grant funds were used to organize professional development activities for educators in public schools. Eighty-two teachers participated in at least one of the activities. The paper is organized into several parts. The first section contains a brief review of the literature related to professional development and student achievement. Section two then provides a description of Project TAHOE and the wide range of professional development activities that teachers can select from. In section three the paper discusses the methodology and data used. Section four presents the empirical results of our research and offers some suggestions based on the research findings.
Brief Review of Previous Literature
     While many researchers claim that professional development is “essential to efforts to improve our schools” (Borko, 2004), the link between professional development and student achievement is elusive. There are perhaps four reasons why the relationship between professional development and student achievement is not very strong.
     The professional development activities might not be effective. The prescriptions for effective professional development include providing a setting where teachers have the time to interact, discuss teaching, assist one another to implement new strategies and participate in study groups as they form a true learning community seems to be critical. (Joyce and Showers, 2002). The National Commission on Mathematics and Science (2000) contended that professional development cannot succeed without strong content. According to Hirsh (2005) in order for professional development to be effective, it must align “most clearly with the assumptions and beliefs” of the educators.
     Even if the professional development is effective there may still not be a link between professional development and student achievement. First, teachers may be too busy to implement the changes in their curriculum and/or pedagogy. This implementation lag seems to be a frequent occurrence in professional development activities especially if there is no accountability for implementation.
     However given time teachers may begin to implement new material into their classroom curriculum or change pedagogical techniques. When teachers implement new material or pedagogy, it could take several years before the true effect of the change is related in student test scores. It is even possible that test scores might go down initially as teachers refine their new material and techniques. Since many of the professional development grants only last for a short period of time, evaluation of the project might not be illustrating increased student achievement because the test to measure the impact on student achievement is given too soon after the professional development experience.
     Finally the professional development might not be related to what is measured by the student achievement data. Teachers still teach material that is not necessarily tested. For example teachers might be trained in a professional development workshop to teach their students not to use illicit drugs, but this will not necessarily lead to increases in student achievement as measured by test scores since questions regarding usage of illicit drugs are not asked in the test.
Project TAHOE
     Project TAHOE is a professional development grant for teachers of U.S. History. The grant from the U.S. Department of Education was administered by the West Kentucky Education Cooperative for a consortium of 15 relatively small school districts. Teachers could choose from a menu of professional development opportunities. These opportunities included seminars that ranged from one day seminars to a two week seminar that was offered for graduate credit at Murray State University. There were also trips to visit selected historic sites across the nation. These trips varied in length from one day to as long as a summer trip of ten days. Finally there were technology assistance workshops. Often these workshops were tied to the trips. For example before one trip teachers were given video cameras. Teachers were taught before the trip how best to utilize the cameras and then after the trip how to take their videos and integrate them into their lesson plans and units of study along with other material they may have purchased on the trip, their texts and previous handouts.
     Eighty-two teachers participated in at least one of the professional development opportunities. Most teachers participated in several of the 27 activities. Of the 82 teachers 31 were elementary school teachers, 20 taught middle school and 31 taught in high school.
Survey Results
     Immediately after every professional development activity the participants were asked to fill out a survey. While there were a number of questions on the survey related to the activity, one important question that is used in this study was “How will you use what you have learned to further student achievement in your school/district?”
     After the professional development activities were completed in Project TAHOE and teachers had a reasonable amount of time to implement the activities, they were emailed an individualized electronic survey listing the professional development activities that they had participated in. Teachers who did not respond to the electronic survey were then sent a paper survey. The survey contained a question about the classroom implementation from the professional development activity: “Have you transferred what you have learned in the above activity to your classroom or school? (Y/N)….”If yes, please write a brief description on how you accomplished the transfer.”
     The response rate to the follow-up survey was 46.8%. The response rate for elementary school teachers was 48.3%. The response rate for middle school teachers was 40.0%. The response rate for high school teachers was 50.0%.
     In Table 1 the implementation results are displayed. The professional development experiences offered in Project TAHOE included 11 discussion seminars, 14 trips and two technology assistance workshops. The discussion seminars had a total attendance of 119. This number reflects the fact that teachers may have attended more than one of the discussion seminars. The historic site trips had a total participation of 172. Once again this number reflects participation of some teachers on more than one trip. The same is true of the technology assistance workshops where 27 teachers attended.
     The follow-up survey sample accounted for half of the teachers who attended the discussion seminars. Once again it is important to remember that some of the respondents to the follow-up survey may have attended more than one discussion seminar so in this table a teacher who attended 3 discussion seminars will be counted three times in the calculation of the response rate in Table 1. The same is true of the response rates for the trips and technology assistance workshops. Implementation rates are also displayed in Table 1. As illustrated in Table 1 the implementation rates are high for all three classifications of professional development activities. (If teachers indicated that they did not implement changes in their classroom after attending a professional development activity, the follow-up survey asked them why. The teachers’ responses for not having implemented changes in their classrooms are listed in the appendix. The comments in the appendix support some of the points raised in the literature review.)
Empirical Results
     An educational production function is used to estimate the effectiveness of Project TAHOE,

 
 
where CATS06 is the U.S. History test score of the Spring 2006 Commonwealth Accountability Testing System (CATS), CATS02 is the U.S. History test score of the Spring 2002 CATS, Activity is the number of PD activities attended by teachers, Implement is the number of professional development activities that resulted in an implementation or change in pedagogy or curriculum, G5 and G8 are dichotomous variables for elementary schools and middle schools respectively, and e is error term.
     Kentucky administers a social studies test every spring to students in the fourth, eighth, and eleventh grade. All of these tests contain both open response and multiple choice questions, which cover a number of different topics including U.S. history. Our U.S. history scores are based on the multiple choice questions on the Commonwealth Accountability Tests given in Spring 2002 and Spring 2006 relating to U.S. history. These tests results are the tests for the individual schools during that year. In our empirical results, the unit of observation is the individual school. Since many of the schools are small, they may have only had one U.S. history teacher. The Spring 2002 test scores serve as the benchmark scores since Project TAHOE had not started at that time. The Spring 2006 test scores are the measure of student achievement. (The last professional development activity ended in the Summer of 2005.)
     The estimated regression function reported in Table 2 uses CATS06 as the dependent variable and uses CATS02 as the independent variable. This function form is advocated by Hanushek (1979) since using the pretest score as an independent variable may control for some of the omitted variable bias. The empirical results seem to indicate that this is a successful strategy since our R-square is equal to .3427 which is high for a pooled – cross sectional educational production function.
     The results reported here have only two statistically significant variables besides the intercept term. The first is CATS02 (the pretest score) and the other is Implement. The statistically significant and positive coefficient for the variable Implement along with the statistically insignificant negative coefficient for the Activity variable indicate that just participating in professional development activities is not enough to increase student learning. What is necessary is to implement what is learned from the professional development activities in the classroom.
     In order to make future professional development more effective, the professional development should provide guidance and encouragement to the teachers about how the professional development activity could be implemented in their classrooms. This could take the form of asking teachers to enhance or develop new lesson plans as part of the professional development activity.
     Professional development providers could also assist in developing small learning communities that could help sustain the relationships that the teachers develop during the professional development activities. These learning communities would help encourage teachers to implement new material and learning activities in their curriculum and pedagogy.

TABLES

Table 2. Educational Production Function Estimate

APPENDIX

Reasons for not having transferred the knowledge from PD

• Not yet, but I plan to.
• It isn't a complete no. I am currently teaching geography, so what I learned at the American Indian Museum is going to some use in passing. I haven't been lucky enough to use it to its fullest.
• Not yet - I am currently teaching this period and will be incorporating these lessons soon.
• Most of the information/handouts were too difficult for fourth graders; however, my understanding of the geography and geology of KY was vastly expanded and has been beneficial to me as I've been prepared lesson plans. I thoroughly enjoyed this workshop!
• Some - I am in the process of requesting new technology that will aid in doing this.
• The information from the museum itself has not really transferred itself to my class. I don't think it had to do with TAHOE at all, but the nature of the American Indian Museum just wasn't what I anticipated and was not able to use it well in my classroom
• I have World Civilization classes therefore I don't get to use this information.
• Not personally because I do not cover the content, but I have shared it with the 8th grade teacher.
• Technology is still limited for our classes. The program and training was great. We simply have no computer to run the program.

REFERENCES

Borko, Hilda. (2004). Professional development and teacher learning: Mapping the terrain. Educational Researcher, 8, 3-15.
Hanushek, Eric A. (1979). Conceptual and empirical issues in estimation of educational production functions.
       Journal of Human Resources, 3, 351-388.
Hirsh, Stephanie. (2005). Professional development and closing the achievement gap. Theory Into Practice, 1, 38-44.
Joyce, B., & Showers, B. (2002). Student achievement through staff development.
       Association for Supervision and Curriculum Development. Alexandria, VA.
National Commission on Mathematics and Science Teaching for the 21st Century. (2002). Before it’s too late: A report to the
       nation from the National Commission on Mathematics and Science Teaching for the 21st Century. Retrieved at
       http://www.ede.gov/americacounts/glenn/report.doc on June 21, 2006.