The Necessity of Culturally Responsive Assessment
Yukari Takimoto Amos
Cory Gann
Don Woodcock
Central Washington University
On January 8, 2002, President Bush signed into law the No Child Left Behind Act of 2001, reauthorizing the Elementary and Secondary Education Act. This Act aims to improve the performance of U.S. primary and secondary schools by increasing the standards of accountability for states, school districts, and schools (PL 107-110, 2001). The theories of standards-based education reform (outcome-based education) have heavily influenced the Act. These theories are based on the belief that high expectations and setting of goals will result in success for all students. As a result, the Act requires states to develop assessments in basic skills to be given to all students in certain grades, if those states are to receive federal funding for schools.
In line with the Act, the state of Washington established Essential Academic Learning Requirements (EALRs) and have advocated for standards-based education. All high school students are expected to earn a Certificate of Academic Achievement (CAA) by demonstrating proficiency in reading, writing, and mathematics before graduation. Most students earn a CAA by passing the High School Washington Assessment of Student Learning (WASL), which is a large-scale and criterion-based standardized test. Yet, to assist those students who may need to use an assessment other than the WASL to demonstrate their skills, there are state-approved alternatives to earn a CAA. The Collection of Evidence (COE) is one of them.
The COE is a classroom-centered collection of work that features examples of assignments that align with the state standards. The state rules dictate that the COE must be comparable (or exceed) in content and rigor to the WASL. The characteristics of the COE are 1) it contains examples of student work that show accurate demonstrations of student performance; 2) it can be collected over time; 3) it can include teacher assistance; 4) it can include the use of approved resources; 5) it allows opportunities for students to review, revise, and select their best work; 6) it can be geared towards a student’s particular interests, cultural background, and/or specific academic or technical area of focus. Typically the COE takes the following steps: 1) a student and teacher(s) identify appropriate tasks; 2) a student completes specific tasks, adds to collection; 3) Teacher monitors student work; 4) Teacher/counselor and student review work.
The Office of Superintendent of Public Instruction (OSPI) in the state of Washington has recently proposed the idea of culturally relevant COE with a particular focus on Native American students. This is due to the fact that there is a clear achievement gap between white/Asian students and other minority students. For example, according to the 2006-2007 WASL performance result (Office of Superintendent of Public Instruction, 2008), 84.6% of White 10th grade students met the reading requirement while 68.4% of Native American students did so. In writing, 87.4% of white students were at standard versus 72.4% of Native American students. In math, the gap was wider. 56.3% of white students met the standard, while only 31.3% of Native American students did so. African American and Latino students followed a similar pattern to that of Native American students. The aim of culturally relevant COE is to use culture as a way to raise the rate of earning a CAA among minority students who may be struggling academically.
This paper analyzes the potential benefits and damages of culturally relevant COE with a particular focus on Native American students.
Theoretical Backgrounds: Culturally Relevant Pedagogy
In the 1980s, anthropologists examined the way to develop a closer match between minority students’ home culture and the school in order to improve the students’ academic achievement. The line of research is based on the theory of cultural discontinuity. The underlying assumption of this theory is that minority students’ academic struggle lies in discontinuity between their culture and the school, particularly the invisible culture of the home and the invisible culture of the school, and that if school incorporates minority students’ culture into curriculum and instruction, the students will succeed. Cultural discontinuity may be seen in many aspects of classroom activity, including “the way teachers interact with students and the way students are asked to complete their assignments” (Au, 1993, p. 8). Several researchers conducted research based on this theory. Au and Jordan (1981) incorporated the language interaction pattern familiar to Native Hawaiian students. Erickson and Mohatt (1982) did so with Native American students by incorporating the participation pattern of their culture.
The cultural discontinuity approach, however, was criticized later by the advocates of the theory of structural inequality. Villegas (1988) suggested that the source of cultural discontinuity is located in larger social structures and that schools as institutions serve to reproduce inequalities. She argued as follows:
As long as school performs this sorting function in society, it must necessarily produce winners and losers…Therefore, culturally sensitive remedies to educational problems of oppressed minority students that ignore the political aspect of school are doomed to failure. (pp. 262-263)
In other words, minority students’ academic struggle is situated not in cultural differences themselves but in the power relation between minority groups and the larger historical, political, economic, and social forces. Unlike the theory of cultural discontinuity that defines student success within the current social structures represented at school, the theory of structural inequality challenges the process of inequality formation at school.
It is Ladson-Billings (1995b) that established the concrete theory of culturally relevant pedagogy. She first questioned the followings:
What constitutes student success? How can academic success and cultural success complement each other in settings where student alienation and hostility characterize the school experience? How can pedagogy promote the kind of student success that engages larger social structural issues in a critical way? (p. 469)
According to the theory of culturally relevant pedagogy, teachers “attend students’ academic needs, not merely make them ‘feel good’” (Ladson-Billings, 1995a, p. 160) so that students develop skills, such as literacy and numeracy, as well as technological, social, and political wherewithal necessary to participate actively in a democratic society. Teachers also need to make sure that students maintain cultural competence using their culture as a vehicle for learning. Most importantly, teachers need to help students develop critical consciousness and challenge inequities that schools perpetuate. Culturally relevant pedagogy derives from critical theory developed by Freire (1974), and it is critical in nature. Although the COE is an assessment, not a pedagogy, the idea of culturally relevant COE definitely derives from the above line of theories and research.
Culturally Relevant COE
The examples of culturally relevant COE for Native American students proposed by OSPI (2007a) include: 1) reading a passage about the origin of the Chickasaw and Choctaw tribes and answering questions in writing in the reading section, 2) using canoes as an example to solve problems in the math section. These examples clearly indicate that OSPI assumes that the mere insertion of Native American students’ culture into the content of the COE is sufficient for it to be called ‘culturally relevant.’ Integrating content from minority cultures in a subject area is one of Banks’s (2004) five dimensions of multicultural education and it is commendable. Native American students are most likely to be more motivated if the content is familiar to them. This effort is congruent with the theory of cultural discontinuity that sought a closer match between students’ home culture and school culture. What is missing, however, is the fact that it is minority students’ invisible culture, not visible culture, that is most likely to cause problems when they learn. The distinction between visible and invisible culture is important. According to Erickson (2007), examples of visible culture are language, dress, food, religion, and aesthetic conventions. Yet, visible culture is only “the tip of the iceberg of culture” (p. 42). Implicit or invisible aspects of culture need to be taken into consideration as well. Erickson (2007) lists the example of invisible culture as follows:
How long in clock time one can be late before being impolite, how one conceives or experiences emotional or physical pain, how one displays such pain behaviorally, what topics should be avoided at the beginning of a conversation, how one shows interest or attention through listening behavior, how loud is too loud or not loud enough in speaking, how one shows that one would like the speaker to move on to the next point. (p. 42)
Invisible culture is something we cannot clearly see and detect unlike visible culture and includes values, customs, traditions, etc. Since it is not seen, it is easy for others to misinterpret.
What influences minority students’ academic performance most negatively is cultural discontinuity between their invisible culture and school’s invisible culture in the process of teaching and assessment. The research based on the theory of cultural discontinuity (Au & Jordan, 1981; Erickson & Mohatt, 1982) actually incorporated the students’ invisible culture into teaching, such as language interaction pattern and classroom participation pattern, and in these studies, the students demonstrated a higher academic performance. It was not the mere insertion of visible culture into the content.
A story or folktale and a canoe are definitely aspects of visible culture. Although the integration of visible culture into the content will surely make Native American students ‘feel good’ and the familiarity with the content will give the students an advantage, it is highly doubtful that the students will actually perform better because of this. As long as the pedagogy and assessment do not consider and do not incorporate the minority students’ invisible culture, the students’ actual learning will not be culturally relevant and the students’ demonstration of their learning will not be culturally relevant, either.
Culturally Relevant Assessment for Native American Students
Then, how do we assess students in a culturally relevant way? How do we incorporate Native American students’ invisible culture into assessment? Although in 1988, the Indian Education Act made provisions for “a program of research and development to provide accurate and culturally specific assessment instruments to measure student performance in cooperation with Tribes and Alaska Native entities,” (Chavers & Locke, 1989, pp. 18-19) the above questions have not been answered clearly yet.
The quote below shows us some hint towards culturally relevant assessment for Native American students:
For White-Man-Runs-Him, as for all youth, games were real-life situations in the miniature that taught important cultural values. His youth was filled with play designed to educate and prepare him to fulfill his future role as an adult Crow warrior. In the Crow way it seemed everyone was a teacher, including his father, grandfathers, uncles, and a variety of interested educators. (Harcey & Croone, 1993, p. 35)
Among Native American tribes, performance-based assessment, as the above quote suggests, was common and used to determine how each individual could best contribute to the survival of the tribe (Bordeaux, 1995). Children were observed by adults and assessed whether or not they exhibited knowledge and skills necessary to survive and assume spiritual leadership. This kind of assessment is similar to today’s performance-based assessment in concept. Performance-based assessment could vary in forms. It could include student portfolios, student performances, teacher observations, self-assessment, work samples, group assessment, and extended tasks (Bordeau, 1995) and could reflect the context of the students’ educational experiences. This is the point at which performance-based assessment could potentially become culturally relevant.
Teacher’s Panel (1994) advocates for portfolio assessment as a form of performance-based assessment and declare that “for Native students, the portfolio’s emphasis on success and growth rather than on what the student has failed to learn is especially important.” The emphasis on students’ growth rather than failure in portfolio assessment is similar to Madhere’s (1998) concept of growth-oriented assessment in the context of culture. Since portfolio exhibits students’ best work samples, it really shows what the students know, not what the students do not know in the process of learning. The idea of growth is particularly culturally relevant to Native American students because it is relevant to their “holistic circle of learning” (Pewewardy, 1994). Quileute Tribal School in Washington and Chuska Boarding School in New Mexico now use portfolio assessment (Bordeaux, 1995).
Estrin and Nelson-Barber (1995) state the link between performance-based assessment and students’ culture as follows:
in theory, generic tasks that call on predictable sets of skills (such as reading and writing about one’s response to a book or investigating and reporting on an environmental topic of importance to the community) can be designed with local contexts and student needs in mind. They can be embedded in instruction and “administered” in flexible ways. (p. 7)
Estrin and Nelson-Barber’s ‘flexible’ assessment for Native American students include the following:
Tailor content of assessments to students’ experiences in and out of school. Use cultural resources with which students are familiar.
Use open- ended formats (not T/F or multiple choice).
Allot time for students to process instructions and tackle various aspects of a task.
Allow students opportunities to practice; give guided practice with multi-step problems.
Allow time for reflection and deliberation.
Allow students choices about when they will be assessed and how.
Provide for cooperative as well as individual assessment opportunities. Allow cooperative problem-solving.
Use forms of assessment that do not rely entirely on language or mastery of standard English (or uses of language unfamiliar to students).
Treat students as whole people with valid experiences; language and culture are part of a student’s identity and way of viewing the world. (p. 6)
These flexible ways effectively incorporate Native American students’ invisible culture into assessment and may assist in ‘redefining how success is measured’ as has already been articulated by leading First Nations, Metis and Inuit scholars and leaders, through the Canadian Council on Learning (CCL). CCL (2007) has established the areas of vision, relationships, reason and action as performance-based assessment measures in a more holistic fashion.
Because the COE is fundamentally performance-based assessment, it already holds the potential to be culturally relevant. In order for it to be culturally relevant in a real sense, we need to consider what factors affect Native American students negatively in this particular type of assessment. The four important factors that cause cultural discontinuity in the COE are: 1) reliance on verbal information and representation to the near exclusion of nonverbal, visual information and representation, 2) a heavy emphasis on writing in all three areas (reading, writing, and mathematics), 3) individual assessment only, and 4) the on-demand nature of assessment. These are all related to differences in invisible aspects of culture between Native American and European American cultures.
It is a well known fact that Native American students “learn and retain knowledge better through experience, touching, and active participation in educational activities” (Pewewardy, 1994, p. 81). Observation is also important in their learning. Wax, Wax, and Dumont (1989) gave examples of learning in Oglala Sioux society, where individuals must observe tasks in actual practice before attempting performance. Communicating information and representing knowledge through language as required in the COE is simply not a cultural norm in many Native communities. A related problem is the way Native American students communicate and write. Elliott, Adams, & Sockalingam (1999) show how apart the Native American culture and the mainstream European American culture are in terms of communication styles. In their cross-cultural comparison, the Native American culture manifests very little directness of questions and answers, and rhetorical style, ‘getting to the point,’ while the mainstream European American culture manifests those styles very much. In addition, academic writing in English is linear, which means it has one central point or theme with every part contributing to the main line of argument, without digressions or repetitions, while writing in the Native American way tends to be narrative and cyclical. Individual assessment causes another problem for Native American students who are accustomed to cooperating with each other and to sharing information. Estrin and Nelson-Barber (1995) cite one principal whose school is located on the Navajo Reservation in Arizona saying that Native American students need explicit practice with similar tasks before they can write on demand since formal, on-demand testing is alien to Native ways of demonstrating learning.
Based on the above invisible cultural discontinuity between Native American and European American cultures, culturally relevant COE should include 1) forms of assessment that do not rely on language exclusively, such as careful listening and observation; 2) an oral rather than a written response; 3) a narrative and cyclical rather than a linear writing style; 4) cooperative pair rather than individual assessment; and 5) allowing time for students to practice for any on-demand testing.
Culturally Relevant Assessment and Standards
Culturally relevant COE is promising in assessing Native American students in a culturally appropriate way, yet there still remains a fundamental problem: Native American students are assessed against the state standards. The federal government defines standards as “concrete examples and explicit definitions of what students have to know and be able to do to demonstrate that such students are proficient in…skills and knowledge” (PL 103-227, 1994, p. 129). The COE, although performance-based, needs to align with the standards the state has created. The potential damage embedded in this fact is simply the discontinuity between the Native American culture and the standards themselves.
Let’s take a look at the state of Washington’s writing standards here to further analyze the cultural discontinuity. The checklists for the writing standards include the followings:
For expository writing:
Narrowed topic, focus (idea), well-chosen details, logical organizational pattern, transitions to connect ideas, topic commitment (voice), and introduction and conclusion
For persuasive writing:
Topic or issue clearly stated, a clear, consistent position, a more than one supporting argument, a detailed supporting evidence, anticipating & refuting opposing arguments, a clear, logical organizational strategy, transitions to connect position, arguments, and evidence, persuasive language & techniques, and conclusion with a call for action. (OSPI, 2007b)
The above state standards are clearly not congruent with the communication and writing styles among Native Americans. Rather, these are “normed” based on the mainstream European American culture. Performance standards were created and implemented by the Tribal members themselves over millennia, while the performance standards are created and forced upon the Native American students by the state today. Culturally relevant COE for Native American students proposed in the previous section will not meet these standards. In other words, as long as Native American students are assessed against the standards that are not culturally relevant to them, the COE that aligns with the standards should not be culturally relevant to them if the students are to meet the standards. To be more clear, as long as the standards are not relevant to the students’ culture, the idea of culturally relevant COE or assessment is an oxymoron. If it exists, that means that the students will not meet the standards. This is a good example of the vicious cycle by which minority students, in general, are trapped with the ways schools perpetuate inequality. Minority students are measured within the mainstream standards, in other words, the current social structure represented at school. Content and pedagogy could be culturally relevant without changing the standards, while assessment could not be so since the standards are directly measured in assessment.
In order to break the vicious cycle of inequality creation at school, the standards themselves need to be reconsidered and more inclusive for cultural relevancy. Fox (2001) says that the development of new content and performance standards in the various states has created an opportunity for Indian educators to provide input about their appropriateness for Indian students. For example, North Dakota and Minnesota reached out to Indian communities for input into standards and outcomes development. Yet, Fox (2001) also adds that in many states, there has been little or no input into the development or review of standards by Indian people.
Worthen (1993) defines performance-based assessment as the assessment that directly examines student performance on specific tasks that are important for life. Since what are important for life vary from one cultural group to another, having a Native American community develop their own standards, in other words, localizing standards, will ensure compatibility with community values (Estrin and Nelson-Barber, 1995). Fox (2001) shows the examples of Indian schools that had the opportunity to develop local standards that infuse Indian cultural and other locally defined outcomes. Sleeter (2005) cites the state of Nebraska as an example of local control. In Nebraska, although “the state established content standards, districts may either adopt them or develop equally rigorous standards of their own” (Sleeter, 2005, p. 20). To help this process, the Bureau of Indian Affairs has provided a set of Indian content standards for the various academic areas to infuse Indian culture into new standards-based curricula (ORBIS Associate, 1998).
The Center on Education Policy (2001) warns us that tests serving “important collective educational goals can sometimes produce negative consequences for individuals” (p. 28). This warning is related to a fundamental question: Is one set of standards appropriate for all students? Gordon (1992) answers this question suggesting that education should strive for “universal standards and differential indicators” (p. 5). As a counter to imposing culturally biased standards on Native American students and other minority students, Estrin and Nelson-Barber (1995) propose that it is possible to “set some very broad academic standards for all students and to measure success according to a common set of criteria, while, at the same time, remaining flexible about the specific means for addressing standards and determining student achievement” (p. 6). In this way, Native American students can demonstrate their cultural competence, such as careful listening and observation, an oral response, and cooperative pair assessment, while also meeting the standards.
All of the above reconsiderations of standards, however, are hindered by, among other things, the shortage of Native American teachers and input from tribal leaders and elders. With so few Native American teachers, and the devaluing through exclusion of tribal elders and leaders and the traditional knowledge, values and wisdom they possess, it is virtually impossible to provide culturally relevant input into the standards, to localize the standards, and to assess culturally specific means to meet the standards. If those who assess are ignorant about Native American culture, the students’ demonstration of cultural competence will not be comprehended as the students intended and will most likely be misinterpreted. Thus, involving tribal leaders and elders, recruiting Native American teachers and training non-Native American teachers also must be a priority when we discussing culturally relevant assessment. Pewewardy (1994) decries the fact that “there seems to be a lack of culture-specific understanding (i.e., identity and values) of communities, culturally responsible pedagogy, parent-teacher communication methods, and relating the concepts of tribal sovereignty to self-determination” (p. 89). In addition to the standards, teacher training programs are also at fault for neglecting to provide appropriate training with the teacher candidates for culturally relevant education of Native American students.
Conclusions
Standards-based school reform provides potential benefits for Native American education since schools are held accountable for the students’ learning. In this accountability system, schools are required to provide appropriate education so that the students make progress. And these students include, of course, Native American students.
The COE as an alternative way to earn a high school diploma already marks some progress. The idea of culturally relevant COE is highly commendable. Yet, culturally relevant COE in the current form proposed by OSPI will not do any good for Native American students. Culturally relevant COE as proposed in the previous section is possible if the vicious cycle of inequality creation at school is seriously examined - with the standards as an example. This is where the critical nature of culturally relevant pedagogy/assessment comes in. Whose standards are we talking about? Whose cultural values are reflected in the standards? These two questions need to be addressed by both Native American communities and teacher training programs in higher education. Pewewardy (1998) states, “All of the restructuring in the world will be of no benefit to children if the philosophy, theory, assumptions, and definitions are flawed or invalid. Indigenous educators and parents know the problems and their causes” (p. 30). The vicious cycle of inequality creation at school will continue to exist unless Native American voice is heard and honored. When such voice is heard, cultural responsiveness can be a reality, and culturally relevant assessment becomes possible.
References
Au, K. H. (1993). Literacy instruction in multicultural settings. Orlando, FL: Harcourt Brace & Company.
Au, K. H., & Jordan, C. (1981). Teaching reading to Hawaiian children: Finding a culturally appropriate solution. In H. Trueba,
G. Guthrie, & K. Au (Eds.), Culture and the bilingual classroom: Studies in classroom ethnography (pp. 69-86). Rowley,
MA: Newbury House.
Banks, J. A. (2004). Multicultural education: Historical development, dimensions, and practice. In J.A. Banks & C. A. M.
Banks (Eds.), Handbok of research on multicultural education (2nd ed.) (pp.3-29). San Francisco: Jossey-Bass.
Bordeaux, R. (1995). Assessment for American Indian and Alaska native learners. ERIC Digest. (ERIC Document Reproduction
Service No. 385424)
Canadian Council on Learning. (2007). Redefining how success is measured in First Nations, Inuit and Metis Learning.
Ottawa, ON: Author. Retrieved March, 2008 from
http://www.ccl-cca/CCL/Reports/RedefiningSuccessInAboriginalLearning/?Language=EN
Center on Education Policy. (2001). It takes more than testing: Closing the achievement gap. Washington, D.C.:Author.
Retrieved April, 2008, from http://www.ctredpol.org/pubs/closing_achievement_gap
Chavers, D., & Locke, P. (1989). The effects of testing on Native Americans. Paper commissioned by the National Commissio
n on Testing and Public Policy, April 4.
Elliott, C., Adams, R. J., & Sockalingam, S. (1999). Normative communication styles and values. Retrieved April 27, 2008 from
http://www.awesomelibrary.org/multiculturaltoolkitstyleschart-normative.html
Erickson, F., & Mohatt, C. (1982). Cultural organization and participation structures in two classrooms of Indian students. In
G. Spindler (Ed.), Doing the ethnography of schooling (pp. 131-174). New York: Holt, Rinehart & Winston.
Erickson, F. (2007). Culture in society and in educational practices. In J. A. Banks & C. A. M. Banks (Eds.), Multicultural
education: Issues and perspectives (6th ed.) (pp. 33-61).
Estrin, E. T., & Nelson-Barber, S. (1995). Issues in cross-cultural assessment: American Indian and Alaska Native students.
Knowledge Brief, 12, 1-8.
Fox, S. J. (2001). American Indian/Alaska Native education and standards-based reform. ERIC digest. (ERIC Document
Reproduction Service No. ED459039)
Freire, P. (1974). Pedagogy of the oppressed. New York: Seabury.
Gordon, E. (1992). Implications of diversity in human characteristics for authentic assessment. CSE Technical Report 341,
National Center for Research on Evaluation, Standards, and Student Testing (CRESST). Los Angeles: UCLA graduate
School of Education.
Harcey, D. W., & Croone, B. R. (1993). White-Man-Runs-Him. Evanston, IL: Evanston Publishing.
Ladson- Billings, G. (1995a). But that’s just good teaching! The case for culturally relevant pedagogy. Theory Into Practice,
34(3), 159-165.
Ladson- Billings, G. (1995b). Toward a theory of culturally relevant pedagogy .American Educational Research Journal, 32(3),
465-491.
Madhere, S. (1998). Cultural diversity, pedagogy, and assessment strategies.Journal of Negro Education, 280-295.
Office of Superintendent of Public Instruction. (2007a). Book2: Collection of evidence work samples for mathematics, reading,
and writing. Olympia, WA: Arthur.
Office of Superintendent of Public Instruction. (2007b). CAA options collection of evidence. Olympia, WA: Arthur.
Office of Superintendent of Public Instruction. (2008). Washington state report card. Retrieved April 2, 2008, from
http://reportcard.ospi.k12.wa.us
ORBIS Associates. (1998). American Indian standards for arts education. Washington, D.C.: Bureau of Indian Affairs. (ERIC
Document Reproduction Service No. ED420478)
Pewewardy, C. (1994). Culturally responsible pedagogy in action: An American Indian magnet school. In E. Hollins, J. King, &
W. Hayman (Eds.), Teaching diverse populations: Formulating a knowledge base (pp. 77-92). Albany, NY: State University
of New York Press.
Pewewardy, C. (1998). Our children can’t wait: Recapturing the essence of indigenous schools in the United States. Cultural
Survival Quarterly., 22(1), 29-34.
PL 103-227. (1994). Goals 2000: Educate America Act. Washington, D.C.: Government Printing Office.
PL 07-110. (2001). No child left behind act. Washington, D.C.: Government Printing Office.
Sleeter, C. E. (2005). Un-Standardizing curriculum: Multicultural teaching in the standards based classroom. New York:
Teachers. College Press.
Teacher’s Panel. (1994). Remarks of 10 American Indian teachers at a day-long meeting held at Far West Laboratory,
June 29, 1994.
Villegas, A. (1988). School failure and cultural mismatch: Another view. The Urban Review, 20, 253-265.
Wax, M., Wax, R., & Dumont, R., Jr. (1989). Formal education in an American Indian community. Prospect Heights, IL:
Waveland Press.
Worthen, B. R. (1993). Critical issues that will determine the future of alternative assessment. Phi Delta Kappan, 74, 444-448. |
