Assessment and Team Learning in a
Graduate Managerial Economics Course

Martin Milkman
Murray State University

1. INTRODUCTION
     There are two major motivations for using team projects in training business students. One of these reasons is that teaming offers students the opportunity to practice working cooperatively, a skill that is critical in today’s workplace. For example, Young and Henquinet (2000) note that “as organizations continue to decentralize decision making and to deal with today’s complex and changing environment there is greater reliance on teams…. Group projects enable students to practice important interpersonal skills, including conflict resolution, the ability to reach consensus on decisions, and effective interpersonal communication techniques.” The other reason is pedagogical. Studies indicate that group or collaborative learning enhances student achievement in college level courses. (Johnson, et al., 1998) However it seems that collaborate learning enhances student achievement mostly in courses where a clear strategy exists regarding establishing group goals and requiring individual accountability. (Slavin, 1995.) This paper discusses how these issues were addressed in a group project that is a major portion of an MBA level Managerial Economics class.
     The paper is divided into various sections. After this brief introduction, the paper discusses the Managerial Economics course, the diversity of the students enrolled in the class, and the role of the team project. This is followed by a description of how the teams are formed. The paper continues with an examination of the process of establishing group goals and individual accountability. The paper then addresses project outcomes and closes with some additional suggestions for implementing collaborative learning.
2. MANAGERIAL ECONOMICS
     Managerial Economics is a required class for MBA students at Murray State University. It is also an elective for students in the Master’s program in Economics and the Master’s program in Telecommunications Systems Management. During the spring 2001 semester 36 students were enrolled in the class. Thirty-three of these students were enrolled in the MBA program, two students were enrolled in the Master’s in Economics program, and one student was enrolled in the Master’s in Telecommunications Systems Management program.
     The students came from diverse backgrounds. Nine of the students were German and six of the students were Thai. Other international students were from Columbia, Hong Kong, Russia, Saudi Arabia, Korea, India, and Japan. Five of the United States students worked full-time and were enrolled in the MBA program on a part-time basis.
     The semester long class met one night per week for three hours. The team project is a major portion of the class. The score that students receive on the project accounts for thirty-five percent of the class grade. Each team is required to pick a particular good or service and estimate either the industry or a firm’s demand using multivariate regression analysis as well as various trend analysis techniques. The project requires students to collect data, enter the data in a spreadsheet, and analyze the data. In addition the team must research various factors that may influence the demand for the product or service they choose. Once a demand function is estimated, the team must develop a forecast for income using information from the Wall Street Journal and other sources. This forecast is then used along with some trend analysis for other independent variables to develop a forecast for future demand. Each team is responsible for a written analysis that details their team’s analysis, estimated demand function, and their forecast of future demand.
     The project begins at the beginning of the semester and each team is required to hand in their project at the last class meeting, one week before the final exam. This semester long project is designed to help students master three major topics central to the course: demand theory and analysis, multivariate regression analysis, and forecasting.
3. DIVIDING THE CLASS INTO TEAMS
     There are many different options when it comes to dividing the class into various teams. There are many considerations including diversity, fairness, and wanting the students to feel an ownership of their team. Because of these considerations most of the literature suggests that teams be formed by the instructor. (Cooper, et al., 1990, Nurrenbern, 1995; Fiechtner and Davis, 1992) In the past, after collecting information about the students’ educational and work backgrounds, I assigned students to their teams. The advantage of this approach was that by personally assigning the students to the teams, I could assure that each team was diverse and that the more prepared students were divided across the teams.
     There are several drawbacks to assigning the students to teams. Sometimes there are problems within the team and some of the students blame the instructor for these problems. Problems are likely to come up when the teams are assigned to maximize diversity. This is often because students from various cultures have different working styles and different work ethics. More mature students may also have different objectives than students entering the class with little or no full-time work experience. These drawbacks and the adverse consequences that resulted from these drawbacks helped me to realize that a different method was necessary.
     During the first class meeting an “Application for Employment” was distributed to each class member. Each class member was asked to apply for either a team leader position or a team member position. The application contains questions relating to previous work and educational experience. The educational background questions include queries about grades in specific courses, which are prerequisites for the Managerial Economics class. The application for employment also asks the applicants to rate their skills in various categories including organizing work, coordinating work, interpersonal relations, analytical skills, writing reports, oral reports, and producing zero defects work. In addition the survey asks applicants the best times for them to meet. (A copy of the application for employment is included as an appendix.) Nine of the 36 class members applied to be team leaders.
     All of the applications for team members were copied and distributed to each team leader. During the second class period each team leader set up an interview station. Each applicant for the team member position was then asked to visit each of the interview stations and be interviewed for approximately five minutes by the team leader. This worked reasonably well although one of the team leaders from Germany only interviewed the German students in the class. Also some of the German students did not interview with non-German team leaders.
     After all of the interviews were completed, the team leaders met privately. Each team leader picked a folded sheet of paper that determined the team leader’s team number. These numbers from one through nine determined the selection order. The team leader from team one selected one student first followed by the team leader from team two. After team leader nine selected a student, the process was reversed with team leader nine selecting another student, followed in descending order by team leader eight. After team leader one selected in round two, the order was once again reversed for round three. Team leaders were asked not to disclose the order in which students were picked for the various teams. Each team contained four team members.
     Four team members is an appropriate team size. The literature seems to indicate that large groups tend to fail. (Fiechtner and Davis, 1992) Larger teams often do not provide an opportunity for all members to fully participate and enhance their skills. Another problem possible problem with large teams is problems with coordination of team meetings and activities. This in the past has been a problem even with small groups given all of the time conflicts that students in the class have in balancing professional and family responsibilities along with their academic commitments. In an effort to reduce these problems the Managerial Economics course utilized a web based technology, Blackboard, that allows for discussion and coordination without physically meeting in the same place. In order to encourage the teams to utilize the Blackboard technology, all team leaders were required to notify their team members about which team they had been drafted by via their team members’ email address, which is listed on the Blackboard software.
     The Blackboard software was also used for other aspects of the class. All student grades were posted on the Blackboard. Students could access and download all supplemental class readings through the Blackboard. In addition, the Blackboard was used to post announcements. This was very useful especially since the class only met once a week. For example if an article appeared in the Wall Street Journal on Thursday, I could post an announcement on Thursday asking students to read the article before the next class meeting on the following Tuesday.
     As mentioned above, another goal is establishing diversity among team members. Of the nine teams, eight of the teams had members from at least two countries and five of the teams had team members from at least three countries. Eight of the teams contained both male and female team members. The one group that lacked diversity contained four German males.
4. INDIVIDUAL ACCOUNTABILITY
     Each team member must be individually accountable if the team project is going to be successful. Two key issues are involved. First since one of the goals of the project is to enhance student knowledge in three key topics of the course, it was important that all team members are engaged in these areas. Second there is the issue of the “free rider” problem. This occurs when a team member shirks responsibilities in the hope that other team members will work hard to make the team project a success. The shirking member hopes to share in the success of the project through a high grade. Since these problems are somewhat related the strategies to deal with these problems can be somewhat complementary.
     One way to try and motivate each student is to include quiz and exam questions that are related to the material covered in the team project and inform students in advance that these types of questions will be included on quizzes and exams. This is a strategy that I used in the Managerial Economics class. This strategy at least encourages students to attempt to learn the material even if they do not actually produce the team project. For example, on the quizzes students are asked to use output from an excel spreadsheet regression to answer questions related to topics such as statistical significance, price elasticity of demand, and prediction intervals. These types of questions can motivate students to attempt to master the material covered in the team project, however they do not help motivate students to participate in the “grunt” work which is necessary to successfully complete the project. This involves tasks such as gathering and entering the data, and proofreading the final report.
     Each team member not doing their fair share of the work is more than just a fairness issue. Smith and MacGregor (1992) note that learning flourishes in a social environment where conversation with learners takes place. Free riding destroys the social environment and therefore hinders the learning of the team members. The other team members become frustrated by the free riders and ultimately this often results in the team loosing focus on the team project. Sometimes this results in a lower quality team project. Hansen, Owan, and Pan (2006) report that differentiated point allocation which provides a way to punish free riders results in a positive and significant effect on group performance compared with autonomous groups where all members of the group receive the same grade.
     One way to attempt to deal with the free rider problem is through assessment. Cramer (1994) and Freeman (1995) both suggest using peer evaluation to deal with the problem of “free riding”. However, while instructors recognize that there is value in using peer evaluation; surveys indicate that pragmatic considerations limit their use and that most instructors only rely on their own evaluation efforts. (Young and Henquinet, 2000)
     Over the last several years I have attempted to use student assessment of the team projects to deal with free rider problem. The method is relatively simple. After the team project is handed in but before the team members receive a grade for the team project, each team member is asked to complete a team project evaluation. The first section of the evaluation asks each student to “briefly describe each of the group member’s contribution to the project. [Include yourself]. Most students write a paragraph about each team member’s contribution to the project. After completing that question, the team project evaluation gives the following prompt:

Now after thinking about how each team member contributed to the project, please assign points to the other team members based on their relative contributions. Your total points to assign are equal to 100 times the number of other team members. For example if your team had three members (yourself plus to others) you would have 200 points to assign. You might assign team member A 105 points and team member B 95 points. This would indicate that team member A did 105% of the work compared to the other members while team member B did only 95% of the work. [NOTE: Do not include team members who audited the course.]

After collecting all team members team project evaluations, I compute the average number of points that each person receives. I then grade the projects and assign a numerical score on a 100-point scale to each project. To determine each team member’s score I multiply the team member’s score by the team project score and divide by 100. For example, suppose a team project received a score of 90 and team member A received an average score of 103. Team member A would receive a score of 92.7 {[(103)(90]/100} for the team project.
     In previous semesters I used the same scoring system but I did not include the question that asks the team members to briefly describe each group member’s contribution to the project. Including this question improves the assessment. It forces the team members to reflect on each team member’s contribution and helps each team member to assign points to the other team members. As an instructor it also provides me with some sense of each team member’s contribution to the project. Normally one or two students will complain that they feel that the other team members evaluated them unfairly. The narrative descriptions also help to provide me with background information as I discuss this issue with the aggrieved student.
5. PROJECT OUTCOMES
     There are several ways to evaluate the team project. One way is examine the quality of the team project. By this standard the projects were moderately successful. Six of the nine projects received a good grade (a score in the 80s) and three of the projects received an excellent score (a score in the 90s or a score of 100).
     A second way to evaluate the team project is to ask students what they learned from the team project. On the team project evaluation one question asked was “What are the three most important things you learned while working on this project?” Over sixty percent of the students indicated that they learned something about working in teams. Examples of these types of student comments include: “groups are not always functional, but you have to pick up the slack and get the project done”, “delegate authority-don’t try to be in charge of everything”, “avoid ‘assumptions’ about people, if you don’t give enough work to someone on the team they might loose interest in the project”, and “cooperation, cooperation, cooperation”.
     A third way to evaluate the tem project is by examining exam scores on questions related to the team project and comparing them to exam scores on questions, which did not relate to the team project. Students scored 20 percent higher on the questions related to the project than they did on the other questions on the final. Perhaps, as one student noted, the project “fit well with the course material and was very helpful as far as applying the stuff we learned in class to something practical.”
6. CONCLUSIONS
     This paper has presented a case study of a team project used in an MBA Managerial Economics class. The project seemed to successfully enhance students’ ability to function in teams and to enhance students’ understanding of the course topics related to the project. Some of the project’s success may be attributed to the assessment strategy that was developed in part to deal with the free rider problem that has been recognized in previous research as one of the key barriers to implementing successful team projects.
     It would be possible to enhance the team project described in this paper by including an oral presentation. The oral presentation could serve as “rough draft” allowing other students and the instructor to give some early feedback on the analysis that might be used to enhance the final project. If an oral presentation is included all team members should be involved in the presentation and perhaps it would be useful to video tape the presentations and have each team member independently review the video as part of the group project evaluation.
     It is also important to note that research about the role of team learning in graduate business education is very limited despite the fact that many graduate business programs make extensive use of team projects. Perhaps this paper will encourage other researchers to investigate some of the issues related to team learning in graduate business programs. Specifically more investigation is necessary to understand the relationship between assessment and the success of team learning. In addition, more research would be helpful in examining the effectiveness of various strategies related to the free rider problem.

REFERENCES

Cooper, J., Prescott, S., Cook, L., Smith, L. Mueck, R., & Cuseo, J. (1990). Cooperative learning and college instruction:
       Effective use of student learning teams. Long Beach,   CA: California State University Foundation.
Cramer, S. F. (1994). Assessing effectiveness in the collaborative classroom in new directions for teaching and learning.
       San Francisco, CA Jossey-Bass publishers.
Fiechtner, S. B., & Davis, E. A. (1992). Why some groups fail: A survey of students’ experiences with learning groups.
       In Goodsell, A. S., Maher, M. R., & Tinto, V. (Eds.), Collaborative learning: A sourcebook for higher education.
       New York: National Center on Postsecondary Teaching, Learning, & Assessment, Syracuse University.
Freeman, M. (1995). Peer assessment by groups of work. Assessment & Evaluation in Higher Education, 20(30). 289-300.
Hansen, Zeynep, Owan, Hideo & Pan, Jie (2006). The Impact of Group Diversity on Performance and Knowledge Spillover ;
       an Experiment in a College Classroom. National Bureau of Economic Research, Cambridge, MA.
Johnson, E. W., Johnston, R. T., & Smith, K. A. (1998). Cooperation in the college classroom. Edina, MN:
       Interaction Book Company.
Nurrenbern, S. (1995). Experiences in cooperative learning: A collection for chemistry teacher. Madison, WI: Institute
       for Chemical education, University of Wisconsin Board      of Regents.
Slavin, R. E. (1995). Cooperative learning: Theory, research, and practice (2nd ed.). Boston:Allyn & Bacon.
Smith, B. L., & MacGregor, J. T. (1992). What is collaborative learning? In Goodsell, A. S., Maher, M. R., & Tinto,
       V. (Eds.), Collaborative learning: A sourcebook for higher education. New York: National Center on Postsecondary
       Teaching, Learning, &   Assessment, Syracuse University.
Young, C. B., & Henquinet, J. A. (2000). A conceptual framework for designing group projects.
       Journal of Education for Business, 76(1), 56-61.

APPENDIX

APPLICATION FOR EMPLOYMENT

Name: ____________________________________________________________________

Place of permanent residence: _________________________________________________

Current employer (if any): ____________________________________________________

Position for which you are applying (team leader or team member): ___________________

Undergraduate degree: _______________________________________________________
                                                    school                                                        major
Number of graduate-level business courses taken: ______________________

Number of graduate-level quantitative analysis courses taken:_____________________

Grade in CIS 443: _____    Grade in ECO 230: _______ Grade in ECO 231: _____

Work Experience:

       Employer                                    Position Held                            Responsibilities
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________

Self-Rating on the Following Skills:

Please summarize your strengths and weaknesses as you perceive them:
___________________________________________________________________________
___________________________________________________________________________

Best Times to Meet: _________________________________________________________