Breaking New Ground: The Comprehensive Social Studies Assessment Project (CSSAP)

 

Fred Czarra

At various times in recent decades, social studies has appeared to be a mere stepchild of educational reform. The needs of subjects such as mathematics and science have taken precedence over the social studies disciplines of civics, economics, geography, and history. This has also been true for national assessment efforts, which have emphasized reading and mathematics. However, recent events are moving social studies and its assessment into the educational limelight.

A collaborative effort by 23 states to develop assessments in history, geography, civics, and economics for elementary and secondary students is nearing completion. This consortium effort, known as the Comprehensive Social Studies Assessment Project (CSSAP), was initiated in November 1997 with a $3.5 million grant from the Office of Educational Research and Improvement, U. S. Department of Education. This project grew out of an earlier collaborative effort by five states—the SCASS (State Collaborative on Assessment and Student Standards) Social Studies project—which concluded three years of similar assessment development early in 1998.1

The primary purpose of the CSSAP project is to develop assessments for upper elementary, middle, and high school students based on major themes from civics, economics, geography, and history. These themes, drawn from state standards adapted from national standards in the various disciplines, provide the Concensus Framework for the project. Unlike many traditional tests in social studies, the CSSAP assessments involve high-level thinking processes that include constructed responses and a performance element asking students to apply their knowledge and skills. The second purpose of the CSSAP project is to initiate a professional development program that explains and illustrates appropriate use of social studies assessments to teachers in schools in the participating states.

Closely linked with the assessments and professional development plan are two more purposes: the design and implementation of a portfolio assessment system at the elementary, middle, and high school levels, and the development of a CD-ROM. The portfolio is meant to showcase student work in the four social studies disciplines, with separate scoring rubrics for the items in the portfolios. The CD-ROM constitutes the delivery system for the professional development process, and will include all the necessary information and materials developed by the project to help teachers develop and use quality social studies assessments.

The CSSAP project is directed by a management team consisting of representatives from the Missouri Department of Education, the Council of Chief State School Officers, and ACT as the assessment development contractor. For operational purposes, such as the teacher workshops for developing assessments, the 23 collaborating states have been divided into five national regions: States currently involved in the grant are (by region): Region One: Kansas, Louisiana, Missouri, and Nebraska; Region Two: New Mexico, Oregon, Utah, and Washington; Region Three: Connecticut, Delaware, New Jersey, and Pennsylvania; Region Four: Massachusetts , Maine, New Hampshire, Rhode Island, and Vermont; and Region Five: Kentucky, Michigan, Minnesota, South Carolina, West Virginia, and Wisconsin. Other states are invited to join the project at any time.

Additionally, there is a portfolio committee and a professional development committee created to carry out these respective tasks. All of the state social studies and assessment staff members meet twice a year for a project update and to hear from the management team about issues and ideas that need to be decided.

 

Teachers at the Forefront

A basic assumption of the CSSAP project was that assessments need to reflect the authentic content and activities that characterize good social studies instruction. Therefore, the plan called for classroom teachers to be engaged in writing the assessment activities.

As a first step, six outstanding teachers from each state in the consortium were identified. The selection process carefully balanced the make-up of the teams by content expertise, teaching level, gender, ethnicity, and type of school (rural, urban, suburban, etc). The group of about 135 teachers was then divided by region and assigned specific content themes according to their level of teaching (elementary, middle school, or high school).

Each teacher was asked to propose two sets of assessment exercises based on the content themes assigned to that teaching level. After the proposals were balanced as to type by the project staff, the teachers received instruction in creating good assessment items. This took place at two 3-day workshops held in each of the five regions over a period of nine months. In addition to the training and support for travel, the teachers received stipends for their completed work.

The assessment exercises (modules) were designed to be completed by students in 40 to 50 minute periods, and included the following:

> Stimulus materials for the assessment module. This might be a cartoon, a paragraph, primary source material, a chart, or any other material to engage the student in the activity and content to be assessed.

> Three or four multiple-choice questions related to the stimulus material or prior knowledge.

> A short-answer question.

> A question requiring an extended response.

> A performance task calling for the student to work on a project involving the application of knowledge and skills to the discipline and theme involved over a two to four week period.

The training, writing, and feedback experience provided by the ACT personnel was an eye-opener for most of the teachers. Although assessment has recently become a hot issue in education, little instruction in assessment techniques is provided by most teacher training programs, and the amount of quality professional development in this field is still minimal. The assessment development process forced the teachers to focus clearly on curriculum and instruction issues. One of its high points was the use of content specialists to assist the teachers in item development.2 These specialists brought the structure of the state and national standards to bear on the content of the assessments.

An unexpectedly difficult part of the assessment process was writing multiple choice items. On the surface, multiple choice writing may appear to be a simple task. If you are writing factual questions, such as identifying the name of a river shown on a map, the question is simple. However, using that river as a basis to broaden student understanding or application of a geographic concept such as place or region, makes the task far more challenging. One Rhode Island teacher said after her assessment was completed, “I’ll never look at multiple-choice questions again without thoroughly analyzing them.”

The development of rubrics for the short answer and extended response items was another challenging task for teachers. Rubrics are used as scoring guides to help analyze and rate student responses. While questions must obviously be clear and unambiguous, the scoring of a student’s open-ended response must also be based on a clear set of criteria that can be consistently applied by the scorers. You may think you are asking a good question, but if you cannot define what a good, fair, or poor response is, you may not have a good question. The writers were asked early on to try out the rubrics in order to gauge the competence of their questions, and to change or refine them as needed.

Final changes to the teachers’ drafts were made after the second workshop, and the completed work was returned to ACT to be readied for field testing in the 23 states. After that testing is completed late in 1999, the units will be returned and their results compiled. ACT will then complete the final preparation of the items for use by the participating states.

The portfolio component of the CSSAP project involved teachers in designing models for scoring examples of student work performed and collected over time. Models will be developed for elementary and secondary schools in the four disciplines. Additionally, a Student and Teacher Portfolio Handbook and a Portfolio Implementation Guide including samples of student work will be published in 2001.

Finally, the professional development committee is charged with overseeing the delivery of the project materials to the states and local school systems and teachers. Vehicles for the delivery system will include a CD-ROM and the development of an interactive website. The CD-ROM and website will contain manuals for using the items, model portfolios for teachers and students, and professional development/staff development delivery models. It will also include the content standards from each of the disciplines, the NCSS standards, and “hot links” to other useful websites.

 

Summary

Testing in social studies has been around for many decades and has taken many forms. Some states are currently contracting with testing companies to develop items based on their standards. But many assessment experts believe that classroom teachers are the best source for authentic assessment ideas. We have found that teachers who deal with their state standards and the real world of students everyday bring a high level of authenticity and professionalism to the process.

The investment made by the teachers involved in this project has provided them with many dividends in the form of insights about the connected nature of curriculum, instruction, and assessment. Their experience will shape, form, and mediate all their future experiences with assessment specifically and with teaching in general. As the potter shapes the clay, the clay shapes the potter.

This article has described a project that is very much in process, with a two-year timetable remaining to be completed. Examples of assessment modules developed for the various social studies disciplines and for different grade levels are included on pp. 362-63. Most assessment modules are considered secure to assure that participating states and local school districts can use these items in their large-scale assessment of students. All of the materials created by this project will be available to all states for use after 2001. Stay tuned for word about our progress!

 

Notes

1. The five states in the original SCASS Social Studies Project were Colorado, Delaware, Missouri, Oregon, and Pennsylvania.

2. These content specialists included Margaret Branson and Elaine Craig-Segal in Civics; Bonnie Maszeros, Mary Suter, Sarah Page McCorkle and Peter Moore in Economics; Joe Stoltman and Joe Manzo in Geography; Liping Bu and Loretta Lobes in History; and Pat Nickell representing NCSS.

 

Fred Czarra is the International Educational Consultant and Specialist in Social Studies and Interdisciplinary Learning at the Council of Chief State School Officers in Washington, DC. For more information about the CSSAP project and how your state can get involved, call either Jim Friedebach, the grant director, at (573) 751-3545, or Fred Czarra, the project coordinator, at (301) 863-2124.

Consensus Framework: Themes For Assessment

 

Civics

1. Civic life, politics, and governance systems.

2. Purpose, structure and functions, of governments in the U.S.

3. Principles and ideals, of democracy in the US

4. Roles, rights, privileges, and responsibilities of citizens in the U.S.

5. Relationships among governments and people that cross national boundaries.

 

Economics

1. Limited resources and choice.

2. Markets.

3. Economic Systems.

4. Economic Interdependence.

 

Geography

1. Places, Regions, Locations.

2. Physical Systems (Spatial Perspective)

3. Human Systems (Spatial Perspective)

4. Environment and Society (Spatial Perspective)

 

History

1. Change and continuity in political systems.

2. Interactions of people, cultures, and ideas.

3. Economic and technological changes.

4. Comparative history of major developments.

Portfolio Assessment

An important focus of CSSAP has been the development of a portfolio assessment system to measure student knowledge and skills in four social studies content disciplines: history, geography, civics, and economics. The design provides a framework for the assessment of outcomes related to specific standards at elementary, middle, and high school levels, and is intended to be rigorous enough to meet the demands of statewide or district-level assessment, while functioning as an effective assessment tool for the classroom teacher.

The structure of the CSSAP Portfolio is built around five entry categories. These categories encompass important instructional approaches widely employed in social studies classrooms, and provide students with varied opportunities to demonstrate their knowledge and skills in the subject. The entry categories are:

1. Interpretation: Students will interpret aspects or parts of a subject, issue, event, experience or idea pertinent to one or more social studies disciplines.

2. Issue Analysis: Students will examine and analyze an issue pertinent to one or more social studies disciplines.

3. Problem Solving: Students will solve a problem pertinent to one or more social studies disciplines.

4. Reasoned Persuasion: Students will take a position on an issue or subject and effectively defend it.

5. Research/Investigation: Students will research and/or investigate a subject, issue, event, or experience relevant to one or more social studies disciplines. Students will conduct research using relevant and appropriate resources, cite those sources, and effectively present the research.

A complete CSSAP Portfolio will include one student product in each of the five entry categories. Using a template designed for the purpose, students will include a reflective summary with each entry. The reflective summaries are designed to facilitate student thinking relative to the four scoring features embodied in the analytic rubrics:

> Content Evidence and Support

> Conceptual Understanding

> Skills and Processes

> Communication and Presentation.

Finally, students are required to write an overall portfolio reflection in which they analyze their work from multiple perspectives, thinking critically about the portfolio as a whole, and expressing what it demonstrates about their development as a learner in social studies. As currently planned, each portfolio entry will be assessed using a four-point analytic rubric; in addition, a portfolio score will be determined through the use of a four-point holistic rubric.

The CSSAP Portfolio is currently undergoing pilot implementation in 23 classrooms. The outcome of this pilot will inform additional development and refinement of the model, and will be followed by a second pilot in 2001-2002. The model is scheduled for release and availability to social studies teachers in CSSAP member states in September 2002. At that time, individual states participating in CSSAP will determine the appropriate level and purpose for use of the CSSAP Portfolio.

©1999 National Council for the Social Studies. All rights reserved.