Target: The unit, with the involvement of its professional community, is implementing an assessment system that reflects the conceptual framework(s) and incorporates candidate proficiencies outlined in professional and state standards. The unit continuously examines the validity and utility of the data produced through assessments and makes modifications to keep abreast of changes in assessment technology and in professional standards. Decisions about candidate performance are based on multiple assessments made at multiple points before program completion. Data show the strong relationship of performance assessments to candidate success. The unit conducts thorough studies to establish fairness, accuracy, and consistency of its performance assessment procedures. It also makes changes in its practices consistent with the results of these studies.
Introduction. Many elements of the assessment system for professional education candidates at EMU--particularly those associated with entrance, retention, and exit requirements--have been in place for years and even decades. Others--particularly those associated with mid-program assessments of candidates and post-completion assessments--have been added in more recent times. In the development of the mid-program assessments, the Teacher Education Advisory Board has been substantially involved in examining and providing feedback on draft statements of outcomes and on assessments as well as in identifying important dispositions.
For both the initial preparation program and the advanced programs, there are six major points where assessment occurs: (1) admission to the university, (2) admission to the program, (3) retention in the program, (4) mid-program, (5) near-exit, and (6) post-completion. The assessment system includes both assessment of individuals and assessment of the program.
Assessment System related to conceptual frameworks and national/state proficiencies. Exhibit 2.1.A demonstrates how the components of the assessment system are aligned with the components of the conceptual framework, particularly the outcomes/benchmarks of the initial teacher preparation program and the common goals of the advanced programs. Inasmuch as the outcomes/benchmarks (for the initial program) are well-aligned with INTASC and state Entry-Level Standards, as shown in Exhibit CF-26, the assessment system is aligned with national, state, and local standards. Similarly, inasmuch as the common goals for the advanced programs for teachers are aligned with NBPTS Standards (see Exhibit CF-25), and there are few state standards for advanced programs, the assessment system can be considered to be aligned with all relevant national, state, and local standards.
Exhibit 2.1.B provides detail about the implementation of the EMU assessment system. This table has been derived by grouping the Outcome/Benchmark items (for the initial program) and the common goals (for the advanced programs), as given in EMU Professional Educator Assessment System (Exhibit 2.1.A), then listing the assessment devices for that goal, then providing information about the status of data collection and use.
Validity and utility of the data; modifications made (technology, changes in standards). The ultimate validity check is, of course, learning by P-12 students taught (or provided other professional services) by those who have completed our program. To date, we have only fragments of that information. Quilter's work ("A Study of the Relationship Between Teaching and Context Variables in Student Outcomes," Renaissance Group, 2001) showed that "taking teaching experience, and level of educational attainment (BA, MA, etc.) into account, having credentials from EMU has a positive impact on student achievement [in the district studied]. . . ." In particular, "[H]igher scores on the MEAP [state 4th grade standardized achievement test] mathematics test were associated with students who had EMU-affiliated teachers in 3rd grade. . . . Higher scores on the MEAP story telling test were associated with students who had EMU-affiliated teachers in 3rd grade. . . . Finally, higher scores on the MEAP informational reading test were associated with students who had EMU-affiliated teachers in 3rd grade. . . ." (See Exhibit 2.9). There are indicators of student learning from the units that our student teachers teach (Exhibit 2.3.E) and from the work of our students who have participated in the Teacher Quality Program (Exhibit G-22).
At the present time, the best single available indicator of the validity of our assessments consists of the ratings provided by the supervisors of our recent program completers. Exhibit 2.3.H provides data from the supervisors of our recent program completers. The alignment of each of the items rated with items assessed during the preparation program is provided in Exhibit 2.1.B, such that the reader can see that, in general, the "on the job" ratings of various characteristics of our professionals are similar to the "during the program" assessments. Exhibit 2.1.B also includes timetables for reports that will eventually establish a statistical relationship between measures of "on-the-job" performance and "during the program" assessments.
The data generated through the assessment system have long had great utility for decision-making related to admission, retention, and exit. However, insufficient data have been gathered from "during the program" assessments to justify major program changes, although numerous minor changes have occurred. A major exception to the previous statement is the number and nature of program changes that have been made as a result of analysis of Michigan Tests of Teacher Certification data. See the minutes of the Basic Programs Committee, the minutes of the Advanced Programs Committee, and the minutes of the College of Education Council for examples of these curricular changes affecting the "content" portions of the program.
Traditionally, assessment data have been maintained in paper files or in the computers of a variety of faculty and staff members. However, during 2002-2003, under the leadership of COE Associate Dean Jim Berry and the Data and Assessment Advisory Committee, steps were taken to create a web-based "warehousing" of educator preparation program assessment and other student-related data. (See Exhibit 2.6). This effort is being undertaken with the support and assistance of the EMU Office of Institutional Research and Information Management. As of Fall 2003, there is a full-time staff member (Joan Quinlan) serving within the College of Education as the coordinator of data management. This person reports to the Associate Dean. Central databases are under construction.
The changes in standards with which we have been most concerned in recent times are content standards of SPA's and selected subject-area standards at the state level. See the minutes of the Basic Programs Committee, the minutes of the Advanced Programs Committee, and the minutes of the College of Education Council for examples of these curricular changes affecting the "content" portions of the program.
Multiple assessments, multiple points. As may be seen in Exhibit 2.1.A, and as indicated above, there are six major points where assessment occurs. Several of the major assessment points include both candidate assessment and program assessment, e.g., mid-program assessments and near-exit assessments. Each of the six assessment points includes multiple items. The total assessment system includes a wide variety of assessment modes, such as tests, papers, projects, and observations.
Relationship of assessments to candidate success. As mentioned above, the current best measure of "success" of our program completers is the proxy for P-12 student achievement that is reflected in assessments by the supervisor of the first-year practitioner. Exhibit 2.3.H contains the ratings of the supervisors of first-year practitioners. These ratings can be compared with the results of assessments on similar topics. Exhibit 2.1.B shows the alignment among various matters that are being assessed.
Studies of validity, reliability, fairness, etc. of performance assessments. In general, formal studies of the validity, reliability, fairness, etc. of the various performance assessments will be undertaken according to the timetable given in Exhibit 2.1.B. However, in the case of all locally-developed instruments, especially those of the mid-program assessments, there have been lengthy and detailed processes of development (including to assure face validity), pilot testing, and revision before putting into full use. Our participation in the Teacher Quality (TQ) consortium of The Renaissance Group (see Exhibit G-22) has provided support to develop and enhance our mid-program assessments. See Exhibit 2.10 for a brief history of the development of our instrumentation for mid-program assessments. In the case of standardized instruments such as the Michigan Tests for Teacher Certification, the validity, reliability, fairness, etc. have been assured by a national professional testing company (National Evaluation Systems for the MTTC).
Changes made in practice. As the mid-program assessments have been implemented, there have been modifications in courses, especially in the courses where key assessments occur. Extensive discussion has occured among the instructors. Additional information has been provided to students in terms of expectations of the program (see, for example, Exhibit G-29, the CPED2S book). Extensive discussion has occurred between the faculty of the Department of Teacher Education and the student teaching supervisors, such as to incorporate elements of the assessment system into the student teaching experience.