Standard
2-C
Use of Data for Program Improvement
Target: The unit has fully developed evaluations and continuously searches for stronger relationships in the evaluations, revising both the underlying data systems and analytic techniques as necessary. The unit not only makes changes when evaluations indicate, but also systematically studies the effects of any changes to assure that the intended program strengthening occurs and that there are no adverse consequences. Candidates and faculty review data on their performance regularly and develop plans for improvement.
Development of evaluations. The EMU Professional Educator Assessment System has been evolving for many years, although at an accelerating rate in recent years. For example, admission and retention requirements (to the university, to the program) and Student Teaching evaluation forms have been in place, modified from time to time, for many years. The Michigan Tests for Teacher Certification have been in place, with modifications, for approximately a decade. Educational Benchmarking Inc. survey data have been gathered for four years. The post-completion self-assessments by new professionals and the post-completion assessment by the supervisor of the new professional now include two years worth of data. See the Exhibits for Standard 2.
Most of the mid-program assessments of students have been in development for around four years. See Exhibit 2.10. They have been pilot-tested and revised. In general, the first data collection from the "final" versions of these was done during the 2002-2003 year. See the results posted in the Exhibits for Standard 2. Clearly, there is more to be done: samples need to be extended to populations, exceptions/outliers in the initial program need to be incorporated into the mainstream assessment activities, advanced programs need to be brought into greater parallelism with the initial preparation program, and the like. These activities are stated or implied in the EMU Professional Education Assessment System Implementation Plan. See Exhibit 2.1.B.
Changes Made. Numerous changes in recent years have been made as a result of assessment information. The most significant of these are changes that have been made in the academic majors and minors for the initial teacher preparation program as a result of feedback from the results of the Michigan Tests for Teacher Certification. A comparison of the academic requirements as found in the 1997 Professional Education Supplement to the Catalog with those of the 2003 Professional Education Supplement to the Undergraduate Catalog will reveal many changes made to bring curricula into greater alignment with State Standards. The need for these changes has been revealed because of relatively poor test performance in certain fields. These changes are reflected, in the order in which they occurred, in the minutes of the Basic Programs Committee, the minutes of the College of Education Council, and, in a few instances, the minutes of the Advanced Programs Committee. Numerous other changes are expected to be made as the reporting called for in the EMU Professional Educator Assessment System Implementation Plan ( Exhibit 2.1.B) is accomplished.
Review of Performance. As the College of Education Data Information System is implemented in 2003-04 and thereafter (See Standard 2.B), several opportunities for review of performance will exist that have existed only in very limited form to date.
(A) Review of performance of an individual. At any given time, advisers will be able to review a number of performance indicators for any given student. Based on this information, advisers might undertake to counsel a candidate out of the program or into another teaching field for which the candidate is better suited, might suggest remediation in areas of individual weakness, or take other action. Longer-range, if a first-year professional is not performing at a high level (either by self-assessment or assessment of the immediate supervisor), this can trigger a review of that former student's professional preparation record with an eye toward "what should we have seen in this case that we need to be alert to in future cases."
(B) Review of performance of groups of individuals similarly situated. Many times, the circulation of assessment data (e.g., results from the EBI survey) is met with comments to the effect that "these data may be true for our candidates in general, but there is evidence to suggest that the data are not very applicable to students in _______ (teaching field)." A more comprehensive Data Information System will permit the disaggregation of assessment data, in many instances, by teaching field, by gender, by ethnicity, by student status (e.g., undergraduate vs. post-baccalaureate), and the like. This will permit a greater identification of program strengths and weaknesses and target areas for improvement.
(C) Review of program components. The evolving Data Information System will permit the identification of program components that are and are not working sufficiently well. For example, if the various measures of, say, initial program outcome/benchmark IV-N "Larger Community" are not at desired levels, then program changes can and will be made.