Target:
The unit, with the involvement of its professional community,
is implementing an assessment system that reflects the
conceptual framework(s) and incorporates
candidate proficiencies outlined in professional and
state standards. The
unit continuously examines the validity and utility of
the data produced through assessments and makes modifications
to keep abreast of changes
in assessment technology and in professional standards. Decisions about
candidate performance are based on multiple assessments
made at multiple points before program completion. Data show the strong
relationship of performance assessments to candidate
success. The unit
conducts thorough studies to establish fairness, accuracy,
and consistency of its performance
assessment procedures. It also makes changes in its practices consistent
with the results of these studies.
Introduction. Many
elements of the assessment system for professional education
candidates at EMU--particularly those associated
with entrance, retention, and exit requirements--have
been in place for years and even decades. Others--particularly
those associated with mid-program assessments of candidates
and post-completion assessments--have been added in more
recent times. In
the development of the mid-program assessments, the Teacher
Education Advisory Board has
been substantially involved in examining and providing
feedback on draft statements of outcomes and on assessments
as well as in identifying important
dispositions.
For
both the initial preparation program and the advanced programs,
there are six major
points where assessment occurs: (1) admission to the
university, (2) admission to the program, (3) retention
in the program, (4) mid-program, (5) near-exit,
and (6) post-completion. The assessment system
includes both assessment of individuals and assessment
of the program.
Assessment
System related to conceptual frameworks and national/state
proficiencies. Exhibit
2.1.A demonstrates how the components of the assessment system
are aligned with the components of the conceptual
framework, particularly the outcomes/benchmarks
of the initial teacher preparation program and
the common goals of the advanced programs. Inasmuch as the outcomes/benchmarks
(for the initial program) are well-aligned with
INTASC and state Entry-Level Standards, as
shown in Exhibit
CF-26, the assessment system is
aligned with national, state, and local standards. Similarly, inasmuch
as the common goals for the advanced programs
for teachers are aligned with NBPTS Standards
(see Exhibit
CF-25), and there are few state
standards for advanced programs, the assessment
system can be considered to be aligned with all
relevant national, state, and local standards.
Exhibit
2.1.B provides
detail about the implementation of the EMU assessment
system. This table
has been derived by grouping the Outcome/Benchmark items
(for the initial program) and the common
goals (for the advanced programs), as
given in EMU Professional Educator
Assessment System (Exhibit
2.1.A), then
listing the assessment devices for that goal, then providing
information about the status of data collection and use.
Validity
and utility of the data; modifications made (technology,
changes in standards). The
ultimate validity check is, of course, learning
by P-12 students taught (or provided other professional
services) by those who have completed
our program. To date, we have only fragments of that information. Quilter's
work ("A Study of the Relationship Between
Teaching and Context Variables in Student Outcomes," Renaissance
Group, 2001) showed that "taking teaching experience, and
level of educational attainment (BA, MA, etc.)
into account, having credentials from EMU has
a positive impact on student achievement [in
the district studied]. . . ." In particular, "[H]igher scores on the MEAP [state
4th grade standardized achievement test] mathematics
test were associated with students who had EMU-affiliated
teachers in 3rd grade.
. . . Higher scores on the MEAP story telling
test were associated with students who had EMU-affiliated
teachers in 3rd grade. . .
. Finally, higher scores on the MEAP informational reading test were
associated with students who had EMU-affiliated
teachers in 3rd grade. . . ." (See Exhibit
2.9). There
are indicators of student learning from the units
that our student teachers teach (Exhibit
2.3.E) and
from the work of our students who have participated
in the Teacher Quality Program (Exhibit
G-22).
At
the present time, the best single available indicator
of the validity of our assessments
consists of the ratings provided by the supervisors of
our recent program completers. Exhibit 2.3.H provides
data from the supervisors of our recent program completers. The alignment
of each of the items rated with items assessed during
the preparation program is provided in Exhibit
2.1.B, such that the reader can see that, in general, the "on the job" ratings
of various characteristics of our professionals are similar
to the "during
the program" assessments. Exhibit
2.1.B also includes timetables for reports that will eventually
establish a statistical relationship between measures
of "on-the-job" performance
and "during
the program" assessments.
The
data generated through the assessment system have long
had great utility for decision-making related to admission,
retention, and exit. However,
insufficient data have been gathered from "during the program" assessments
to justify major program changes, although numerous minor
changes have occurred. A major exception to the previous statement is
the number and nature of program changes that have been
made as a result of analysis of
Michigan Tests of Teacher Certification data. See the minutes of
the Basic Programs Committee, the minutes of
the Advanced Programs Committee, and the minutes of
the College of Education Council for examples of these
curricular changes affecting the "content" portions
of the program.
Traditionally,
assessment data have been maintained in paper files or
in the computers of a variety of faculty and staff
members. However, during
2002-2003, under the leadership of COE Associate Dean
Jim Berry and the Data and Assessment
Advisory Committee, steps were taken to create a web-based "warehousing" of educator preparation program
assessment and other student-related data. (See Exhibit
2.6). This
effort is being undertaken with the support and assistance
of the EMU Office of Institutional Research and Information
Management. As of Fall 2003, there is a
full-time staff member (Joan Quinlan) serving within
the College of
Education as the coordinator of data management. This person reports
to the Associate Dean. Central databases are under construction.
The
changes in standards with which we have been most concerned
in recent times are content standards of SPA's
and selected subject-area standards at the state level. See the minutes of
the Basic Programs Committee, the minutes of
the Advanced Programs Committee, and the minutes of
the College of Education Council for examples of these
curricular changes affecting the "content" portions
of the program.
Multiple
assessments, multiple points. As may be seen in Exhibit
2.1.A, and as indicated above, there are six major points where
assessment occurs. Several of the major assessment points include
both candidate assessment and program
assessment, e.g., mid-program
assessments and near-exit assessments. Each of the six assessment
points includes multiple items. The total assessment system includes
a wide variety of assessment modes, such
as such
as tests, papers, projects, and observations.
Relationship
of assessments to candidate success. As mentioned above,
the current best measure of "success" of
our program completers is the proxy for
P-12 student achievement that is reflected
in assessments by the supervisor of
the first-year practitioner. Exhibit
2.3.H contains the ratings of the supervisors of first-year
practitioners. These
ratings can be compared with the results
of assessments on similar topics. Exhibit
2.1.B shows the alignment among various matters
that are being assessed.
Studies
of validity, reliability, fairness, etc.
of performance assessments. In
general, formal studies of the validity,
reliability, fairness, etc. of the various
performance assessments will be undertaken
according to the timetable given in Exhibit
2.1.B. However, in the case
of all locally-developed instruments,
especially those of the mid-program assessments,
there have been lengthy and detailed
processes of development (including to assure face
validity), pilot testing, and revision
before putting into full use. Our participation
in the Teacher Quality (TQ) consortium
of The Renaissance Group (see Exhibit
G-22) has provided support to develop and enhance our mid-program
assessments. See Exhibit 2.10 for
a brief history of the development of
our instrumentation for mid-program assessments. In the case of
standardized instruments such as the
Michigan Tests for Teacher Certification,
the validity, reliability, fairness,
etc. have been assured by a national professional testing company (National
Evaluation Systems for the MTTC).
Changes
made in practice. As the mid-program assessments have been
implemented, there have been modifications in
courses, especially in the courses where
key assessments occur. Extensive discussion has
occured among
the instructors. Additional information has
been provided to students in terms of expectations
of the program (see, for example, Exhibit
G-29, the CPED2S book). Extensive discussion has
occurred between the faculty of the Department
of Teacher Education and the student
teaching supervisors, such as to incorporate
elements of the assessment system into the student
teaching experience.
Previous
Page
