Program Review 2008 (Appendix F)

ENGL 101 Assessment

Critical Skills in English BA Program

Instructor Grading Practices

Range of Required Reading for Graduate Courses

MA Comprehensive Examination

ENGL 101 Assessment: The English department assessed ENGL 101 to observe the knowledge and skills students exhibit when they complete the course. To measure student progress in the course, two sets of essays were collected, the first formal draft of the first essay and the final draft of the final essay, written approximately eight weeks apart. To score each essay, analytic scoring of primary traits, an assessment measure developed by the National Assessment of Educational Progress, was used. This assessment indicates that students are acquiring the skills necessary for success in ENGL 101 and that the program is meeting its institutional requirements. Students did particularly well in focusing, developing, and organizing their papers, suggesting the success of the program's orientation toward thesis-driven arguments. They did less well—though still above the expected mean—in sentence-level clarity, complexity, and precision. Perhaps the most challenging result was the survey indicating students have the most difficulty in getting started on their papers.

Critical Skills in English BA Program: In 2005-2006, Maria Karafilis (undergraduate adviser), Michelle Hawley (graduate adviser), and Hema Chari (associate chair) were awarded an assessment mini-grant for their project to assess whether students were improving in critical skills throughout the program and whether the BA in English enabled students to meet the outcomes outlined in the department’s mission statement. The department assessed students’ ability to construct and support persuasive arguments in the field of English studies. The results indicated that student performance improved from "developing" in the entry-level ENGL 340 to nearly "strong" in culminating course ENGL 492. The data also suggested students would benefit from additional help in developing original and compelling arguments and articulating their arguments in clear, fluent, and precise prose.In response to these findings, the department sought to enforce the sequencing of these courses through advisement, engage in ongoing discussions on the learning outcomes of ENGL 340, and go forward with modifications to the MA program.

Instructor Grading Practices: All undergraduate course descriptions include course objectives, which are used to assess student progress through the course and ultimately substantially determine the student’s grade. In Fall 2007, the department undertook a review of grading practices to evaluate their consistency. First department grading practices were compared with those of other departments in the university. Grade data for Winter 2007 through Summer 2007 supplied by Institutional Research yielded a university GPA for all courses of 3.07. When these totals are adjusted for ABC/NC courses, which eliminates from average GPA all grades less than C, the university GPA for all courses was 2.74. The unadjusted English department GPA for this same period was 3.07. The adjusted GPA for the English department was 2.86. These values are consistent with university practice. Because of the often wide variance in grades between lower and upper division, undergraduate and graduate courses, the department undertook an analysis of average GPA by course level. The results from Winter 2007 are shown in the table below:

These results from Winter 2007 demonstrate that grading practices in the department are in line with university practice.

Finally, the department considered the grading practices of individual instructors against department averages. During the examination period (Winter 2007-Summer 2007), 37% of all university instructors had GPAs greater than one standard deviation from their department’s GPA, while 39% of English instructors had GPAs greater than one standard deviation from the department’s GPA. This percentage places the department very close to the median. For the same period, 35% of all university instructors had GPAs within 0.5 standard deviation from their department’s GPA, while 39% of English instructors had GPAs less than one-half standard deviation from the department’s GPA. These findings suggest two facts about grading practices in the department: a greater percentage of department faculty are in agreement about grades than suggested by the university average; and a portion of the department, comparable to that found in the university as a whole, is less in agreement.

Range of Required Reading for Graduate Courses: In 2006, John Cleman, then department chair, went through syllabi for variable topic graduate seminars offered since 2002 and compiled a list of readings required. This list indicated both the range of readings required and the variety of courses offered, and was distributed to department faculty prior to the 2007 faculty retreat. By showing what had been taught and what hadn’t been taught in graduate courses, this list helped illustrate the need for new faculty in specific program areas.

MA Comprehensive Exam: Assessment of the graduate program relies on periodic review of graduate student progress by the graduate advisor, and on student performance on the program’s culminating activity. At the beginning of 2007, the department conducted a review of pass rates on the department’s comprehensive examination. The review showed consistency over the review period as well as consistency between the review period and earlier periods.

The MA comprehensive examination is divided into two parts; the first assesses a student’s knowledge of a selected literary historical period; the second assesses a student’s ability to respond effectively to a topic on a single text. From 2002-2006, 71% of first-time examinees and 66% of all examinees passed Part 1 of the comprehensive examination. During the same period, 79% of first-time examinees and 78% of all examinees passed Part 2 of the comprehensive examination. From 1987-2001, 71% of first-time examinees and 67% of all examinees passed Part 1, while 80% of first-time examinees and 80% of all examinees passed Part 2. The differences between the pass rates for 2002-2006 and 1987-2001 are statistically insignificant and suggest a remarkable consensus among faculty concerning both what is being assessed and how to assess it.

Also, in Winter 2006, the department’s principal graduate adviser solicited and collated detailed comments from exam readers. Substantial agreement between readers down to the level of comment suggested departmental consensus on what was being assessed by the examination.

To assess student learning outcomes in the single subject credential option, the English Department will review teaching portfolios that the students develop as they proceed through the program. For these portfolios, students will be asked to provide artifacts, accompanied by a reflective commentary, that demonstrate their competencies in each of the required English Content Domains. The Credential Option Assessment Committee (COAC) will assess the portfolios of credential option students who have applied for graduation or have requested certification of subject-matter competency in English to determine if they have met the learning outcomes for the preparation of teachers. These assessments will be conducted quarterly, and the prospective teachers reviewed will normally be enrolled in the third quarter prior to the quarter in which they plan to graduate. The COAC will report on the results of their assessment in two ways:

For individual prospective teachers, the results will be reported to the Principal Credential Adviser (PCA), who will, when necessary, identify measures to meet program requirements (e.g., revising the portfolio and/or taking additional coursework to meet standards or improve GPA).

For the program generally, the COAC, chaired by the English Education Coordinator (EEC) and in consultation with the PCA, will report annually to the Department Chair on perceived strengths and weaknesses in the program that were revealed through these assessment activities.

The Chair will then forward the COAC report and any recommendations for change to the department and proceed as follows:

a. If the recommendations for change can be addressed by adjustments in the syllabi of individual instructors, the Chair will notify the instructors concerned;

b. If the recommendations require course or program modifications, the Chair will forward the recommendations to the Department Undergraduate Studies Committee (USC).

c. The Undergraduate Studies Committee (USC) will develop the curricular modifications necessary and forward them to the Department and subsequently to the College of Arts and Letters Instructional Affairs Committee for approval.

In addition, the COAC reports, along with other information such as surveys of program graduates and of credential program colleagues in the Charter College of Education, will become part of the Department’s Self Study Report for the University Program Review conducted every five years. Any recommendations for further program changes made by external reviewers will be forwarded by the University Program Review Committee to the Provost and Vice-President for Academic Affairs, who will, in turn, request any necessary changes by the Department.