Background
Kat is a PhD student in the school of Social Work. This semester she is teaching two sections of Social Work Research Methods II, a required course for MSW students at NYU. The overall goal of Research II is to provide students with an understanding of how research is used to evaluate social work practice and programs. The course is divided into two 7-week units, focusing on practice evaluation and program evaluation respectively. While teaching the course last fall, Kat noticed that students struggle to integrate component skills and knowledge to correctly apply what they have learned. This semester, Kat is seeking to incorporate more student-centered teaching methods into in-class activities and out-of-class assignments to support student mastery of course material.

The section that Sofia will be observing is composed of 24 students, and meets on Tuesdays from 11:00am-12:50pm. Sofia will observe a class session that falls near the beginning of the second unit. The session covers two main topics: formative evaluation and process evaluation.

GOALS

Instructor's learning goals for the evaluated class session or assignment:
  • Learning goals for observed class session:
    • Students acquire component skills related to evaluation:
      • Students understand the differences between formative evaluations and process evaluations (purpose and goals of each type of evaluation)
      • Students know the basic approaches and/or steps involved in each method
      • Student develop an accurate conceptual understanding of standard deviations
    • Students move toward mastery of the by material through aappropriately applying the above component skills to real-world case studies and published research articles


At least one way that this learning goal is helping the instructor define his or her teaching:
  • Kat is experimenting with activities aimed at developing mastery into her class session and assignments, as a way to encourage students to apply material from the lecture and readings in real-world case studies.
  • Kat is committed to developing a classroom environment where students feel comfortable and safe to ask questions about the material and take risks as they practice moving from theory to application.

Instructor's vision of how students will step towards that goal:
  • In-class activities: Students need to actively interact with material to develop a working knowledge of concepts and practice integrating theory and application that connects skills and application – ways that Kat hopes to encourage this is through promoting in-class discussion and using small group case study activities.
    • Kat’s teaching concerns:
      • Creating a safe environment where students are comfortable is necessary to students engaging in trying new skills. Kat wonders if the ways that she asks questions (e.g., open/closed) and responds to students (e.g., praise/criticism) during class supports student engagement?
      • Kat struggles with transitioning into the application-focused activities and is nervous about how well she explains assignments. She would like to test whether modeling the activity first helps students to better understand the activity, and/or to develop better understand of the process of applying the material.
      • Readings: Students are assigned two types of readings each week: a methodology chapter from the textbook, and 1-2 research articles that provide real-world examples of the methodology in action. During the first half of the semester, students participated in small groups to analyze and critique the research articles. In the groups, students rotate through a series of roles requiring them to practice applying knowledge gained about evaluation research to deciphering the material. However, a midterm survey showed that this assignment wasn’t effective in leading students to a deeper understanding of the concepts. The workload proved too overwhelming for students to complete thoroughly. In this half of the semester, Kat has assigned a smaller number of articles and instead asked all students to keep in mind the features each reading group role encouraged them to seek out (key terms, strengths, weaknesses, and questions).
        • Kat is curious how this change will affect student comprehension of concepts and data.


Observer's understanding of what exactly she is tracking and how the tracked session or assignment will be measured:
  • In-class observation:
    • Sofia will use an observation checklist created to track how Kat is asking questions (e.g., yes/no,open-ended, essential, leading) and students responses (e.g., factual, opinion, brief answer, conversation, silence), and the length of time Kat waits for a response before stepping in. The checklist will be designed to enable Sofia to capture both counts of the categories and how the pairings match up across categories. (See attached file.)

  • Because the class is covering two topics (formative & process evaluation), Kat will experiment with her teaching approach during the observed session.
    • Readings: there is one assigned research article for the session that is an example of a process evaluation study. During the lecture on process evaluation Sofia will note whether students actively bring up the article and associated terms and concepts (provided by Kat beforehand) for discussion. This will provide insight as to whether students are beginning to link methodology to application using the added reading. Sofia will also ask about student’s preparation of the article during the focus group.
    • Application activities: Kat will run two application activities during the session. During the first activity, she will proceed as usual; during the second activity, she will model the activity before having the students proceed. Sofia will note how long it takes for students to settle into the activity and whether there seems to be confusion. During the focus group, Sofia will also check in with students about which activity they found more helpful and whether the instructions were clear. Both activities involve students completing questions on a written template. The template will be handed in and Kat will rate and compare the quality of student responses in each instance.


  • Sofia will also conduct a 30-minute anonymous focus group directly following the observed session, delving into students' process in preparing for class (e.g., in what order do they complete readings, do they try to apply information from the methods to the research article), and the effectiveness of the application activities.

TIMELINE:
  • 9/29: Observation Planning workshop
  • 10/1: Meet with Kat to write observation plan
  • 10/6: Present observation plan to Teagle Fellows
  • 10/14: We meet to revise observation plan, including survey questions
  • 10/21: Kat will give her students a survey to get midterm feedback and to assess whether the Reading Groups assignment is achieving learning goals
  • 11/1: Kat will send a draft of focus group questions for me to revise
  • 11/4: Kat will alert students of upcoming observation, introduce the focus group opportunity and call for volunteers (with some kind of snack-based incentive)
  • 11/8: Finalized focus group protocol decided on
  • 11/11: Observe Kat’s classroom
    • o 10:30AM – Sofia arrives at NYU
    • o 11AM – Class begins (Kat presents a lecture about the method used by the article (process evaluations), the group discusses the research article that employs this methodology)
    • o 12:50PM – Class ends, Kat invites focus group participants to stay and leaves the room
    • o 1PM-1:45PM – I run the focus group with willing students

  • 11/14: Kat and I meet to do a general debrief
  • 11/24: I send Kat a write-up of survey and focus group responses, as well as my general observations, to be revised in the first week of December
  • 12/8: Present our reflections about the observation to Teagle Fellows

Attached files:


Midterm Evaluation.docx
Midterm Evaluation.docx

Midterm Evaluation.docx

Focus Group Guide rev 11-6-14.docx
Focus Group Guide rev 11-6-14.docx

Focus Group Guide rev 11-6-14.docx

In-Class Observation Worksheet.docx
In-Class Observation Worksheet.docx

In-Class Observation Worksheet.docx