Instructor: Kirk Fiereck
Observer: Matthew Sanger

Class: Qualitative Data Analysis

Instructor's learning goal and vision
Students often miscalculate the influence their preconceived notions and predilections can have on the gathering, analysis, and interpretation of data, particularly when that data is qualitative in nature. The learning goal of this class session is to highlight the potential dangers of personal biases. These biases will be highlighted through both a class lecture as well an in-class activity (described in detail below). The class lecture will revolve around 1) a set of readings that consider the ways in which the students’ subjectivities and positions influence the kinds of data they collect and the way they interpret this data; and 2) the instructor’s own research and how their predilections were affecting the way he was observing in the field. Students will then split up into pairs or groups of three for about 20-25 minutes so that they can then discuss the types of personal biases that might be affecting how they are conducting their data analyses that they are all working on as part of a course-long project. After this, one person from three groups will report back to class so that students can hear about others’ experiences of accounting for their bias/subjectivity and what implications this will have for their research. This will allow students to use the instructor's and other students’ experiences as a model for thinking about and discussing their own biases in their assignments that they will complete after this class session. Many of the students in this class are not accustomed to including a self-reflexive account of their personal biases in their research and analyses given that the overarching pedagogical approach in the school of public health tends towards positivist research approaches and statistical reasoning.
The goal of the lecture is to show that subjective biases are potential forces even in the work of much more experienced practitioners and require personal reflection and self-critique. The in-class activity will further these goals by having students reflect on their own biases. By working in pairs/groups after the lecture component, this will also give each student a chance to discuss their biases while also receiving feedback from other students in the course before they work on their assignment based on this learning goal. As part of their grade for the class, students are required to write a 1-2 page "subjectivity memo" where they reflect in narrative form about the biases they have that may be affecting the way in which they interpret and analyze the data they are working on as part of the course-long project.

Observer's understanding of what is being tracked and how progress will be measured
The goal of this session is to advance students’ appreciation for the efficaciousness of their preconceived biases. Observation will focus on measuring how, and to what degree, students become more self-reflexive and are able to identify potential biases in their own work. While the overall effect of the session will be considered, the observation will also attempt to determine to what extent the two different learning points (the lecture and the in-class activity) were individually responsible for advancing toward increased self-critique so that particular suggestions for improvement could be made.
Prior to the session, the entire class (36 students) will fill out a questionnaire designed to determine their level of appreciation for their own biases and the potential danger of those preconceived notions. This questionnaire will be distributed and collected two weeks prior to the session being observed.
During the session, the lecture and in-class activity will be observed. Notes on class reaction to each will be made as well as general effectiveness of delivery.
After the session is complete, students will be given questionnaire in order to track any changes in their self-reflexivity and appreciation for personal biases. Additionally, a sub-sample of five students will be briefly interviewed in order to get a more detailed reaction to the presentation and in-class activity.
The observation will complete by giving this data to the students. After this lecture and discussion session students have been assigned to write a 1-2 page "subjectivity memo" putting what they have learned into a narrative form. This will offer the students another opportunity to engage with the concepts brought out in class as well as utilize the skills they are learning to analyze qualitative data.

PRE-CLASS QUESTIONNAIRE

The following is the questionnaire that will be given to students before class so that a benchmark determination can be made about their understanding of their subjectivity and biases and how this would affect their data analyses and formation of their research questions.

1) During this course, there may have been ways in which your subjectivity might potentially have influenced the kind of research questions you were asking and the way you interpreted your data. If you have experienced this, what kinds of personal biases did you notice?

2) Please describe when this happened. If a class exercise, discussion or assignment helped you to account for these biases, which activity specifically helped you with this?

3) There are many kinds of biases or ways that your subjectivity can influence your research. What kinds of biases can you think of that might be influencing your research?

4) There are good and bad things to do with being self-reflexive about your approach to research (e.g., accounting for your biases). How would your conceptual framework change if you account for the biases you described in #1 in this conceptual model?


IN-CLASS GROUP ACTIVITY

This in-class activity will be used so that students will have a chance to discuss their subjectivities and potential biases that relate to their own research projects. The activity will give them a chance to use what they learned during the lecture and model the lecturer’s personal fieldwork example in identifying and addressing the biases in their own work. The class will split up into the groups of two or three that they have been working in throughout the semester. The lecturer will preface the activity, describing what the students should discuss during the group activity. He will put on the overhead projector the following discussion topics that each person in the group should have a chance to discuss with the others:

1) Brainstorm the main issues, problems, or impasses that you have noticed in your data/research. In other words, think about any glaring paradoxes or contradictions that have arisen between how you think about your research topic/data/concepts and how people in your interviews and focus groups discuss these topics/concepts. Discuss how these may be influenced by your subjectivity or biases.

2) Discuss the issues/contradictions/biases related to your own subjectivity that are most important for your research question or your conceptual framework.

3) Discuss how accounting for these aspects of your subjectivity may affect how you approach your research question and analyses moving forward.

After 20-25 minutes, the class will reassemble. Three groups will be asked to present their initial ideas about their subjectivity and biases; how they are important for their particular research question/analysis; and what implications this will have for their research going forward. Each group will choose one person from their group to report back to the class on their subjectivity/bias as it relates to their research.


POST-CLASS QUESTIONNAIRE

The following is the questionnaire that will be given to students after the class session so that a follow-up determination can be made about the effect the in-class lecture and activity had on their understanding of their subjectivity and biases and in what ways this would affect their data analyses and formation of their research questions.

1) During this class session, we discussed a number of ways in which an individual’s subjectivity might potentially have influenced the kind of research questions they were asking and the way they interpreted their data. Before, you were asked what kinds of personal biases you were aware of in your research. After this session, are there any other kinds of biases, or ways in which you might understand your subjectivity would affect your research?

2) What part of the class session today helped you best to understand how subjective biases affect your research and analysis and why? (i.e., lecture on subjectivity, lecturer’s own experience with subjective bias, group activity, group “report-back” presentations)

3) There are many kinds of biases or ways that your subjectivity can influence your research. What kinds of biases that you did not think of before, did this class session help you to become aware of that might be influencing your research?

4) There are good and bad things to do with being self-reflexive about your approach to research (e.g., accounting for your biases). How would your conceptual framework change if you account for the biases you became aware of as a result of this class session (if any)?


POST-CLASS FOCUS GROUP QUESTIONNAIRE (FIVE STUDENTS)

The focus group will allow us to ask a sub-set of students in the class about their thoughts regarding the pedagogical approaches used. We will use this feedback to improve the methods associated with the overarching teaching goals.

1) Which of the in-class activities did you feel was most useful in helping you to account for aspects of your subjectivity that may produce potential biases in your own research? Please explain why. (i.e., lecture, lecturer’s field experiences of bias, group activity, group “report-back”)

2) What about any of the in-class activities do you feel could have been improved?

3) What other activities or approaches to addressing the issue of subjectivity and potential biases as a force in research that you think might be have helped you understand these topics?

Timeline:


9/25: Observation Planning workshop



9/30: Request permission for observation from Professor Smith.



10/11: Present observation plan to Teagle Fellows



10/13: Meet with Kirk to revise observation plan after presentation



10/21: Distribute questionnaire to students (in-class)



10/28: Meet with sub-sample of students for group interview (before/after class?)



11/4: Observe Class

5:00 Arrive at Hammer Health Sciences Building, Room LL 203, introduced to class, lecture begins

6:00 lecture ends, students begin in-class activity

6:45 activity ends, students given questionnaire and 15 minutes to complete it

7:00 students give Kirk their questionnaires as they leave class, he gives them to me after session is over



11/9: I compile and record questionnaire results as well as observations which I then share with Kirk through the project page



11/11: Sub-sample of students interviewed (before/after class?)



11/17: All data complied, meet with Kirk and debrief.



11/18: Students given data as part of an in-class activity analyzing qualitative data.



11/21: Final debrief with Kirk, observation report written



Early December: Reflections about the observation offered to Teagle Fellows.