On Tuesday, August 7th, Carrie Fisher, PhD, Research and Evaluation Manager at the Institute for Community Health, presented to over 20 people on qualitative data analysis at the Jewish Family and Children’s Services in Waltham. Ms. Fisher is an anthropologist with interests in applied and evaluation research, innovative research methods, health and public health, and research with difficult-to-reach populations.
Ms. Fisher provided a comprehensive overview of qualitative data analysis that began with a very interesting discussion about how concepts and meanings of “truth” and “reality” can vary in qualitative data analysis. The presentation then focused on qualitative data analysis choices and methods, data management, data analysis, and key issues around data interpretation. Below are a few key summary points from the discussion.
Reality and Truth in Qualitative Data Analysis
There is not just one truth when dealing with qualitative statements and there are things that can affect how truth and reality are defined such as interviewer, analytical, and interpretive bias. Even false statements can provide important lessons in the analysis. It’s important to remember that qualitative data analysis is always selective and can be impacted by individual backgrounds, education, and even the mission and vision of the organization sponsoring the qualitative data project.
Qualitative Data Analysis Choices and Methods
Starting a qualitative data analysis project can be challenging as there are many questions that need to be considered such as: quantity and type of data, validity of analysis, how will the findings be used, identifying the audience, staff capacity to conduct the analysis, time and money for the analysis, and whether or not to use technical qualitative data analysis software.
Ultimately, a qualitative data analysis project starts when you decide on a data collection method. Qualitative analysis is an ongoing iterative process and researchers and evaluators should meet weekly with those doing the data collection to review responses to questions to determine if a change to question design is necessary.
Qualitative Data Management
It is critical to plan ahead for data management of qualitative data and to consider small, yet important, factors such as how you are going to record the data (written, audio recordings, video recordings), how you are going to conduct data entry (Excel, Word, other database), and which tools and/or software you may use to manage and analyze the data.
For example, one key data management consideration for in-depth interviews and focus groups is whether or not to use transcription services. Verbatim transcription can be expensive – just one hour of audio or video can require up to four hours of transcription, which can cost hundreds of dollars. The website www.rev.com is a cost-effective alternative for transcription with rates around $1 per minute.
If you are using qualitative data analysis software like Dedoose, NVivo, ATLAS.ti, or others, it’s important to remember that these software only manage data and do not automatically perform analysis for you. Coding and analysis of qualitative data using these software are additional steps which require sufficient time and technical training in the software itself.
Qualitative Data Analysis
It’s critical to choose your analytical approach with the end in mind. Key issues to consider include: level of detail, level of rigor, audience, how the information may or will be used, time and resources for analysis. There are many different approaches to analyzing qualitative data including pragmatic thematic analysis, case studies, content analysis, and coding trees to name a few.
Once your data has been collected and managed, schedule extra time to “swim in the data.” Get to know your data well by reading over all notes and transcripts before doing any coding. Write down preliminary thoughts on main themes, points of interests, and gaps in the data.
Lastly, coding of qualitative data is a critical step. It’s important to organize codes into meaningful categories and to create a descriptive codebook to document definitions and themes in codes. This is particularly important if your project has more than one analyst conducting coding in order to maintain consistency.
Qualitative Data Interpretation
When you are ready to begin interpreting your data findings, begin by listing key points and themes such as: key confirmations, major lessons, new ideas, and applications to other settings and/or programs. Some evaluators choose to summarize qualitative findings using quantitative outcomes (i.e., “9/10 respondents agreed with this idea…”). While this can increase confidence in results, it should be used with caution.
Ultimately, the evaluator should always ask: “Would I feel comfortable with the participant reading this.” “Would the participants agree with my interpretation of the findings?”
Ms. Fisher’s full presentation slides can be found here (members only).
