Week 4: The top ten developments in qualitative evaluation over the last decade – part 1

Michael Quinn Patton
mountain and canyon

Over the next two weeks, Michael Quinn Patton joins us to give his top ten trends in qualitative evaluation over the last decade. This week, he sets out the first five.

Over the next two weeks, Michael Quinn Patton joins us to give his top ten trends in qualitative evaluation over the last decade. This week, he sets out the first five.

A lot has changed in the decade since I wrote the 3rd edition of Qualitative Research and Evaluation MethodsWhile working on the 4th edition, I’ve pulled out ten highlights to sum up the state of qualitative evaluation methods, and some emerging challenges. To help put together this list, I also consulted the three qualitative colleagues listed below (and they in turn with their colleagues and students) about their sense of the major trends - though they bear no responsibility for the final top ten list I’ve constructed.

  • Sharon Rallis, incoming editor of the American Journal of Evaluation and co-author of two qualitative books
  • Leslie Goodyear and Jennifer Jewiss, co-chairs of the AEA Qualitative Topical Interest Group (TIG) and editors of the forthcoming book, Qualitative Inquiry in the Practice of Evaluation.

So here are the first five of my top ten developments in qualitative evaluation inquiry over the last decade

10. Powerful Qualitative Software

Software features and capabilities have expanded greatly, but the learning curve remains steep. There is also confusion that qualitative software actually analyzes data: it doesn’t.  Software is a data management tool.  Humans beings still have to organize, interpret and make meaning from the data.

9. Social Media as a qualitative tool: increasingly used for both data collection and sharing findings

The rise of social media in every aspect of modern life is a hallmark of the information age. Qualitative inquiry is already being profoundly influenced by social media opportunities as social media becomes a tool both for data collection and communicating results.

8. Ethical challenges abound

The in-depth, engaged, interactive, and interpersonal nature of qualitative fieldwork increases dramatically the challenges of creating and following appropriate ethical standards. Institutional Review Boards are struggling to determine how to apply traditional standardized ethical research standards to qualitative designs that are emergent, naturalistic, and dynamic. Issues include:

  • Anticipating impact on participants
  • Confidentiality with small sample sizes
  • Appropriate compensation – particularly in circumstances where their involvement is more than simple data collection. Where does the principle of reciprocity lead us?
  • Lack of qualitative expertise and experience on review boards.

7. Mixed Methods

The former qualitative-quantitative debate has realized rapprochement around the triangulated value of mixed methods. However, in actual implementation, mixed methods manifest more as parallel play (like two-year olds not yet able to play together) than as genuinely integrated inquiry and analysis.

6. Data visualization

Qualitative inquiry is rapidly incorporating a variety of data visualization techniques and tools. The two most recent issues of New Directions for Evaluation are devoted to Data Visualization (Issues 139 and 140, fall and winter, 2013).  The year 2013 saw the organization of a Data Visualization TIG.  Dr. Stephanie Evergreen has led AEA in pioneering data visualization as a key competency for evaluators.

Coming up next week: the final five top trends in qualitative evaluation methods.



For an overview of specialist tools for qualitative data analysis, see the CAQDAS site at the University of Surrey which compares ten packages including Atlas.Ti, HyperResearch and NVivo.  Learn how to use commonly available non-specialist software to analyse qualitative data. 

50 Top Tools for Social Media Monitoring, Analytics, and ManagementFind out how social media can be used as a data collection tool. 

Define ethical and quality evaluation standards: Read about some ethical issues to consider when planning an evaluation 

Mixed methods in evaluation part 2: exploring the case of a mixed-method outcome evaluation: See how one outcome evaluation used mixed methods to gain rich and meaningful insight.


Image source: Grand Canyon, by Mark on Flickr