Week 4: The top ten developments in qualitative evaluation over the last decade – part 1

MQuinnP's picture 16th January 2014 by MQuinnP

Over the next two weeks, Michael Quinn Patton joins us to give his top ten trends in qualitative evaluation over the last decade. This week, he sets out the first five.

A lot has changed in the decade since I wrote the 3rd edition of Qualitative Research and Evaluation MethodsWhile working on the 4th edition, I’ve pulled out ten highlights to sum up the state of qualitative evaluation methods, and some emerging challenges. To help put together this list, I also consulted the three qualitative colleagues listed below (and they in turn with their colleagues and students) about their sense of the major trends - though they bear no responsibility for the final top ten list I’ve constructed.

  • Sharon Rallis, incoming editor of the American Journal of Evaluation and co-author of two qualitative books
  • Leslie Goodyear and Jennifer Jewiss, co-chairs of the AEA Qualitative Topical Interest Group (TIG) and editors of the forthcoming book, Qualitative Inquiry in the Practice of Evaluation.

So here are the first five of my top ten developments in qualitative evaluation inquiry over the last decade

10. Powerful Qualitative Software

Software features and capabilities have expanded greatly, but the learning curve remains steep. There is also confusion that qualitative software actually analyzes data: it doesn’t.  Software is a data management tool.  Humans beings still have to organize, interpret and make meaning from the data.

9. Social Media as a qualitative tool: increasingly used for both data collection and sharing findings

The rise of social media in every aspect of modern life is a hallmark of the information age. Qualitative inquiry is already being profoundly influenced by social media opportunities as social media becomes a tool both for data collection and communicating results.

8. Ethical challenges abound

The in-depth, engaged, interactive, and interpersonal nature of qualitative fieldwork increases dramatically the challenges of creating and following appropriate ethical standards. Institutional Review Boards are struggling to determine how to apply traditional standardized ethical research standards to qualitative designs that are emergent, naturalistic, and dynamic. Issues include:

  • Anticipating impact on participants
  • Confidentiality with small sample sizes
  • Appropriate compensation – particularly in circumstances where their involvement is more than simple data collection. Where does the principle of reciprocity lead us?
  • Lack of qualitative expertise and experience on review boards.

7. Mixed Methods

The former qualitative-quantitative debate has realized rapprochement around the triangulated value of mixed methods. However, in actual implementation, mixed methods manifest more as parallel play (like two-year olds not yet able to play together) than as genuinely integrated inquiry and analysis.

6. Data visualization

Qualitative inquiry is rapidly incorporating a variety of data visualization techniques and tools. The two most recent issues of New Directions for Evaluation are devoted to Data Visualization (Issues 139 and 140, fall and winter, 2013).  The year 2013 saw the organization of a Data Visualization TIG.  Dr. Stephanie Evergreen has led AEA in pioneering data visualization as a key competency for evaluators.

Coming up next week: the final five top trends in qualitative evaluation methods.

Related resources on the BetterEvaluation site:

For an overview of specialist tools for qualitative data analysis, see the CAQDAS site at the University of Surrey which compares ten packages including Atlas.Ti, HyperResearch and NVivo.  Learn how to use commonly available non-specialist software to analyse qualitative data.

Read more

50 Top Tools for Social Media Monitoring, Analytics, and Management

Find out how social media can be used as a data collection tool.

Read more

Define ethical and quality evaluation standards

Read about some ethical issues to consider when planning an evaluation

Read more

Mixed methods in evaluation part 2: exploring the case of a mixed-method outcome evaluation

See how one outcome evaluation used mixed methods to gain rich and meaningful insight.

Read more

Image source: Grand Canyon, by Mark on Flickr

52 Weeks of BetterEvaluation. Click here to view past features

A special thanks to this page's contributors
Founder and Director, Utilization-Focused Evaluation.
United States of America.


rickjdavies's picture
rick davies

Re "Software is a data management tool.  Humans beings still have to organize, interpret and make meaning from the data." YES, BUT....

I suggest that an important upcoming trend/development will be the increased use of data mining algorithms that can help with this "organising" bit. Data mining algorithms are used primary for inductive purposes i.e. pattern finding, via three main means (as far as I can see) : finding clusters, identifying one to one dependencies, and finding more complex association rules. So far data mining methods have been used to analyse "big data" sets, where there may not be a usable/relevant off the shelf theory readily available to direct the analysis of that data. But this does not mean they cant be also used with small data sets, such as coded qualitative data (e.g. text material).  In making this argument I am not tacitly claiming that "hypothesis free" research is possible or desirable via data mining methods. But these methods do help us search what can be at times a very large haystack of possibilities, and the results can then be further refined by application of relevant theories of what might be happening

Add new comment

Login Login and comment as BetterEvaluation member or simply fill out the fields below.