From evaluation practice to publication: Lessons from turning workplace research into a journal article

This blog by Bobby Thomas Cameron outlines key lessons he learned while researching the research process, such as the need to plan for publication from the start and the use of literature to identify strategic gaps. He also presents unique risks, including caution when researching colleagues, considerations for interview design, and approaches to balance credibility and accessibility in writing.  

How can evaluators turn day-to-day practice into something that contributes to the wider field?

That question came to me after a colleague and I interviewed fellow federal government evaluators in Canada about the use of expert panels in evaluation. The project eventually led to a peer-reviewed journal article, but just as importantly, it showed me that evaluation practice can generate knowledge worth sharing beyond one organization.

For many evaluators, peer-reviewed publication can feel separate from everyday work. Yet practitioners are often well placed to notice emerging methods, identify practical gaps in the literature, and develop grounded guidance for others. In our case, a workplace research project on expert panels became an opportunity to reflect on how publication starts much earlier than the writing stage. It begins with planning, ethics, literature review, interviewing, analysis, and then translating practice into a form that others can use.

Here are a few lessons from that process.

1. Plan carefully from the start if there is any chance the work could later become a publication.

One of the clearest lessons from this project was that publication starts at the design stage, not at the end of the work.

Because we thought the project might eventually become an article, we approached the work in a more structured way from the outset. We developed a project plan for the publication, clarified timelines, designed a semi-structured interview guide, identified risks, and sought ethics approval through our department’s institutional review board. That process helped ensure the study was rigorous enough to support external publication later.

We were also influenced by work on practitioner inquiry and reflective research. The Describe, Analyze, Theorize, Act Model (Peters, 2009) helped shape how we thought about structuring both the research process and the eventual article. Furthermore, the concept of evaluator reflexivity was a useful reminder that evaluators do not simply collect and interpret data from a neutral distance; they also need to examine how their own location, assumptions, and experience shape the work (Smith et al., 2015).

2. Treat ethics as central, especially when researching with colleagues or inside your own organization.

Our study involved interviewing colleagues in the same organization.

The literature on insider research was especially helpful here. Researchers studying their own setting need to be careful not to rely on assumptions simply because the setting feels familiar (Asselin, 2003). Research involving one’s own colleagues requires particular attention to ethics, workplace norms, and possible conflicts of interest (Fleming & Zegwaard, 2018). Insider status can be a strength because it offers access, contextual understanding, and practical relevance, but it also brings responsibilities related to power, anonymity, and participant protection (Hanson & Allin, 2020). Finally, an important consideration is that closeness and friendship can complicate insider research in ways that need to be thought through explicitly (Taylor, 2011).

Those insights mattered in our case. Because we had pre-existing professional relationships with some participants, we needed to reduce any possibility that colleagues would feel pressured to participate or worry that saying no could affect our relationship. We addressed this by making voluntariness explicit in recruitment emails and consent forms, reaffirming that participation decisions would not affect professional relationships, and anonymizing data and direct quotations.

This was not just good ethics. It also improved the study. Careful ethical design helped create conditions where participants could speak more candidly, which in turn strengthened the quality of the findings.

3. Use the literature strategically to identify the gap and sharpen your contribution.

The literature review did more than provide background. It helped us identify what our article could add.

We reviewed published work on expert panels in evaluation and related methods. For example, the Hierarchical Decision Model is a participative way to structure expert judgment (Kocaoglu, 1983). Another method is impact identification for development program evaluation using expert input, including forms of panel-based judgment (see Dietz and Pfund, 1988). We also internalized the importance of credibility and relative independence when expert panels are used in evaluation-related priority setting (O’Brecht, 1999). Practical considerations, such as validity, reliability, time, cost, and clarity for users when expert judgment is incorporated into evaluation, are also important (Averch, 2004).

The articles reviewed were valuable because they laid out work already completed on expert panels and because they showed a gap. Much of the literature was dated, and there was relatively little that directly captured evaluators’ own perspectives on how expert panels are used in practice, what challenges arise, and what others should consider when planning them. That gap shaped our focus. Rather than simply documenting examples of expert panels, we wanted to ask evaluators what they had experienced and what guidance they would offer others.

That shift was important. It gave the article a clearer contribution and helped us move from “here is a method” to “here is what practitioners say about using it.”

4. Design interviews for candour and analysis for practical usefulness.

Because our study centered on practitioner perspectives, interviews were at the centre of the project.

We developed interview questions using both the literature and our own practical interests. One useful strategy was to map recurring concepts from the literature and use them to shape questions. That helped ensure the interviews were grounded in published knowledge while still leaving room for participants to raise issues that mattered in practice.

We were also mindful of the interview setting. Since participants were fellow evaluators in the same organization, we wanted to support candid discussion. For that reason, we chose not to audio record interviews. Recording can affect what participants are willing to say in some contexts (Harvey, 2011; Peabody et al., 1990). Instead, we used detailed note-taking.

We also had two researchers present during interviews. One of us asked questions while the other took notes and occasionally probed further. After each interview, we debriefed immediately. Those debriefs were useful analytically because they helped us compare impressions, notice patterns early, and check each other’s assumptions.

For analysis, we used a version of thematic analysis (Braun & Clarke, 2006; 2017). Thematic analysis provides a clear and practical approach to identifying themes in qualitative data. In our project, thematic analysis helped us move systematically from interview notes to patterns, themes, and practical implications.

One of the most useful outputs from that analysis was a checklist of promising practices for using expert panels in evaluation. That mattered because it made the final article more actionable. It translated practitioner experience into something other evaluators could actually use.

5. Write for both credibility and accessibility, so the article is rigorous but still useful to practitioners.

Once the research was complete, the challenge became writing it in a way that journals would recognize as a meaningful contribution.

Practitioners generate valuable knowledge through reflection on action and should be seen as important contributors to scholarship (Kernaghan, 2009). Looking at John Mayne’s legacy illustrates the value of the “thinking practitioner” in evaluation (Nielsen et al., 2023). Outside of evaluation, other fields also underscore the importance of practitioner-authored publications and the distinctive contribution they make to the literature (see Chang, 2021; Finlay et al., 2013; Novick & Moore, 2018).

At the same time, publishing practice-based work requires care. Practitioner scholarship should be relevant, actionable, comprehensible, and ethically reasoned (Robey et al., 2018). Clear writing and strong implications for readers is essential (Boyle et al., 2020).  For practice to become publishable, it often needs to be translated into a form that fits academic expectations, including a clear research contribution, explicit use of theory or concepts, and a level of abstraction beyond a single workplace story (Wolfenden et al., 2019).

That was true for us. The work began as a practical research project, but publication required us to show how the study connected to broader methodological conversations in evaluation and what it contributed beyond our own context.

Our article on expert panels was eventually published, but the bigger lesson for me was that evaluators already sit close to valuable knowledge. The challenge is not whether practice contains insights worth publishing. It often does. The challenge is how to design, reflect on, and write up that work so it can travel beyond one project or one organization.