Info: Join us for an AEA Coffee Break conversation, featuring an upcoming pre-conference workshop taking place at Eval 23 in Indianapolis, Indiana. In this session, you will hear about the topic from the presenter and have the opportunity ask the presenter a question before heading out to Eval 23. You can find registration information for pre-conference workshops here:
https://www.evaluationconference.org/Registration/Registration-2023

Description:

Do you strive for a cutting-edge evaluation strategy that promotes utility in formative and developmental evaluation, in a way that some traditional methods seem less suited for? Do you firmly believe in “The Power of Story” but worry that your thematic analysis might not be considered robust by certain audiences? Our values drive us to engage stakeholders through our collection methods to elevate the voice of those that are often not heard. We also attempt to make visible those that go unseen, however, many traditional analysis methods reduce them to means or regress them to averages, completely ignoring the important nuance we know exists. Qualitative Comparative Analysis may be the method that comes to the rescue, purposefully investigating complexity, asking in what ways, and to what extent did multiple pathways influence desired implementation across cases.
This full-day workshop will provide food for thought for participants but will also challenge advanced evaluative thinkers. Please join us for a hands-on and engaging walk through this new method and leave with all of the tools you need to activate and accentuate a more realist perspective and meaningful utility through evaluation. Moreover, participants will be invited to an ongoing community of practice consisting of evaluators interested in QCA as a method. This community developed from a 2022 AEA workshop on the same topic, provided to a packed house of 50+, including roughly a dozen that wanted to continue their study and conversation around the topic.
Attendees will:
• Remember the value of seeking nuanced answers from complex data
• Understand the rationale for Qualitative Comparative Analysis (QCA) as an evaluative tool
• Apply their understanding of the value of QCA in evaluative planning and decision making
• Analyze additional resources that will properly scaffold the process of completing a QCA at home
• Evaluate their current practices and the possibilities present in innovative methods for attending to complexity.
• Create an action plan for continued learning related to QCA, set theory, and non-parametric opportunities for analysis.
• Receive further coaching from the presenter at one- and three- month intervals, and entrance into an existing community of practice of practitioners developing skills in this area
Pre-requisites: Attendees should have some nominal working knowledge in the areas of logic, program theory, set theory, basic algebra and qualitative design. Participants should have experience with quantitative analysis using parametric or non-parametric statistics, and at least basic qualitative thematic analysis.


Learning Outcomes:

• Attendees will remember the value of seeking nuanced answers from complex data
• Attendees will apply their understanding of the value of QCA in evaluative planning and decision making
• Attendees will create an action plan for continued learning related to QCA, and set theory opportunities for analysis

This workshop is aligned to AEA’s Competencies and Guiding Principles as follows:
QCA was first rooted in political science analysis and then began to spread to other areas of concentrated study, especially with the advent of fuzzy-set QCA which allowed for its use in complex environments. A study completed almost a decade ago showed that the number of published studies per year had increased 14 times in the decade ending in 2015 (Roig-Tierno, Gonzalez-Cruz, & Llopis-Martinez, 2017). This increase continued in the years to follow and included more frequent uses in the area of program evaluation. For example, just last year, evaluands as diverse as renewable energy adaptation, community garden programs, and the use of COVID 19-era PP funds were published and are listed on the QCA bibliography hosted by the Compasss organization: https://compasss.org/bibliography/.
A QCA approach has become an essential tool for evaluators seeking to understand complex and nuanced systems using data collected through mixed methods. Social science researchers and program evaluators are taking note and making efforts to provide training. To this end, SAGE devoted Volume 6 of its 2018 Mixed Methods Research Series exclusively to the topic (Kahwati & Kane, 2018). In that resource, the authors make the argument that qualitative data collected with an exploratory aim, and quantitative data collected with an explanatory aim can be seamlessly integrated for a configurational study using QCA, even when data collection are sequenced across multiple years or purposes. They then argue that such a technique, through capitalizing on the complementary strengths inherent in a mixed approach and assumptions of causal complexity provides a dynamic complement to findings generated through more traditional methods.
At any rate, the prominence of the approach is increasing in the ecosystem of our membership. At the same time, it is not a topic that is easy to dive into using just a few seminal texts, or by scouring journals, or YouTube for instructional manuals. Instead, as a rather large topic, it requires a big dose of scaffolded learning, a curated resource library and instructions for how and when to use certain resources, and a plan of action for implementation, along with some coaching to help move from theoretical learning to practice.
As a result, this workshop would be hands-on from the very beginning and would attend to those with auditory, kinesthetic, and visual learning preferences with intention. The presenter will facilitate a deep dive into a specific strategy, serving as a "guide on the side," rather than a "sage on the stage." As such, participants will begin by working collaboratively to express current strategies and needs using arts-based methods. We will continue to capitalize on neuro-aesthetic learning principles as participants sketch out how a new process might look from start to finish. Third, the presenter will coach participants to work towards a solution in groups on a sample dataset. Finally, all will actively action plan for future learning before the close of the session, and interested participants will receive personalized coaching from the presenter at one-month and three-month intervals to promote change in behaviors and beliefs.

Presenter:
Jason R. Torres Altman, M.A.
Evaluator and Collaborator
TerraLuna Collaborative

Date:
September 28, 11:30 AM - 12:00 PM ET

Note:
Once you purchase the eStudy you must register for each session. Recordings will be made available to registrants unable to attend sessions live. Recordings will be made available to all registrants for 90 days.