News & Publications



APPAM Institutional Member Forum - Bringing Rigor and Intentionality to Mixed Methods Evaluations of Social Programs

February 24, 2019 08:38 AM
hosted by:
June 5, 2019 at The Conference Center at Heart House, American College of Cardiology, 2400 N Street Northwest, Washington, DC 20037

Mixed methods evaluation seeks to integrate a body of qualitative and quantitative evidence to form conclusions about the success, or impact, of a social policy intervention that neither piece alone—nor even the concatenation of their findings—can produce.  Despite its promise, concern persists that too often the qualitative and quantitative “mixing” occurs only after investigation ends and that mixed methods approaches lack the scientific rigor to be trusted.  This day-long forum sponsored by Westat and APPAM focused researchers on sharpening and expanding the mixed method paradigm to respond to these concerns and to enable impact evaluations to make larger contributions to policy learning.  Major themes included the roles of mixed methods evaluation in explaining/understanding impact findings from randomized control trials and in strengthening quasi-experimental analytic methods to ward off selection bias.  A view of how government agencies can benefit from the approach followed.  Closing perspectives—both skeptical and hopeful—considered where mixed methods evaluation may take us going forward.  Audience participation opportunities interlaced the day.
You can read the content highlights via live tweets!
The half-day agenda for 200 registrants was as follows:

8:00 am - 9:00 am Registration & Morning Refreshments 
9:00 am - 9:15 am


Scott Royal, Westat


9:15 am - 9:25 am
Overview of Day
Patty Troppe, Westat
9:25 am - 9:45 am
Mixed Methods Evaluations: Purpose, Scope, and Challenges
Demetra Nightingale, Urban Institute
9: 45 am - 10:45 am
Pushing the Envelope of Mixed Methods Evaluation to Learn from Prospective Impact Studies 
Virginia Knox, MDRC 
Debra Rog, Westat (Reaction, followed by open discussion)
10:35 am - 11:00 am
11:00 am - 11:45 am
Using Mixed Methods Evaluation to Support Better Non-Experimental Analysis
Stephen Bell, Westat

Jeffrey Smith, University of Wisconsin-Madison (Reaction, followed by open discussion)

11:45 am - 12:30 pm
What Would Benefit Funders and Policymakers?
Maria Woolverton, DHHS/ACF/OPRE 
Marsha Silverberg, ED/IES/NCEE
12:30 pm - 1:20 pm
Lunch Service/Announcement of Breakroom Assignments
Patty Troppe, Westat
1:20 pm - 2:20 pm
Sharing of Examples and Real-World Challenges (breakout sessions)
• "Mixed Methods Evaluation for Better Understanding of RCT Results" - Michelle Derr, Mathematica Policy Research
• "Mixed Methods Evaluation to Better Support Non-Experimental Impact Analysis" - Douglas Walton, Abt Associates 
• "Policy Questions Besides Intervention Impact Worth Addressing by Mixed Methods" - Irma Perez-Johnson, American Institutes for Research 
2:25 pm - 3:10 pm
Reports from Breakout Sessions
3:10 pm - 3:35 pm
3:35 pm - 3:55 pm
Haven't We Been Here Before?  Convince Me
Larry Orr, Johns Hopkins University
3:55 pm - 4:15 pm
Potential and Predicted Progress for the Next Decade 
Carolyn Hill, MDRC
4:15 pm - 4:45 pm
Open Discussion 
Share your memories from this event on Twitter using the hashtag #APPAMWestat!
Back to news