Wednesday, July 3, 2019

Video: APPAM/Westat Forum - Bringing Rigor and Intentionality to Mixed Methods Evaluations

A day-long forum sponsored by Westat and APPAM will focus researchers on sharpening and expanding the mixed method paradigm to respond to these concerns and t enable impact evaluations to make larger contributions to policy learning. Major themes include the roles of mixed methods evaluation in explaining/understanding impact findings from randomized control trials and in strengthening quasi-experimental analytic methods to ward off selection bias.


2019 Spring Conference - Knowing Enough to Be Dangerous

May 15, 2019 03:37 PM

By Ramon Robinson   

Wednesday morning of our triennial Spring Conference opened with a plenary addressing one of the most critical questions for graduate students and employers alike: what can graduate programs (and their students) do to build the skills needed for policy jobs?  

This question was addressed with a multitude of data- and experience-based insights from Dora Kingsley Vertenten of the University of Southern California, Heather Campbell of Claremont University, Juliet Musso of the University of Southern California, Logan O’Shaughnessy of the Privacy and Civil Liberties Oversight Board, and Rob Seidner of the U.S. Office of Management and Budget, who collectively formed a panel representing diverse professional backgrounds and methodological interests.   Together, their insights provide students with the following tips on how to be more competitive job candidates and (as panelist Juliet Musso says) “know enough to be dangerous” in the research profession, whether they’re pursuing opportunities in academia, government or the private sector:

• Prepare to use your knowledge of “wicked problems” and program evaluations both in your day to day work, and in your navigation of the job market – perhaps especially if you pursue a career in federal government.  As panelist Heather Campbell explained, the Executive, Judicial and Legislative branches of the federal government are “infinitely complex systems” with wicked problems.   The more recent graduates can use their knowledge to navigate those systems, and understand the systems within the systems, the better their chances of career advancement. 

• For your research outside of academia, and perhaps within academia as well, learn to be concise.  Panelist Rob Seidner advises that you learn to put the “bottom line up front” and relegate background and theory to supporting documents when too profuse.  

• Prepare to work with multiple stakeholders, glean knowledge from them, and identify the most influential (and potentially loudest) voice among them.  Graduate students may grow accustomed during their studies to working in silos, but to be successful in the workforce and, in government in particular, they need to learn to collaboratively work across those silos and identify champions outside of their own networks.    

• Be prepared to work desks with set responsibilities, but learn how to shift focus very suddenly.  Graduates who demonstrate capability and accountability with this are more likely to find opportunity knocking.

• Be aware of where career opportunities dominate.   Consulting firms such as Deloitte may offer the most impressive technology for their employees, but government will have significantly more job opportunities and more employees with diverse degrees, which enables career progress and interdisciplinary research.  Additionally, considering that the average age of government employees is 50, that may indicate that government is especially welcoming of individuals across a variety of ages and experience levels. 

• Understand the role that privacy and technology play in research issues across policy areas, and consider going into privacy as a career.  Logan O’Shaughnessy reports that questions about what constitutes “personal privacy” are pervasive in policy jobs: “Data becomes digital currency.  Now we have this tension where consumers are not able to protect themselves, so we need a real way to think about this problem.   If you as a company are collecting privacy vs collecting data, you now have a responsibility for how that data is used or transferred.   You need to look at how policy and technology are going to interact.  We need to know what the potential effects of certain data-based decisions will be.  Are we potentially going to have some risk to consumers that are not intended?  What are the measurable impacts that certain decisions are going to have on privacy policy?  Privacy is looking really good as a profession!” 

• As students, you must learn how to create and understand data manually, rather than relying on technology exclusively for both purposes.  In this case knowing enough to be dangerous means knowing what your professors and other predecessors knew.  Sometimes, the oldest tools are the most powerful.   

In short, so that students can “know enough to be dangerous,” graduate education must refocus teaching on both the ability to decisively function in highly technical, big data environments and on the need for interpretation of high-quality analytic outputs, both by new and traditional means. There is also a need for administrators, faculty and students to consider the technical skills required of policy analysts and public managers dealing with machine learning and rapid technological change. The curricula of the future will necessarily incorporate nontraditional technical skills such as coding and privacy education, as well as traditional skills such as basic data modeling.  And take note graduate programs and students: this vision is for the immediate future of the profession.

View slides from the plenary here.


« Back

Association for Public Policy Analysis and Management (APPAM)
NEW ADDRESS! 1100 Vermont Avenue, NW, Suite 650 Washington, DC 20005
Phone: 202.496.0130 | Fax: 202.496.0134
Twitter Facebook LinkedIn Subscribe to me on YouTube

Home|About APPAM|Membership|Public Policy News|Conference & Events|Publications| Awards|Careers & Education|Members Only

Web site design and web site development by

© 2019 Association for Public Policy Analysis & Management. All Rights Reserved.
Site Map | Privacy Policy | Terms of Use | Events | Add Your Event