- PREDICTION FACTORS OF U.S. SOUTHWEST BORDER ARRIVALS
Full Question: What “push” and “pull” factors at the national and local levels predict increases/decreases in the numbers of noncitizens arriving at the Southwest Border?
Secure, well‐managed borders are needed to protect the U.S. against threats from abroad and to safeguard and expedite the flow of lawful trade and travel. DHS employs near real‐time trend analysis to produce operational planning profiles that inform short‐term staffing requirements, but the Department lacks a long‐term model of the underlying social, economic, security, U.S. policy, and demographic factors that influence immigration. Understanding and anticipating the numbers of expected encounters at the SWB is critical for appropriately resourcing DHS operations and coordinating partnerships to reduce unauthorized flows. Understanding factors that influence migration to the SWB, including root causes like economic and security conditions as well as dislocation associated with environmental change and natural disasters, may inform other DHS strategic program efforts, such as interventions within noncitizens’ communities of residence.
Proposed Approach and Method: Evidence review of scholarly research publications and other reports to assist researchers in identifying potential country‐and local‐level factors and data to include in the statistical analysis. Relevant data will be acquired and integrated so that statistical analysis can be conducted to (1) examine relationships among noncitizen border encounters at the Southwest Border (SWB), noncitizens’ residence, and country‐and local‐level factors and (2) develop a model that accurately forecasts land‐based SWB encounters.
Proposed Data: Critical data systems include the U.S. Border Patrol (USBP) E3 System and the Office of Field Operations (OFO) Unified Secondary system. These and other administrative/operational data include historic land border encounters and noncitizens’ prior residence (i.e., country, state, city) before encounter. New data to acquire are country‐and local‐level economic, security, and environmental conditions from researchers and other federal agencies (e.g., U.S. Department of State (DOS) and U.S. Agency for International Development) through open data and data sharing agreements. Analytic approaches may include descriptive and inferential statistics as well as advanced data mining and analytics. Third party research support is anticipated.
- IMPLEMENTATION OF RISK MITIGATION ACROSS CYBERSECURITY IN U.S. TRANSPORTATION
Full Question: What are the most significant sources of variation in transportation security sector (TSS) operators’ implementation of cyber security technology and processes? What do TSS operators report as enablers and barriers to implementation, and what gaps/unmet needs could TSA address?
The U.S. transportation system’s cyber environment and infrastructure are vulnerable to a wide range of continuously evolving risks stemming from both cyber and physical threats and hazards. DHS assesses and provides security and mitigation guidance through several outreach and information sharing activities. The intended result is that transportation systems sector (TSS) owners and operators mitigate the most significant cyber risks to transportation infrastructure that could impact national security, public health and safety, and economic security and improve the resiliency of the TSS. DHS monitors how TSS owners and operators implement risk mitigation in accordance with the National Institute of Standards and Technology (NIST) Cybersecurity Framework.18 DHS is currently building measures to monitor cybersecurity incidents that have an actual or potential operational impact and has issued three security directives and three national security program amendments, which require owners and operators of certain pipeline, railroad, public transit, and aviation sectors to report cybersecurity incidents to DHS. Understanding factors that influence implementation of risk mitigation across different sectors and operators will enable DHS to improve its approach (including exercising its statutory and regulatory authorities in different ways and to the extent necessary) for ensuring the resilience of the transportation security sector.
Proposed Approach and Method: A needs assessment that examines the needs of the target populations that, when addressed, result in improved implementation of risk‐mitigating critical cybersecurity technology and processes across TSS operator characteristics. The assessment will explore the nature and causes of those needs, set priorities for the future, and consider what adjustments in the Department’s approach could better support TSS operators in advancing resilience of the TSS.
Proposed Data: Critical data for evidence building come from the TSS NIST Cybersecurity Framework Survey and Validated Architecture Design Reviews (VADRs).These existing data include TSS operator characteristics (sector, size, ownership), security posture, and number and nature of recommended risk mitigation activities undertaken. Collectively these data may provide initial insights to challenges of adopting the NIST Cybersecurity Framework, including mitigation activities, and serve as the foundation of case studies. New data to collect from TSS operators through a quantitative survey and qualitative interviews/focus groups include barriers and challenges to implementation of risk mitigation. Analytic approaches may include descriptive and inferential statistics, qualitative data analysis, and case study analysis. Third party research support is anticipated.
- VEHICLE FLEET ELECTRIFICATION COSTS/BENEFITS
Full Question: What are the estimated costs and benefits of DHS vehicle fleet electrification given specialized/law enforcement requirements?
Section 205 “Federal Clean Electricity and Vehicle Procurement Strategy” of the Executive Order 14009 of January 27, 2021, Tackling the Climate Crisis at Home and Abroad, calls for clean and zero‐emission vehicles for federal and SLTT government fleets. DHS is developing a 10‐year plan to electrify a portion of the Department’s vehicle fleet and must consider goals, strategies, and performance measures for its motor vehicle fleet, fuel procurement, and related sustainability and environmental programs. Motor vehicle fleet electrification is intended to improve sustainability and reduce environmental impact; however, the full extent of the costs and benefits of this policy are currently unknown, especially for vehicles that must meet specialized law enforcement requirements. Understanding the costs and benefits of DHS vehicle fleet electrification will enable DHS to determine long‐term bulk fuel requirements; identify strategies to offset impacts of vehicle fleet electrification on building energy use; and set goals, strategy and performance measures for both fleet electrification and building/facility sustainability.
Proposed Approach and Method: Formative economic analysis, specifically benefit‐cost analysis, of fleet vehicle electrification as an alternative to combustion vehicles, including costs of ownership, petroleum fuel use, hazardous waste disposal, building/facility energy efficiency, and disposal requirements for vehicles that meet specialized law enforcement requirements. Market research will be conducted as needed to estimate total cost of ownership, determine charging requirements, and understand hazardous waste and vehicle disposal for electric vehicles that meet specialized law enforcement requirements.
Proposed Data: Critical data sets include the Department’s Asset Management Data Warehouse, Consolidated Asset Portfolio, and Sustainability Information System and the General Services Administration Federal Automotive Statistical Tool. These data include information on DHS combustion engine motor vehicle fleet characteristics, cost of ownership, petroleum fuel use, and building energy efficiency. New data to collect or acquire from industry or federal research and reports may include the estimated total cost of ownership, hazardous waste and vehicle disposal requirements, specialized/law enforcement requirements for vehicles, and charging requirements. Analytic approaches may include descriptive and inferential statistics, benefit‐cost analysis, and qualitative data analysis. Third party research support is anticipated.
Website for Learning Agenda
Office of Disability Employment Policy
Priority Questions from the Agency's Learning Agenda
- RACIAL/ETHNIC DIFFERENCES OF EMPLOYMENT OUTCOMES FOR PEOPLE WITH DISABILITIES
Full Question: As a group, persons of color with disabilities have lower employment levels and earnings than non-Hispanic whites with disabilities and may not be equitably served by the workforce system and other key disability employment programs. Yet there is little research on racial differences in employment and earnings for people with disabilities; on racial inequities in policy and program design and implementation; on program participation and outcomes for persons of color with disabilities; or on level of program access and cultural competence to ensure inclusive programs and services.
Are there racial differences in access to, participation in, and outcomes of employment programs for people with disabilities?
Proposed Approach and Method: This research project would review extant data on racial differences in employment, earnings, program participation, and program outcomes for people with disabilities, and consult experts and conduct field research to identify current gaps and inequities and promising practices.
- LONG COVID ILLNESS AND EMPLOYMENT--EFFECTIVE STRATEGIES
Full Question: How does prolonged COVID-19 (Long COVID) illness interfere with employment, and what are effective strategies for serving these individuals to support employment and wellbeing? How have disabilities and functional limitations changed during and after the COVID-19 pandemic based on requests for reasonable accommodations?
Proposed Approach and Method: This research project would investigate how long COVID symptoms have impacted people's ability to work some or all of the time, their interest or ability to look for work, and the extent to with long COVID patients have used public assistance programs as income replacement. Among those employed, the project would investigate the supports and accommodations that they may need to work. The analysis should be segmented by the type of long COVID symptom or groupings of related symptoms.
- SOCIAL CAPITAL & EMPLOYMENT CHOICES FOR PEOPLE WITH DISABILITIES--INTERACTIONS & IMPACTS
Full Question: How does social connectedness and social capital affect the employment choices of people with disabilities?
Proposed Approach and Method: This research project would examine how proximity to family or other strong social connections could influence people with disabilities' choice to work, their ease or difficulty in finding and keeping employment, their choice in geographic/residential location, and the nature of those social connections. For example, a person with a disability may receive transportation assistance from a family member, which enables them to work onsite. This research project could also examine the extent to which people with disabilities choose to work in order to build or maintain social connections in their community, and for how many that is a primary motivation to work.
Website for Learning Agenda
Bureau of Budget and Planning; Office of Foreign Assistance
Priority Questions from the Agency's Learning Agenda
- TOOLS TO ADDRESS THE CLIMATE CRISIS
Full Question: The first question we would like to discuss is, "How can the Department’s tools best address the climate crisis?" However, because this question is too large to be answered without endless resources, we have broken it down into several sub-questions, including "What cost-effective measures to reduce carbon emissions from the Department’s operations can serve as models for other governments and demonstrate U.S. commitment to reducing emissions?"; "How is the Department adapting to current and projected climate hazards?"; "Which diplomatic and programmatic interventions have the greatest effect on decisions/commitments by government and private sector actors to reduce emissions?"; and "How can the Department and interagency prioritize and effectively respond to the climate adaptation and resilience needs of vulnerable countries and marginalized populations?"
Proposed Approach and Method: The Department is planning the following activities to answer these sub-questions. To evaluate cost-effective ways of reducing greenhouse gas emissions (GHGe) from the operations of Department facilities, the Department will first develop a global baseline estimate and then track GHG reduction activities. The Department’s monitoring tools, such as our EnergyCAP (utility data) and MeterNet (the Department’s smart metering program) will be used to evaluate impact. Implications from possible changes to fleet, shipping, and travel will be studied. These analytics will help the Department prioritize its efforts to further reduce emissions from its operations. To assess progress on adapting to current and projected climate hazards, the Department will identify posts most susceptible to natural hazards and implement surveys to begin to assess overall preparedness. The Department will integrate key performance goals into the Climate Adaptation and Resilience Plan. To assess the effectiveness of our mitigation-focused diplomacy, the Department will use the Agency Priority Goals (APG) and programmatic reviews. On both mitigation and adaptation analysis, State will also rely on post reporting, internal trackers of host government climate commitments, and external data sources (e.g., Climate Action Tracker, National Adaptation Plan Global Network (NAPGN), Race to Resilience, Race to Zero, etc.). Our ability to respond to the adaptation needs of vulnerable countries will be assessed through the annual data quality assessment (DQA), progress on APG goals, and supporting indicators.
Proposed Data: On Department mitigation and adaptation efforts, State will collect data on the Department’s GHGe and internal adaptation activities. On climate diplomacy, State will track its diplomatic and programmatic interventions and the climate mitigation commitments and implementation actions of key actors (national and subnational governments, private companies, etc.) connected to our interventions. On adaptation diplomacy, State will track the adaptation and resilience needs of countries and the effectiveness and impact of our adaptation programs. On public diplomacy, State will collect and learn from monitoring existing data tracking the effectiveness of public engagement initiatives around efforts to mitigate the climate crisis.
- GLOBAL MISINFORMATION AND U.S. SECURITY AND PROSPERITY
Full Question: The second question for discussion is “How should the Department confront the rise of global disinformation and its negative effects on the security and prosperity of the United States?” This question focuses on State Department diplomatic and programmatic efforts to confront the rise of global disinformation and its negative effects on U.S. security and prosperity, including the effect on allies and partners. It also accounts for the need to optimize interagency coordination and cooperation with allies, partners, and the private sector to shape the broader information environment in order to mitigate disinformation’s negative effects on the security and prosperity of the United States.
Proposed Approach and Method: The Department will analyze cases where it has (1) taken preventative, deterrent, or punitive actions against state and non-state actors who spread disinformation and/or (2) engaged in activities to limit the spread of disinformation or to shape the information environment. Bureaus will create a typology of measures and outcomes and conduct a meta-assessment of these actions and activities to determine where it is possible to assess their effectiveness and which measures were most effective based on the available evidence. The Department anticipates conducting a literature review and consulting with outside experts, including allied governments engaged in combatting disinformation. Using the data from the meta-assessment, the literature review, and expert roundtables, State will identify cases that illustrate the different approaches the Department could take to effectively counter disinformation. State will select a smaller number of cases for more detailed analysis, identifying trends related to effectiveness while controlling to the degree possible for other variables.
Proposed Data: Data collection and data sources this question will encompass a review of current literature; data calls to State bureaus and independent offices, and posts that implement disinformation programs for program evaluations; analysis of research conducted on the effectiveness of communications efforts to counter disinformation; surveys of participants, practitioners, program officers, and managers to identify activities that may not have been categorized as intended to counter disinformation; as well as workshops with external partners to solicit external viewpoints and current research. Additionally, team members will review the Department's Evaluation Registry and the Evaluation Management System, which house reports and summaries of evaluations funded by foreign assistance and diplomatic engagement sources.
- PROMOTING A SAFE AND SECURE WORK ENVIRONMENT--MANAGING RISKS
Full Question: Our third question, "How can the Department more effectively analyze and manage risks to promote a safe and secure working environment for its staff and partners," will examine how the Agency can best adapt policy and procedures to manage the risks our foreign affairs professionals face at home and abroad while conducting in-person diplomacy and advancing U.S. interests around the world. The focus will be on the policy, guidance, programs, and training that are needed to: support safe and secure conduct of in-person diplomacy that includes the possibility of locations and workspaces outside of U.S. embassies and consulates; to foster a culture that accepts needed risks and manages them appropriately; and to ensure the physical and information readiness and mobility of our global workforce amid constantly changing circumstances.
Proposed Approach and Method: The Department will consider ways to adapt existing risk mitigation strategies to allow for more in-person diplomacy overseas. These will include enhancing existing processes that evaluate the risks, costs, and benefits associated with U.S. presence – and absence – in a given location; and adapting Foreign Affairs Counter Threat (FACT) training to risks associated with varied threat environments, and encouraging more participation in Overseas Security Advisory Council (OSAC) Chapters, including from outside of capitols or consulate cities through small or single officer posts engaging with U.S. private sector interests.
Proposed Data: Data sources will include recent literature, relevant national and agency-level guidance, and bureau-level strategies including relevant Joint Regional Strategies (JRS) and Functional Bureau Strategies (FBS); integrated country strategies (ICS); the Department’s Evaluation Management System (EMS), which serves as the system of record for all evaluations funded by diplomatic engagement funds; and diplomatic cables. State will collect relevant information from allies, interagency partners, and the private sector related to operating in low-threat, small footprint, and expeditionary platforms. Similar studies related to facility and information security will be conducted. This includes evaluating the collocation of U.S. personnel in allies’ facilities in environments where the U.S. does not have appropriate physical structures; the use of commercially leased facilities overseas for small missions in permissible environments; and the expansion of the number of public diplomacy venues that are not collocated within hardened Embassy or Consulate structures as appropriate. Bureaus and offices will conduct data calls and continue to collect emergency action cables, after action reports, and Accountability Review Boards data to understand the efficiency, effectiveness, and risk-benefit trade-offs of different solutions in different environments. Bureaus and offices will also continue to identify additional data sources as appropriate throughout the course of the learning activities.
Website for Plans Performance Budget
Priority Questions from the Agency's Learning Agenda
- TRIBAL RELOCATION PILOT CAUSED BY SEA LEVELS
Full Question: DOI has multiple learning questions around supporting tribes. In the Bipartisan Infrastructure Law spend plan, DOI’s Bureau of Indian Affairs is proposing to conduct a pilot of community relocations necessitated by rising sea levels. Effective financial and technical assistance in this process will be important.
Proposed Approach and Method: Questions for APPAM researchers could include how to gather and apply existing knowledge on best practices for relocations, or how to set up a longitudinal study on the pilot program’s impact.
- EFFECTIVENESS OF ECOSYSTEM RESTORATION TECHNIQUES
Full Question: DOI’s learning agenda has two priority questions related to restoration: on restoration techniques’ effectiveness over the long-term and on nature-based solutions.
Proposed Approach and Method: In the Bipartisan Infrastructure Law spend plan, there is funding associated with ecosystem restoration. APPAM researchers could help DOI explore how to set up an experimental design on restoration activities and/or develop cost-benefit analyses related to ecosystem services.
- SUPPORT TO TRIBES--COST/BENEFITS OF CHANGING FUNDING MECHANISMS
Full Question: DOI has multiple learning questions around supporting tribes. During recent tribal consultations, DOI has heard from tribal members that there is a preference for DOI to issue payments using a mandatory funding scheme rather than requiring tribes to go through a time-consuming grant request process.
Proposed Approach and Method: APPAM researchers could help DOI determine what the federal government’s obligation to tribes is and what the impact would be to moving to a mandatory funding distribution for Bureau of Trust Funds Administration payments.
Website for Learning Agenda
Departmental Office and Internal Revenue Services
Priority Questions from the Agency's Learning Agenda
- CAPITAL PROJECTS FUND AND STATE & LOCAL RECOVERY FUND IMPACTS
Full Question: Did Capital Projects Fund (CPF) funds have the intended effect on communities?
What is the impact of specific State and Local Fiscal Recovery Fund (SLFRF) projects on priority policy areas?
Proposed Approach and Method: Treasury is considering various approaches and methods for answering questions about implementation, impact, and outcomes of American Rescue Plan programs. We have identified two potential approaches for the CPF and SLFRF questions above.
Potential CPF approaches:
- Impact evaluation to understand how improving internet access or speed through broadband infrastructure projects impacts health, employment, or educational outcomes using a difference-in-differences design to measure how outcomes change over time for areas that receive investment compared to those that do not.
- Randomized evaluation of loaning digital connectivity devices to members of the public on community outcomes, such as Medicaid uptake, telehealth access, graduation, student use, and individual unemployment status. Note that some nuance will be required to identify communities without CPF investment (as others may see investment from IIJA, etc.) - Treasury is interested in getting input on how to think about this research design.
Potential SLFRF approaches:
- In partnership with local governments and research community (universities, non-profits, consultancies), examine the impact of funds on key outcomes of interest with one or a collection of grantees (e.g., evaluation using several lotteries in different jurisdictions to compare UBI programs implemented by different counties, partner with health researchers to evaluate outcomes of spending on lead remediation).
- Grantee level randomized evaluation of community engagement processes on project selection and community satisfaction. Randomize outreach strategies (e.g., sender, message, areas that we are reaching out to) to gather input on preferences for spending/project selection from citizens. The outcome would then be the ranked preferences/selected projects, satisfaction, etc.
Proposed Data: CPF and SLFRF reporting data; Census data; publicly available data on health, employment, and educational outcomes (e.g. CDC places, Medicare, graduation data)
- DELIVERING A BETTER TAXPAYER EXPERIENCE
Full Question: How can the IRS address taxpayer needs and preferences to deliver a better taxpayer experience?
Proposed Approach and Method:
- Conduct surveys -- Customer satisfaction surveys, focus groups, and structure interviews to determine taxpayer preferences, pinpoint weaknesses, evaluate the opinion(s)
- Conduct field experiments, lab experiments and pilot studies, as needed
- Conduct cognitive and usability studies
- Utilize named-entity recognition (NER) to identify people, places and organizations in documents
- Utilize sentiment analysis and natural language processing (NLP) to infer meaning from text
- Use of Artificial Intelligence (AI) techniques
Proposed Data:
- CDW (Compliance Data Warehouse)
- Call Transcripts
- Customer questions, inquires, and other feedback
- Business Objects (BOE) data
- Other operational data sets that are not in CDW
- Taxpayer Burden Surveys
- REUCING THE TAX GAP AND IMPROVE COMPLIANCE--EFFECTIVE STRATEGIES
Full Question: How can the IRS use information reporting, artificial intelligence, data, and advanced analytic techniques to reduce the tax gap, improve compliance and diminish repeat noncompliance?
Proposed Approach and Method:
- Network and Link analysis
- Anomaly detection and Machine Learning
- Cluster analysis
- Tax harm modeling
- Pattern Detection
- Data synthesis with external sources
- External data sources
- Longitudinal statistics
- Payment, filing and third-party matching profile and then clustering
- Interactive Mapping
- Graph Analytics (Clustering, Centrality Metrics, etc.)
- Text Analytics found within IRS Forms, schedules, attachments
Proposed Data:
- CDW
- Operational data not in CDW or system extracts
- External data sets
- SEC data
- Attachments from tax return filings
- Foreign bank disclosures
- Bank transactions data from the Federal Reserve
- Data from IRS Employment Tax-related systems
- Data regarding botnets -- may be available from IRS Criminal Investigations division or other law enforcement agency
- Social Media data as appropriate=Social Security Administration data
Website for Learning Agenda
Priority Questions from the Agency's Learning Agenda
- FUTURE OF WORK
Full Question: Looking to the future of work, what organizational structures for work, workplaces, and workforces create a competitive advantage for recruitment and retention? Looking to the future of work, what workplace flexibilities and strategies, including related to pay and benefits, promote equity, diversity, inclusion, engagement, and performance?
Proposed Approach and Method: Various (including evidence reviews). OMB is interested in learning from researchers about relevant, current research activities and proposed approaches.
- TRUST IN GOVERNMENT RELATED TO EXPERIENCES WITH GOVERNMENT
Full Question: How do various touch points between a person and Government services affect trust in Government (speed? transparency? knowledgeable service)?
Proposed Approach and Method: Various (including evidence reviews at all levels of government). OMB is interested in learning from researchers about relevant, current research activities and proposed approaches.
- IMPROVING EQUITABLE DISTRIBUTION OF FEDERAL GRANTS, LOANS, AWARDS, AND BENEFITS--EFFECTIVE STRATEGIES
Full Question: What approaches effectively improve equitable distribution of grants, loans, or other time-sensitive Federal funding and benefits programs from enacted laws (such as the CARES Act, the American Rescue Plan Act, or the Infrastructure Investment and Jobs Act) or in response to disasters, climate-related crises, or public health emergencies?
Proposed Approach and Method: Various approaches to be considered. OMB is interested in learning from researchers about relevant, current research activities and proposed approaches.
Website for Learning Agenda
ARP Equity Learning Agenda
Priority Questions from the Agency's Learning Agenda
- EFFECT OF TELE/REMOTE/HYBRID WORK ON EMPLOYEE ENGAGEMENT AND RETENTION
Full Question: What is the relationship between telework, remote work, and hybrid environments and employee engagement and retention? How do these effects vary by employee demographic?
Proposed Approach and Method: Analysis of OPM FEVS data (special project analyses)
Analysis of OPM administrative human resources and payroll data (analysis of large panel dataset to assess trends in telework and remote work uptake and effects on employee retention, performance, and advancement)
Proposed Data: OPM FEVS; EHRI (HR and payroll)
Data are not public but we are looking for IPAs and developing a process for data sharing agreements
- FINANCIAL LITERACY FOR RETIREMENT - EFFECTIVE STRATEGIES
Full Question: What strategies are effective in increasing financial literacy for retirement?
Proposed Approach and Method: Literature review; audit of agency approaches to employee education around financial literacy and retirement readiness; analysis of TSP data; identification, pilot, and evaluation of a promising educational program
Proposed Data: TSP data (access pending); FEBS data; other survey data
Data are not public but we are looking for IPAs and developing a process for data sharing agreements
- INCREASING EARLY CAREER TALENT IN GOVT--EFFECTIVE STRATEGIES
Full Question: What policies and programs are effective in increasing early career talent into government?
Proposed Approach and Method: Analysis of USAJOBS applicant data to assess applicant interest in fellowships, internships, and recent graduate hiring mechanisms
Analysis of EHRI human resources panel data to assess conversion rates and retention in government for interns, fellows, and recent graduates hires
Experiments/evaluations assessing ROI on recruitment efforts
Proposed Data: USAJOBS applicant data; EHRI (HR and payroll)
Data are not public but we are looking for IPAs and developing a process for data sharing agreements
Website to Learning Agenda
Website to Action Plan
Office of Performance, Planning, and the Chief Financial Officer (OPPCFO)/ Office of Program Performance, Analysis, and Evaluation (OPPAE)
Priority Questions from the Agency's Learning Agenda
- EQUITABLE ACCESS TO SBA PROGRAMS
Full Question: Are there barriers that prevent underserved small businesses from accessing SBA programs, and if so, how can the SBA address these barriers to increase equitable distribution of services?
Proposed Approach and Method: SBA is open to a variety of methodologies to address this research question. This could include but is not limited to a mixed-method, formative process evaluation to assess the barriers that prevent underserved small businesses from accessing SBA programs; a literature review to determine if any existing evaluations, studies, or analyses of the programs have been conducted; focus groups of underserved small businesses; and/or surveys of underserved small businesses. Qualitative and quantitative analyses could also be performed on the existing and collected data. An external researcher with expertise in studying equity would greatly contribute to capacity building of the SBA Evaluation Team and provide needed insights to improve equitable distribution of services, which directly supports Goals 1 and 2 of the SBA Strategic Plan.
Proposed Data: Yes, there are data files available, and these may include demographics of program applicants and awardees. SBA also foresees the need for further data collection to understand who is not accessing the SBA programs and to ascertain why. Additional data can also be found at U.S. Small Business Administration (SBA) Open Data: https://data.sba.gov/dataset/
- PROMISING PRACTICES IN CUSTOMER-CENTRIC PROGRAM DELIVERY
Full Question: What best practices in customer-centric design and delivery of government programs could the SBA integrate into its program delivery?
Proposed Approach and Method: SBA is open to a variety of methodologies to address this research question. This could include but is not limited to a mixed-method, formative process evaluation to assess how best practices in human-centered design could be integrated into SBA program delivery; a literature review to determine if any existing evaluations, studies, or analyses of best practices in human-centered design and delivery of government programs have been conducted; benchmarking of other federal agencies using human-centered design; and focus groups or other data collection from SBA customers and CX experts. Qualitative and quantitative analyses could also be performed on the collected data. An external researcher with expertise in studying human-centered design would greatly contribute to capacity building of the SBA Evaluation Team and provide needed insights to improve the delivery of SBA programs, which directly supports Goals 1 and 2 of the SBA Strategic Plan.
Proposed Data: The SBA could provide access to application forms, processes, and other related items for analyses. Additional data can also be found at U.S. Small Business Administration (SBA) Open Data: https://data.sba.gov/dataset/
- HELPING SMALL BUSINESS RECOVER/ADAPT POST COVID-19
Full Question: What policies, technologies, or best practices should the SBA implement to help small businesses recover and adapt to a post-COVID-19 environment? If businesses have emerging needs, what are they?
Proposed Approach and Method: SBA is open to a variety of methodologies to address this research question. This could include but is not limited to a mixed-method, outcomes or impact evaluation to assess the policies, technologies, or best practices to help small businesses post-COVID; a literature review to determine if any existing evaluations, studies, or analyses of previous relief efforts for small businesses have been conducted; benchmarking of other federal, state, or local agencies providing relief efforts; focus groups of small businesses; surveys of small businesses; and/or econometric studies. An external researcher with expertise in studying COVID relief efforts or econometrics would greatly contribute to capacity building of the SBA Evaluation Team and provide needed insights to improve the economic impact of SBA programs, supporting Goal 2 of the SBA Strategic Plan.
Proposed Data: The SBA could provide access to data on applicants and awardees as well as other related items for analyses. Additional data can also be found at U.S. Small Business Administration (SBA) Open Data: https://data.sba.gov/dataset/
Website to Learning Agenda
Website to Strategic Plan
ARP Implementation Team (WH) & Office of Evaluation Sciences (GSA)
Priority Questions from the Agency's Learning Agenda
- EQUITABLE DISTRIBUTION OF FUNDS
Full Question: Which strategies are more or less effective at ensuring funds (eg, grant and loan programs) are distributed equitably?
Proposed Approach and Method: RCTs or retrospective analysis
- EFFECTIVE STRATEGIES TO INCREASE AWARENESS OF GOVERNMENT PROGRAMS AND RESOURCES
Full Question: How have programs attempted to increase awareness of and access to available ARP programs & resources, particularly among historically underserved individuals & communities? What were the effects of those efforts?
Proposed Approach and Method: RCTs or retrospective analysis
- CONTINUOUS IMPROVEMENT OF GOVT PROGRAMS, WITH VOICES OF UNDERSERVED GROUPS
Full Question: How have programs sought opportunities for continuous improvement, including by soliciting and incorporating feedback from historically underserved communities? What were the effects of these efforts?
Proposed Approach and Method: Retrospective analysis, process and outcome evaluation