PROGRAM, POLICY, AND ORGANIZATIONAL EVALUATIONS

Directive #069-03

Revision #01

Department of Homeland Security

DHS Directives System

Directive Number: 069-03

Revision Number: 01

Issue Date: 5/18/2023

PROGRAM, POLICY, AND ORGANIZATIONAL

EVALUATIONS

I. Purpose

The Department of Homeland Security (DHS) is committed to ensuring a strong culture of

evaluation, evidence building, and organizational learning. A robust and coordinated

evaluation function is essential to the Department’s capacity to build rigorous evidence for

better decision making. This Directive provides the overall policy, responsibilities, and

principles for the conduct of evaluation within the Department.

II. Scope

This Directive is applicable to all evaluation and evaluation activities conducted,

sponsored, or funded by DHS with the exception of the Office of Inspector General.

III. Authorities

A. Public Law 115-435, 132 Stat.5529, “Foundations for Evidence-Based

Policymaking Act of 2018” (Evidence Act)

B. Office of Management and Budget (OMB) Memorandum M-19-23 “Phase

1 Implementation of the Foundations for Evidence-Based Policymaking Act of

2018: Learning Agendas, Personnel, and Planning Guidance”

C. OMB Memorandum M-20-12 “Phase 4 Implementation of the Foundations

for Evidence-Based Policymaking Act of 2018: Program Evaluation Standards

and Practices”

D. OMB Memorandum M-21-27 “Evidence-Based Policymaking: Learning

Agendas and Annual Evaluation Plans”

E. Deputy Secretary of Homeland Security Policy Statement 076-01

“Evidence Act Title 1 and Evaluation Implementation”

F. Public Law 107-296, 116 Stat. 2135, “Homeland Security Act of 2002,”

codified in Title 6, United States Code; amended Public Law 108-330, “The

Department of Homeland Security Financial Accountability Act”

Directive #069-03

Revision #01

G. Designation 00-01001, “Designation to the Director of Program Analysis

and Evaluation” as the DHS Evaluation Officer

IV. Responsibilities

A. DHS Evaluation Officer (EO) reports to the Chief Financial Officer and

has authority and responsibility for providing leadership over the Department’s

evaluation activities. The EO:

1. Establishes and oversees Department-wide implementation of this

Evaluation Policy.

2. Coordinates and engages with Department stakeholders in carrying

out the responsibilities assigned to the EO in the Evidence Act Title 1

requirements (i.e., development, implementation, and scientific integrity of

evaluations, quadrennial Capacity Assessments, quadrennial Learning

Agendas, and Annual Evaluation Plans) and related implementing

guidance.

3. Chairs and acts as the secretariat for the DHS Evaluation Officer

Council (EOC).

4. Champions capacity building opportunities for DHS as a whole,

while serving as an institutional source of guidance on evaluation and

evidence building.

5. Coordinates or commissions Department-wide, cross-cutting and

other priority evaluations, systematic reviews, meta-analyses, or policy

research to address DHS priority learning questions.

6. Coordinates with Department stakeholders for the oversight of

Department-wide evaluation activities, including but not limited to

prioritization of topics to be evaluated, implementation of external

requirements, development of evaluation procurement requirements,

selection of appropriate evaluation designs and methods, and reporting of

evaluation findings, to ensure adherence to evaluation standards,

scientific integrity principles, and other administrative requirements.

7. Coordinates with appropriate Department officials on the release of

findings from evaluation and other evidence building conducted for the

DHS Learning Agenda and Annual Evaluation Plans.

8. Supports internal and public release of evaluation study plans,

evaluation reports/summaries, and action plans/reports for the use of

findings, including advising on principled exceptions (exemptions or

limitations) to the requirement of public disclosure of evaluations.

Directive #069-03

Revision #01

9. Supports the integration of evaluation and other evidence-building

plans and findings from the Department’s Evidence Act Title 1

deliverables with the Department’s strategic plans, regulatory agendas,

strategic reviews, annual program performance measures and plans,

annual program performance reports, budget submissions, federal

assistance program funding announcements, and foreign assistance

programs.

B. Component Heads advance and prioritize evaluation and other evidence

building, creating a culture of evidence and support for staff in undertaking

evaluation. The Component Head:

1. Designates a Component Evaluation Officer with senior-level

technical expertise in evaluation methods and practices who participates

in the EOC and is responsible for supporting the Component Head in

carrying out all activities in this section in coordination with the EO.

2. Ensures compliance with this Evaluation Policy across the breadth

of the Component, including meeting the standards outlined in this

Evaluation Policy and related published instructions and guidance for

planning, conduct, dissemination, and use of evaluations.

3. Develops the Component’s capacity for evaluation management

and methods, including making investments needed to support evaluation

and its scientific integrity, which should extend to allocating adequate

funding for third-party evaluation studies and to hiring, retaining, and

developing qualified employees to design and oversee evaluation

activities.

4. Leads Component engagement in annual and quadrennial

Department-wide processes led by the EO for Evidence Act Title 1

deliverable development and updates.

5. Develops Component Evidence Act Title 1 deliverables that

address Administration priorities and external requirements, are informed

by robust engagement with internal and external stakeholders, and align

with Department-wide plans.

6. Undertakes, accelerates, and reports on evaluation and other

evidence-building activities contained in Component and Department-wide

Learning Agendas and Annual Evaluation Plans.

7. Establishes a strategic portfolio of evaluations to examine and

generate recommendations for improving the efficiency, effectiveness, and

equity of programs, policies, regulations, or organizations at a rate

Directive #069-03

Revision #01

commensurate with the scale of the Component’s work, the scope of their

portfolio, and the size of their resources.

8. Ensures Component evaluations maintain scientific integrity while

drawing on a full range of evaluation types and approaches, including

quantitative, qualitative, and mixed- methods approaches, as appropriate to

address evaluation questions, and evaluations making claims about

efficacy, effectiveness, or impact are supported by appropriate quasi-

experimental or experimental designs and quantitative methods.

9. Consults and coordinates with the relevant DHS Chief and

Component officers (e.g., for Evaluation, Statistical, Data, Information,

Privacy, Human Subjects Protection) to ensure that evaluations and those

who conduct and manage them comply with the requirements of federal

laws, regulations and Department policies for evidence building,

information collections, data privacy protections, human subjects research

protections, and information dissemination.

10. Integrates evaluation and other evidence-building plans and

findings from the Department’s Evidence Act Title 1 deliverables with the

Component’s strategic plans, regulatory agendas, annual strategic

reviews, annual program performance measures and plans, program

performance reports, budget submissions, federal assistance program

funding announcements, and foreign assistance programs.

11. Uses evidence from evaluation to further mission and operations,

and commits to building evidence of effectiveness to strengthen and

advance the Department’s mission.

C. DHS Chief Data Officer (CDO) and Component Data Officers advise

and collaborate with the EO, Component Evaluation Officers, and other DHS

officials to ensure appropriate data governance, data stewardship, and data

lifecycle management processes that align with scientific integrity principles to

support systematic data collection and use for evaluation and other evidence-

building.

D. DHS Statistical Official (SO) and Component Statistical Officials

advise and collaborate with the EO, Component Evaluation Officers, and other

Department officials to enable access, acquisition, and linkage to federal

statistical data and ensure that statistical methods or techniques provide optimal

levels of data quality and confidentiality.

VI. Policy and Requirements

A. Policy: DHS conducts well-designed evaluations to more fully characterize

and account for the ways the Department uses resources to achieve its goals and

Directive #069-03

Revision #01

objectives. DHS generates and uses rigorous evidence from evaluations to inform

decisions about programs, policies, regulations, and organizations, better

enabling the Department to achieve the most effective U.S. homeland security

outcomes and greater accountability to our primary stakeholders, the American

people. At DHS, evaluation supports:

1. Organizational learning. Evaluations answer questions that

produce valuable knowledge and inform the Department’s understanding

of and ability to respond to mission needs and changes in the

environment.

2. Performance improvement. Evaluations identify when and how

the Department has met its goals, providing leaders with evidence they

need to make decisions about changes to programs, policies, regulations,

or organizations that should be made to increase efficiency, effectiveness,

and equity.

3. Resource priority determinations. Evaluations help DHS allocate

resources, highlighting where they are needed for greatest impact. This

includes decisions about the future of programs, policies, regulations, or

organizations, such as whether to continue as is, enhance/scale up, or

reduce/scale down.

4. Stakeholder engagement. Evaluations engage and exchange

valuable information with a broad range of internal and external

stakeholders, promoting transparency and accountability for stewardship

of public funds and leading to advances in research, policy, and practice

in and beyond the Department.

B. Requirements: This Evaluation Policy presents core standards that guide

the conduct of evaluation at DHS to ensure credibility and high-quality evidence

for learning and decision making. These standards align with published federal

evaluation standards. DHS evaluations should always adhere to standards of:

1. Relevance and utility. DHS evaluations address questions that

are important and provide findings that are actionable and available in

time for use. DHS evaluations consider the learning priorities related to

programs, policies, regulations, or organizations, and the potential impact

on the Department’s strategic priorities. Evaluation findings inform and are

integrated into the Department’s activities, such as budgeting, program

improvement, management, accountability, and the development of

programs, policies, and regulatory actions.

2. Rigor. DHS evaluation findings are credible and mean what they

purport to mean. DHS conducts evaluations to the highest standards:

those who conduct DHS evaluations have appropriate expertise for the

Directive #069-03

Revision #01

designs and methods undertaken, evaluation designs and methods are

appropriate for the questions asked, documentation of evaluation

processes and findings are clear and accurate, and limitations of findings

are transparent so that internal and external stakeholders can act on

evaluation findings with confidence.

3. Transparency. DHS is committed to ensuring that the leadership

and staff, collaborators, policymakers, researchers, and the public at large

are able to learn from the Department’s work. DHS is transparent in the

planning, implementation, and reporting of evaluations to enable learning

and accountability. DHS issues a public record of significant evaluations

conducted and shares findings for those evaluations in a timely way

(including null results and results that run counter to the Department’s

expectations and goals) in accordance with applicable law.

4. Independence and objectivity. DHS evaluations are conducted

with an appropriate level of independence from program, policy,

regulation, and stakeholder activities. Those who conduct DHS

evaluations demonstrate objectivity, impartiality, and professional

judgment throughout the evaluation process.

5. Ethics. DHS evaluations meet the highest ethical standards and

safeguard the dignity, rights, safety, and privacy of participants,

stakeholders, and affected entities. DHS evaluations comply with relevant

professional standards and requirements, such as laws, regulations, and

DHS policies governing data privacy and confidentiality, human subjects

research protections, and administrative burden to the public.

6. Equity. DHS evaluations are equitable, fair, and just, and account

for cultural and contextual factors that could influence findings and the use

of those findings. Those who plan, implement, disseminate, and use DHS

evaluations seek and gain understanding of the diversity of perspectives

and experiences that stakeholders bring to an evaluation, including those

not usually represented. Evaluations examine equity of access,

experiences, benefits, and unintended consequences of programs and

policies across relevant groups, including underserved communities, of

the affected populations.

7. Scientific integrity. As evaluation is a scientific activity, those who

conduct DHS evaluations must uphold scientific integrity principles and

requirements.

Directive #069-03

Revision #01

VII. Questions

Address any questions or concerns regarding this Directive to DHS Evaluation Officer,

Director of Program Analysis and Evaluation, Office of the Chief Financial Officer.

R.D. Alles

Date

Senior Official Performing the Duties of the

Under Secretary for Management