Developmental Test, Evaluation and Assessment

About the Office of Developmental Test, Evaluation and Assessment

Developmental Test and Evaluation (DT&E)

Developmental Test and Evaluation DT&E is the deliberate exercising of virtual or real components or systems, and the evaluation of responses to generate information on capabilities and limitations. DT&E's value is creating credible, reality-based information to inform decisions that enable progress in providing new or enhanced capabilities. DT&E provides one perspective of system capabilities and limitations and operates in a continuum of contractor verification, developmental T&E.

Adaptive Acquisition Framework

USA-from-space-with-telecom-network_secondary-section

DT&E Provides Timely Knowledge to Inform Decisions Related to:

Test & Evaluation Process Chart

System concepts
Technology and technical maturity
System and subsystem design trades
Production or manufacturing
Operation and sustainment

DT&E Supports Efficient and Adaptive System Acquisitions

A robust DT&E effort provides the information needed for independent evaluations and risk managed decisions:

DT&E is applicable from concept to combat.

DT&E starts with concept refinement and continues through capability requirements definition; technology maturity; system development; manufacturing; deployment; operations; and sustainment.

DT&E provides feedback on progress toward capability development and improvements, and achieving contractual technical performance.

DT&E stresses systems to validate system functionality and capability to operate within the intended operational mission contexts.

DT&E assesses readiness for initial operational test and evaluation (IOT&E) or limited deployment using production or deployment-representative test articles.

More information on Adaptive Acquisition Framework (AAF) and Pathways can be found here: https://aaf.dau.edu/

Adaptive Acquisition Framework

DT&E Policy

DoDI 5000.02

OPERATION OF THE ADAPTIVE ACQUISITION FRAMEWORK

- Establishes the Adaptive Acquisition Framework (AAF)

Learn More

DoDI 5000.80

OPERATION OF THE MIDDLE TIER OF ACQUISITION (MTA)

- Establishes policy, assigns responsibilities, and prescribes procedures for the management of the MTA for rapid prototyping and rapid fielding

Learn More

DoDI 5000.88

ENGINEERING OF DEFENSE SYSTEMS

- Establishes policy, assigns responsibilities, and provides procedures to implement engineering of defense systems.

Learn More

DoDI 5000.89

TEST AND EVALUATION

- Establishes policy, assigns responsibilities, and provides procedures for test and evaluation (T&E) programs across five of the six pathways of the adaptive acquisition framework: urgent capability acquisition, middle tier of acquisition (MTA), major capability acquisition, and software acquisition.

Learn More

DoDD 5137.02

UNDER SECRETARY OF DEFENSE FOR RESEARCH AND ENGINEERING (USD(R&E))

- Establishes the position, responsibilities and functions, relationships, and authorities of the USD(R&E).

- Authorizes the USD(R&E), as a Principal Staff Assistant (PSA) reporting directly to the Secretary of Defense, to promulgate DoD policy within the responsibilities, functions, and authorities assigned in this issuance.

Learn More

OSD T&E Oversight List

T&E Oversight

The Director of Operational Test and Evaluation (DOT&E) and the Under Secretary of Defense for Research and Engineering (USD(R&E)) jointly publish the Office of the Secretary of Defense (OSD) Test and Evaluation (T&E) Oversight List. The list identifies programs designated for developmental test (DT) oversight or engagement, operational test (OT), and live fire (LF) T&E oversight.

DTE&A actively monitors acquisition programs to determine the need for DT&E oversight or engagement and makes recommendations to the USD(R&E) for adding or removing programs from the list. To determine the need for DT&E oversight or engagement, OUSD(R&E) considers the following

Developmental Test Oversight

Developmental Test Engagement

Adaptive Acquisition Framework programs that are national security systems providing intelligence activities, cryptographic activities related to national security, and command and control of military forces.
- Programs for which an Independent Technical Risk Assessment determined open or unresolved high technical risk rated areas.
- ACAT IB and IC programs conducting DT&E activities prior to initial operational T&E or follow-on DT&E to verify a military capability:
  - Linked to the successful execution of an ACAT ID program Concept of Operations.
  - Supporting a USD(R&E) Assistant Director Modernization Road Map or investment area.
  - Receiving high-level interest and attention as communicated by congressional or Department principals.
- In coordination with USD(A&S), critical 804 Middle Tier of Acquisition Programs.

Oversight List

DOT&E maintains the joint OSD T&E Oversight List, which is designated Controlled Unclassified Information (CUI) and requires a Common Access Card for access, on an OSD collaboration site. The OSD T&E Oversight List does not include highly classified and sensitive programs that are subject to T&E oversight. Oversight of such programs will be identified directly to the Services.

OSD T&E Oversight List (CAC Required)

List of Available DT&E Documents and Guidance

Full Library of DTE&A Documents Located in the DTE&A Library (under construction)

T&E Management Guide

The Test and Evaluation Management Guide (TEMG) is kept updated with the cooperation and support of the Director of Developmental Testing, Evaluation, and Assessments (DDTE&A), the Director of Operational Test and Evaluation (DOT&E), the President of the Defense Acquisition University (DAU), and the DoD Component T&E representatives. The TEMG is one of many technical management educational guides developed for use by DAU. Although some Service-specific processes are included as illustrative examples, the TEMG, like all DAU products, is written primarily from a Department of Defense (DoD) perspective, i.e., non-Service specific.

The objective of a well-managed T&E program is to provide timely and accurate information to decision makers and program managers (PMs). The TEMG was developed to assist the acquisition community in obtaining a better understanding of who the decision makers are and determining how and when to plan T&E events so that they are efficient and effective.

DoD Cybersecurity T&E Guidebook

The purpose of this updated guidebook is to promote data-driven mission-impact-based analysis and assessment methods for cybersecurity T&E and to support assessment of cybersecurity, survivability, and resilience within a mission context by encouraging planning for tighter integration with traditional system T&E. Cybersecurity T&E starts at acquisition initiation and continues throughout the entire life cycle. The guidebook supplements information provided in the Test and Evaluation Master Plan (TEMP) Guidebook, which is linked later on this page.

DoD Cyber Table Top Guidebook

The Cyber Table Top (CTT) process is a best-practice and includes an intellectual wargame-like exercise followed by analysis. The exercise and analysis in the CTT facilitates identification and comprehension of risks from potential cybersecurity vulnerabilities. The purpose of this guidebook is to provide an overview of the CTT process, guidance on performing a CTT, and instructions for generating actionable information on potential cyber threats for the Program Manager. This process and the templates can be tailored to meet individual organization or program needs.

Integrating T&E into DoD Acquisition Contracts

This guide is designed to help the DoD and industry T&E professionals identify T&E items to consider for inclusion when drafting a statement of objectives (SOO), a statement of work (SOW), and a request for proposal (RFP), and during solicitation and contract execution. This guide presumes the reader has an understanding of T&E and the DoD systems acquisition processes as described in DoD Instruction 5000.02 (DoDI 5000.02) and the Defense Acquisition Guidebook (DAG), particularly, DAG Chapter 8, Integrated Test and Evaluation. This guide follows some of the content of the DoD Guide for Integrating Systems Engineering into DoD Acquisition Contracts and T&E topics and issues.

DoD Guide for Integrating Systems Engineering into DoD Acquisition Contracts

The target audience for this guide is the Government program team responsible for (1) incorporating program technical strategy and technical planning into the Request for Proposal (RFP) and (2) performing pre-award functions, including source selection, as well as post-award contractor execution activities. The guide is of most use to the Program Manager (PM), the Lead (or Chief) Systems Engineer (LSE), the Contracting Officer (CO), and the solicitation team.

The primary purpose of this guide is to aid the PM and LSE to effectively integrate SE requirements into appropriate contracting elements in support of system acquisition; however, all Government and industry personnel involved in a program can benefit from this guide.

TEMP Guidebook

T&E Evaluation Guidebook (in development)

DevSecOps Test and Evaluation Guidebook

DevSecOps is an organizational software engineering culture and practice that aims at unifying software development (Dev), security (Sec) and operations (Ops). The main characteristic of DevSecOps is to automate, monitor, and apply security at all phases of the software lifecycle: plan, develop, build, test, release, deliver, deploy, operate, and monitor. In DevSecOps, testing and security are shifted to the left through automated unit, functional, integration, and security testing - this is a key DevSecOps differentiator since security and functional capabilities are tested and built simultaneously.

T&E Guidebook for Autonomy

The purpose of this handbook is to equip testers with tools, approaches, and insights to confidently test autonomous systems. Due to the breadth of challenges with testing autonomy and the overall immaturity of existing test capabilities, the content of this handbook does not explicitly define specific best practices. Rather, this handbook outlines the considerations for autonomy test, which need to take place at each phase of the test process to ensure a safe, secure, effective, and efficient test. This handbook is intended to be more of a compass than a map.

Handbook on Statistical Design and Analysis Techniques for Modeling and Simulation Validation

Published by the Institute for Defense Analysis, this handbook focuses on methods for data-driven validation to supplement the, vast existing literature for Verification, Validation, and Accreditation (VV&A) and the emerging references on uncertainty quantification (UQ). The goal of this handbook is to aid the test and evaluation (T&E) community in developing test strategies that support model validation (both external validation and parametric analysis) and statistical UQ.