Developmental Test, Evaluation and Assessment

About the Office of Developmental Test, Evaluation and Assessment

developmental Test & Evaluation as a Continuum

Digital Engineering, Modeling and Simulation Header

The paradigm of developmental Test and Evaluation as a Continuum (dTEaaC) drives continuous mission validation across the capability lifecycle to deliver capability to the warfighter at the speed of need.

dTEaaC differs from the current serial test execution process by enabling an integrative continuum for Test and Evaluation approaching a paradigm of continuous mission validation to promote an understanding of the operational outcomes that become available when engineered systems are deployed as early as possible.
dTEaaC leverages model-based engineering, digital mission threads, and advanced statistical techniques to enable accelerated learning across the capability lifecycle via focused mission validation, from pre-acquisition efforts (S&T, prototyping, experimentation) through post-fielding efforts (learning and agile SW systems).
dTEaaC establishes logical, agile, iterative test, capability evaluation, and decision support through a decision support enterprise framework – integrated decision support key.

dTEaaC unites ME, SE, and T&E in pursuit of data-informed decision making as part of a campaign of learning. Leveraging model-based engineering, digital threads, and statistical techniques to amplify T&E, dTEaaC drives continuous mission validation across the acquisition lifecycle to deliver capability to the warfighter at the speed of need.

dTEaac is Built on 3 Core Tenets

Links a Campaign of Learning to Warfighter Need across the Capability Lifecycle

Establishes a Campaign of Learning based on data, information, and knowledge across the Capability Lifecycle.
- Promotes continuous mission validation via an Agile, Iterative, Data-Driven and Knowledge-Based Framework for capability delivery, moving away from an event-based, serial approach to a data-driven approach.
- Facilitates critical testing and validation of emerging technologies, including Artificial Intelligence (AI) – enabled and software-intensive systems.
- Optimizes S&T, prototyping, experimentation, and test execution in support of pre-acquisition technology maturation, readiness, transition, and risk reduction.
- Emphasizes assessment of warfighting capabilities across the testing continuum.
- Supports continuous validation of fielded agile development and learning systems.
Provides Timely Decision Support
- Promotes a Decision Support Enterprise Framework that integrates insights from test and evaluation early in the lifecycle in support of complex decision making, ensuring an enhanced level of validation of warfighting needs through the alignment of data, analysis and evaluation to support capability delivery decisions.
- Leverages the Integrated Decision Support Key framework applied both pre-acquisition and post-acquisition, integrating into an enterprise-level integrated data framework.
- Enables earlier discovery and mission focused decision making.
- Provides ongoing and continuing learning and feedback to build better understanding for decision makers.
- Enables near real time decision dashboards and visualizations directly supported by the specific relevant authoritative data.
Aligns Test and Evaluation with the Digital Innovation Ecosystem
- Fully integrates and aligns Test and Evaluation with Model-Based Digital Engineering development and application, ensuring an integrative environment between Mission Engineering, Systems Engineering, and Test and Evaluation models and processes across the capability lifecycle.
- Closely integrates with ongoing Digital Workforce Initiatives and Curriculum to develop the requisite test and evaluation engineering skills and experiences for the workforce.
- Builds the foundation for a M&S “Digital Thread” continuum documenting the core testing required to validate and accredit models and leverages the modeling environment to compliment and improve live testing.
- Establishes the foundational integrated data and information architecture that is essential for seamless knowledge sharing and enhanced decision making.

dTEaaC defined

A critical change in how Test and Evaluation (T&E) supports capability delivery is needed
Making this change requires a new paradigm in which:
- T&E provides focused and relevant information supporting decision-making continually throughout capability development from the earliest stage of Mission Engineering (ME) through Operations and Sustainment (O&S)
  - Starting at the earliest phases of S&T, prototyping, and experimentation to develop and mature technology
  - Into traditional program of record developmental test (both contractor and govt)
  - Beyond fielding for systems that continue to evolve (learning and agile software)
- A foundation of data and analytics across the capability development continuum is established
This new approach moves T&E from a serial set of activities conducted largely independently of Systems Engineering (SE) and ME activities to a new agile and integrative framework focused on a campaign of learning termed developmental T&E as a Continuum (dTEaaC)
dTEaaC has key attributes and enablers critical in the conduct of T&E as a continuum and delivery of capability at the Speed of Need

Capability Delivery Spectrum

Knowledge Management Cloud Computing Environment

Decision Support Enterprise Framework

Tying it all together by putting in place a framework that links the decisions that need to be made to the data that supports those decisions and the test events and resources that generate that data.

Capability Life Cycle Focus

The term “capability life cycle” means the span of time from when an organization recognizes the need for the ability to execute a new task, to meet its roles, functions, and missions in current or future operations, until that task is no longer required. Capability lifecycles are enduring in nature, with the material and non-material aspects of the capability varying over time.

Looking across the entirety of the continuum from gap identification through sustainment

Alignment between Mission Engineering, Systems Engineering, Discipline Engineering, T&E

dT&E as a Continuum

The term “developmental test and evaluation” means deliberately exercising virtual, constructive, or live systems to evaluate their responses, generating information on capabilities, affordances, and limitations while developing credible, reality-based insights to inform decisions for new or enhanced capabilities, operating within a developmental test and evaluation continuum that includes contractor verification, developmental test and evaluation, developmental test and evaluation concurrent with operational test and evaluation, and developmental test and evaluation occurring throughout operations and sustainment.

Providing continuous feedback via validation

M&S as a Continuum

Leveraging the modeling environment early and continually as models increase in fidelity

Data as a Continuum

Take full advantage of shared data and analytics

TEaaC and the Decision Support Evaluation Framework (DSEF)

Decision Support Enterprise Framework (DSEF) provides strategy for:

Top level mapping of enterprise decisions and capability needs to define components’ performance expectations and contributions
Enterprise-level decision support and capability evaluation using enterprise and/or component-level data
Capability components can be lifecycle time phases, systems, or capability portfolio elements

Integrated Decision Support Key (IDSK) Identifies experimentation, test, and M&S event data for…

Evaluation of technical capabilities and operational effectiveness, suitability, survivability, lethality to…
Inform technical, acquisition, programmatic, operational decisions
Expanded to (DoDI 5000.DT policy)
Component capability development & fielding decision-evaluation-data
Data and evaluation flows to DSEF for enterprise evaluation and decisions

Decision Support – Evaluation Framework

Full Capability Delivery Continuum decisions (technical, transition, operational, enterprise architecture, acquisition, investment) to be informed by evaluation.
Operational and technical capabilities to be evaluated to generate the knowledge needed to inform decisions.
Wargames, experiments, exercises, tests, M&S and LVC events provide data for evaluation and decision support.

Evaluation drives TEAAC event design

Capability evaluation and decision-support data needs drive TEAAC event design (wargame, experiment, exercise, test, M&S).

Model-based DSEF enables agility

Digital Engineering DSEF enables agility in decision-support and test execution.

dTEaaC and Systems Engineering

The SE community has historically used a serial interpretation of the SE “V”.

Executing the dT&E as a Continuum will integrate ME, SE, and T&E into parallel, collaborative, and combined efforts through a dynamic, connected new model-based SE “V” Environment.

Using this model-based environment, DoD can transition to a “model-test-validate-design-test” process providing early and continual information on expected mission capability.

“Collapsing” of the SE “V” does not negate good SE practices. The model-based continuum will allow the SE community to manage the many complex activities being conducted simultaneously within this continuum across the “V”.

Connecting to Mission Engineering through Systems Engineering

Shifting the Traditional Testing Paradigm to a "Testing Continuum"

dT&E as a Continuum builds off the DoD 2018 Digital Engineering Strategy and DOT&E 2022 Strategy Implementation Plan making T&E an integral part of the Systems Engineering and Mission Engineering processes.