DoD-Wide Oversight and Monitoring

Position

The ten elements

Read in order, these answer a different question from the report they come from: not what happened, but what a system built to prevent it would have to measure.

Where it appeared, and what it was called

One row per published roster. A year absent from this table is a year in which this weakness was not on the roster, or — for FY2023 — a year for which no roster was published at all.

From this root cause to a system that can be audited

The first three steps come from the record above. The rest is a design, and is marked as one: nothing on this site evidences that any of it was built or that it works.

1. 1

   Root cause

   from the record
   Closure is reported by the organisation that owns the corrective action, against milestones it set. A plan can complete on schedule while the control it was meant to install never operates, and nothing in the reporting chain distinguishes the two. This is the weakness that makes the others durable, and it is why an NFR count falling is not evidence that anything was fixed.
2. 2

   The business relationship that should hold

   from the record
   A finding should close only when the control it concerns is shown to operate, in a period after the remediation.
3. 3

   Data the relationship requires

   from the record
   NFR population with status history, corrective action milestones, and control performance evidence after closure.This site holds the published counts on this page and nothing at notice level, which is the whole of the problem: closure cannot be distinguished from downgrade or from reissue under a new number without the notice population, and that population is not public.
4. 4

   Rule

   design
   State the relationship as a testable condition over that data and run it over the whole population, not a sample. Every break is an exception with a transaction behind it. A rule that can only be evaluated on one side of the relationship is not a test of it, which is why step 3 has to come first and has to be honest about what is missing.
5. 5

   Machine learning

   design
   Patterns the rule does not express: a break that appears only at a particular period end, a counterparty whose exceptions cluster, a value distribution that moves before a reconciliation fails. Trained on the exception history the rule produces, so the model has a labelled population rather than an unsupervised guess at what “unusual” means for this account.
6. 6

   Language model

   design
   Explain a specific exception against the source records it was raised from, quoting them. Grounded in the retrieved evidence, never in the model’s own account of how the process works — an explanation that cannot name the record it rests on is not audit evidence.
7. 7

   Automation

   design
   Route the exception to the accountable office, collect the supporting document, open the correction, and record what was done and by whom. The automation is the part that makes the control operate on a schedule rather than at year end under an auditor’s deadline.
8. 8

   Monitoring

   design
   Measure whether the control performs: exception rate, time to clear, ageing of what is unresolved, and recurrence after closure. Recurrence after closure is the one that matters here, because the oversight weakness on this roster is precisely that a corrective action can be reported complete while the control it was meant to install never operates.
9. 9

   Audit evidence

   design
   Retain the tested population, the exceptions, the investigation, the remediation and the control-performance history, each immutable and timestamped. The deliverable is not a dashboard. It is a package an auditor can test that demonstrates the control operated across the period.

The order is the argument. Building an anomaly detector for this account without steps 1 to 3 gives a model trained on whichever side of the relationship happens to be in a data lake, and it will find anomalies there — reliably, and without any of them being the failure the auditor reported. Materiality decides whether the work is worth doing, and the public record sizes this one only this far: Not published at this grain. 2,473 NFRs were issued and 1,004 closed in FY2025, against 2,972 open.