Configuration Management

Position

The ten elements

Read in order, these answer a different question from the report they come from: not what happened, but what a system built to prevent it would have to measure.

Where it appeared, and what it was called

One row per published roster. A year absent from this table is a year in which this weakness was not on the roster, or — for FY2023 — a year for which no roster was published at all.

From this root cause to a system that can be audited

The first three steps come from the record above. The rest is a design, and is marked as one: nothing on this site evidences that any of it was built or that it works.

1. 1

   Root cause

   from the record
   The change population itself is not reliably captured. Where the change log is generated by the same system being changed, and the log can be edited by those who make the changes, there is no evidence that survives the control failing.
2. 2

   The business relationship that should hold

   from the record
   Every production change should have an approved, tested request that predates it.
3. 3

   Data the relationship requires

   from the record
   Change request records, migration logs, production deployment timestamps.IT general control evidence is internal to the systems and appears in no published financial file.
4. 4

   Rule

   design
   State the relationship as a testable condition over that data and run it over the whole population, not a sample. Every break is an exception with a transaction behind it. A rule that can only be evaluated on one side of the relationship is not a test of it, which is why step 3 has to come first and has to be honest about what is missing.
5. 5

   Machine learning

   design
   Patterns the rule does not express: a break that appears only at a particular period end, a counterparty whose exceptions cluster, a value distribution that moves before a reconciliation fails. Trained on the exception history the rule produces, so the model has a labelled population rather than an unsupervised guess at what “unusual” means for this account.
6. 6

   Language model

   design
   Explain a specific exception against the source records it was raised from, quoting them. Grounded in the retrieved evidence, never in the model’s own account of how the process works — an explanation that cannot name the record it rests on is not audit evidence.
7. 7

   Automation

   design
   Route the exception to the accountable office, collect the supporting document, open the correction, and record what was done and by whom. The automation is the part that makes the control operate on a schedule rather than at year end under an auditor’s deadline.
8. 8

   Monitoring

   design
   Measure whether the control performs: exception rate, time to clear, ageing of what is unresolved, and recurrence after closure. Recurrence after closure is the one that matters here, because the oversight weakness on this roster is precisely that a corrective action can be reported complete while the control it was meant to install never operates.
9. 9

   Audit evidence

   design
   Retain the tested population, the exceptions, the investigation, the remediation and the control-performance history, each immutable and timestamped. The deliverable is not a dashboard. It is a package an auditor can test that demonstrates the control operated across the period.

The order is the argument. Building an anomaly detector for this account without steps 1 to 3 gives a model trained on whichever side of the relationship happens to be in a data lake, and it will find anomalies there — reliably, and without any of them being the failure the auditor reported. Materiality decides whether the work is worth doing, and the public record sizes this one no further than the roster it sits on: the reports name no population or dollar exposure at material weakness grain, so the decision to build has to be made on the balance, not on the finding.