SOX change management evidence: proving your change population is complete
The hardest question in an ITGC audit is not whether changes were approved - it's how you know nothing else reached production. A ticket export cannot answer it. Here's why completeness is the control that fails most often, and how a second, independently generated population closes it.
The AlertKick team
Most teams preparing for a SOX audit brace for the wrong question. They gather approvals, tidy the ticket queue, and get ready to defend individual changes. Then the auditor asks something else entirely:
How do you know this is all of them?
Approvals are the easy half. Completeness is the half that generates findings, and it is genuinely hard, for a structural reason rather than a diligence one.
Why a ticket export cannot answer it
The population you hand over comes out of the change management system. The control being tested is that changes go through the change management system. Those are the same system, so the evidence is circular: it can show that everything recorded was approved, and it can never show that everything that happened was recorded. A change made by someone with production access and no ticket is invisible to it by construction.
Auditors know this, which is why the standard technique is to obtain the population a second time from somewhere else. In application terms that might be a deployment log or a database of installed versions. At the infrastructure layer it means asking the host what actually changed, and comparing.
Anything present in the observed population and absent from the declared one is an exception. That exception list is the deliverable. Producing it is what separates “our change control is effective” from “our change control is documented”.
The three buckets
Reconciling two populations over a period sorts every change into one of three states, and all three are worth reporting.
Authorised. An observed change that falls inside an approved window on the host it targeted. This is the bulk of the list on a healthy estate, and it is the part that lets the auditor stop worrying.
Unauthorised. An observed change with no approved window covering it. The exception list. Each one needs an explanation written at the time, not reconstructed in Q4 from a chat search.
Unobserved. An approved change that ran and altered nothing on disk. Usually a no-op deployment, occasionally a change whose target host was never actually monitored. The second case matters: it means the observed population had a hole in it, and a hole in the population is a hole in the completeness argument.
Reporting only the middle bucket makes the report look cleaner and quietly weakens it. The third bucket is what tells you whether to trust the other two.
Timing is the control, not a detail
A subtlety that decides whether the reconciliation is meaningful: match against the window the change was actually executed in, not the window it was planned for.
A change approved for a 02:00 Saturday window and executed at 14:00 on the following Tuesday is an unauthorised change at 14:00. It had an approval; it did not have an approval for what happened. Reconciling against the planned window launders it into compliance and produces a report that is worse than useless, because it now certifies something false.
The same logic applies to changes that were approved but never started. An approval that was never acted on opened no window and authorises nothing, so activity during its planned slot is still an exception.
Turning “unauthorised” into something computable
The reason unauthorised change is usually a matter of debate rather than a number is that nothing declares the authorised state in advance. If change is only ever recorded after the fact, then every observed change has to be argued about individually, months later, by someone reconstructing intent.
Declaring the change first inverts that. A change request with a window and a target host list, approved before it starts, is a machine-readable statement of what is expected to happen and where. Once that exists, “unauthorised” stops being a judgement call: it is any observed change outside a declared window. The exception list computes itself.
That declaration also makes the preventive control possible. If production access is closed by default and opens only inside an approved window, then the segregation of duties requirement stops being a policy sentence and becomes an enforced technical control with a continuous evidence trail. Auditors rate enforced preventive controls far more highly than documented detective ones, and an automated control backed by effective general controls can often be tested with a sample of one instead of forty.
What this looks like in practice
The loop is: declare, approve, unlock only for the window, observe, re-lock and rebaseline, then verify what actually changed against what was declared.
- Declare and approve. A change request carries a window, the target hosts, and an approver, with a strictly forward lifecycle so an approval cannot be applied after the work started.
- Scoped access. Starting the change opens production access for that window only. Completing it closes access and rebaselines file integrity.
- Observe independently. Throughout, kernel-level file and configuration change capture records what happened without relying on the deployment tooling to report it honestly.
- Verify and reconcile. Per change, compare declared scope to observed changes. Per period, reconcile the whole declared population against the whole observed one and export the exceptions.
The output is the artefact the auditor is actually asking for: two populations generated independently of each other, a reconciliation between them, and a documented explanation for every difference. Most organisations cannot produce the second population at all, which is why this question is where the findings cluster.
A last point worth being blunt about: a completeness report has to state its own limits. If part of the period falls outside the window where change records were retained, the report cannot speak to it, and saying so is the difference between “no unauthorised changes happened” and “no unauthorised changes are visible”. Those are very different sentences to put in front of an auditor, and a report that blurs them is a finding waiting to be found.
For the wider picture of what else gets requested, see SOX evidence gathering, and for the year-round sequence, the SOX audit preparation checklist.
Frequently asked questions
- What is change population completeness in a SOX audit?
- It is the auditor's test that the list of changes you handed over contains every change that reached production in the period, not just the ones that were ticketed. Sampling from an incomplete population proves nothing, so auditors test completeness before they test approvals. The standard technique is to obtain a second population directly from the target system and reconcile it against the ticket population.
- Why is a ticket system export not enough evidence for SOX change management?
- Because it is self-referential. A ticket export proves what was requested through the ticket system, but a change made without a ticket leaves no trace in it by definition. The population and the control being tested come from the same source, so the export cannot demonstrate that nothing is missing from it.
- How do you get a second, independent change population?
- From the target systems themselves rather than the process that was supposed to record the change. File integrity and configuration change records captured at the kernel level work because they do not depend on the deployment tooling cooperating: whatever wrote to disk is recorded regardless of whether a pipeline, a person, or a script did it.
- What counts as an unauthorised change in an ITGC audit?
- Any change observed on an in-scope system with no approved change covering it at the time it happened. Timing matters as much as existence: a change approved for a Saturday window but executed on the Tuesday is unauthorised at the point it was executed, and reconciling against the planned window rather than the actual execution window hides exactly that.
