Proving compliance over time, not just once

A technician turns on MFA for a client's Microsoft 365 account, takes a screenshot, and files it in the evidence folder. Eight months later the client's cyber insurer — or its ISO 27001 auditor — asks for evidence that MFA has been continuously enabled since the last renewal. The screenshot does not prove that. It only proves that at the instant it was taken, the setting was on.

That is the most common gap between what a technician calls evidence and what an auditor calls evidence — and it costs the most at exactly the moment it comes to light, which is during the audit.

What does an auditor call evidence?#

A dated observation, repeated over time, showing that a control held throughout the period concerned — not a one-off statement showing it was working at a given instant. ISO/IEC 27001:2022 formalises this in clause 9.1: the organisation must monitor and measure the effectiveness of its controls, and retain documented information as evidence of the results — not as evidence that a technician once checked the setting.

For a technician, evidence answers "is it enabled". For an auditor, it answers "was it enabled throughout the period under review, and how do you know without having gone and checked every single day".

The audit profession has kept those two questions apart for a long time, and the vocabulary is worth borrowing even if you never sit a certification. On one side, evidence of design: the control exists, it is configured the way the policy says it should be, somebody can show it on screen. On the other, evidence of operating effectiveness: the control produced the expected effect, with no unexplained interruption, for the whole of the period under review. A screenshot answers the first question and leaves the second entirely open. That is exactly the difference between an attestation report covering an instant and one covering a period — and it is the second that an insurer or a regulated client always ends up asking for.

Why is a point-in-time attestation never enough?#

Because it says nothing about what happened before or after the day it was produced. An ISO 27001 certification audit, or an inspection carried out by a supervisory authority, covers a period — usually the past year — not the instant of the audit. A single attestation, however honest at the moment it is signed, literally covers one day out of three hundred and sixty-five.

The risk is not even dishonesty. It is that a point-in-time attestation collected just before the audit describes exactly the least representative period of the year: the one where everybody has just checked, and fixed, whatever was not in order.

There is a more mechanical reason still, and it catches a lot of providers out the first time. An auditor does not read the whole period: they sample it. They pick dates inside the period under review — often without telling you in advance which ones — and ask for evidence of the state of the control on those dates. If your file holds nothing but a screenshot dated the day before the audit, every sampled date is a hole. You cannot fill them in afterwards: a management console shows today's state, not the state on 14 March. The 14 March reading either existed on the day or it does not exist at all.

How does a security setting drift without anyone meaning to break it?#

Through normal use of the system, not through a fault. A user gets locked out of their account, a technician disables MFA temporarily to unblock them on a Friday evening, and nobody thinks to turn it back on come Monday. A management console update resets a group policy to its default value. A new device joins the estate before the compliance policy has been applied to it. A security rule targets a group of users whose membership shifts with hires and departures, with nobody resynchronising the scope.

Each of these events is trivial taken on its own. Added up over twelve months, across dozens of accounts and several tools, they produce exactly what an auditor fears: a control that was true on the day it was checked, and has not been true for three months without anyone noticing.

What makes this kind of drift expensive is not how serious it is, but how quiet. None of it raises an alert, because none of it is a failure: the system is doing precisely what it was told to do. Set side by side, all of it has the same signature.

The right-hand column is the only one that answers the auditor's question. The other three describe today's state, which is useful for running the estate and worth nothing as proof.

What does evidence need in order to survive an audit?#

Five properties, none of them optional: drop one and the file falls over on the next question. The rule that sums them up fits in a sentence — a missing reading is an absence of evidence, not favourable evidence.

Undated evidence proves nothing about a period. Evidence overwritten by the next reading — a dashboard that only shows the latest snapshot — loses the trace of when the control stopped holding. And a day without a measurement must never be counted as a compliant day by default, because the first auditor to notice will call the rest of the file into question too, including the parts that were right.

What does an evidence reading actually look like?#

A row per measurement, not a document. Here is the minimal shape of a usable series for a single control — MFA enforcement on a client's Microsoft 365 accounts — over a handful of days.

Six rows, and the whole story is already legible. On 14 March, one account fell outside the scope of MFA. On the 15th the collection did not run — and the row says so, instead of quietly carrying yesterday's value forward. On the 16th an account was created, which is why the denominator moves from 52 to 53 and coverage drops further even though no additional account was excluded. On the 17th the gap is closed.

A screenshot taken on 17 March would have shown 100% and would have been perfectly honest. It would also have erased the three days before it, including the one where the collection did not run — the single point in the file a conscientious auditor will ask you to explain.

What does an auditor do with a series of dated readings?#

Three things, in this order, and you are better off having anticipated them. First they check the denominator: where the number 52 comes from, how the scope is put together, and what would happen to a machine that appears in no inventory at all. A compliance rate calculated over a population you choose yourself has no probative value, and it is the first thing an experienced auditor tests.

Then they sample. They take two or three dates inside the period and ask you to retrieve, at source, the state your reading asserts. If the row for 14 March says 51 out of 52, they want to know which account was missing and why. A series of readings that does not let you drill back down to the detail does not survive that step.

Finally they look at the gaps and the dips — and this is where a lot of providers have the intuition the wrong way round. A curve sitting at 100% for three hundred and sixty-five days without a single trough does not inspire confidence: it suggests the measurement is not measuring much. A series that shows a dip on 14 March, its cause, and its correction on the 17th describes a control regime that works. ISO/IEC 27001 explicitly provides for that case at clause 10.2: a non-conformity that has been identified must be dealt with, its causes examined, and the corrective action retained as documented information. A gap that is documented and closed is a piece of compliance, not a confession.

The problem is never the gap. It is the gap you discover during the audit because nobody was measuring.

Which controls can be captured automatically, and which never will be?#

Only some of them, and it is better said plainly than left to the implication that a tool replaces governance. Controls whose state lives inside a console you can query can be read continuously; those that rest on a human act or a document have to be proved another way, and there is no honest automation for them.

The right-hand column is not a fallback: it is the same requirement applied to evidence of a different nature. An undated access review report has exactly the same defect as an undated console screenshot.

What should be collected continuously rather than reconstructed at audit time?#

Regular technical readings of every control that can be checked automatically — MFA enforced, backup completed successfully, endpoint protection agent present, patches up to date — kept individually rather than aggregated into a single indicator that flattens the history. The practical difference is easy to state: instead of answering "yes, MFA is on" once a year, you can answer "MFA was enforced on at least 95% of endpoints on 91 of the last 92 days", with the date of the day that slipped and what happened that day.

The second answer is verifiable. The first is not — it asks the auditor to take your word for it, which is neither their job nor yours to ask of them.

The same reasoning applies to decisions, not only to measurements. A compliance register that shows nothing but its current state has the dashboard's defect: it does not say when an entry moved from "non-compliant" to "compliant". A history that keeps the old status, the new status and the date of the switch answers a question auditors ask as a matter of routine — how long has this gap been open, and what has been done about it in the meantime.

How long should these readings be kept?#

At minimum the period the audit covers, and in practice longer. The period under review for an ISO 27001 audit is usually the past year, but the length of a certification cycle and the exact content of surveillance audits depend on the scheme applied by the certification body: ask them which period to cover, not a blog article. On the insurance side, the question comes up at renewal, so every year, over the preceding twelve months.

The practical rule that keeps you out of trouble: keep one full period more than you are asked for today. A file that begins on exactly the day the audit begins gives the impression — often an unfair one — of having been assembled for the occasion. And keep the readings in the form they were produced in, with their original timestamps: an export reprocessed, recalculated or reformatted at audit time loses precisely the property that made it evidence.

That is the principle behind Vigicap's continuous drift module: every connector reading is kept, never overwritten, which makes it possible to produce a dated coverage figure like the one above rather than an isolated screenshot. The compliance register follows the same rule: every change of status on an objective keeps the old status, the new one and its date, instead of replacing the previous value.