If you sell software services to any organisation of size, you know the document. It is the spreadsheet with between eighty and four hundred rows. It asks about encryption at rest, acceptable use of removable media, and a long list of other controls.
It usually arrives late in a sales process. It is filled in under time pressure. The person answering is often whoever is available, rather than whoever knows the details best.
We got tired of answering the same questions forty times a year, so we published the answers. That decision had more interesting consequences than we expected. Most of those consequences are about how we work, rather than about sales.
The spreadsheet gives buyers a repeatable process
A spreadsheet cannot establish, by itself, whether a supplier is secure. It establishes whether the supplier has a person who can fill in a spreadsheet. That does correlate with organisational maturity. It correlates much more weakly with whether the software will be built well.
That is a limitation of the format, rather than a criticism of the people who send it. A procurement function needs a repeatable, defensible process. At scale, a standard questionnaire is the only instrument available.
The problem is on the supplier side. The format rewards answers that pass the process, even when a fuller answer would be more true. When the buyer needs a yes or no answer and the supplier needs the work, the safest answer is often the least specific one.
Vague answers are easier to pass
Take a typical row: do you encrypt data at rest. The answer that passes is yes. The answer that is true is longer.
A useful answer says which data is encrypted, at which layer, how the keys are managed, when those keys are rotated, and who can access them. That detail matters because the same word can describe very different controls.
A supplier who writes yes is answering the question as asked. The question was not precise enough to separate two very different situations. Full disk encryption on a cloud volume protects against a stolen disk and nothing else. Application-level encryption of specific fields with keys the client controls is an enormously different security posture.
The questionnaire scores those two answers identically. That means the document rewards the supplier who is least specific, although the buyer usually needs the most specific answer.
Public answers make vague claims harder to use
When the answers are public and permanent, vagueness stops being safe. You cannot write something convenient for one client and something different for another, because both clients can read the same page. You cannot claim a certification you are working toward, because somebody will check.
Your own team can also read what you have committed to. That effect mattered more than we expected.
Once the answer to a question is published, it becomes a specification. Our documented incident response window is a number engineers can be held to. If a client can quote that number back to us, it becomes a stronger enforcement mechanism than any internal policy.
We include the answers that do not flatter us
The temptation when publishing these answers is to include only the flattering ones. We include the others. There are several.
The unflattering answers fall into a few categories:
- Certifications we do not hold.
- Controls that are appropriate for our size and would be inadequate for an organisation of two thousand people.
- Areas where we rely on a cloud provider’s control rather than operating our own.
That last point is entirely normal, and it is better to say it clearly than to imply otherwise. If a cloud provider operates the control, the honest answer names that dependency instead of making our role sound larger than it is.
Being explicit about those gaps has cost us work, twice that we know of. In both cases, a procurement policy required something we do not have. Both times the process ended in week one rather than week nine. That was a better outcome for everyone, including us.
Many rows assume the wrong kind of supplier
A large proportion of any standard questionnaire assumes the supplier is a SaaS product hosting client data in a multi-tenant platform, meaning one system that serves multiple customer tenants. We are frequently a different shape of supplier.
We build software that the client owns and runs, on the client’s own infrastructure, and hand it over. That changes how many rows should be answered.
Perhaps a third of the rows become either inapplicable or actively misleading. Questions about our data centre are answered by naming the client’s. Questions about our backup schedule for their data are answered by explaining that we do not hold it. Questions about tenant isolation have no meaning when there is one tenant and they own the servers.
The tempting answer is to write not applicable and move on. That is the wrong instinct, because a wall of not applicable reads as evasion to somebody who has to defend the assessment internally.
What works better is answering the concern behind the row. If the underlying concern is whether client data is adequately protected at rest, the honest answer explains who holds it, what we recommended, what was implemented, and what our access to it is during and after the engagement.
That takes four sentences instead of two words. It also ends the exchange rather than triggering a follow-up call.
Access during a build matters most for suppliers like us
The question almost nobody asks is the one we think matters most for a supplier like us: what access we hold and how it ends.
During a build we typically have credentials to a staging environment, sometimes to production for a deployment window, and access to a repository. A staging environment is a test or pre-release system used before production. That is still a meaningful amount of authority over somebody else’s business.
So we publish the shape of that access:
- Access is granted per engagement and per person rather than to the firm.
- Production access is time-boxed to a deployment window.
- The client grants production access rather than us holding it standing.
- Everything is revoked at handover as a checklist item.
- The checklist item has a named owner on the client side.
That last detail matters because revocation performed by the departing supplier is a control with an obvious flaw in it. The supplier should not be the only party responsible for removing the supplier’s own access.
We also state what we retain afterwards. We retain the source we wrote and the decision records. We retain nothing else.
The items we do not retain are specific:
- No copies of production data.
- No exports kept for convenience.
- No long-lived credentials in a password manager that somebody forgot about.
That last category is where most supplier-related incidents actually originate. It is also almost never covered by a questionnaire row.
A few questions tell you far more than the long spreadsheet
Having answered a great many of these questionnaires, we have found that a small number of questions do most of the useful work. They are rarely the questions with the most rows devoted to them.
These five questions have told us the most about a counterparty:
- Who has production access, how is it granted, and how is it removed when somebody leaves? The last clause is where organisations fail. Access accumulates.
- Where are the secrets? This means the actual credentials, keys and tokens, rather than the policy about them. A great many breaches begin with a credential in a repository.
- What happens when a dependency has a critical vulnerability on a Friday afternoon? This tests whether there is a process or a hope.
- Has a restore from backup been performed, and when? A backup that has never been restored is a belief.
- What was your last incident and what changed afterwards? A supplier who says they have never had one is either very new or not counting properly.
Those five questions have told us more about a counterparty than the other three hundred rows combined.
Third parties touching client data need a live list
One row deserves more attention than it usually gets: the list of third parties who will touch client data. These third parties are often called subprocessors, meaning vendors that process data for the supplier as part of the service.
Most suppliers answer with the obvious ones and forget the long tail. The long tail includes services that may receive client data as a by-product of normal work:
- The error tracking service that receives stack traces containing user data.
- The analytics tool.
- The support desk.
- The transcription service used on recorded calls.
We maintain that list as a live document rather than a point-in-time answer. We notify clients when it changes.
A subprocessor list that was accurate at contract signature and has drifted for two years is worse than no list at all. It is a claim of control where none exists.
AI vendors add questions the old format does not ask
The questionnaire format has not caught up with model providers, and it shows. The rows assume a supplier who processes data on request and returns a result. That is broadly true for model providers, but the missing questions matter.
The standard rows usually do not ask about behaviour change, versioning or training data. Those questions do not exist because there was never a category of supplier for whom they mattered.
We answer them anyway, in the same document. Our answer covers:
- Which model providers we use.
- What reaches them.
- What the retention terms are.
- Whether inputs are used for training.
- How we detect when a model’s behaviour changes.
Nobody asks for that last one yet. They will.
Certification is useful, but it leaves important gaps
The most common gap between a serious security posture and a passing questionnaire score is certification. It is better to be clear-eyed about what certification proves and what it does not prove.
A certification demonstrates that an organisation has documented processes and had them audited. It does not demonstrate that the software is well built. The correlation is weaker than procurement frameworks assume.
We have inherited systems from certified suppliers with credentials committed to the repository. We have also worked with uncertified small teams whose engineering discipline was excellent.
Neither observation is an argument against certification. Certification is a reasonable proxy at scale. These observations are an argument for asking at least a few questions that a certificate cannot answer.
Buyers would get more from evidence than from more rows
If we were designing the buyer’s side of this process, we would keep the questionnaire and shorten it substantially. We would add one thing: a request for evidence rather than assertion, on three rows of the buyer’s choosing.
The buyer could ask for evidence on rows like these:
- Show me an access review from last quarter.
- Show me the ticket from your most recent dependency vulnerability, with timestamps.
- Show me the report from your last restore test.
Those requests take a supplier with real practices about ten minutes. They take a supplier without those practices considerably longer. The difference in response time is itself the answer.
Public commitments make later vagueness harder
Publishing the answers is one example of something we try to do more generally. We put the awkward information in front of people before they ask for it.
It is the same reason we state the conventional cost next to our own in proposals. It is also the same reason we publish our method rather than treating it as proprietary.
The mechanism is the same in each case. When you commit to something in public, you remove your own ability to be vague later. That is uncomfortable at the moment of publication and useful every day afterwards.
Our answers are on the trust centre, including the ones we would rather not have written down. More on how we approach security work in cybersecurity services.
