The Limits of Vendor Security Questionnaires
The security questionnaire is the most widely used instrument in third-party risk management and the least interrogated. It is self-attested, point-in-time, unverified and easy to answer aspirationally — and yet it still has two jobs that nothing else can do. Knowing which two is the whole skill.
Where questionnaires came from
Questionnaires solved a real problem in a world with no alternatives. If you needed to know whether a supplier encrypted data at rest and had no way to observe them, asking was the only option — and putting the answer in writing created a record and a degree of accountability. ISO/IEC 27036 formalized this kind of supplier information exchange as part of managing supplier relationships.[4]
What has changed is not that asking became useless. It is that asking is no longer the only way to know, while questionnaire volume has grown to the point where the instrument collapses under its own weight — vendors receiving hundreds a year, buyers receiving answers nobody has capacity to read.
The four structural limits
These are not implementation problems to be fixed with a better template. They are properties of the instrument itself.
- Self-attestation — The answers come from the party with an interest in the outcome, usually from someone who cannot personally verify them. "Do you enforce MFA?" is answered from policy, not from a coverage report.
- Point-in-time — A questionnaire describes the day it was completed. Vendor posture changes weekly — acquisitions, staff departures, new internet-facing systems, newly disclosed vulnerabilities. The document ages into fiction while remaining on file as evidence.
- Unverifiable at scale — Verifying a single answer properly requires evidence review or audit. Across 400 vendors, verification does not happen, so the whole population is effectively trusted on assertion.
- Aspiration bias — Questions written as controls invite answers describing intent. Policies exist; enforcement coverage, exception volume and tested recovery times are what actually matter — and questionnaires rarely distinguish them.
What external evidence sees instead
Externally observable evidence is the mirror image of a questionnaire: it is unverifiable-by-assertion proof, continuously refreshed, requiring no cooperation — but blind to everything not visible from outside.
| Property | Questionnaire | External evidence |
|---|---|---|
| Source | The vendor | Direct observation |
| Verification | None, unless separately audited | Inherent — the finding is the evidence |
| Freshness | Annual at best | Continuous |
| Cooperation needed | Yes — and often slow | No |
| Scales to 400 vendors | No | Yes |
| Sees internal controls | Claims to | No |
| Sees actual exposure | No | Yes |
| Gaming risk | High | Low — findings are observed, not declared |
Notice that neither column dominates. Evidence cannot tell you whether a vendor tests its backups; a questionnaire cannot tell you that the vendor has an unpatched, known-exploited vulnerability on an internet-facing server right now — which is precisely the class of finding most strongly associated with real incidents.[3][5]
What questionnaires are still for
Two jobs, and they are genuinely important.
- Seeing inside — Internal segmentation, backup testing cadence, joiner-mover-leaver process, privileged access management, subprocessor governance, incident response rehearsal. None of these are externally observable. If you need to know them, you must ask — and then verify through evidence or audit.
- Creating obligation — A written answer is a contractual and legal artefact. It establishes what the vendor represented, which matters for liability, and it forces an internal conversation on the vendor's side. NIST's supply chain guidance treats this documentation role as part of a broader programme, not as the assessment itself.[1]
Third-party attestations occupy a middle position worth understanding: a SOC 2 Type II report is independently examined over a period, which is stronger than self-attestation — but only within the scope and system boundary the vendor defined.[2] Always read the scope section before the opinion.
A better operating model
The sequence matters more than the instruments. High-functioning programmes invert the traditional order:
- 1. Score the whole population externally — Every vendor gets a measured baseline immediately, with no vendor effort at all. For the majority of vendors, this is the entire assessment.
- 2. Tier by consequence — Rank by what the vendor could cost you, not by spend — the logic set out in building a third-party programme.
- 3. Question only what evidence cannot answer — For critical vendors, replace the 300-question template with a short, pointed set aimed at internal controls — and at anything the external evidence flagged.
- 4. Demand artefacts, not assertions — Coverage reports, dated recovery test results, penetration test summaries with remediation status.
- 5. Monitor continuously thereafter — The questionnaire's expiry problem is solved by monitoring, not by asking again next year.
Twelve questions worth asking
When you do question a critical vendor, ask for numbers and dates rather than yes/no. What percentage of administrative accounts enforce phishing-resistant MFA, and what is on the exception list? When did you last restore production data from backup, and how long did it take? What is your median time to patch internet-facing critical vulnerabilities, measured over the last quarter? Which subprocessors can access our data, and how are they assessed? What was your last security incident, and what changed as a result? Who can access our data, and how is that access reviewed?
Twelve questions of that character, sent to a vendor you have already measured externally, will tell you more than any standard template — because you already know where to press. That combination of continuous external measurement under our patented method[6] and targeted human enquiry is the model Xcigence is built around; see third-party cyber risk scoring.
Scoring vendors, suppliers, and downstream providers on one methodology.
References
- [1]NIST. SP 800-161r1 — Cybersecurity Supply Chain Risk Management Practices
- [2]AICPA. SOC 2 — Trust Services Criteria and report scope
- [3]Verizon. Data Breach Investigations Report (DBIR)
- [4]ISO. ISO/IEC 27036 — Information security for supplier relationships
- [5]CISA. Known Exploited Vulnerabilities (KEV) Catalog
- [6]USPTO. Granted patent US 11,418,531 (application US 16/822,691) — Global Dossier record
Continue reading
A six-stage blueprint: inventory, tiering, assessment depth, contractual controls, continuous monitoring and offboarding — with what to automate at each stage.
Your vendors have vendors. Concentration risk in shared cloud, identity and file-transfer providers is now the dominant mode of systemic cyber failure.
A cybersecurity risk score turns observable security evidence into a standardized number decision-makers can compare, price, and act on. Here is what goes into one and what makes it defensible.
See your own cyber risk score
Xcigence computes a standardized 300–850 cyber risk score for your organization and every vendor in your ecosystem — no agents, no questionnaires.