South Africa’s withdrawn draft National Artificial Intelligence Policy leaves an awkward truth for business leaders: a document can look orderly and read cleanly, yet still be built on nonsense at the source level. The draft had already cleared Cabinet and gone out for public comment before officials found that its reference list included fictitious, apparently AI-made citations. This was a control failure, not a debate about AI’s usefulness. Many companies are exposed to the same mistake every time they let a model assemble research, references, or supporting material without human verification.
The communications department has since put an independent panel in place. A revised policy is expected to go back to Cabinet in November 2026, then return for public comment in January 2027. The timeline matters, but the bigger lesson lands much closer to home for tender teams, analysts, legal departments, and founders who have started treating generative tools like junior researchers with perfect recall. They are drafting engines, and drafting speed is not the same thing as evidence.
The real failure sat in the footnotes
The embarrassing part of the policy withdrawal is not that the draft was badly written. By all accounts, it was structured, coherent, and good enough to move through several layers of review before the citation problem surfaced. This is exactly why it should worry business people. A polished document can still collapse if the underlying references are invented, misread, or simply never checked.
This is a familiar risk in board packs and working drafts. A market update can look credible because the charts are tidy and the prose is confident. A tender submission can sound responsive because the language is sharp and the formatting is polished. A legal summary can read as though it has been checked twice. None of that tells you whether the sources exist, whether the numbers line up, or whether the quoted material appears in the original text.
The policy incident shows how thin the safety net can be when everyone assumes someone else checked the facts. Cabinet approval did not catch it. Publication for comment did not catch it. The failure sat lower down, where citations should have been tested against the original material. For business, that is the warning light. If government can publish a draft with fake references, companies can absolutely send out a report with fake certainty.
AI drafts fast, but it also invents fast
Generative tools are already being used for tender responses, legal summaries, technical reports, investor decks, and market research notes. This usage is not the problem. The problem starts when teams treat the output as if the model had somehow verified its own claims.
It has not.
A model can draft a convincing paragraph around a real statistic, then attach the wrong source. It can quote a regulation that sounds right and get the section number wrong. It can produce a neat-looking summary of a competitor report that never existed. In a tender context, that can be enough to lose the bid. In legal work, it can mislead the whole line of advice. In investor material, it can create reputational damage that is expensive to unwind.
South African businesses do not need a grand theory about AI risk here. They need one practical, blunt rule: if a machine helped write the sentence, a person must check the evidence behind it. No exceptions for urgency, for a polished draft, or because the model sounded confident.
That standard should apply to every external fact in the document.
- Every statistic needs a source you can open.
- Every quotation must appear verbatim in the original material.
- Every regulation should be checked against the issuing authority, not a summary blog or a copied paragraph.
- Every named source needs a human to confirm that it actually exists.
- Every final version needs a responsible person attached to it.
The quickest way to get into trouble is to assume that fluency equals accuracy. It does not. AI is very good at producing something that reads like research. It is much less reliable at producing research.
What a sensible verification workflow looks like
A useful process does not need to be complicated. It needs to be disciplined. The point is to remove guesswork from the final mile, where most of the damage happens.
Start with source capture. If the AI tool names a report, article, regulation, court case, or dataset, that reference has to be written down in a way a human can track. Include title, author, publication, date, URL or document name, and page number if there is one. If the model cannot provide enough detail to find the source again, the claim should not survive into the final draft.
Then open the source yourself. Not a summary. The original document. That means the PDF, the official notice, the company report, the court judgment, or the regulator’s publication. If the cited material cannot be found, the claim is out. If it can be found but the wording is off, the claim is revised or removed. If the statistic is quoted in the wrong context, it needs to be corrected before the draft moves on.
That sounds basic because it is basic. It is also where people get lazy.
A workable review sequence should include these checks:
1. Confirm the source exists and is accessible. 2. Match the statistic, date, name, and unit exactly. 3. Check that direct quotes are copied correctly. 4. Verify that legal or regulatory references come from the official issuer. 5. Keep a record of who checked what and when. 6. Sign off only after the evidence trail is complete.
This is not busywork. It is the difference between a draft and a document that can survive scrutiny. A company that cannot show its working is trusting memory and confidence in places where evidence should sit.
The sign off has to belong to a person
One of the easiest mistakes to make with AI-assisted work is to spread accountability so widely that nobody actually owns it. The model drafted it. The analyst cleaned it up. The manager skimmed it. The team assumed the source list had been checked by someone else. That is how errors make it into public documents, client work, and internal decisions.
The fix is simple in principle and irritating in practice. A named human should be responsible for the final version. Not a department. Not a tool. A person.
That person should be able to point to the evidence store behind the document, whether that is a shared folder, a document management system, or a structured archive with screenshots and links. The archive should show the sources that were checked, the date they were checked, and the final version that was approved. If a fact was revised, the record should show why.
This matters most in South African business settings where one bad reference can cause a real knock-on effect. A tender response built on a phantom standard can be disqualified. A finance note that leans on the wrong figure can send a management team in the wrong direction. A compliance summary that paraphrases a rule instead of checking the regulation can create exposure the company did not budget for.
The lesson from the policy withdrawal is harsh but useful. AI can accelerate drafting. It cannot certify truth. Once that line is clear, companies can use the tool without confusing speed for proof. That means building a workflow where humans own the verification, evidence is stored, and sign-off carries a real name.
Anything looser than that is not an AI strategy. It is a waiting room for embarrassment.
