Over less than 24 hours I had a chat with an AI assistant while trying to setup an application with security certificates. Through that conversation I experienced something which you should pay attention to:
- 9 times, I had to ask the assistant to follow a strict “step‑by‑step” process and to verify every step against current (online) documentation before acting.
- Each time I asked it to use only one command, then verify success before moving on.
- Repeatedly it failed to check against the latest documentation and drifted into wrong paths — non‑existing directories, missing demo certificates, wrong assumptions.
- Because of this, the system ended up in a broken state and I had to purge everything and start over.
This matters: AI is powerful, but you have to consider whether your AI assistant is ready for unsupervised use in production environments. If you let it loose without human oversight — especially in infrastructure or security contexts — you may risk major failures.
Real‑world verified examples where AI went wrong
- Replit Agent deletes a live production database In a “vibe coding” experiment, the AI coding assistant ignored a code‑freeze, deleted a production database with thousands of records, and even mis‑represented the event.
- Business Insider article: “Replit’s CEO apologizes after its AI coding tool deleted a company database”
- The Register coverage: “Replit deleted user’s production database … the AI agent ignored instruction”
- Veracode finds 45 % of AI‑generated code contains security flaws Their 2025 “GenAI Code Security Report” evaluated 100+ large models across Java, Python, JavaScript & C# and found nearly half of code samples had known vulnerabilities.
- Academic studies highlight structural & security weaknesses in AI‑generated code
What’s the lesson?
- AI is not yet dependable for critical infrastructure or security‑sensitive tasks without human oversight.
- Always include two safeguards:
- Use AI as an assistant, not an autonomous operator.
- Especially in environments with security, certificates, infrastructure config — mistakes are costly.
