{"id":93,"date":"2026-11-02T22:35:00","date_gmt":"2026-11-02T22:35:00","guid":{"rendered":"https:\/\/mumbaiorbit.in\/blog\/?p=93"},"modified":"2026-11-02T22:35:00","modified_gmt":"2026-11-02T22:35:00","slug":"aiops-for-security-five-wrong-ideas","status":"publish","type":"post","link":"https:\/\/mumbaiorbit.in\/blog\/tech-guides\/aiops-for-security-five-wrong-ideas\/","title":{"rendered":"Five Things People Get Wrong About AIOps for Security"},"content":{"rendered":"

Ask around about machine learning in security and you will collect confident answers that contradict each other. Part of the confusion is the name itself: AIOps for Security<\/strong> is an editorial label that grew up around machine-learning and LLM-assisted security work \u2014 spotting odd behaviour, scoring alerts, helping investigate \u2014 and it has no standards body, no launch date, and plenty of people using it loosely. Myths thrive in that kind of fog, so here are five of the common ones, each stated the way a colleague would say it at a whiteboard and then corrected with what the published record shows. None of the corrections requires a mathematics degree, and none requires buying anything. If you have nodded along to one or two of these in a meeting, you are in good company; the fixes, happily, are mostly about asking better questions rather than spending money.<\/p>\n

“It’s Just Antivirus With Extra Steps” \u2014 What AIOps for Security Actually Covers<\/h2>\n

The first mistake is treating the whole thing as one product category. It is a bundle of quite different jobs: behaviour-based anomaly detection and user-entity analytics, statistical scoring of incoming alerts, and now LLM-driven investigation agents that read case files and draft conclusions. Each job fails in its own way, and each needs its own evidence. There is also a real dividing line from the better-known IT-operations meaning of “AIOps”: security data is produced by adversaries who adapt to being watched, and a missed detection can cost a breach, not a queue backlog. Those two facts \u2014 hostile, adaptive inputs and breach-scale penalties for misses \u2014 change how any model must be evaluated before it is trusted with decisions, and they explain why borrowed guarantees from the IT world arrive pre-broken.<\/p>\n

“If It Looks Weird, It Must Be an Attack”<\/h2>\n

Anomaly detectors flag whatever differs from a baseline, and most of what differs day to day is life, not crime. People change roles, travel, start jobs, install tools; the baseline has to catch up and, in the meantime, they glow on a dashboard. The operational records repeat this: the majority of anomalies turn out to be ordinary changes and new starters, not intrusions. The correct reading of an anomaly score is “worth a look”, never “guilty”. Teams that let raw anomaly scores page people directly burn out their on-call rotation within a quarter and teach everyone to ignore the pager \u2014 which is the worst outcome, because the one night the alert is real, nobody answers. Context fixes most of this: pair the score with a change calendar, recent joins, and planned maintenance, and half the glow disappears on its own.<\/p>\n

“The Vendor’s Accuracy Figure Will Follow the Product Into Our Building”<\/h2>\n

This one sounds reasonable and fails twice. First, published accuracy often does not survive contact with independent evaluation: the TESSERACT research programme showed headline F1 scores near 0.99 collapsing once spatial and temporal bias were removed from the test setup, and the Arp et al. survey found sampling bias in 90% and data snooping in 73% of thirty top-tier security ML papers. Second, even an honest figure measures someone else’s network, someone else’s attackers, someone else’s mixture of easy and hard cases. Your environment is not their environment, and baselines drift as teams grow and tooling changes. A score close to perfect is itself a warning \u2014 in this field, numbers near 1.0 usually indicate evaluation bias rather than excellence. Ask instead for results on your own labelled sample, however small: pull a hundred recent cases, mark the truth yourself, and see what the product does. Until that exists, treat the vendor’s number as marketing material with decimal points.<\/p>\n

A related trap hides in “helper” features. Submitting suspicious files or links to multi-scanner services feels safe, yet whatever you upload can travel \u2014 VirusTotal’s own terms caution that what you hand over can end up in front of its premium subscribers as well. The harmless-seeming convenience feature is a disclosure channel, and attacker-supplied files are the last thing you want in someone else’s telemetry.<\/p>\n<\/p>\n

“Prompt Injection Is Just a Filtering Bug” in AIOps for Security Tools<\/h2>\n

Familiarity with SQL injection makes people reach for the same toolbox, and the UK’s NCSC has explained at length why that toolbox does not fit: their December 2025 note is literally titled “Prompt injection is not SQL injection”. Language models process instructions and data in the same channel, so an attacker can hide orders inside the very content the assistant was asked to read \u2014 a forwarded mail, a document, a web page. Rephrasing defeats deny-lists endlessly, and the NCSC’s assessment is candid: the model is an inherently confusable deputy, and the weakness may never be fully mitigated. Practical defence therefore shifts to what surrounds the model \u2014 deterministic checks, least privilege, and no outbound channels on anything that reads attacker-controlled text. In 2026, the reminder matters more, because agents now sit closer to real actions than ever before.<\/p>\n

“Once It’s Automated, Nobody Needs to Check Its Work”<\/h2>\n

The last myth dresses up as a budget plan: automate the queue and headcount falls. What actually works is granting autonomy per action class, with evidence calibrated to the risk of each rung. Recommending is cheap \u2014 a system can earn that once it has studied around two hundred labelled alerts, thirty-plus of them genuine. Auto-closing is expensive to prove: errors must already have been bounded below 2% on 150-plus confirmed threats, with decoy cases and a held-out evaluation set in place before the first silent close. Reversible actions require mistakes measured under 1% across 300-plus decisions where acting would have been right, rollback that has been rehearsed, and pace limits that actually bite. Irreversible actions stay human, always \u2014 no sample size, benchmark, or contract clause changes that. Chasing a headline “percent automated” figure skips all of this arithmetic, and MITRE’s assessment of the “AI will run the SOC” dream is blunt: a person still has to evaluate the machinery’s output. Automation removes typing, not accountability, and the accountability is the part that was never redundant.<\/p>\n

A Friendlier Way to Hold These Ideas<\/h2>\n

Notice the pattern across the five corrections: each myth replaces a question with an assumption. The repairs are short enough to keep on a sticky note:<\/p>\n