The Rocky Path of Managing AI Security Risks in IT Infrastructure

Written by David Balaban.

Today, most people associate artificial intelligence (AI) with its generative facet manifested through services that create images, text, videos, and software code based on human input. While that’s arguably the most popular option among end-users, AI shows a lot of promise across many other areas due to its unmatched power in traversing and analyzing heterogeneous clusters of information, automating processes, and making intelligent decisions.

One of these areas is data center management in enterprise environments, including servers, network components, and everything in between. The tech can do all the heavy lifting for predictive maintenance, resource allocation, and security. It reduces the grunt work in helping to identify and resolve issues before they affect entities at the core of an organizational IT infrastructure.

The benefits are clear, but as is often the case, there’s another side to these booming technological advancements. AI introduces new security risks that companies must manage. For IT and security teams, understanding and mitigating these risks is essential to ensure that AI works as an asset, rather than a liability, in the IT ecosystem.

Let’s explore the pitfalls of AI adoption across digital infrastructures and the ways for organizations to detect and actively prevent them.

New Tech, New Threats

AI systems, particularly those used for machine learning (ML), process large volumes of data to make predictions and automate decisions. As these systems become more deeply ingrained into modern IT infrastructures, their vulnerabilities create a new threat landscape and can be exploited by threat actors in ways that were previously unthinkable.

One major source of concern is adversarial attacks, where malefactors subtly modify input data to hoodwink AI models. For instance, they could alter malware signatures just enough to slip below the radar of AI-based detection systems, or they could trick facial recognition software by changing images fed into it. To top it off, AI models themselves can end up in the crosshairs, which leads to inaccurate or unsafe decision-making.

Another challenge lies in data poisoning attacks, where cybercriminals game the data used to train an AI model. The fallout from this exploitation is a corrupted model that behaves unpredictably. Detecting this foul play is an arduous task because compromised systems may not immediately exhibit signs of attack.

Lastly, there’s AI bias. This is an oft-overlooked risk, yet it can have significant repercussions in security contexts. If an AI model is trained on biased data, its predictions might lead to discrimination, flawed threat detection, or unintended consequences that may go unnoticed without proper oversight.

From Detection to Prevention

Many organizations rely on AI for threat detection, and clearly, that’s half the battle. However, adversaries always stay a step or two ahead. Instead of focusing on detection, IT and security teams must build their AI risk management around proactivity, which translates into a prevention-centric approach. Here’s how the organization can gain the upper hand.

Conduct Regular AI Audits

AI systems evolve over time, and their behavior changes as they process new data. Periodic audits help ensure that AI models are working as expected and aren’t being influenced by threat actors. These activities should focus on the following aspects:

• Model integrity: Verify that AI models haven’t been tampered with or corrupted.
• Input/output analysis: Examine data inputs and model predictions for signs of adversarial manipulation or bias.
• Performance evaluation: Monitor the accuracy of AI systems and make sure that they continue to meet security standards.

Regular assessments allow security teams to spot signs of compromise early and take corrective action before a full-fledged attack happens.

Develop a Robust AI Governance Framework

A well-thought-out governance framework enforces security controls around AI usage within the organization. This boils down to setting policies for how AI models are developed, deployed, and monitored. The fundamentals of this framework are as follows:

• Access controls: Restrict who can modify AI models and the data used to train them.
• Model versioning: Keep track of different iterations of AI models to pinpoint changes and quickly roll back to previous versions if issues arise.
• Incident response plans: Ensure that security teams know how to act if an AI model is compromised, which includes identifying potential threats and recovering corrupted models.

Effective AI governance is a joint effort by data scientists, developers, and security teams aiming to design AI models with security in mind from the beginning.

Enhance Model Transparency and Interoperability

One of the challenges in managing AI security risks is the black-box nature of these systems. The reasoning behind decisions made by complex models, such as deep neural networks, can be hard to interpret. This, in turn, complicates the detection of malicious behavior or biases.

To prevent risks, organizations should prioritize enhancing the transparency and interoperability of AI models in use. Techniques like explainable AI (XAI) offer insights into how models make decisions so that IT teams can tell when models exhibit unexpected behavior. Not only does XAI implementation help detect vulnerabilities, but it also provides clues on how to step up the robustness of AI models.

Shield the AI Supply Chain

AI development tends to rely on third-party tools, open-source libraries, and pre-trained models. All of them can introduce vulnerabilities. Here are a few tips to mitigate these risks:

• Vet AI vendors: Ascertain that third-party providers follow rigorous security standards and offer transparency around their development workflows.
• Exercise due diligence with open-source tools: Regularly monitor such building blocks of your AI systems for security gaps and apply patches as soon as they become available.
• Test pre-trained models: Thoroughly evaluate third-party models to spot potential biases or weaknesses before deploying them in production environments.

Vulnerabilities are often inherited from external sources these days, so it’s in the organization’s best interest to stay on top of the AI supply chain.

Implement Continuous Monitoring and Anomaly Detection

An AI model shouldn’t be treated as a “set it and forget it” type of thing. Once deployed, it should be continuously monitored to detect anomalies that might signal an attack. Interestingly, security teams can leverage AI itself to perform this real-time checking.

Anomaly detection can unveil subtle changes in AI model performance, data inputs, or outputs that might suggest tampering. For instance, if an AI-powered malware detection system suddenly starts allowing suspicious files through, this could indicate an attack.

In addition to monitoring, organizations must establish logging and reporting mechanisms to track model behavior over time. This provides a trail that security teams can analyze if an incident occurs, allowing them to find the root cause more quickly.

Endnote

AI is an invaluable tool for modern IT infrastructure, but its power comes with responsibility. Managing the associated risks requires a paradigm shift from reactive detection to proactive prevention. Regular audits, constant efforts to improve model transparency, AI supply chain security, continuous monitoring, and a well-thought-out governance framework are the pillars of this strategy.

About the Author

David Balaban is a cybersecurity analyst with two decades of track record in malware research and antivirus software evaluation. David runs Privacy-PC.com and MacSecurity.net projects that present expert opinions on contemporary information security matters, including social engineering, malware, penetration testing, threat intelligence, online privacy, and white hat hacking. David has a solid malware troubleshooting background, with a recent focus on ransomware countermeasures.