LLM Security: Going Beyond Firewalls

By Neal Swaelens, Head of Product, LLM Security, Protect AI

If 2023 is the opening act for large language models (LLMs) in the enterprise, 2024 would be the main event, focusing on scaling production use cases, increasing investments, and delivering initial returns on investment for businesses. According to industry projections, spending on LLMs will grow 2.5x from $7M in 2023 to $18M in 2024. 

This adoption pattern shows a split in usage between LLM enterprise solutions provided by third-party vendors, such as GitHub Copilot and JetBrains, and the internal applications built using API-based models (GPT-4.5, Claude, etc.) or internally deployed open-source LLMs (e.g., Mistral, LlaMa-2, etc.). To date, most applications have focused on internal-facing use cases driven by concerns over security and data protection associated with customer-facing implementations.

Nevertheless, we have already seen several security incidents affecting external-facing applications – and one that involved a lawsuit. While most of these have resulted in temporary brand blemishing, it is a warning of what the future holds. The blast radius of these attacks will expand significantly as upstream and downstream services, such as Retrieval-Augmented Generation (RAG), and Agents become more readily adopted. Current attacks show the relative ease with which LLMs can be forced to do what they should not. 

Related:Hundreds of LLM Servers Expose Corporate, Health & Other Online Data

To address these risks, an in-line approach to LLM security has emerged, typically referred to as an LLM Firewall. Like a reverse proxy that sits in between an application and LLMs, an LLM Firewall performs real-time inspection, using multiple scanners, to detect security risks ranging from sensitive data leakage, adversarial prompt attacks, and integrity breaches. 

Limitation of LLM Firewalls

Despite their benefits, LLM Firewalls are struggling to scale security efficiently to keep pace with the usage and growth of LLM applications within organizations for the following reasons:

As organizations continue to scale monthly interactions within their LLM applications, a new approach to security is needed that shifts from active blocking to LLM threat detection and response across the entire value chain of the LLM. This will involve expanded detection on multi-modality and the upstream and downstream detection of threats within RAG and Agency.

Related:LLM Hallucinations Are Inevitable. Limiting Them Requires a Trusted Data Foundation.

What is LLM Threat Detection and Response?

Unlike LLM Firewalls, which primarily focus on blocking known threats, LLM Threat Detection and Response is an advanced security framework designed to identify and mitigate threats continuously within LLM environments. This approach uses threat detection scanners to scrutinize interactions within and around LLMs. It ensures comprehensive protection beyond threats within the input and output of the LLM and the action a model takes in connected systems.

LLM Threat Detection and Response provides several advantages over LLM Firewalls, including:

Related:Master AI Cybersecurity: Protect and Enhance Your Network

Implementation Best Practices

As LLM Threat Detection and Response evolves, organizations must adopt best practices for security. This requires proactive integration of solutions to capture end-to-end telemetry on all LLMs, including sessions, API calls, and generated code. This approach future-proofs complex use cases like retrieval augmented generation and agency in connected systems.

Here are some practices for implementing robust LLM Threat Detection and Response:

By moving beyond traditional firewalls and adopting LLM Threat Detection and Response, organizations can ensure robust protection for their LLM applications and full visibility into threats across the entire value chain. This will allow them to confidently and securely adopt advanced LLM use cases at scale.

About the Author

Neal Swaelens is Head of Product, LLM Security for Protect AI. He is an expert on securing large language models (LLMs), machine learning, and AI. Previously, he founded Laiyer AI, a leading open-source LLM security product acquired by Protect AI. Laiyer AI's threat detection models for Large Language Models on HuggingFace, the leading platform for building ML models, have been downloaded more than 2.5M times.