My setup for monitoring prompt injection attempts using regex rules on logged inputs.

Hey everyone! 👋 I've been experimenting with PromptLayer's logging features for the last few months, specifically to add a layer of defense against prompt injection attempts in our customer-facing chat application. While we have guardrails in our main application code, I wanted a secondary, observability-focused safety net that could alert us in real-time. The goal was to catch anything that slipped through before it could cause issues or rack up huge LLM costs.

I realized that by combining PromptLayer's excellent logging of every prompt/response with their dashboard and webhook features, I could set up a simple but effective monitoring system. The core idea is to scan the logged **inputs** for known injection patterns using regex rules. Here's a breakdown of my setup:

**The Components:**
1. **PromptLayer Logging:** Already in place for all our OpenAI calls.
2. **Regex Rule Set:** A curated list of patterns targeting common injection techniques (ignore case, multi-line).
3. **PromptLayer Dashboard Alert:** Configured to trigger on a regex match.
4. **Webhook Integration:** To send alerts to our internal incident management channel.

**My Current Regex Rules (constantly evolving):**
I started with a basic list and have been adding to it based on what we see. These are applied to the `input` field of the logged request.

```python
# This is a simplified version of the regex patterns I'm monitoring for.
# They are joined with the | operator in the dashboard rule.
INJECTION_PATTERNS = [
r"(?i)ignore.*previous|ignore.*instructions", # Classic instruction override
r"(?s)system:.*user:|assistant:.*user:", # Role switching attempts
r"....*instructions.*...", # Encapsulated instructions
r"[.*](.*javascript:", # Basic markup/js injection
r"(?i)your.*goal.*now|from.*now.*on", # Goal redefinition attempts
r"b(do not|don't)b.*b(as an ai|you are)b", # Negation of identity
]
```

**Configuring the Alert in PromptLayer:**
1. Went to the "Monitoring" section in the dashboard.
2. Created a new "Prompt Alert" with the type "Regex Match."
3. Set the field to `request.input`.
4. Pasted in my joined regex pattern.
5. Set the threshold to "Trigger on any match."

**The Payoff - Webhook to Incident Channel:**
This was the best part. I configured the alert to send a webhook to a small Lambda function that formats the alert and posts it to our PagerDuty service. The payload from PromptLayer gives you everything you need:

```json
{
"alert_name": "Prompt Injection Match",
"prompt_id": "pl_...",
"input_snippet": "...the user input that triggered the rule...",
"timestamp": "...",
"permalink": "https://promptlayer.com/..." // Direct link to inspect the full log
}
```

**What This Gets Us:**
* **Real-time visibility:** We get an incident ticket within seconds of a match.
* **Forensic capability:** The `prompt_id` link lets my team instantly see the full conversation chain, model used, costs, and response.
* **Trend analysis:** We can refine our app-level guards based on what patterns are *actually* being attempted.
* **Team awareness:** It gets our support and engineering teams on the same page about the types of adversarial inputs we're seeing.

It's not a silver bullet—determined attackers can craft novel injections—but it has already caught several automated probing attempts and helped us tune our system prompts to be more resistant. It feels like having a security camera on our LLM traffic.

Has anyone else built similar monitoring on top of PromptLayer's logs? I'm particularly curious if you've found other useful regex patterns or have taken a different approach, like statistical anomaly detection on input length or token counts.

—jr