Skip to main content

Pillar Security

Use Pillar Security for comprehensive LLM security including:

  • Prompt Injection Protection: Prevent malicious prompt manipulation
  • Jailbreak Detection: Detect attempts to bypass AI safety measures
  • PII Detection & Monitoring: Automatically detect sensitive information
  • Secret Detection: Identify API keys, tokens, and credentials
  • Content Moderation: Filter harmful or inappropriate content
  • Toxic Language: Filter offensive or harmful language

Quick Start​

1. Get API Key​

  1. Get your Pillar Security account from Pillar Security
  2. Sign up for a Pillar Security account at Pillar Dashboard
  3. Get your API key from the dashboard
  4. Set your API key as an environment variable: 
    export PILLAR_API_KEY="your_api_key_here"
    export PILLAR_API_BASE="https://api.pillar.security" # Optional, default

2. Configure LiteLLM Proxy​

Add Pillar Security to your config.yaml:

🌟 Recommended Configuration (Dual Mode):

model_list:
  - model_name: gpt-4.1-mini
    litellm_params:
      model: openai/gpt-4.1-mini
      api_key: os.environ/OPENAI_API_KEY

guardrails:
  - guardrail_name: "pillar-minitor-everything"     # you can change my name
    litellm_params:
      guardrail: pillar
      mode: [pre_call, post_call]                   # Monitor both input and output
      api_key: os.environ/PILLAR_API_KEY            # Your Pillar API key
      api_base: os.environ/PILLAR_API_BASE          # Pillar API endpoint
      on_flagged_action: "monitor"                  # Log threats but allow requests
      default_on: true                              # Enable for all requests

general_settings:
  master_key: "your-secure-master-key-here"

litellm_settings:
  set_verbose: true                          # Enable detailed logging

3. Start the Proxy​

litellm --config config.yaml --port 4000

Guardrail Modes​

Overview​

Pillar Security supports three execution modes for comprehensive protection:

ModeWhen It RunsWhat It ProtectsUse Case
pre_callBefore LLM callUser input onlyBlock malicious prompts, prevent prompt injection
during_callParallel with LLM callUser input onlyInput monitoring with lower latency
post_callAfter LLM responseFull conversation contextOutput filtering, PII detection in responses
  • ✅ Complete Protection: Guards both incoming prompts and outgoing responses
  • ✅ Prompt Injection Defense: Blocks malicious input before reaching the LLM
  • ✅ Response Monitoring: Detects PII, secrets, or inappropriate content in outputs
  • ✅ Full Context Analysis: Pillar sees the complete conversation for better detection

Alternative Configurations​

Best for:

  • 🛡️ Input Protection: Block malicious prompts before they reach the LLM
  • ⚡ Simple Setup: Single guardrail configuration
  • 🚫 Immediate Blocking: Stop threats at the input stage
model_list:
  - model_name: gpt-4.1-mini
    litellm_params:
      model: openai/gpt-4.1-mini
      api_key: os.environ/OPENAI_API_KEY

guardrails:
  - guardrail_name: "pillar-input-only"
    litellm_params:
      guardrail: pillar
      mode: "pre_call"                       # Input scanning only
      api_key: os.environ/PILLAR_API_KEY     # Your Pillar API key
      api_base: os.environ/PILLAR_API_BASE   # Pillar API endpoint
      on_flagged_action: "block"             # Block malicious requests
      default_on: true                       # Enable for all requests

general_settings:
  master_key: "your-master-key-here"

litellm_settings:
  set_verbose: true

Configuration Reference​

Environment Variables​

You can configure Pillar Security using environment variables:

export PILLAR_API_KEY="your_api_key_here"
export PILLAR_API_BASE="https://api.pillar.security"
export PILLAR_ON_FLAGGED_ACTION="monitor"

Session Tracking​

Pillar supports comprehensive session tracking using LiteLLM's metadata system:

curl -X POST "http://localhost:4000/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer your-key" \
  -d '{
    "model": "gpt-4.1-mini",
    "messages": [...],
    "user": "user-123",
    "metadata": {
      "pillar_session_id": "conversation-456"
    }
  }'

This provides clear, explicit conversation tracking that works seamlessly with LiteLLM's session management.

Actions on Flagged Content​

Block​

Raises an exception and prevents the request from reaching the LLM:

on_flagged_action: "block"

Monitor (Default)​

Logs the violation but allows the request to proceed:

on_flagged_action: "monitor"

Examples​

Safe requset

# Test with safe content
curl -X POST "http://localhost:4000/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer your-master-key-here" \
  -d '{
    "model": "gpt-4.1-mini",
    "messages": [{"role": "user", "content": "Hello! Can you tell me a joke?"}],
    "max_tokens": 100
  }'

Expected response (Allowed):

{
  "id": "chatcmpl-BvQhm0VZpiDSEbrssSzO7GLHgHCkW",
  "object": "chat.completion",
  "created": 1753027050,
  "model": "gpt-4.1-mini-2025-04-14",
  "system_fingerprint": null,
  "choices": [
    {
      "index": 0,
      "finish_reason": "stop",
      "message": {
        "role": "assistant",
        "content": "Sure! Here's a joke for you:\n\nWhy don't scientists trust atoms?  \nBecause they make up everything!",
        "tool_calls": null,
        "function_call": null,
        "annotations": []
      },
      "provider_specific_fields": {}
    }
  ],
  "usage": {
    "completion_tokens": 22,
    "prompt_tokens": 16,
    "total_tokens": 38,
    "completion_tokens_details": {
      "accepted_prediction_tokens": 0,
      "audio_tokens": 0,
      "reasoning_tokens": 0,
      "rejected_prediction_tokens": 0
    },
    "prompt_tokens_details": {
      "audio_tokens": 0,
      "cached_tokens": 0,
      "text_tokens": null,
      "image_tokens": null
    }
  },
  "service_tier": "default"
}

Support​

Feel free to contact us at support@pillar.security

📚 Resources​