Allow/deny lists

Allow/deny lists are access control mechanisms that explicitly define which URLs, domains, or actions an agent is permitted or forbidden to access. These lists serve as the foundation for implementing security boundaries in autonomous systems by creating explicit allowlists (permitted resources) or denylists (forbidden resources) that constrain agent behavior.

Why it matters

Allow/deny lists provide essential scope control for autonomous agents operating in production environments. Without explicit boundaries, agents can inadvertently access sensitive internal systems, attempt operations on unauthorized domains, or trigger actions that violate compliance requirements.

Security boundaries established through allow/deny lists prevent agents from navigating to admin panels, internal dashboards, or payment processing pages unless explicitly authorized. A customer support agent, for example, should be restricted from accessing */admin/* paths even if the underlying action planning model suggests navigating there.

Data leakage prevention represents a critical use case. Deny lists can block agents from submitting data to external logging services, analytics platforms, or third-party APIs that weren't part of the intended workflow. An agent extracting customer data should be denied access to domains like webhook.site, requestbin.com, or other data exfiltration vectors.

Compliance enforcement becomes tractable when agents operate within well-defined boundaries. Healthcare agents can be restricted to HIPAA-compliant domains, financial agents to PCI-DSS approved systems, and data processing agents to GDPR-compliant infrastructure through explicit allow/deny lists that encode regulatory requirements.

Concrete examples

Domain restrictions

The most common implementation uses domain-level controls to constrain agent navigation:

{
  "allowlist": [
    "example.com",
    "*.example.com",
    "docs.partner.com"
  ],
  "denylist": [
    "admin.example.com",
    "*.internal.example.com",
    "localhost",
    "127.0.0.1"
  ]
}

This configuration permits the agent to access the main domain and all subdomains except those explicitly denied. The denial of localhost and internal addresses prevents the agent from probing local services or internal infrastructure.

URL pattern matching

More granular control requires path-level matching that considers the full URL structure:

const denyPatterns = [
  /^https:\/\/.*\/admin\/.*/,
  /^https:\/\/.*\/api\/delete\/.*/,
  /^https:\/\/.*\/settings\/billing/,
  /^https:\/\/.*\?.*[&?]token=.*/
];

function isUrlAllowed(url: string): boolean {
  return !denyPatterns.some(pattern => pattern.test(url));
}

This pattern-based approach blocks access to administrative interfaces, destructive API endpoints, billing pages, and URLs containing authentication tokens that could be leaked through logs or traces.

Action filtering

Beyond navigation, allow/deny lists can control specific agent actions:

ALLOWED_ACTIONS = {
    "click": ["button", "a", "input[type='submit']"],
    "type": ["input[type='text']", "input[type='email']", "textarea"],
    "scroll": ["*"],
}

DENIED_ACTIONS = {
    "click": ["a[href*='delete']", "button[data-action='remove']"],
    "type": ["input[type='password']", "input[name='ssn']"],
}

def can_perform_action(action_type: str, selector: str) -> bool:
    if selector in DENIED_ACTIONS.get(action_type, []):
        return False
    return selector in ALLOWED_ACTIONS.get(action_type, [])

This action-level filtering prevents the agent from clicking delete buttons, typing into password fields, or interacting with sensitive form elements regardless of the page domain.

Common pitfalls

Overly broad patterns

Wildcard patterns that are too permissive undermine security controls. A pattern like *.com allows access to virtually any commercial domain, while *example* would inadvertently match malicious-example-phishing.net. Always anchor patterns and use specific matching criteria.

Missing subdomain rules

A common error is allowing example.com without considering subdomain implications. If the allowlist contains only the apex domain, agents may be blocked from legitimate subdomains like app.example.com or api.example.com. Conversely, allowing *.example.com without denying admin.example.com creates security gaps.

Regex complexity

Complex regular expressions become maintenance burdens and introduce performance penalties when evaluating every agent action. A pattern like /^https?:\/\/(?!.*(?:admin|internal|staging)).*\.example\.com(?::\d+)?\/(?!(?:api\/delete|settings\/billing)).*$/ is difficult to audit, slow to execute, and prone to subtle bypass vulnerabilities. Prefer simple, explicit patterns over clever regex.

Evaluation order issues

When both allowlists and denylists exist, the evaluation order determines behavior. Without a clear precedence rule, an entry might appear in both lists, creating ambiguity. Most implementations should evaluate denylists first (deny-by-default) or allowlists first (allow-by-default), but never both without explicit precedence.

Forgotten protocol variations

Patterns that match only https:// URLs fail to block http://, ws://, wss://, or ftp:// variants of the same domain. Always consider protocol variations or normalize URLs before evaluation.

Implementation

List structure

Effective allow/deny lists balance simplicity with expressiveness:

interface AccessControlList {
  // Simple domain strings for exact matching
  domains: {
    allow: string[];
    deny: string[];
  };

  // Pattern-based matching for complex rules
  patterns: {
    allow: RegExp[];
    deny: RegExp[];
  };

  // Path-specific rules that apply to specific domains
  pathRules: {
    domain: string;
    allowPaths: string[];
    denyPaths: string[];
  }[];
}

This structure separates concerns: simple domain matching for performance, pattern matching for flexibility, and path-specific rules for granular control within trusted domains.

Evaluation order

Implement a clear precedence hierarchy to resolve conflicts:

function isAccessAllowed(url: string, acl: AccessControlList): boolean {
  const parsedUrl = new URL(url);

  // 1. Check explicit denylists first (highest priority)
  if (acl.domains.deny.includes(parsedUrl.hostname)) {
    return false;
  }

  if (acl.patterns.deny.some(pattern => pattern.test(url))) {
    return false;
  }

  // 2. Check path-specific deny rules
  const pathRule = acl.pathRules.find(r => r.domain === parsedUrl.hostname);
  if (pathRule && pathRule.denyPaths.some(p => parsedUrl.pathname.startsWith(p))) {
    return false;
  }

  // 3. Check allowlists
  if (acl.domains.allow.includes(parsedUrl.hostname)) {
    return true;
  }

  if (acl.patterns.allow.some(pattern => pattern.test(url))) {
    return true;
  }

  // 4. Check path-specific allow rules
  if (pathRule && pathRule.allowPaths.some(p => parsedUrl.pathname.startsWith(p))) {
    return true;
  }

  // 5. Default deny if no explicit allow found
  return false;
}

This deny-first, then allow, then default-deny approach prevents bypass through conflicting rules.

Wildcard patterns

Implement wildcard matching that handles common cases without regex complexity:

function matchesWildcard(domain: string, pattern: string): boolean {
  // Convert wildcard pattern to regex
  // *.example.com matches api.example.com but not example.com
  // **.example.com matches both subdomains and apex

  if (pattern.startsWith("*.")) {
    const baseDomain = pattern.slice(2);
    return domain.endsWith("." + baseDomain);
  }

  if (pattern.startsWith("**.")) {
    const baseDomain = pattern.slice(3);
    return domain === baseDomain || domain.endsWith("." + baseDomain);
  }

  // Exact match
  return domain === pattern;
}

This wildcard implementation covers 90% of use cases without requiring users to write regular expressions.

Dynamic list updates

Support runtime updates for responding to emerging threats:

class DynamicAccessControl {
  private allowlist: Set<string>;
  private denylist: Set<string>;
  private listVersion: number;

  constructor(initial: AccessControlList) {
    this.allowlist = new Set(initial.domains.allow);
    this.denylist = new Set(initial.domains.deny);
    this.listVersion = 0;
  }

  addToDenylist(domain: string): void {
    this.denylist.add(domain);
    this.listVersion++;
    this.logUpdate("deny", domain);
  }

  removeFromAllowlist(domain: string): void {
    this.allowlist.delete(domain);
    this.listVersion++;
    this.logUpdate("revoke", domain);
  }

  private logUpdate(action: string, domain: string): void {
    console.log(`[ACL v${this.listVersion}] ${action} ${domain}`);
  }
}

Dynamic updates enable security teams to respond to incidents without redeploying agents, while version tracking supports audit trails.

Key metrics

Blocked request rate

The percentage of agent requests blocked by allow/deny lists indicates both security effectiveness and potential false positives:

blocked_request_rate = (blocked_requests / total_requests) × 100

A rate < 1% suggests either very permissive lists or well-aligned agent behavior. A rate > 10% indicates either overly restrictive rules or agents attempting inappropriate access. Track this metric segmented by block reason (domain, path, action) to identify patterns.

False positive blocks

Legitimate agent operations incorrectly blocked by access controls:

false_positive_rate = (incorrect_blocks / total_blocks) × 100

This requires human review or agent failure analysis to identify cases where blocks prevented valid task completion. A false positive rate > 5% suggests rules need refinement. Track specific patterns that cause false positives to iteratively improve list quality.

Coverage percentage

The proportion of agent actions validated against allow/deny lists:

coverage = (validated_actions / total_actions) × 100

Perfect coverage (100%) means every agent action passes through access control. Coverage < 95% indicates gaps where agents operate without constraints, creating security blind spots. Identify which action types or workflows bypass validation.

List staleness

Time since last allow/deny list update:

staleness_days = current_date - last_update_date

Lists not updated within 30 days may miss newly identified threats or fail to incorporate evolved application structure. Automate periodic reviews and integrate threat intelligence feeds to maintain current lists.

Enforcement latency

Time required to evaluate allow/deny rules per request:

latency_p95 (microseconds)

P95 latency > 100 microseconds suggests overly complex patterns or inefficient data structures. High latency adds up across thousands of agent actions, degrading user experience. Optimize hot paths and cache evaluation results for repeated patterns.

Related concepts

Allow/deny lists operate as one layer within a broader security architecture. Guardrails provide the overarching framework that incorporates allow/deny lists alongside other safety mechanisms. Policy engines offer more sophisticated decision-making that can dynamically evaluate context beyond static lists.

The distinction between policy and playbook becomes relevant when allow/deny lists are embedded in playbooks as procedural steps versus policies as declarative constraints. Selector stability impacts the effectiveness of action-level deny lists, as unstable selectors may allow agents to circumvent restrictions through alternate element targeting.

Together, these mechanisms create defense-in-depth for autonomous agent systems, with allow/deny lists serving as the foundational access control primitive.