How to secure Anthropic SDK applications with an AI gateway

The first version of a Claude application is usually simple: install Anthropic's SDK, add an API key, and send a message. That is exactly how a prototype should start.

Production is less forgiving. The same key ends up in several services. Nobody is sure which workload drove a usage spike. A leaked credential cannot be revoked without interrupting unrelated applications. Security wants an audit trail, while engineering wants to keep the SDK code that already works.

An AI gateway solves that operational problem. The application still speaks Anthropic's Messages API, but it connects through a controlled endpoint instead of calling Anthropic directly.

The use case: a support summarisation service

Consider an internal service that turns long customer-support threads into short case summaries. It runs all day, handles text that may contain personal information, and uses Claude through the Anthropic Python SDK.

The prototype has one ANTHROPIC_API_KEY in its environment. That works, but the credential carries the provider account's authority rather than the application's intended scope. If a second service copies the key, the provider bill no longer tells you which system made a request. If the summariser is compromised, rotating the key may also break every other service that shares it.

The production version needs a narrower contract:

• the summariser gets its own revocable credential;
• its requests stay within a defined rate and spend envelope;
• configured guardrails inspect text before and after the model call;
• each call is attributed to the virtual key, model, provider, usage, and outcome;
• the Anthropic SDK remains the application interface.

That is the part Agent Access Manager plays. It sits in the request path between the SDK and the provider.

The request path

A request moves through five practical stages.

1. The Anthropic SDK sends a Messages API request to Agent Access Manager's /v1/messages endpoint.
2. The gateway authenticates the application's virtual key and resolves its organisation, team, and project scope.
3. Budget, rate-limit, and request-guardrail checks run before the provider call.
4. The configured model alias routes to an approved deployment. For an Anthropic target, the native Messages body can pass through without cross-format translation.
5. Usage and policy events are recorded for reporting and investigation. Configured response guardrails can inspect the model output before it returns to the application.

The provider key stays in the gateway's encrypted catalogue. The support service never receives it.

Connect the Anthropic Python SDK

Anthropic's client SDK accepts a custom base URL. Point it at the gateway and use the virtual key as the SDK credential.

AAM_BASE_URL=https://gateway.example.com
AAM_VIRTUAL_KEY=sk-replace-with-issued-virtual-key
AAM_MODEL=support-summary

import os
from anthropic import Anthropic

client = Anthropic(
    api_key=os.environ["AAM_VIRTUAL_KEY"],
    base_url=os.environ["AAM_BASE_URL"],
)

message = client.messages.create(
    model=os.environ["AAM_MODEL"],
    max_tokens=500,
    system=(
        "Summarise the support case for an internal agent. "
        "Keep dates, product names, and unresolved actions."
    ),
    messages=[
        {
            "role": "user",
            "content": support_thread,
        }
    ],
)

summary = "".join(
    block.text for block in message.content if block.type == "text"
)

The application change is deliberately small. The base URL and credential now point to the gateway, and support-summary is a model alias managed by the platform team. The application does not need to know which Anthropic model sits behind that alias.

For the SDK's supported constructor options and Messages API behaviour, use Anthropic's client SDK documentation as the upstream reference. The Agent Access Manager application guide covers the gateway-side connection.

Give the service its own virtual key

A production service should not share a gateway credential with a notebook, a developer laptop, or another workload. Issue one virtual key for the summariser and attach it to the relevant team or project.

This gives operations a useful control point. They can revoke the summariser without rotating the Anthropic provider key or interrupting other applications. An expiry date can limit temporary workloads, and the non-secret key hint helps an operator identify the credential later without displaying it again.

The gateway stores the issued key as a hash. Provider credentials are kept separately and encrypted at rest. The virtual keys guide explains the credential boundary in more detail.

Put a ceiling on mistakes

A summarisation worker can loop because of a queue bug, retry too aggressively after a timeout, or receive a sudden batch of old tickets. None of those failures is exotic. They are ordinary production mistakes, and each one can turn into an unexpected provider bill.

Apply controls at the narrowest useful scope:

• a requests-per-minute limit to stop a tight retry loop;
• a tokens-per-minute limit to contain unusually large inputs;
• a monthly cost or token budget for the support project;
• an organisation-wide budget as a final backstop.

Agent Access Manager evaluates the applicable key, project, team, and organisation scopes before the call. Usage accrues after the provider reports it. That means one request can cross a hard budget before the updated total is available, so the limit is a control boundary rather than a promise of invoice-level precision.

See budgets and rate limits for the supported scopes and reset periods.

Screen the data that should not travel

Support conversations are messy. Customers paste passwords, access tokens, billing details, and personal data into tickets even when a form tells them not to.

A request policy can inspect the text before it reaches Anthropic. Depending on the configured policy, a finding can be flagged, redacted, or blocked. A response policy can apply similar checks to the model's answer. For this use case, a sensible starting point is to block secrets, redact approved categories of personal information, and monitor lower-confidence findings before enforcing them.

Do not switch every detector to blocking mode on day one. False positives in a support workflow create their own operational problem. Run representative, synthetic tickets through the policy first, review the findings, and then tighten enforcement.

The guardrails documentation covers policy scope, direction, and actions.

Make the audit trail useful

A useful record answers practical questions:

• Which virtual key made the call?
• Which model alias was requested?
• Which provider and model served it?
• How many tokens did it consume, and what did it cost when pricing was configured?
• Did the request succeed, fail, or trigger a guardrail?

That is enough to investigate a usage spike without searching application logs across several hosts. It also lets the platform team compare activity by model and governance scope.

An audit log is not a reason to collect prompts forever. Decide which payloads, metadata, and retention periods are appropriate for the organisation before production traffic begins. Agent Access Manager records governed activity, but the deployment owner still sets the surrounding privacy and retention policy. The audit and usage guide describes the available records and reports.

A production checklist

What changes for the engineering team

Very little changes in the application. It still uses Anthropic's SDK and Messages API. The operational decisions move out of each service and into one administered layer: credential issuance, model routing, budgets, rate limits, guardrails, and audit.

That separation is useful when the second and third Claude applications arrive. Each service gets its own key and limits, while the provider credential remains behind the gateway. Platform teams gain a consistent control path without asking developers to replace the SDK they chose.