CloudWatch API key: OpenTelemetry traces without SigV4

Since November 3, 2026, Amazon CloudWatch also accepts OpenTelemetry traces authenticated with a CloudWatch API key: a plain bearer token in the HTTP header instead of a SigV4 signature. It reads like a footnote in the AWS news feed. For anyone running applications outside AWS who still wants to see in CloudWatch what those applications are doing, it is a real relief.

This topic matters to me because observability in mixed environments often gets stuck on authentication. The service on a rented server, the script in a build pipeline or a third-party tool simply cannot sign AWS requests. So the traces end up somewhere else, or nowhere at all. In this article you set up the new path step by step, with a small TypeScript app, the Serverless Framework for the IAM part and a clean strategy for rotation and security. I will also tell you plainly when I would stay away from it.

In short

With a CloudWatch API key, OpenTelemetry clients can send traces to the X-Ray OTLP endpoint using a bearer token, with no SigV4 involved. That helps applications outside AWS and tools that only support a fixed header. The biggest pitfall is the key itself: it is a long-lived secret and needs an expiry date, tight permissions and a rotation plan.

What you will learn in this article

• When an API key makes sense and when SigV4 remains the better choice
• How to enable Transaction Search and create a tightly scoped IAM user
• How to instrument a Node.js app with OpenTelemetry and a bearer token
• How to store, rotate and, in an emergency, block the key

What is new with the CloudWatch API key for traces?

First, a few terms. OpenTelemetry (OTel for short) is an open standard that applications use to produce telemetry: traces, metrics and logs. A trace shows the path of a request through your system, split into individual sections called spans. This data travels over OTLP, the OpenTelemetry protocol, usually via HTTP.

AWS provides its own OTLP endpoint for traces: https://xray.<region>.amazonaws.com/v1/traces. Until now, this endpoint required every request to be signed with SigV4, the signing process AWS clients use to sign each request with their credentials. That works nicely when your code uses the AWS SDK or the AWS Distro for OpenTelemetry (ADOT). Many exporters and third-party tools cannot do it, though.

Now there is a second door. You create an API key for the X-Ray service in IAM and send it as Authorization: Bearer <key>. Any OTLP HTTP client that can set a header is ready to go. The spans end up in the same place as signed spans and are available for Transaction Search, Application Signals and the other CloudWatch features.

Two details matter. First, the key only works for sending spans to the OTLP endpoint. You cannot read anything with it, and all other X-Ray APIs still require SigV4. Second, AWS calls this “CloudWatch API keys” in the announcement, but technically it is a service-specific IAM credential for xray.amazonaws.com. In the IAM console you will find it under “API keys for AWS services”.

When is a CloudWatch API key better than SigV4?

AWS is refreshingly clear here: SigV4 with short-term credentials remains the recommended approach. The API key is the exception for cases where that is not practical. I have sorted the typical situations below.

My decision path: the API key is deliberately the last option.

So for Serverless applications on Lambda, nothing changes. If you already use Powertools or ADOT there, keep doing so. I show what that looks like in TypeScript in Lambda Powertools TypeScript: logs, metrics, traces. The new path gets interesting where your Lambda functions work together with services outside AWS and you finally want to see the full path of a request in one trace.

How traces flow from your server into CloudWatch

Before we build anything, here is the flow at a glance. Your application creates spans, the OpenTelemetry exporter batches them and sends them via HTTPS with a bearer header to the X-Ray endpoint. IAM checks the key and the permissions of the user it belongs to. X-Ray then writes the spans to the aws/spans log group, and Transaction Search makes them searchable.

🎬 Flow: sending traces to X-Ray with a CloudWatch API key – this animated diagram only runs on my site: watch it on sascha-ms.de

Prerequisites

• An AWS account with permissions for IAM, X-Ray, CloudWatch Logs and Secrets Manager
• AWS CLI v2, signed in locally (only for the setup, not for the app)
• Node.js 22 or newer and npm
• Serverless Framework v4 (npm i -g serverless) for the IAM part
• A Region where the endpoint supports API keys. Frankfurt (eu-central-1) is included; the full list is in the X-Ray documentation.

The example is a fictional bike rental. Its booking service runs on a server at another hosting provider and should still deliver its traces to CloudWatch.

Step 1: Enable Transaction Search

The OTLP endpoint for traces only works when Transaction Search is enabled. It stores spans as structured logs in CloudWatch Logs and makes them searchable. If you have already turned it on in your account, simply skip this step.

First, allow X-Ray to write to the two log groups. Replace the account ID and Region with your own values:

export AWS_REGION=eu-central-1 export ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text) aws logs put-resource-policy --policy-name xray-transaction-search --policy-document "{ \"Version\": \"2012-10-17\", \"Statement\": [{ \"Sid\": \"TransactionSearchXRayAccess\", \"Effect\": \"Allow\", \"Principal\": { \"Service\": \"xray.amazonaws.com\" }, \"Action\": \"logs:PutLogEvents\", \"Resource\": [ \"arn:aws:logs:${AWS_REGION}:${ACCOUNT_ID}:log-group:aws/spans:*\", \"arn:aws:logs:${AWS_REGION}:${ACCOUNT_ID}:log-group:/aws/application-signals/data:*\" ], \"Condition\": { \"ArnLike\": { \"aws:SourceArn\": \"arn:aws:xray:${AWS_REGION}:${ACCOUNT_ID}:*\" }, \"StringEquals\": { \"aws:SourceAccount\": \"${ACCOUNT_ID}\" } } }] }" 

Then switch the destination for trace segments to CloudWatch Logs and check the result:

aws xray update-trace-segment-destination --destination CloudWatchLogs aws xray get-trace-segment-destination 

You should see "Destination": "CloudWatchLogs" and "Status": "ACTIVE". Allow about ten minutes until the first spans are searchable. By default, AWS indexes 1% of spans as trace summaries free of charge. You can change that share later with aws xray update-indexing-rule.

Step 2: Create the IAM user with the Serverless Framework

An API key always belongs to an IAM user. That is why you create a dedicated user that can do exactly one thing: send spans. AWS offers the managed policy AWSXRayAPIKeyAccess for this. I prefer my own inline policy here because I can additionally restrict it to the public IP address of the server. A stolen key is then worthless anywhere else.

The Serverless Framework also works for pure infrastructure without functions. The template deliberately does not set the value of the key, because a secret does not belong in a CloudFormation template.

# infra/serverless.yml service: bike-rental-tracing-access frameworkVersion: "4" provider: name: aws region: eu-central-1 params: default: # Public IP address(es) of the server that sends traces allowedIps: - 203.0.113.10/32 resources: Resources: TracesSenderUser: Type: AWS::IAM::User Properties: UserName: ${self:service}-${sls:stage}-traces Policies: - PolicyName: send-traces-only PolicyDocument: Version: "2012-10-17" Statement: - Sid: TracesPerApiKey Effect: Allow Action: - xray:CallWithBearerToken - xray:PutTraceSegments Resource: "*" Condition: IpAddress: aws:SourceIp: ${param:allowedIps} TracesApiKeySecret: Type: AWS::SecretsManager::Secret Properties: Name: /bike-rental/${sls:stage}/xray-api-key Description: X-Ray API key for OTLP traces (set the value via CLI) Outputs: TracesSenderUserName: Value: !Ref TracesSenderUser TracesApiKeySecretArn: Value: !Ref TracesApiKeySecret 

cd infra serverless deploy --stage dev 

In the outputs you will see the user name bike-rental-tracing-access-dev-traces and the ARN of the still empty secret. If your server sends from changing IP addresses, leave out the Condition and rely more on expiry and rotation.

Step 3: Create the CloudWatch API key and store it safely

Now you create the actual key. This only works with “real” AWS credentials, so via the console or the CLI, never with an API key itself. Always set an expiry date: without --credential-age-days the key never expires.

USER_NAME=bike-rental-tracing-access-dev-traces API_KEY=$(aws iam create-service-specific-credential \ --user-name "$USER_NAME" \ --service-name xray.amazonaws.com \ --credential-age-days 90 \ --query ServiceSpecificCredential.ServiceCredentialSecret \ --output text) aws secretsmanager put-secret-value \ --secret-id /bike-rental/dev/xray-api-key \ --secret-string "$API_KEY" unset API_KEY 

The value from ServiceCredentialSecret is your API key. AWS shows it exactly once and you cannot retrieve it afterwards. That is why it goes straight into Secrets Manager, without a detour via the clipboard or a file. If it gets lost, just create a new one.

On the server you fetch it at startup, for example with a start script that sets the value as the XRAY_API_KEY environment variable. Fetching it requires AWS access, of course. If the server has none, your deployment tool passes the key in as a secret, for example as an encrypted variable of your pipeline.

Step 4: Instrument a Node.js app with OpenTelemetry

Now for the application. You need the OpenTelemetry Node.js SDK and an OTLP exporter. I use the protobuf exporter because it transfers data more compactly than JSON. The endpoint accepts both.

mkdir bike-rental-tracing && cd bike-rental-tracing npm init -y npm pkg set type=module npm i @opentelemetry/sdk-node @opentelemetry/exporter-trace-otlp-proto @opentelemetry/api @opentelemetry/resources @opentelemetry/semantic-conventions npm i -D typescript tsx @types/node 

The tsconfig.json is simple. The important part is NodeNext, so that top-level await and ES modules work together:

{ "compilerOptions": { "target": "ES2022", "module": "NodeNext", "moduleResolution": "NodeNext", "strict": true, "outDir": "dist", "skipLibCheck": true }, "include": ["src"] } 

Configure the exporter with a bearer header

The tracing.ts file starts the SDK before any other code runs. The crucial part is the headers entry on the exporter. The file also fails immediately if the key is missing. Otherwise missing telemetry tends to surface only when you urgently need it, right in the middle of an incident.

// src/tracing.ts import { NodeSDK } from "@opentelemetry/sdk-node"; import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-proto"; import { resourceFromAttributes } from "@opentelemetry/resources"; import { ATTR_SERVICE_NAME, ATTR_SERVICE_VERSION } from "@opentelemetry/semantic-conventions"; const region = process.env.AWS_REGION ?? "eu-central-1"; const apiKey = process.env.XRAY_API_KEY; if (!apiKey) { // Fail loudly instead of silently sending nothing throw new Error("XRAY_API_KEY is missing – traces cannot be sent."); } const exporter = new OTLPTraceExporter({ url: process.env.OTLP_TRACES_URL ?? `https://xray.${region}.amazonaws.com/v1/traces`, headers: { Authorization: `Bearer ${apiKey}` }, timeoutMillis: 10_000, }); export const sdk = new NodeSDK({ resource: resourceFromAttributes({ [ATTR_SERVICE_NAME]: "bike-rental-booking", [ATTR_SERVICE_VERSION]: "1.0.0", "deployment.environment.name": process.env.APP_ENV ?? "dev", }), traceExporter: exporter, }); sdk.start(); // Flush open spans on shutdown const shutdown = async () => { await sdk.shutdown(); process.exit(0); }; process.on("SIGTERM", shutdown); process.on("SIGINT", shutdown); 

Create spans in the application

The sample app books a bike and creates two nested spans along the way. Errors are added to the span with recordException and an error status, so you can filter for them in CloudWatch later.

// src/app.ts import "./tracing.js"; import { trace, SpanStatusCode } from "@opentelemetry/api"; import { sdk } from "./tracing.js"; const tracer = trace.getTracer("bike-rental-booking"); async function checkAvailability(bikeId: string): Promise<boolean> { return tracer.startActiveSpan("availability.check", async (span) => { span.setAttribute("bike.id", bikeId); await new Promise((r) => setTimeout(r, 40)); // simulated database query span.end(); return true; }); } async function bookBike(bikeId: string, customerId: string): Promise<string> { return tracer.startActiveSpan("booking.create", async (span) => { try { span.setAttribute("bike.id", bikeId); span.setAttribute("customer.id", customerId); const available = await checkAvailability(bikeId); if (!available) throw new Error("Bike is already rented out"); const bookingId = `B-${Date.now()}`; span.setAttribute("booking.id", bookingId); return bookingId; } catch (err) { span.recordException(err as Error); span.setStatus({ code: SpanStatusCode.ERROR }); throw err; } finally { span.end(); } }); } const id = await bookBike("bike-042", "customer-7"); console.log(`Booking ${id} created`); // Short-lived processes (scripts, CI jobs): flush explicitly await sdk.shutdown(); 

For a long-running server, the SIGTERM handler in tracing.ts is enough. For short-lived processes such as a CI job, call sdk.shutdown() yourself, otherwise the process ends before the exporter has sent its last batch.

Dry run without AWS

Before real data flows, a quick local test shows whether the header arrives at all. A tiny receiver that logs every OTLP request is all you need:

// src/mock-receiver.ts // Local dry run: accepts OTLP requests and shows whether the header is right. import { createServer } from "node:http"; createServer((req, res) => { const auth = req.headers.authorization ?? ""; const ok = /^Bearer \S{20,}$/.test(auth); let bytes = 0; req.on("data", (chunk: Buffer) => (bytes += chunk.length)); req.on("end", () => { console.log( `${req.method} ${req.url} | ${req.headers["content-type"]} | ${bytes} bytes | Bearer header: ${ok ? "ok" : "MISSING"}`, ); res.writeHead(ok ? 200 : 403, { "content-type": "application/json" }); res.end("{}"); }); }).listen(4318, () => console.log("Mock receiver listening on http://localhost:4318/v1/traces")); 

npx tsx src/mock-receiver.ts & XRAY_API_KEY=testkey-abcdefghijklmnopqrstuvwxyz \ OTLP_TRACES_URL=http://localhost:4318/v1/traces \ npx tsx src/app.ts 

The expected output looks like this:

Mock receiver listening on http://localhost:4318/v1/traces Booking B-1793889441350 created POST /v1/traces | application/x-protobuf | 1068 bytes | Bearer header: ok 

Afterwards, remove OTLP_TRACES_URL, set the real key and AWS_REGION=eu-central-1, and start the app again. The exporter then talks directly to https://xray.eu-central-1.amazonaws.com/v1/traces.

Step 5: Use the OpenTelemetry Collector instead

If several services run on the server, I prefer to put an OpenTelemetry Collector in front of them. It is a standalone process that receives, batches and forwards telemetry. The key then lives in exactly one place, and your applications send to localhost:4318 without authentication. You need the contrib distribution of the collector, because only it includes the bearertokenauth extension.

# collector/otel-collector.yaml extensions: bearertokenauth/xray: token: "${env:XRAY_API_KEY}" receivers: otlp: protocols: http: endpoint: "0.0.0.0:4318" grpc: endpoint: "0.0.0.0:4317" processors: memory_limiter: check_interval: 1s limit_percentage: 80 spike_limit_percentage: 20 batch: send_batch_size: 512 timeout: 5s exporters: otlphttp/xray: traces_endpoint: "https://xray.eu-central-1.amazonaws.com/v1/traces" compression: gzip auth: authenticator: bearertokenauth/xray retry_on_failure: enabled: true initial_interval: 5s max_elapsed_time: 120s service: extensions: [bearertokenauth/xray] pipelines: traces: receivers: [otlp] processors: [memory_limiter, batch] exporters: [otlphttp/xray] 

The key comes in through the environment variable and never appears in the file. Alternatively, the extension can read it from a mounted file with filename, which suits containers. The retry_on_failure block handles short outages, since AWS explicitly recommends retrying with backoff on a 503.

Step 6: Check the traces in CloudWatch

In the CloudWatch console, open Application Signals and then Transaction Search. Filter by service.name = bike-rental-booking. After a few minutes you should see the spans booking.create and availability.check, including attributes such as bike.id.

If you want more detail, enable CloudTrail data events for AWS::XRay::Trace. Requests with an API key show up there as PutTraceSegments, with the IAM user as the identity and "callWithBearerToken": true in the additional event data. Data events cost extra, but they are worth it during a trial phase.

Rotate a CloudWatch API key without downtime

Each IAM user can hold up to two X-Ray API keys at once, and that is precisely the point: you create the new key, switch over and only then turn off the old one.

1. Create a new key with create-service-specific-credential and write it to Secrets Manager.
2. Restart the collector or application so it picks up the new value.
3. Deactivate the old key and watch for a few minutes whether spans keep arriving.
4. Only then delete the old key.

aws iam list-service-specific-credentials \ --user-name bike-rental-tracing-access-dev-traces \ --service-name xray.amazonaws.com aws iam update-service-specific-credential \ --user-name bike-rental-tracing-access-dev-traces \ --service-specific-credential-id <ID-of-the-old-key> \ --status Inactive 

You can automate this as well, for example with the rotation feature of Secrets Manager and a small Lambda function. I show how such a rotation is structured in Secrets Manager rotation with Lambda for API keys. Only the order matters: distribute the new value first, then block the old one.

Common errors and how to fix them

Keeping an eye on cost and security

The API key itself is free. You pay the same as for signed spans: with Transaction Search enabled, AWS bills span ingestion at CloudWatch prices, and stored spans in aws/spans take up log storage. Check the current CloudW...