Secrets Manager rotation is my answer to an uncomfortable question: how old is the API key your Lambda function has been sending to a partner for months? In many projects the honest answer is “as old as the project”. The key lives in a secret, the secret is read encrypted, and still nobody has ever replaced it. If it slips into a log, a support email or a ticket once, it stays valid until somebody notices.
AWS Secrets Manager can renew such keys automatically. For databases that is almost a single click. For an API key at an external service you need your own rotation function in Lambda, and that is what we build in this article, step by step: with TypeScript, tests and the Serverless Framework. Afterwards you will understand what happens in the four steps, why staging labels matter so much and how your other functions pick up the new key without downtime.
In short
Secrets Manager invokes your rotation function four times: create a new value, set it at the target service, test it, mark it as current. To avoid downtime, the target service has to accept the old and the new key in parallel for a while. The most common pitfall is a stuck AWSPENDING label after a failed rotation.
What you will learn in this article
- How Secrets Manager rotation works with versions and staging labels
- When you do not need your own rotation function
- A rotation function for an API key in TypeScript, including tests
- Deployment with the Serverless Framework, schedule and alarm included
- How Lambda functions read rotated keys without downtime
What is Secrets Manager rotation?
Rotation means a secret is replaced regularly, and in two places at once: in the secret itself and at the service that checks it, such as the database or the partner API. If you only change one side, you lock yourself out.
Secrets Manager solves this with versions. A secret can have several versions, and each version carries so-called staging labels. These are simple text markers that say which version currently plays which role:
-
AWSCURRENTis the version your applications read by default. -
AWSPENDINGis the new version that is being created and has not been released yet. -
AWSPREVIOUSis the last valid version. After a successful rotation it stays around as a fallback.
Before, during and after rotation: how AWSCURRENT, AWSPENDING and AWSPREVIOUS move.
In the end, a rotation is nothing more than a carefully choreographed move of these labels. Only when the new value works at the target service does AWSCURRENT move to the new version. Until then, every application keeps reading the old value.
When you do not need your own rotation function
Before you write code, take an honest look at the alternatives. According to the documentation, AWS distinguishes three forms of rotation:
| Form | Used for | Lambda needed? |
|---|---|---|
| Managed rotation | Secrets that an AWS service manages itself, e.g. the master password of an RDS database | No |
| Managed external secrets | Credentials at partners that Secrets Manager supports directly | No |
| Rotation by Lambda function | Everything else: your own databases, internal services, API keys at any provider | Yes |
For Amazon RDS I would always start with the master password managed by the service. For many database engines AWS also provides ready-made rotation templates. A custom function is therefore mostly worth it where no standard applies. An API key at a service provider is the classic case.
The four steps of Secrets Manager rotation
When a rotation starts, Secrets Manager invokes your Lambda function four times in a row. Every invocation gets the same secret ID, the same ClientRequestToken (it becomes the version ID of the new version) and a different value in the Step field.
🎬 The four steps of a rotation – this animated diagram only runs on my site: watch it on sascha-ms.de
createSecret creates the new value
First, the function checks whether a value already exists for the token. If so, this was a retry, and it stops. Otherwise it creates a new key with GetRandomPassword and stores it via PutSecretValue with the label AWSPENDING. This check makes the step idempotent, so you can repeat it as often as you like with the same result.
setSecret registers it with the target service
Now the target service learns about the new key. AWS describes the rotation function as a privileged deputy here: it may change both the secret and the credentials at the service. That is why safety checks belong in this step. Do AWSCURRENT and AWSPENDING point to the same target? Does the current key still work at all? If not, the function aborts. This prevents the so-called confused deputy attack, where someone abuses an authorized function for their own purposes.
testSecret checks that it works
The function calls a harmless endpoint with the new key. If the service rejects it, the function throws an error and the rotation ends here. Your applications notice nothing, because they keep reading AWSCURRENT.
finishSecret switches over
Finally, UpdateSecretVersionStage moves the label AWSCURRENT to the new version. Secrets Manager attaches AWSPREVIOUS to the old version automatically. Done.
The example: a bike rental and its lock API
To make this tangible, let us take a fictional bike rental. Its bikes have smart locks, operated by a partner. When a customer wants to unlock a bike, a Lambda function calls the partner lock API and sends an API key in the x-api-key header.
The good news: the (equally made-up) partner API allows two active keys per account. New keys are registered with POST, old ones revoked with DELETE. That is the most important prerequisite for a rotation without downtime, and you should check it with every real provider first. If a service only supports exactly one key, there will inevitably be a moment when running functions fail with the old value.
The flow looks like this: the key from the rotation before last (AWSPREVIOUS) is revoked, the new one is registered, tested and then becomes current. The previously current key stays valid until the next rotation. That gives every function that still has it in its cache plenty of time to switch.
Step by step: rotation with Lambda and the Serverless Framework
Prerequisites
- Node.js 22 or newer and npm
- An AWS account with a configured AWS CLI (region in the example:
eu-central-1) - Serverless Framework v4 (
npm i -g serverless), which builds TypeScript handlers automatically - Basic knowledge of Lambda and IAM; I explain the rest along the way
Step 1: Create the project
Create a folder and install the AWS SDK for Secrets Manager plus the development tools:
mkdir key-rotation && cd key-rotation npm init -y npm install @aws-sdk/client-secrets-manager npm install -D typescript @types/node @types/aws-lambda vitest@3 aws-sdk-client-mock The tsconfig.json stays simple and strict:
// tsconfig.json { "compilerOptions": { "target": "ES2023", "module": "NodeNext", "moduleResolution": "NodeNext", "strict": true, "esModuleInterop": true, "skipLibCheck": true, "types": ["node"] }, "include": ["src", "test"] } In package.json, add "type": "module" and the scripts "typecheck": "tsc --noEmit" and "test": "vitest run".
Step 2: Client for the partner API
The client wraps the three calls we need. It is deliberately built as an interface plus a class, so the test can replace it with a fake.
// src/lock-partner-client.ts // Client for the (fictional) lock API of the partner that runs the smart locks // on the rental bikes. The API allows two active keys at the same time, // which is exactly what we need for a rotation without downtime. export interface LockApiSecret { baseUrl: string; // e.g. https://api.lockpartner.example accountId: string; // the bike rental's customer account at the partner keyId: string; // name of the key at the partner apiKey: string; // the actual secret value } export interface LockPartnerClient { /** Checks whether a key works (true = valid). */ ping(secret: LockApiSecret): Promise<boolean>; /** Registers a new key. Authorized with the currently valid key. */ registerKey(auth: LockApiSecret, newKey: LockApiSecret): Promise<void>; /** Revokes an old key. Authorized with the currently valid key. */ revokeKey(auth: LockApiSecret, keyId: string): Promise<void>; } export class HttpLockPartnerClient implements LockPartnerClient { constructor(private readonly timeoutMs = 5000) {} async ping(secret: LockApiSecret): Promise<boolean> { const res = await this.call(secret, 'GET', `/accounts/${secret.accountId}/ping`); return res.status === 200; } async registerKey(auth: LockApiSecret, newKey: LockApiSecret): Promise<void> { const res = await this.call(auth, 'POST', `/accounts/${auth.accountId}/api-keys`, { keyId: newKey.keyId, apiKey: newKey.apiKey, }); // 409 means the key is already registered (e.g. on a retry), which is fine. if (res.status !== 201 && res.status !== 409) { throw new Error(`registerKey failed: HTTP ${res.status}`); } } async revokeKey(auth: LockApiSecret, keyId: string): Promise<void> { const res = await this.call(auth, 'DELETE', `/accounts/${auth.accountId}/api-keys/${keyId}`); // 404 means the key no longer exists, which is fine as well. if (res.status !== 204 && res.status !== 404) { throw new Error(`revokeKey failed: HTTP ${res.status}`); } } private call(auth: LockApiSecret, method: string, path: string, body?: unknown): Promise<Response> { return fetch(new URL(path, auth.baseUrl), { method, headers: { 'x-api-key': auth.apiKey, 'content-type': 'application/json' }, body: body === undefined ? undefined : JSON.stringify(body), signal: AbortSignal.timeout(this.timeoutMs), }); } } Note the status codes 409 and 404: when a step is repeated, the key may already be registered or already gone. Neither is an error; both are exactly the result we want.
Step 3: The rotation function
Now the core. The handler first checks that rotation is enabled and that the token belongs to an AWSPENDING version. Then it branches into the four steps.
// src/rotate.ts // Rotation function for AWS Secrets Manager: rotates the API key // for the lock API in the four steps createSecret, setSecret, // testSecret and finishSecret. import { SecretsManagerClient, DescribeSecretCommand, GetSecretValueCommand, PutSecretValueCommand, GetRandomPasswordCommand, UpdateSecretVersionStageCommand, ResourceNotFoundException, } from '@aws-sdk/client-secrets-manager'; import { HttpLockPartnerClient, type LockApiSecret, type LockPartnerClient } from './lock-partner-client.js'; export interface RotationEvent { SecretId: string; ClientRequestToken: string; Step: 'createSecret' | 'setSecret' | 'testSecret' | 'finishSecret'; RotationToken?: string; // pass it on, never log it } const CURRENT = 'AWSCURRENT'; const PENDING = 'AWSPENDING'; const PREVIOUS = 'AWSPREVIOUS'; export function createRotationHandler(sm: SecretsManagerClient, partner: LockPartnerClient) { async function readSecret(secretId: string, stage: string, versionId?: string): Promise<LockApiSecret> { const res = await sm.send( new GetSecretValueCommand({ SecretId: secretId, VersionStage: stage, VersionId: versionId }), ); const value = JSON.parse(res.SecretString ?? '{}') as Partial<LockApiSecret>; for (const field of ['baseUrl', 'accountId', 'keyId', 'apiKey'] as const) { if (!value[field]) throw new Error(`Field ${field} missing in ${stage}`); } return value as LockApiSecret; } async function createSecret(secretId: string, token: string, rotationToken?: string): Promise<void> { const current = await readSecret(secretId, CURRENT); try { // If the version already exists (retry), there is nothing to do. await readSecret(secretId, PENDING, token); console.log('createSecret: AWSPENDING already exists'); return; } catch (err) { if (!(err instanceof ResourceNotFoundException)) throw err; } const random = await sm.send( new GetRandomPasswordCommand({ PasswordLength: 48, ExcludePunctuation: true }), ); const pending: LockApiSecret = { ...current, keyId: `rotation-${token.slice(0, 8)}`, apiKey: random.RandomPassword!, }; await sm.send( new PutSecretValueCommand({ SecretId: secretId, ClientRequestToken: token, SecretString: JSON.stringify(pending), VersionStages: [PENDING], RotationToken: rotationToken, }), ); console.log(`createSecret: stored new version ${token} as AWSPENDING`); } async function setSecret(secretId: string, token: string, previousVersionId?: string): Promise<void> { const current = await readSecret(secretId, CURRENT); const pending = await readSecret(secretId, PENDING, token); // Confused deputy protection: both versions must belong to the same target. if (current.baseUrl !== pending.baseUrl || current.accountId !== pending.accountId) { throw new Error('setSecret: AWSCURRENT and AWSPENDING point to different targets'); } // Only rotate if the current key still works. if (!(await partner.ping(current))) { throw new Error('setSecret: AWSCURRENT is invalid, rotation aborted'); } // The partner allows two keys. We remove the one before last (AWSPREVIOUS) // now to make room for the new one. It has not been in use since the // last rotation. if (previousVersionId) { const previous = await readSecret(secretId, PREVIOUS, previousVersionId); if (previous.keyId !== current.keyId && previous.keyId !== pending.keyId) { await partner.revokeKey(current, previous.keyId); console.log(`setSecret: revoked old key ${previous.keyId}`); } } await partner.registerKey(current, pending); console.log(`setSecret: registered new key ${pending.keyId}`); } async function testSecret(secretId: string, token: string): Promise<void> { const pending = await readSecret(secretId, PENDING, token); if (!(await partner.ping(pending))) { throw new Error('testSecret: partner rejects the new key'); } console.log('testSecret: new key works'); } async function finishSecret(secretId: string, token: string, currentVersionId?: string): Promise<void> { if (currentVersionId === token) { console.log('finishSecret: version is already AWSCURRENT'); return; } await sm.send( new UpdateSecretVersionStageCommand({ SecretId: secretId, VersionStage: CURRENT, MoveToVersionId: token, RemoveFromVersionId: currentVersionId, }), ); console.log(`finishSecret: AWSCURRENT now points to ${token}`); } return async function handler(event: RotationEvent): Promise<void> { const { SecretId: secretId, ClientRequestToken: token, Step: step } = event; // Important: never log the secret value, only step and version. console.log(`Rotation ${step} for version ${token}`); const meta = await sm.send(new DescribeSecretCommand({ SecretId: secretId })); if (!meta.RotationEnabled) throw new Error('Rotation is not enabled for this secret'); const versions = meta.VersionIdsToStages ?? {}; const stages = versions[token]; if (!stages) throw new Error(`Version ${token} does not belong to this secret`); if (stages.includes(CURRENT)) { console.log('Version is already AWSCURRENT, nothing to do'); return; } if (!stages.includes(PENDING)) throw new Error(`Version ${token} is not AWSPENDING`); const findVersion = (stage: string) => Object.entries(versions).find(([, s]) => s.includes(stage))?.[0]; switch (step) { case 'createSecret': return createSecret(secretId, token, event.RotationToken); case 'setSecret': return setSecret(secretId, token, findVersion(PREVIOUS)); case 'testSecret': Source:https://dev.to/sascha-ms/secrets-manager-rotation-with-lambda-for-api-keys-2kp5 Related posts

