Opening a war room from a CloudWatch alarm

When a CloudWatch alarm goes into ALARM, Landfall opens a war room for it, or attaches to the room that alarm already has open. The alarm reaches Landfall through SNS and a small forwarding Lambda.

This page is not about reading telemetry; see Connecting AWS to Landfall. The two are independent.

How it works

Landfall accepts an automated trigger on POST /triggers, the same public ingest endpoint Datadog, Coralogix and Sentry use. It is authenticated by your organization's bearer ingest token, which identifies the organization. Landfall recognizes a CloudWatch alarm by an AlarmName or NewStateValue string at the top level of the body, which is the JSON CloudWatch publishes to an SNS topic on an alarm action.

The alarm's ARN is its identity, so one alarm keeps to one room while that room is open. Plain SNS HTTPS subscriptions can't attach an Authorization header, so the recipe below puts a small forwarding Lambda between SNS and Landfall.

What you'll need

• An admin seat on your Landfall organization, to mint the ingest token.
• AWS credentials that can create an SNS topic, a Lambda function and its execution role.
• At least one CloudWatch alarm already configured. This guide doesn't create one.
• The Lambda runs on Node.js 20 or later, which has a built-in fetch.

1. Mint the ingest token

If you already minted one for Datadog, Coralogix or Sentry, reuse it. The same token works for every source on this endpoint. The token has no screen in the app yet, so it is two API calls, made once with your own Landfall account.

1. Get a session token or an API key

   If your organization signs in with email and password, sign in with curl:

   TerminalCopy

   SESSION=$(curl -s -X POST https://api.landfalls.ai/auth/login \
     -H "Content-Type: application/json" \
     -d '{"email":"<you>","password":"<your password>","orgSlug":"<your-org>"}' \
     | jq -r .accessToken)

   If your organization signs in through SSO, there is no password to hand curl. Create a personal API key instead, under Your settings → API keys → New key. The key starts with lf_key_ and is shown once. Use the whole key as $SESSION.

   TerminalCopy

   SESSION='<the whole key, starting lf_key_>'

   Either way the credential acts as you, so you must be an admin of the organization. Revoke the API key from the same panel afterwards if you made it only for this.

2. POST /o/<your-org>/triggers/ingest-token

   TerminalCopy

   curl -X POST https://api.landfalls.ai/o/<your-org>/triggers/ingest-token \
     -H "Authorization: Bearer $SESSION"

2. Point CloudWatch at Landfall

1. aws sns create-topic

   If your alarms already notify an SNS topic, skip this and reuse that topic.

   TerminalCopy

   aws sns create-topic --name landfall-incident-triggers

2. Add the topic to your alarm's actions

   --alarm-actions notifies the topic when the alarm goes into ALARM. Add the same topic to --ok-actions if you want recoveries recorded on the room, and to --insufficient-data-actions if you want missing data noted there too.

   TerminalCopy

   aws cloudwatch put-metric-alarm --alarm-name <your-alarm> \
     --alarm-actions <the topic arn> \
     --ok-actions <the topic arn> \
     <the rest of your existing alarm definition>

3. Deploy the forwarder Lambda

   The function forwards the alarm's own JSON unchanged. It already carries every field Landfall reads.

   index.mjsCopy

   export const handler = async (event) => {
     const token = process.env.LANDFALL_INGEST_TOKEN;
   
     for (const record of event.Records) {
       // CloudWatch's own alarm-notification JSON, already the shape Landfall
       // expects, forwarded as-is rather than re-parsed or rebuilt.
       const alarmPayload = record.Sns.Message;
   
       const res = await fetch('https://api.landfalls.ai/triggers', {
         method: 'POST',
         headers: {
           Authorization: `Bearer ${token}`,
           'Content-Type': 'application/json',
         },
         body: alarmPayload,
       });
   
       if (!res.ok) {
         // Log and move on: one bad delivery shouldn't fail the whole batch.
         console.error('Landfall trigger ingest failed', res.status, await res.text());
       }
     }
   };

   TerminalCopy

   zip function.zip index.mjs
   
   aws lambda create-function --function-name landfall-trigger-forwarder \
     --runtime nodejs20.x --handler index.handler --zip-file fileb://function.zip \
     --role <an execution role with basic Lambda logging permissions> \
     --environment "Variables={LANDFALL_INGEST_TOKEN=<the token from step 1>}"
   
   aws lambda add-permission --function-name landfall-trigger-forwarder \
     --statement-id sns-invoke --action lambda:InvokeFunction \
     --principal sns.amazonaws.com --source-arn <the topic arn>
   
   aws sns subscribe --topic-arn <the topic arn> \
     --protocol lambda --notification-endpoint <the function arn just created>

What CloudWatch's fields become

3. Test it

Publish a test notification straight to the topic:

aws sns publish --topic-arn <the topic arn> \
  --message '{
    "AlarmName": "landfall-test-alarm",
    "NewStateValue": "ALARM",
    "NewStateReason": "Threshold crossed",
    "StateChangeTime": "2026-01-01T00:00:00.000Z",
    "Trigger": {
      "MetricName": "5xxErrorRate",
      "Namespace": "AWS/CloudFront",
      "Dimensions": [{ "name": "DistributionId", "value": "E123ABC" }]
    }
  }'

Within a few seconds a new incident titled landfall-test-alarm appears in your organization's Incidents list with the AWS CloudWatch badge, proposed at SEV2 because the metric name contains 5xx. The incident is about CloudFront distribution E123ABC, its first timeline event carries the reason Threshold crossed, and the proposed severity records its basis, cloudwatch state=ALARM metric=5xxErrorRate. Publish the same message again and nothing new is recorded, because the same state change time and state mark it as a redelivery.

Then test a real alarm end to end, for example with aws cloudwatch set-alarm-state --alarm-name <your-alarm> --state-value ALARM --state-reason "Testing Landfall".

With AWS also connected as a source, Beacon can chart the alarm's metric inside the room.

app.landfalls.ai · War room · Acme 42

Repeats and recovery

The alarm's ARN is the attach identity. While a room for that alarm is open, a later ALARM is recorded on it as a repeat and opens nothing new. An OK is recorded on that room as a recovery and never opens an incident. INSUFFICIENT_DATA is recorded on an open room, or ignored when there is none. A redelivered message, with the same state change time and state, is recognized and not recorded twice.

A recovery never resolves the room. The alarm clearing is a fact about the alarm, and closing the war room is a person's decision. Once a person resolves the room, the alarm's next ALARM opens a new one. An ALARM that would open a room while an active maintenance window matches it is suppressed, and no room opens.

Troubleshooting

Reference

How every trigger shares this pipeline: Integrations architecture.