Connecting MySQL to Landfall

The Landfall MySQL agent is a small program you run next to your databases. It dials out to Landfall, reads each database with a read-only user whose password never leaves your network, and answers questions only for an incident the database is part of.

• As a source: an agent token Landfall mints for the agent, plus a read-only MySQL user you create on each database. The user's password stays with the agent.
• As a trigger: not yet. See Opening incidents from MySQL.

How it works

• Outbound only. Landfall never connects to a database. The agent holds a long poll on its own job queue in Landfall, runs each read it is given, and posts the answer back.
• Quiet until involved. A database is read only for an incident it is part of. Until then the agent keeps its snapshots in its own memory and sends nothing.
• No rows, no statement text. The agent reads metadata and performance_schema summaries only. Statements appear as MySQL's normalized digests, with literals already replaced by ?. It never reads a table row, statement samples, lock data, column defaults or comments.
• Read-only, checked. Every hour the agent reads its user's grants. If the user holds anything beyond the published set, the database reads as degraded, naming the extra grant, and the agent refuses to read it.
• Standard parts. The agent is an OpenTelemetry Collector distribution: the upstream MySQL receiver and SQL query receiver do the collecting.

What you'll need

• MySQL 8.0 or later (RDS for MySQL, Aurora MySQL, or your own) with performance_schema on. On RDS it is a parameter-group setting, and the micro and small burstable classes cannot run it.
• Somewhere to run a container that can reach the database and api.landfalls.ai over HTTPS: a VM, a container service, or any Kubernetes cluster.
• An administrator on each database, to create the agent's user.
• An admin on your Landfall organization.

1. Add the agent in Landfall

1. Integrations → Context sources → MySQL → Add agent

   In the left menu, open Integrations and choose MySQL under Context sources. Under Agent name, type the name the room will use (for example orders-prod) and click Add agent. Landfall mints the agent's token and shows three things under Run the agent: the database user to create, a docker run command and a helm install command. Copy what you need now. The token is shown once. Then click I have copied it.

   app.landfalls.ai · Integrations · MySQL

   

2. Create the agent's user on each database

CREATE USER 'landfall'@'%' IDENTIFIED BY '<password>'
  REQUIRE SSL WITH MAX_USER_CONNECTIONS 10;
GRANT PROCESS, REPLICATION CLIENT, REFERENCES ON *.* TO 'landfall'@'%';
GRANT SELECT ON performance_schema.* TO 'landfall'@'%';
GRANT SELECT, EXECUTE ON sys.* TO 'landfall'@'%';

-- Optional, per migration history table, so applied migrations show with their time:
GRANT SELECT ON shop.flyway_schema_history TO 'landfall'@'%';

REFERENCES lets the agent see every table's columns and indexes without reading a row. Pick the password yourself; Landfall never sees it.

3. Run the agent

With Docker, one database

docker run -d --name landfall-mysql-agent --restart unless-stopped \
  -v landfall-mysql-agent:/var/lib/landfall \
  -e LANDFALL_API_URL=https://api.landfalls.ai \
  -e LANDFALL_CONNECTION_ID=orders-prod \
  -e LANDFALL_TOKEN=lfa_... \
  -e LANDFALL_AGENT_NAME='orders-prod' \
  -e LANDFALL_MYSQL_NAME=orders-prod \
  -e LANDFALL_MYSQL_ENDPOINT=orders.abc123.eu-west-1.rds.amazonaws.com:3306 \
  -e LANDFALL_MYSQL_PASSWORD=<the landfall user's password> \
  -e LANDFALL_MYSQL_TLS=required \
  public.ecr.aws/r9i3z7v6/landfall-mysql-agent:0.1.0

The volume keeps the agent id the token binds to, so a replaced container reconnects as the same agent. For several databases (a primary and its replica, say), set LANDFALL_MYSQL_TARGETS to a JSON list of {"name","endpoint","tls","role"} and give each password as LANDFALL_MYSQL_PASSWORD_<NAME> (upper case, dashes as underscores).

With Kubernetes

helm install landfall-mysql-agent oci://public.ecr.aws/r9i3z7v6/charts/landfall-mysql-agent \
  --namespace landfall --create-namespace \
  --set landfall.apiUrl=https://api.landfalls.ai \
  --set landfall.connectionId=orders-prod \
  --set token=lfa_... \
  --set mysql.name=orders-prod \
  --set mysql.endpoint=<host:3306> \
  --set mysql.password=<the landfall user's password> \
  --set mysql.tls=required

The chart keeps the token, the agent id and each password in one Secret (or use existingSecret), gives the agent no Kubernetes permissions, and lists several databases under mysql.targets.

4. Watch it connect

Within a minute the agent reads Connected in the same panel, and each database shows its own state from the agent's heartbeat: Connected, Sign-in refused, Unreachable, or Degraded with the reason.

app.landfalls.ai · Integrations · MySQL

What it looks like

Once a person adds the agent to an incident, the room's chat shows a card saying MySQL name is part of this investigation, with a Remove from investigation button. On a real run, a migration that dropped orders.legacy_price while the app still read it showed up as four pieces at once: the dropped column between two snapshots, Flyway's V2 with its time, a spike of 1054 errors, and the app's own statement leading the errors ranking. The ALTER waiting behind a long transaction showed as a metadata lock wait.

app.landfalls.ai · War room

What Beacon can read

A database answers only for an incident its agent is part of. The room has no control for adding one yet. A member of the organization adds the agent with one call to Landfall's API, using a personal API key from Your settings → API keys. The connection id is the LANDFALL_CONNECTION_ID value in the run command. An agent's own session is refused.

curl -X POST https://api.landfalls.ai/o/<your-org>/incidents/<incident id>/agents/mysql/<connection id>/involve \
  -H "Authorization: Bearer lf_key_..."

An agent cannot be added to a resolved incident, and adding one twice is refused. Remove from investigation on the room's card takes it out at once.

These operations reach Beacon and every agent that joins the room through get_signal_catalog and query_signals. How the agents fit with every other integration is on Integrations architecture.

Opening incidents from MySQL

Not yet. The agent answers questions for incidents that already exist. Open incidents from the alerts you already have, for example through Alertmanager or the other pages under Triggering integrations, and add the database's agent to the incident.

Troubleshooting

The agent's own reads are left out of the error counts it reports: on MySQL 8, scanning information_schema without data-dictionary access raises an internal error the server would otherwise count.

Reference