CloudTrail Lake With AI: Detect Risky Console Clicks Immediately

Console clicks that open security groups or attach admin policies are still how incidents start. The unfair advantage is CloudTrail Lake queries plus an LLM that narrates with citations — every claim tied to eventTime, eventName, userIdentity, and sourceIPAddress — so investigations finish in minutes without hallucinated blame.

⚡ TL;DR: Continuously query Lake for high-risk eventNames; feed row-level JSON (redacted) to a model with “cite or silent” rules; page humans with a narrative + deep links. Never let the model invent events. Pair with LLM Incident Runbooks and IAM Access Analyzer verify-before-attach patterns.

High-risk event watchlist

-- CloudTrail Lake — illustrative
SELECT eventTime, eventName, recipientAccountId,
       userIdentity.arn AS principal,
       sourceIPAddress, userAgent,
       element_at(requestParameters, 'policyArn') AS policyArn
FROM example_event_data_store
WHERE eventTime >= date_add('minute', -15, current_timestamp)
  AND eventName IN (
    'AuthorizeSecurityGroupIngress',
    'PutUserPolicy', 'AttachUserPolicy', 'CreateAccessKey',
    'DeleteTrail', 'StopLogging', 'PutBucketPolicy'
  )
  AND userIdentity.sessionContext.sessionIssuer.userName IS NULL -- console-ish

✅ Prefer Lake SQL over ad-hoc S3 scans when you need interactive investigate loops.

Narrative with mandatory citations

// investigate/narrate.ts
export type TrailRow = {
  eventTime: string;
  eventName: string;
  principal: string;
  sourceIPAddress: string;
  requestParameters?: Record<string, unknown>;
};

export const SYSTEM = `You explain CloudTrail rows.
Every sentence that asserts a fact MUST end with [eventTime|eventName|principal].
If unsure, say UNKNOWN. Never invent API calls.`;

export function buildPrompt(rows: TrailRow[]): string {
  // ✅ Pass compact JSON only — strip secrets from requestParameters
  return JSON.stringify(rows.map(sanitize), null, 2);
}

❌ Bad: "Alice probably opened SSH to the world around lunch."
✅ Good: "AuthorizeSecurityGroupIngress from arn:... at 2026-09-11T07:02:11Z
added 0.0.0.0/0:22 [2026-09-11T07:02:11Z|AuthorizeSecurityGroupIngress|arn:...]."

Detection → page loop

# lambda/trail_watch.py
def handler(event, _ctx):
    rows = run_lake_query(HIGH_RISK_SQL)
    if not rows:
        return {"ok": True}
    narrative = bedrock_converse(SYSTEM, build_prompt(rows))
    if not citations_valid(narrative, rows):  # ✅ reject ungrounded
        raise SystemExit("ungrounded_narrative")
    sns_publish(title="Risky console activity", body=narrative)

Same grounding discipline as AI On-Call Copilots — suggest, don’t auto-remediate IAM.

Closing checklist

✅ Dos
– ✅ Lake SQL on a tight high-risk allowlist of eventNames
– ✅ Cite eventTime/eventName/principal on every claim
– ✅ Redact secrets from requestParameters before LLM
– ✅ Validate citations against source rows
– ✅ Page humans; keep remediations manual

❌ Don’ts
– ❌ Don’t ask the model to “find anything weird” over raw multi-GB trails
– ❌ Don’t auto-revoke sessions from model output alone
– ❌ Don’t trust userAgent strings as proof of console
– ❌ Don’t leave DeleteTrail unmonitored
– ❌ Don’t paste full authMaterial into prompts

Related reading

• LLM Incident Runbooks: Ground On-Call Answers in CloudWatch Signals
• AI On-Call Copilots: Suggest Runbooks Without Mutating Production
• Secret-Aware Context Filters: Stop AI Editors From Shipping Keys
• Deterministic Replay: Agent Sessions You Can Debug in Postmortems

Last updated on September 11, 2026