9 views
This guide covers Claude Computer Use with production-focused examples and trade-offs.
“Fails on CI, passes locally” dies when the agent can reproduce the UI path, dump DOM + HAR, and attach artifacts to the ticket. Computer-use agents are not for auto-healing production—they are for evidence collection at the speed of a flaky suite.⚡ TL;DR: On Playwright/Cypress failure, spawn a sandboxed computer-use session with the failing URL + seed; capture screenshot, accessibility tree, network HAR, and console errors; upload to the job artifacts and Jira. Never grant prod credentials. Pair with CI Failure Triage Bots and Self-Healing Tests Quarantine.
Failure hook → evidence agent — Claude Computer Use
typescript
✅ Domain allowlist + read-only instructions.// ci/on-ui-fail.ts
export async function captureEvidence(fail: {
testName: string;
traceUrl: string;
videoUrl?: string;
seed: string;
}) {
const prompt = [
`Reproduce flaky UI test: ${fail.testName}`,
`Open staging URL from seed ${fail.seed}`,
`Do NOT change application data. Read-only.`,
`Return: steps, console errors, selector that flake, HAR path.`,
].join("\n");
// Computer-use runs in ephemeral ECS — see secure sandboxes
const session = await startComputerUseAgent({
prompt,
allowlistDomains: ["staging.example.com"],
denyWrites: true,
maxMinutes: 8,
});
return session.artifacts; // png, har, a11y.json
}
❌ Agent with prod SSO and write access “to fix the button.”
Artifact contract for tickets
markdown
Sanitize tickets before agents read them—tracker text is untrusted (Prompt Injection in Trackers).
## Flake evidence (auto)
- Test: `checkout.spec.ts:141`
- Screenshot: `s3://ci-artifacts/.../fail.png`
- HAR: `.../network.har`
- Console: 3 errors (first: TypeError cannot read 'id' of undefined)
- Suspected selector: `[data-testid=pay-cta]` intermittent detach
- Agent session: `cu-2026-09-11-ab12`
Quarantine, don’t infinite-retry
yaml
Computer-use cost adds up—sample only failures that already failed twice, or gate on labels. Metrics mindset from AI Pair Programming Metrics: evidence quality over raw retry count.
# After N evidence-backed flakes → quarantine with SLA
if: flake_count >= 3
steps:
- run: pnpm tsx scripts/quarantine-test.ts --sla-days=5
- run: gh issue create --title "Flake: ${TEST}" --body-file evidence.md
Closing checklist
- [ ] Computer-use only on staging + allowlisted domains
- [ ] Artifacts: screenshot, HAR, console, a11y tree
- [ ] No prod credentials in the agent environment
- [ ] Ticket body includes session ID for replay
- [ ] Quarantine SLA after repeated flakes; no silent skip forever
- [ ] Sanitize tracker fields before agent consumption
Related reading
- CI Failure Triage Bots: Separate Flaky Noise From Real Regressions
- Self-Healing Tests: AI Quarantine With Hard Human Fix SLAs
- Prompt Injection in Trackers: Sanitize Jira Before Agents Read Issues
- Secure AI Sandboxes: Ephemeral ECS Tasks for Agent Tool Execution
Related: Vibe Coding: The Professional Developer Guide to Cursor, Claude, and Copilot · Claude Projects vs Cursor: Context Hygiene for Regulated Codebases · Claude Code: The Developer Workflow That Replaces 80% of Manual Coding Tasks
Watch: 100 Days of AI on YouTube
Short videos from CheatCoders, one AI engineering topic per day.
Most viewed
- Python Decorators Explained: From Simple Wrappers to Production Patterns
- Claude Code vs Cursor vs GitHub Copilot: Choose by Workflow, and Set Up One Repo That Works With All Three
- AI Agent Frameworks in 2025: LangGraph vs CrewAI vs AutoGen vs Raw API
- Python String Methods: Every str Method With Real Production Examples
- Css Animation Performance: Performance-Optimized Motion for Modern UIs
Newly added
- AWS Systems Manager Parameter Store: Hierarchical Runtime Config for Coding Agents
- Amazon Cognito Identity Pools: Federate Coding-Agent Tool Callers Without Long-Lived Keys
- Amazon Athena for Coding Agents: Query Tool Audit Logs and Eval Traces on S3 Without a Warehouse
- AWS Batch for Coding Agents: Parallel Overnight PR Reviews and Long-Running Jobs Without Lambda Timeouts
- Amazon Bedrock Prompt Management: Version and A/B Test Coding-Agent System Prompts
Deep-dive PDF
Get the expanded guide for this post — extra diagrams-style checklists, failure modes, and a production walkthrough. Free when you subscribe to CheatCoders.
Already subscribed? or open the subscribe page.
Discover more from CheatCoders
Subscribe to get the latest posts sent to your email.
