Agent screenshot and UI expression

Agent screenshot and UI expression

One-liner

Agents “express visually” three ways: (1) see via Playwright/browser MCP screenshot loops, (2) design-canvas MCP (Doop get_frame_screenshot), (3) emit UI via catalog JSON (json-render / OpenUI) or artifacts/canvas. Field loop: implement → screenshot → multimodal critique → patch; pair with frontend-design + webapp-testing skills. Storybook MCP lets the agent see components, not only pages.

Pin checked 2026-10-02 CST. Primary: X + Exa how-tos. Related: 2026-10-02 Dual-model Sol Codex Claude UI advisor prompts · 2026-10-02 JSON generative UI for agents · 2026-10-02 Suggestive collaborative agent prompting (ungrillable taste → prototype/screenshot).

---

1. Why visual expression matters

Text-only agents invent “looks fine.” Field reports: attach a screenshot and the same model suddenly finds hierarchy/clutter/contrast issues. Joseph Langford: screenshot results → ask Claude/Codex to review + return fix prompts with PRD/spec — josephlangford_/2011113641727398003. Danielle Morrill on Claude browser MCP for frontend PR testing — DanielleMorrill/1985528615094927840.

Ungrillable UX (“how should this feel?”) from grill-me docs → stop talking, ship a throwaway, look, then decide — same loop with browser eyes.

---

2. Pattern A — Playwright / browser MCP screenshot loop

Setup (Claude Code)

Simon Willison TIL + Shipyard:

claude mcp add playwright npx '@playwright/mcp@latest'

Then: “Use playwright mcp to open … take a screenshot.” First turn: name “playwright mcp” so it doesn’t shell raw Playwright. Tools include browser_take_screenshot, browser_snapshot (a11y tree — often better for structure than pixels alone), navigate/click/type.

Refs: til.simonwillison.net/claude-code/playwright-mcp-claude-code · shipyard.build/blog/playwright-mcp-screenshots · playwright.dev/mcp

Field loop shapes

Loop	Who / where	Shape
Implement → screenshot → visual review	Parent lead: emanuel_build; Exa how-tos (Shipyard visual diffs, claude-codex.fr design+Playwright, qaskills screenshot verification)	Code → browser_take_screenshot → critique → patch → repeat
Mockup ↔ screenshot WA	2034_SWE — 2059622186021777907	Imagegen/mockup → implement 10% fidelity → Playwright/browser-use screenshot → compare → refine
Figma/design vs live	biscotti_roti — 2087939118265151921; rfrolov.me Figma MCP compare	Grab design + loop Playwright screenshots until match
Self-check after UI change	Parent lead: DeRonin_; Danielle Morrill; rubenpdegroot wanting browser verify + CodeRabbit	Agent must not claim UI done without a screenshot pass
Manual attach still works	josephlangford_; Cursor tip screenshot→describe→code	No MCP required; slower

Caveat from field: Jess.py — Playwright MCP screenshots alone ≠ ChatGPT agent “browse like a user” for UX judgment (ldadwda/1973391039726502285); use screenshot for layout/bugs, human or stronger multimodal pass for taste.

Anthropic skills

• frontend-design — anti-slop UI rules (CodeByPoonam list — CodeByPoonam/2100580893668893172; Sabrina_Ramonov beginner five skills including frontend-design + webapp-testing).
• webapp-testing — Playwright scripts + screenshot/reconnaissance-then-action (anthropics/skills webapp-testing): wait networkidle, screenshot, discover selectors, act. Prefer helper scripts (with_server.py) over stuffing huge scripts into context.

Also: canvas-design, theme-factory, web-artifacts-builder in Anthropic skill lists.

Storybook + browser MCP

Parent lead arb5z: browser MCP + Storybook so agent sees components. Exa: Claude Code × Storybook CDD playbook; frontend MCP servers with storybook_story_run + page screenshots. Pattern: isolate component in Storybook → screenshot story variants (empty/loading/error) → fix → re-screenshot.

---

3. Pattern B — Design canvas MCP (Doop)

Doop MCP (get_frame_screenshot): render a design frame headlessly → PNG so the agent can iterate on layout without a full app boot (mcpvault Doop tool; parent lead: santhosh_patell). Use when expressing proposed UI before implementation, or live design canvas review.

Complementary: Operator Kit “Screenshot Operator” / annotated screenshots via MCP (operatorkit.ai) — less coding-loop, more annotation.

---

4. Pattern C — Express as UI (not only screenshots)

When the agent should show structure to the human mid-chat:

Approach	Mechanism	Digests / refs
json-render	LLM emits constrained JSON component tree → renderer	2026-10-02 JSON generative UI for agents
OpenUI	Compact DSL (~fewer tokens than JSON)	same
AI SDK / CopilotKit GenUI	Tool result → fixed React component	same
Artifacts / canvas	Chat-hosted interactive preview	Claude artifacts; Anthropic canvas-design skill
Meta-harness mockups	Parent lead: rchodava — UI mockups of changed screens for review before/with PR	Pair with suggestive OK-gate

Dual-model note: Sol implements; Claude advisor critiques with screenshots attached — see 2026-10-02 Dual-model Sol Codex Claude UI advisor prompts.

---

5. Pasteable prompts / CLAUDE.md bits

Snippet 1 — Visual verify gate (CLAUDE.md)

## UI / frontend changes
After any user-visible UI change:
1. Start or use the running app (or Storybook story).
2. Use Playwright MCP: navigate, wait for network idle, take screenshots
   at mobile (390) and desktop (1440) for the changed routes/states.
3. Critique hierarchy, spacing, contrast, clutter, empty/error states.
4. Patch, then re-screenshot. Do not claim "done" without at least one
   screenshot pass attached or saved under /tmp or docs/ui-shots/.

Snippet 2 — Session prompt (implement → review)

Implement the change. Then use Playwright MCP to open the affected page,
capture full-page screenshots for default / empty / error if applicable,
and do a visual review against DESIGN.md / our component library.
Return: (1) what looks wrong, (2) concrete CSS/component patches, (3) re-shot after fix.
Do not skip the screenshot step.

Snippet 3 — Anthropic webapp-testing posture

Use the webapp-testing skill. Reconnaissance-then-action: networkidle →
screenshot + DOM inspect → discover selectors → act. Prefer scripts/with_server.py
helpers; don't dump large Playwright scripts into context.

Snippet 4 — Generative UI expression (when catalog exists)

Do not only describe the UI in prose. Emit a json-render spec using only
catalog components so I can see the layout. Then we approve and implement.

(Details: 2026-10-02 JSON generative UI for agents)

Snippet 5 — Prototype escape from grilling

This decision is ungrillable (taste/feel). Build a throwaway prototype,
screenshot it with Playwright MCP, then I'll pick. One round only.

---

6. Wiring cheat-sheet

suggestive plan (/grill-me or BLUEPRINT)
        ↓ OK
implement (Codex/Claude/Cursor)
        ↓
eyes: Playwright MCP | Storybook story | Doop frame
        ↓
multimodal critique (same agent or Claude advisor)
        ↓
patch → re-screenshot → merge gate

Optional: DESIGN.md / interface-design plugin (system.md) so critique isn’t free-form slop — interface-design.dev · prior dual-model digest.

---

Sources

X

• josephlangford_ — screenshot → review → fix prompt
• DanielleMorrill — Claude browser MCP frontend PR
• 2034_SWE — mockup ↔ Playwright screenshot loop
• biscotti_roti — Playwright screenshot vs designs
• CodeByPoonam — Frontend Design + design skills list
• Sabrina_Ramonov — frontend-design + webapp-testing
• vishwastam — Claude build + browser MCP verify
• Alexnicolai — screenshot → prototype in browser
• ldadwda — Playwright MCP UX limits vs ChatGPT agent
• rubenpdegroot — browser MCP + code review verify

Exa / docs

• Playwright MCP · Willison TIL · Shipyard screenshots · Shipyard visual diffs GHA
• anthropics/skills webapp-testing · frontend-design skill (same org)
• Doop get_frame_screenshot — mcpvault
• Storybook + Claude CDD guides (claudelab.net Exa hit)
• 2026-10-02 JSON generative UI for agents — json-render / OpenUI

Reddit / HN

• Reddit blocked from box; Exa Reddit thin this pass — gap.
• HN: Willison Playwright MCP TIL; Show HN interface-design.dev consistent UI.

Gaps

• Direct X posts from emanuel_build, arb5z, DeRonin_, rchodava, santhosh_patell were rate-limited / not returned in from: queries this session — patterns above reconstructed from parent leads + matching Exa/X corpus; re-pull those handles when X API cools.
• Doop: tool page found; full santhosh thread not fetched.