Research

Watch · narrated whiteboard episodesL2

Computer-Use Agents: The Screen as Untrusted Input

A computer-use agent perceives a screen and acts by clicking and typing — and the moment the display it reads is attacker-influenced, the pixels on screen become an untrusted input that drives real actions. This threat lab maps the perceive-ground-decide-act-observe loop as a trust boundary, showing where hostile display content enters and what it can make an agent do. Each exposure — UI-grounding decoys, action hijack and irreversibility, the environment as adversary — is taught by the assumption it breaks, the mechanism that makes it work, and the containment control that limits the blast radius. Product-agnostic, grounded in the GUI-agent and computer-use attack literature, and tied back to the AI-agent stack every time.

Murali Chillakuru·5 episodes
  1. 17 min Episode 1The Perceive-Act Loop: Why a Screen-Driven Agent Turns the Display Into a Trust BoundaryA moderator and a staff-level security researcher map the computer-use loop as a trust boundary — where hostile screen content enters, how it's laundered into a legitimate-looking click, and why you can only harden the two ends.
  2. 16 min Episode 2UI-Grounding Attacks: Decoy Elements, Overlay and Z-Order Tricks, and Pixel-Level SteeringA moderator and a staff-level security researcher show how an attacker leaves a screen-driven agent's plan pristine and bends only where that plan lands — and why the fix lives at the intent-to-element step, not in the reasoning.
  3. 16 min Episode 3Action Hijack and Irreversibility: Blast Radius When an Agent Can Click AnythingA moderator and a staff-level security researcher analyze the consequence side of computer-use agents — the unbounded action space, the reversibility axis, and the containment rule that keeps a hijacked click from becoming an incident.
  4. 16 min Episode 4The Environment as Adversary: Malicious Apps, Pop-Ups, and Content-as-Instruction on ScreenA moderator and a staff-level security researcher treat a computer-use agent's whole environment as an active attacker — how on-screen content becomes instruction, why telling the agent to ignore it fails, and the structural controls that hold.
  5. 18 min Episode 5Containment for Computer-Use: Sandboxes, Action Gates, Human-in-the-Loop, and ReversibilityA moderator and a staff-level security researcher assemble the whole series into one containment architecture — sandbox, provenance, corroboration, action gate, human oversight, and reversibility — that makes a screen-driven agent's failures survivable.