Research seriesL2offensive security
A computer-use agent perceives a screen and acts by clicking and typing — and the moment the display it reads is attacker-influenced, the pixels on screen become an untrusted input that drives real actions. This threat lab maps the perceive-ground-decide-act-observe loop as a trust boundary, showing where hostile display content enters and what it can make an agent do. Each exposure — UI-grounding decoys, action hijack and irreversibility, the environment as adversary — is taught by the assumption it breaks, the mechanism that makes it work, and the containment control that limits the blast radius. Product-agnostic, grounded in the GUI-agent and computer-use attack literature, and tied back to the AI-agent stack every time.
A computer-use agent acts on what it sees, so the screen stops being an output and becomes an untrusted input that drives real actions.
The attacker does not need to change the agent's plan — only where its correct plan lands, by manipulating the fragile step that turns intent into a click.
The danger of a screen-driven agent is not that it acts, but that a single action can reach arbitrary software and cannot always be undone.
A screen-driven agent reads its environment as both data and instruction, so any hostile party that can put content on screen can try to command the agent.
You cannot prevent every manipulation of a screen-driven agent, so the discipline that makes one deployable is containment: bounding what any failure can reach.