Screen
Give a box a display, watch your agent work on it live, and take control when you need to.
A box can have a screen: a virtual display at 1440 × 900. With it on, your agent can run programs that need a real window, such as a headed browser, and you can watch them live from the dashboard.
Most work doesn't need one. Scripts, builds and headless browsers run without a screen, and box_screenshot takes a picture of a page on any box. Turn the screen on when something has to run in a window: clicking through an app, canvas and WebGL work, a login a headless browser can't get past, or a job you want to watch.
Turn it on
Any of these turns the screen on:
- The Screen switch in the New box dialog, when you make the box.
- Turn on screen on the box's Screen tab. The box has to be awake; the first time can take up to a minute.
- Your agent, by calling
box_screen, or by passingscreen: truetobox_create.
Once it's on, every command your agent runs has DISPLAY set, so programs it starts draw on the screen. The screen stays on until you or your agent turn it off, and it comes back on when the box wakes.
Watch and take control
The box's Screen tab shows the display live. It starts view-only, so you can't disturb your agent by accident. Take control lets you click and type too, for example to sign in to a site yourself or get past a prompt, and your agent carries on from there.
- Up to three people can watch a box's screen at once.
- Watching doesn't keep a box awake. An idle box still sleeps after 10 minutes, so a bigger box isn't billed just because its tab is open. Clicking and typing while you're in control count as activity.
- If the connection drops, the viewer reconnects by itself. When the box goes to sleep, the tab offers Wake box.
Turn off on the Screen tab closes the display and the programs on it.
What your agent does with it
box_screen returns a picture of the whole display. It can also drive it, with a list of actions that run in order before the picture is taken:
| Action | Example |
|---|---|
| Click | {"click": {"x": 200, "y": 120}} (add "button": 3 for a right click, "double": true for a double click) |
| Move the pointer | {"move": {"x": 10, "y": 10}} |
| Type text | {"type": "hello"} |
| Press keys | {"key": "ctrl+l"} |
| Scroll | {"scroll": {"dy": 5}} (positive scrolls down) |
| Wait | {"wait_ms": 500} |
off: true turns the screen off.
A typical job starts a headed browser with box_exec, then looks and clicks with box_screen. Playwright's Chromium is already installed, and a setup with a chrome step adds an exact Chrome build at $CHROME_PATH. Two launch flags matter on the screen: --app=URL opens a window without tabs or an address bar, and --test-type hides the warning bars that would otherwise push the page down and move every click.
Sleep and the screen
The screen comes back on when the box wakes. On a standard box, the programs on it are still there. Large and XL boxes restart their programs on wake, so your agent opens the browser again (Sizes).
Screens and the GPU
On a GPU box, browsers on the screen draw on the GPU when launched with $WACK_CHROME_GPU_FLAGS. On other sizes the screen has no GPU: browsers draw WebGL and WebGPU on the CPU, which is slower but very repeatable. Pixel tests explains how to compare screenshots between runs.
Who can see it
Only you, signed in to the dashboard. Each viewer connection uses a ticket that works once and expires after a minute, and your browser connects to wack, never to the box itself. Security has the rest.
Last updated