Remote desktop automation with RDP and VNC

RDP / VNC

Command-driven remote work

Capture the desktop, click, drag, scroll, enter text and send shortcuts. The agent reads the returned JSON and checks the next screenshot to see what actually happened. A successful input dispatch alone does not confirm the task result.

• JSON results, error codes, operation queries and event subscriptions help your agent decide whether to continue, wait or recover. Inspect a fresh screenshot to verify the application outcome.

Take a screenshot, act using its observation token, then capture again.

● ● ●01 / Commands do the work

rdp-cli clipboard set \
  --session SESSION \
  --text "中文输入验证：远端命令完成。" \
  --observation OBSERVATION

rdp-cli key --session SESSION \
  --keys CTRL+V \
  --observation OBSERVATION

rdp-cli screenshot \
  --session SESSION --out result.png

RDP / The result, in the actual application

After every action, look at the result. Let the remote screen guide the next step.

RDP / VNC

RDP and VNC in one workflow

Connect to an existing RDP or VNC server from Windows, macOS or Linux. Use the same screenshot, input and viewing workflow. Remote access must already be enabled and reachable; this tool does not create a hosted remote desktop.

• Use the same session commands for both protocols. RDP supports local and domain accounts; VNC uses the authentication supported by the server.

Choose RDP for a Windows desktop, or select VNC for an existing VNC server.

Connect from Linux to Windows RDP with one executable →

Connect to VNC and verify input from the command line →

RDP / VNC

Several desktops. Persistent sessions.

Keep multiple sessions connected and address each one by its session ID. Closing a read-only viewer keeps the connection. Reconnection invalidates old observations and leaves control paused; review the current screen before explicitly resuming.

• One command can finish while the remote connection remains open. Session IDs keep later actions attached to the right desktop; reconnecting pauses input for a fresh observation.

List current sessions, inspect one session, and disconnect only the session you no longer need.

Troubleshoot RDP connections, black screens and blocked input →

RDP / VNC

Watch, pause and take over

Open a native window or an authorized browser to view an existing session. Several people can observe, but only one viewer can own human control. Taking control stops agent input; releasing it hands the session back. Losing the controlling viewer pauses input.

• Opening another viewer reuses the existing connection. Several people can watch; one controller acts at a time. After a person hands control back, the agent observes the updated screen.

Watch a task, take control for a correction, then return the same session to the agent.

View original screenshot ↗

See a real handoff +

Recorded on Windows 11 through one RDP session. The agent writes a checklist; a person interrupts and reviews it.

1. 01 Watch the agent work

   Typing hints appear as rdp-cli sends text to the remote application.

   View original screenshot ↗

2. 02 Pause the task

   The viewer pauses input. The unfinished sentence remains visible.

   View original screenshot ↗

3. 03 Take control and review

   The person takes control and adds “Human review: approved.” in the same document.

   View original screenshot ↗

4. 04 Release and verify

   After release, the agent takes a fresh remote screenshot, checks the review and saves the result.

   View original screenshot ↗

The finished file was retrieved through the exchange drive and verified with SHA-256. Download the actual result (504 B)

Original English recordings, captured on 21 September 2026. Screenshot pixels are unchanged by website language or theme.

Menu bar: a second real handoff

Recorded on 23 September 2026. The menu panel and the final observation below belong to one Windows 11 RDP session.

View original screenshot ↗

View original screenshot ↗

Original captures: the Mac panel is in Chinese and the remote desktop is in English.

RDP / VNC

Sessions within reach

On Mac, click the menu bar icon for a searchable list of current sessions. Windows provides a notification-area menu. See the target, connection state, controller and latest action; open a viewer, pause or disconnect from the available session controls. On Linux, use the searchable session panel; a status icon is also available when the desktop supports it. Without a tray, open the panel with presence show --panel.

• Find a Mac session by name or address, or select it from the Windows tray. Pause all sessions from the status menu when you need a break. On Linux, use the searchable session panel; a status icon is also available when the desktop supports it. Without a tray, open the panel with presence show --panel.

Find an active desktop by its name or address. This is a current-session list, not a saved-connection address book.

RDP / VNC

Understand each action

Click rings, drag paths, scroll directions, shortcuts and text-entry progress show what was sent. Each viewer can turn these hints off. Text hints do not repeat the entered content and are separate from the remote pixels.

• Pointer motion, click rings, drag paths, scroll directions and shortcut hints explain what was sent. Text progress avoids repeating the input; the remote screen itself remains visible.

Watch where the agent clicks. The remote screen itself is not automatically redacted.

RDP / VNC

Use your existing access

Connect with a local or domain account. Save and reference credentials using the supported credential store, and grant browser resources explicitly. RDP certificate settings and VNC security modes have different boundaries; choose them for your environment.

• Save a credential in the system credential store and reference it later. Give browser viewers only the resources they need, and revoke their access when the task ends.

Separate permission to view a session from permission to control input, files or microphone.

RDP

Work across screens

For a negotiated multi-monitor RDP session, capture the full desktop or one monitor, target a screen and drag across screens. Native viewers can open separate monitor windows; browser viewers can switch screens. Resizing a viewer changes its presentation, not the remote layout.

• Capture and target individual displays. Drag across screens.
• Open per-display native windows, or switch screens in the browser.
• View a remote dual-display workspace even from a single-screen computer.

Inspect one screen while keeping an overview of the complete desktop.

RDP

Exchange files and folders

Upload to the session exchange drive, process the document remotely, then download it. CLI, native and browser views share transfer progress and cancellation. Overwriting an existing file is explicit. RDP drive redirection must be allowed by the server.

• Upload and download files or folders with their directory structure.
• Follow individual transfer progress and results. Cancel when needed.
• Choose whether to replace existing files; transferred content is verified.

Transfer a report folder, edit a document remotely and retrieve the saved result; this is not arbitrary access to remote disks.

1. Upload

   Files and folders → session exchange drive

2. Process remotely

   Open, edit and save the resulting file

3. Download

   Follow transfer progress and verify the completed file

View original screenshot ↗

See the real spreadsheet and video workflows →

Upload and download files over RDP from the command line →

RDP / VNC

Use the remote text clipboard

Read or set remote clipboard text explicitly, and paste it with a shortcut. This does not automatically synchronize your local clipboard. RDP clipboard access depends on server policy; VNC text encoding depends on negotiation.

• Move an address, a paragraph or tabular text through the remote clipboard. The type command can enter supported text directly without replacing the clipboard.

Set the remote clipboard, send CTRL+V and inspect the application result.

RDP

Hear the remote desktop

Play RDP sound with separate playback controls in each viewer. A person can explicitly enable microphone capture while holding control, subject to system permission and server support. Release or disconnect stops capture; it does not restart automatically.

• Multiple authorized viewers can listen at the same time.
• Take control and explicitly enable a microphone when voice input is needed.
• Handing back control or disconnecting stops capture. Enable it again to resume.

Listen to a remote playback task; turn on the microphone only when you need voice input.