Can an AI assistant click buttons in TradingLead Desktop?

Yes. Since Desktop 0.1.9 for Apple Silicon Mac, native controls such as ui-click, ui-type, ui-scroll and ui-drag operate the selected chart tab. Login, credentials, payments, broker setup, AI permissions and Practice Labs are protected and stay human-only.

Do I have to approve every AI click?

Not in current builds. In 0.1.9 and 0.1.10, UI control needed human approval that lasted 10 minutes. From 0.1.11, control is prompt-free once you have an active account and local assistant access. You can stop it at any time with Help > Stop AI control and recording or Cmd/Ctrl+Shift+Escape.

Where are AI recordings saved, and are they uploaded?

Legacy recordings are saved privately under ~/.tradinglead-desktop/recordings and are never uploaded automatically. They capture only the selected tab, with no microphone or system audio. Screencast recordings save to the local path the assistant requests.

What video quality can it record?

From 0.1.12, recordings use native resolution by default, with optional cropping to an element, at 1 to 60 FPS (30 by default). The 0.1.14 screencast engine writes constant-frame-rate MP4 or WebM, but MP4 needs ffmpeg installed locally.

Can it turn my own clicks into a script?

Yes, on 0.1.15 and later. Record mode captures your real clicks, typing, scrolls, drags and keys and returns a timed ui-run batch. Password, OTP and payment fields are never read, and those steps are left for you.

Is this available on Windows?

Not in the public Windows build, which is still 0.1.7 and predates managed AI integration. These features are in the Apple Silicon Mac build.

Feature

AI UI Control and Demo Recording in Desktop

Since Desktop 0.1.9 for Apple Silicon Mac, an AI assistant can operate the selected TradingLead chart tab with native clicks, typing, scrolling and dragging, and record that tab to a private local video. The current build, 0.1.18, adds scripted ui-run step batches and form, dialog and download handling. It can record your own actions as a replayable script with sensitive fields left out, offers a separate demo browser, and can select and highlight text. Login, credentials, payments, broker setup and Practice Labs stay human-only.

What this feature is

Most AI control of TradingLead goes through named product commands such as chart.open or screening.run; see TradingLead commands. Some tasks need more than that: making a product demo, testing a flow end to end, or pressing a button that has no command. For these, TradingLead Desktop adds native UI controls. An AI assistant such as GitHub Copilot CLI or Claude Code can then click, type, scroll, drag and record inside the selected chart tab.

These controls are part of the Desktop app shell, not the website. They are listed by tradinglead desktop capabilities. TradingLead uses no MCP server, browser-debugging endpoint or arbitrary JavaScript for them. They arrived in Desktop 0.1.9 for Apple Silicon Mac. The current public Mac build is 0.1.18. The public Windows build is still 0.1.7, which predates all AI integration.

How it evolved, release by release

Desktop (Mac)	What it added
0.1.9	Native clicks, right-clicks, scrolling, typing and keys in the selected chart tab, after human approval lasting 10 minutes. Protected areas include login, credentials, payments, permissions and Practice Labs. Local video-only recording with a Stop & save panel
0.1.10	One shared 10-minute approval across actions, so far fewer prompts
0.1.11	Prompt-free control for an active account with local assistant access. Pause, Stop, tab and account isolation and retry protection remain
0.1.12	No floating control window; stop with Help > Stop AI control and recording or Cmd/Ctrl+Shift+Escape. Drag, focused-page keys and two-axis scrolling. Recording at native resolution with optional cropping, 1–60 FPS (30 by default)
0.1.14	Targeting by CSS, text, role or test ID; waits; smooth pointer movement; viewport sizing; screenshots; a chart-coordinate helper. A screencast engine with constant frame rate and a timestamped events sidecar
0.1.15	`ui-run` step batches with optional recording. Form filling, selects, checkboxes, uploads and assertions. Dialogs, download, popup and network waits. A step trace viewer. Record mode, which turns your own actions into a replayable script. Mocked responses for clean demo data
0.1.16	Optional "AI" cursor badge and glowing frame, both off by default
0.1.17	A demo browser for any https website in an isolated session. Page scraping. Network capture with HAR export
0.1.18	Real mouse-drag text selection, including across links and lines. Text highlighting with marker, underline or box, plus optional label and spotlight

(Release 0.1.13 was a loading-spinner fix.) Older builds keep their original behaviour. An assistant should read the installed build's capabilities rather than assume the newest features exist.

What the assistant can do

Operate the chart tab. The assistant can:

click, double-click and right-click;
hover and move the pointer;
drag over the chart canvas (wheel scroll pans and zooms the chart);
type up to 16,000 characters, and press keys.

Targets can be CSS selectors, visible text, roles, test IDs or screen coordinates. ui-chart-point converts a candle time and price into a screen position, so drawings and clicks land on the right bar.

Script whole flows. ui-run executes up to 300 validated steps inside the app with no round trips, so a recording has no dead air. A step can be a click, fill, select, check, upload, wait, assertion (ui-expect) or highlight. If control is lost, the batch stops immediately. ui-trace-start and ui-trace-stop save a step-by-step trace for debugging.

Handle real app behaviour. The assistant can:

answer or hold dialogs (ui-dialog);
capture the next download into ~/.tradinglead-desktop/downloads (ui-download-wait);
follow popups;
wait for specific network requests or responses.

ui-route can mock up to 20 XHR or fetch responses on the current tab for clean paper-trading demo data. Sign-in, session, payment and broker or order URLs are refused and never mocked.

Record. record-start, record-status and record-stop capture only the selected tab. There are two engines:

Engine	Output	Notes
Legacy	VP8 WebM, video only	1–600 seconds (60 s default), 64 MiB per clip, one recording at a time. Saved privately under `~/.tradinglead-desktop/recordings` and never uploaded
Screencast (0.1.14+)	Constant-frame-rate H.264 MP4 or WebM	1–60 FPS at native pixels, optional crop. Writes an `events.json` sidecar that logs every UI action and each `record-mark` cue, which helps when editing. Without ffmpeg installed locally, it saves timestamped frames instead of a video

Neither engine records a microphone, system audio or the whole desktop. Tab or account changes stop the capture. The assistant should not report a clip until its state is completed.

Make demos readable. The assistant can show a styled cursor with click ripples and on-screen keystrokes. It can highlight an element with a label or spotlight, select text with a real drag and mark phrases (0.1.18). The "AI" badge and glowing frame are optional and off by default.

Record your actions as a script. In record mode (ui-codegen-start / ui-codegen-stop), you demonstrate a flow while a "Recording your actions" badge is shown. The assistant gets back a timed ui-run batch it can replay or polish. Password, OTP and payment fields are never read; those steps are left for you. Hover-only movement and cross-origin iframes are not captured.

Demo other websites. The demo browser (demo-browser-open, tab demo:main) is a separate window that can open any https site, or http on localhost and private addresses, for polished recordings.

Isolation. It runs in its own isolated session, so your TradingLead login, broker tokens and app cookies are unreachable from it.
Denied by default. Downloads and permission requests (camera, microphone, location and so on) are refused.
Human-only fields. Password, OTP, card, payment, sign-in and broker controls stay with you.
Local data. Scrapes (ui-scrape) and network captures (JSONL or HAR, with sensitive headers left out) are saved locally under ~/.tradinglead-desktop. Response bodies are kept as-is, so treat these files as private.

Safety and approval model

You enable it. UI control requires an active, visible, ready /charts tab and local assistant access for your signed-in account. Agents cannot grant themselves access.
Approval history. 0.1.9 and 0.1.10 required human approval that lasted 10 minutes. From 0.1.11, control is prompt-free once access is on. Pause and Stop remain.
You can stop it instantly. Use Help > Stop AI control and recording or Cmd/Ctrl+Shift+Escape. Switching or closing the tab, pausing access in the AI Workspace or signing out also ends control.
Protected areas stay human-only. These are login, broker setup, credentials, purchases, AI permissions and Guided Practice Labs. Native OS shortcuts, file pickers, clipboard automation and arbitrary JavaScript are unavailable.
No live trading. Order tickets in TradingLead are paper simulations, and nothing here places a broker order.

Example requests

"Record a 30-second 60 FPS MP4 of adding RSI to this chart, with the reel cursor and on-screen keys."
"I'll show you how I set up my layout. Record my actions and give me a replayable script."
"Open our landing page in the demo browser at 1280×720 and highlight the pricing headline while recording."

More examples are in the AI prompt cookbook. Setup is in control TradingLead with AI, and the full agent overview is on AI agent control. Download Desktop.

Limitations

Apple Silicon Mac only in public builds; Windows public 0.1.7 has no AI integration.
Works on an active, visible, ready /charts tab (or the separate demo browser); tab switches, sign-out or pausing access end control.
Human-only: login, broker setup, credentials, purchases, AI permissions, Guided Practice Labs; native OS shortcuts, file pickers, clipboard automation, cross-origin frames and arbitrary JavaScript are unavailable.
Recordings are video-only with no audio; MP4 screencasts need ffmpeg installed locally.
Legacy recorder: 1-600 seconds, 64 MiB per clip, one recording at a time.
Demo browser denies downloads and permission requests; scrapes and network captures can contain private data and stay local.

Frequently asked questions

Can an AI assistant click buttons in TradingLead Desktop?: Yes. Since Desktop 0.1.9 for Apple Silicon Mac, native controls such as ui-click, ui-type, ui-scroll and ui-drag operate the selected chart tab. Login, credentials, payments, broker setup, AI permissions and Practice Labs are protected and stay human-only.
Do I have to approve every AI click?: Not in current builds. In 0.1.9 and 0.1.10, UI control needed human approval that lasted 10 minutes. From 0.1.11, control is prompt-free once you have an active account and local assistant access. You can stop it at any time with Help > Stop AI control and recording or Cmd/Ctrl+Shift+Escape.
Where are AI recordings saved, and are they uploaded?: Legacy recordings are saved privately under ~/.tradinglead-desktop/recordings and are never uploaded automatically. They capture only the selected tab, with no microphone or system audio. Screencast recordings save to the local path the assistant requests.
What video quality can it record?: From 0.1.12, recordings use native resolution by default, with optional cropping to an element, at 1 to 60 FPS (30 by default). The 0.1.14 screencast engine writes constant-frame-rate MP4 or WebM, but MP4 needs ffmpeg installed locally.
Can it turn my own clicks into a script?: Yes, on 0.1.15 and later. Record mode captures your real clicks, typing, scrolls, drags and keys and returns a timed ui-run batch. Password, OTP and payment fields are never read, and those steps are left for you.
Is this available on Windows?: Not in the public Windows build, which is still 0.1.7 and predates managed AI integration. These features are in the Apple Silicon Mac build.

Last updated: 7 November 2026