Visual Context
Visual Context lets the AI Chat assistant see what is on your screen and, with your permission, interact with other applications. Instead of only reading your TIA Portal project, the assistant can look at an application window, describe what it sees, and, after you confirm, click a button or type into a field for you. This is useful for guiding you through a dialog, filling in a form, or checking what an application is showing.
By default the assistant may look at and act on TIA Portal and AnyAutomation Studio itself, and nothing else. You decide which other applications, if any, it is allowed to touch.
Visual Context requires a Pro plan. On a lower plan the tools are listed but tell you the feature is not included instead of running.
What the assistant can do
Visual Context adds four abilities to the AI Chat, grouped under Visual Context in the chat tool picker. You do not run them yourself; the assistant uses them when your request calls for it, and you can also mention them by name in a message.
| Ability | What it does |
|---|---|
| List Application Windows | Reports the open windows of the applications you allow, so the assistant can find the right one to look at. |
| Capture Screenshot | Takes a picture of a window or element so the assistant can see it. |
| Get Visual Tree | Reads the structure of a window (its buttons, fields, labels and their text) as text, or reads the element you currently have focused. |
| Run Visual Actions | Clicks an element, types text into it, presses a keyboard shortcut, or waits, on the applications you allow. This is the only ability that changes anything, and it always asks you to confirm first. |
A typical flow: you ask the assistant to help with a window, it lists the windows, reads the one you mean, optionally captures a screenshot to see it, and then, if you want it to act, it proposes the actions and asks you to confirm before doing anything.
Choosing which applications the AI may use
Open Settings (Ctrl+,) and search for Visual Context. Two options control what the assistant is allowed to touch:
- Allowed Process Names is a list of the applications the AI may inspect, capture, click, or type into. Add an application's process name to allow it; remove one to block it again. Only the applications on this list (plus this application, if the option below is on) are ever reachable. Everything else stays hidden and blocked, even if the assistant tries.
- Allow Own Process lets the AI inspect and automate AnyAutomation Studio's own windows. It is on by default; turn it off if you do not want the assistant looking at Studio itself.
A window that is not on your allowed list still shows up when the assistant lists windows, but its title is hidden and the assistant cannot click or type into it. If you empty the list and turn off the own-application option, nothing is reachable at all.
Confirming before the AI acts
The read abilities (listing windows, capturing a screenshot, reading a window's structure) run on their own, limited to the applications you allow. The Run Visual Actions ability is different: because it can change another application, it always shows a confirmation prompt before it runs, describing what it is about to do. Nothing happens until you agree, and there is no way to have it skip the prompt, not even in an automated agent run. Decline the prompt and the actions are cancelled.
This means a click or a keystroke in another application never happens by surprise: it is limited to the applications you chose, and it waits for your yes each time.
Seeing screenshots
The Capture Screenshot ability sends a picture to the model. For the assistant to actually read that picture, the model you are chatting with must be able to understand images (a "vision" model). Most current cloud chat models can; if you are using a text-only model, the screenshot is sent but the model cannot interpret it, so pick a vision-capable model in the chat model picker before asking the assistant to look at a screenshot. Reading a window as text (Get Visual Tree) works on any model.
Availability
Visual Context is available on Windows. It is part of the AI Chat and requires a Pro plan; see AI Chat for signing in to a model provider and using the chat.