SCL Unit Testing for TIA Portal PLCs

A unit test in TIA Portal checks a single PLC block (FB/FC) in isolation: write defined inputs (Arrange), let the controller run a few cycles (Act) and check the outputs against expected values (Assert) — against PLCSIM Advanced or a real S7 PLC, automated and repeatable.

Integrated unit test framework for TIA blocks. Write, run and evaluate tests against a live PLCSIM Advanced instance without leaving the app.

Requirements:

Pro+ license or higher
PLCSIM Advanced V3.0+ installed (separate Siemens license)
A TIA project with at least one PLC open

Opening the Unit Testing Workspace

Click the Unit Testing icon (beaker) in the Activity Bar on the far left
The Side Bar then shows the two Unit Testing views:
- Test Suites — the explorer tree of discovered .tia-tests suites, with title-bar actions New Test Suite, Refresh Test Suites, Run All Test Suites / Stop Test Run, and a per-suite inline Run Test Suite
- Test Results — live progress and per-assertion details (Total / Passed / Failed / Errors)
Opening a suite from the tree opens it as a Test Suite Editor in the editor area in the center. The editor shows the same suite in three switchable views — Visual, JSON and SCL — plus an Open as JSON Text escape hatch; its toolbar holds Save, the Visual / JSON / SCL view toggle, Open as JSON Text, Validate, and — in Visual mode — Add Test Case and Generate from Boundaries, and — in SCL mode — Import to TIA.

Test Explorer

The Test Explorer shows the test suites of the TIA project you are currently connected to — every .tia-tests/*.json file appears as a suite node with its test cases as children. While no test folder is chosen and no project is connected, the view reads "Connect a TIA project or choose a test folder to load your test suites."; once a folder is chosen its suites appear automatically (a chosen folder with no suites shows "No test suites yet."; if the chosen folder no longer exists, it shows "Selected test folder not found: <path>").

Status icons (loaded from SQLite on startup):

✓ Pass — green checkmark, test passed in the last run
✗ Fail — red cross, test failed in the last run
⚠ Error — orange warning, test errored (exception, timeout, S7 error)
⊘ Skipped — grey slashed circle, test was filtered out of the last run
○ NotRun — grey empty circle, no persisted result yet

Interactions:

Action	Result
Double-click a suite (or select it and press Enter)	Opens the suite as a Test Suite Editor in the editor area
Hover a suite → inline Run Test Suite	Runs this suite only
Run All Test Suites (view title bar)	Runs all visible suites sequentially
Stop Test Run (view title bar, while a run is in progress)	Cancels the current run at the next safe point
New Test Suite / Refresh Test Suites (view title bar)	Create a new suite, or rescan the `.tia-tests/` folder
Choose Test Folder / Clear Test Folder (view title bar)	Pick which folder your test suites live in (when a folder is chosen the list follows it, otherwise the connected project), or clear the selection again

A filter box at the top of the Test Suites view narrows the tree by suite and case name, a Show Failed Only title-bar toggle hides everything that didn't fail, and a per-case Run Single action runs just one case.

Planned for the in-app workspace. A Run Affected Only action (re-run only suites whose underlying block changed) is planned. Today, single-suite, single-case, and all-suites runs are available as above; on the command line, tia-test-runner run --rerun-affected already runs only the suites whose underlying block changed (see CI/CD Integration).

Planned — Block-Change Badge. A small orange dot next to a suite whose underlying PLC block changed since its last run, with a "Block changed since last run" tooltip, is planned for the Test Suites view.

A test case that has been unstable across its recent runs (passing some times, failing others) carries a Flaky badge next to its name, with a tooltip showing how many times its result changed. This is computed from the run history, so it appears once a case has both passed and failed in recent runs. The badge clears on its own once the case settles back into a steady result, so a test you have fixed stops being flagged after a few consistent runs, and a case that now fails every time is shown as a real failure rather than flaky.

Test-Case Metadata

Each test case in the visual editor of a suite has an expandable Metadata pane (collapsed by default, located below the variable-watch list). Click the section header to expand it and edit the following fields per case:

Field	Purpose
Order	Numeric run order. Leave empty to keep the natural order in the suite
Tags	Comma-separated labels (e.g. `safety, regression`) used for filters in HTML reports
Priority	One of Low / Normal / High / Critical. Drives report sort order
Owner	Free-text owner (team name, e-mail, etc.)
Requirements	Comma-separated requirement IDs (e.g. `REQ-123, REQ-456`) for traceability

Schema-validation issues that target a specific test case (such as an out-of-range priority) are shown in a red panel directly inside the affected case's Metadata pane. Issues that target the suite as a whole still appear in the suite-level validation banner. All edits are auto-saved with the rest of the suite. Suites created before this release load with empty metadata defaults — no migration is required.

Live Progress During a Run

While a test is running, the workspace shows:

Status Bar: A progress indicator shows that the run is in progress (e.g. "Running '<suite>'…") while the runner works through its phases — creating the PLCSIM instance, compiling, connecting, running each case
Test Suites view: Status icons update live per test case (pass/fail appear as each case finishes)
Test Results view: The test case list fills up incrementally as results come in
Stop Test Run: Cancels the current run at the next safe point (the S7 connection and PLCSIM instance are always cleaned up)

Results Panel

After a run completes, the Results panel shows:

Summary: Total / Passed / Failed / Errors / Skipped counts
Test case list: Every case with its status icon and name; a failing case also shows its error message, and a case demoted by flaky-quarantine carries a Quarantined (flaky) badge
Detail grid (per case): Every assertion with Variable, Operator, Expected, Actual, ✓/✗

Variable Timeline chart. The in-app Test Results tab already renders a chart showing how each watch variable changed cycle by cycle during the run, refreshed live as test events arrive. The same watch data is also rendered as a time-series chart in the HTML reports produced by the tia-test-runner command-line tool (see CI/CD Integration).

To collect watch data, add a "watch": ["Var1", "Var2"] array to a test case in the suite JSON and set "cycles" greater than 1. The sampling is capped at 100,000 cycles per test case.

Test results are persisted to %LocalAppData%\AnyAutomation Studio\db\test_results.db — they survive app restarts and reappear in the Test Suites view automatically.

Run History

Run History, Compare Runs, and Trend are live tabs in the in-app Unit Testing full-pane editor (alongside Test Results and a Coverage tab). Open the pane via the Open in Full Pane action in the Test Suites view title bar, or from the Command Palette → Open Unit Testing. The tia-test-runner command-line tool produces the same information as HTML / CSV reports (see CI/CD Integration).

The Run History view lists every run that has been persisted to the database, regardless of which suite executed.

Columns: Status icon, Started At (local time), Duration, Pass / Fail / Error / Skipped counts, Suite, PLC, Hostname, TIA version
Status icon: ✓ green when all cases passed, ✗ red on any failure or error
Filter box: Filters the list client-side by hostname, suite name, PLC, or block name (case-insensitive)
Refresh: F5 reloads the list (or use the Refresh entry in the context menu)

Context menu actions (right-click a run):

Action	Result
Open Results	Loads that run into the Test Results view so you can inspect each case
Compare To…	Marks this run as the baseline and opens the Select run to compare dialog. Pick a second run and the Compare Runs view renders the diff
Delete Run	Permanently removes the run row plus all its test cases and provenance metadata. Block hashes survive so block-change comparisons keep working
Export HTML	Renders the run as a self-contained HTML report (`<run-id>.html`) with the same styling as the in-app Results panel
Open Manifest	Opens the SHA-256 integrity manifest for the run (when one was generated by the report pipeline)

The list shows the most recent 100 runs by default, ordered newest-first. Provenance values such as hostname and TIA version come from the run-provenance metadata captured when the run started; the project path itself is never stored in plain text — only an irreversible SHA-256 hash, so you can correlate runs across machines without leaking workspace paths.

Compare Runs

The Compare Runs view is opened on demand by Compare To… in Run History — you pick a baseline run, the launcher dialog asks for a second run, and the Compare Runs view activates with the diff.

Header: Baseline / Current run-IDs side by side, Swap button reverses the comparison, Export HTML writes the diff as a self-contained HTML report (the same renderer the tia-test-runner compare command uses)
Summary pills: Five colour-coded counters — Regressions (red), Fixed (green), New (blue), Removed (grey), Stable (muted). Stable counts only Still-Passing and Unchanged cases; Still-Failing cases stay flagged as warnings, not as stable
Filter bar: Five radio buttons (All / Regressions / Fixed / New / Removed) reduce the case list in place
Case rows: Each case shows a coloured bullet (red / green / blue / grey / orange) plus <Suite> / <Case>, the failure message (if any), the change-kind label, and the duration delta versus the baseline (+12 ms / −5 ms)

The launcher dialog lists the most recent 100 runs minus the baseline you started from, sorted newest-first. Double-click a row or click OK to commit the pick; Cancel closes the dialog without changing the comparison.

Trend

The Trend view visualises pass-rate, duration, and per-case status over time using historical runs from the database.

Filter bar (row 1): PLC name (required), suite name (optional — narrows the duration chart), case name (optional — narrows the heatmap)
Filter bar (row 2): From / To calendar pickers (clear a picker to remove the bound), Last 30 days / Last 90 days quick buttons, Refresh to re-query, Export CSV to write the current trend to a CSV file (the same export the tia-test-runner trend command produces)
Pass rate chart: Time-series scatter, X = run start time, Y = pass rate %. Visible once the project query returns at least one point; otherwise shows "No data".
Duration chart: Time-series scatter, X = run start time, Y = suite duration in seconds. Requires a suite name; otherwise shows "No data".
Case heatmap: Wraps cells left-to-right, one per run, colour-coded by status (green = pass, red = fail, orange = error, grey = skipped) plus a glyph (✓/✗/!/—) for accessibility. Hover any cell to see the run timestamp. Requires both suite and case names; otherwise shows "No data".

If the project trend query fails (database offline, corrupt range), the bottom banner shows the error message. Suite and case query failures show their own red banner above the affected chart so it is clear which section failed; the other charts still render normally.

Mutation Testing

Mutation testing answers a question coverage cannot: do your tests actually catch a fault? It makes a series of tiny changes to a copy of your block (for example, flipping a > to a >=, swapping an AND for an OR, or removing an output assignment), runs your existing test suite against each changed copy, and reports the changes your tests failed to notice. A change your suite caught is a "killed" mutant; a change that slipped through with every test still green is a "surviving" mutant, and that is a gap in your tests.

Your original block is never touched. Every change is made on a throwaway copy that Studio cleans up after each run.

Running mutation testing

Select a test suite in the Test Suites view.
Make sure you are connected to TIA Portal and your run targets a PLCSIM instance. Mutation testing needs a PLCSIM target, so a suite set up for a direct S7 connection cannot be used here.
Run Run Mutation Testing from the Command Palette.
Studio first runs the suite once against the unchanged block. The suite must pass first (mutation testing measures whether your passing tests are thorough, so it tells you to fix any failing cases before it continues). It then generates the mutants and runs the suite against each one.

The run can take a while because each mutant is compiled, loaded, and tested in turn. You can stop it at any time, and Studio still cleans up the temporary copies it created.

Reading the results

The Mutation tab in the results panel shows:

A mutation score (the percentage of mutants your tests caught, ignoring any that could not run), with the killed, survived, and stillborn counts.
A Surviving Mutants list, one row per gap: the line, the kind of change, and what was changed. Click a row to jump straight to that line in the block source.
Collapsible lists of the killed and stillborn mutants.

Surviving mutants are also highlighted in the editor: the affected lines get a red tint and a red marker in the gutter, with a tooltip describing the change no test caught. Use Toggle Mutation Markers (Command Palette, or the button on the Mutation tab) to show or hide them.

A high score means your tests caught nearly every fault. A low score, or a long surviving list, tells you exactly where to add or strengthen test cases.

Settings

Maximum mutants per run sets how many changes Studio tries in one run. A higher number finds more gaps but takes longer.
Mutation operators lets you choose which kinds of change to apply: relational comparisons, logical operators, constants and arithmetic, and statement deletion. All are on by default.

Block Analysis (before running tests)

When you open a suite, AnyAutomation Studio extracts the block interface from the TIA project so the editor knows the real parameter names and types, and so the inputs and assertions grids can validate against them.

You do not need TIA Portal open for this. If you are not connected to TIA Portal, Studio reads the block from a copy you exported earlier (export it from the project explorer). If the block has not been exported yet, Studio shows a message asking you to export it first or connect to TIA Portal, instead of leaving the interface empty.

Pick a Source Folder. The Source Folder button in the Unit Testing full-pane toolbar lets you tell Studio where your exported blocks are. Pick the folder that holds your export files, and the folder structure inside it no longer matters — whether the files sit directly in the folder or in sub-folders, Studio finds the right file for the block. This lets you analyze and run tests without TIA Portal open. A Source Folder you pick is remembered, so it is ready again right after a restart. Click the button again to change the folder (Change Folder…) or to remove the selection (Clear Source Folder). For the cleanest results, pick the folder you exported into, not a parent folder of several projects.

The Unit Testing full-pane editor has a dedicated analysis surface with Interface, Boundary Values, and Dependencies tabs (open the pane via Open in Full Pane in the Test Suites view title bar, or Command Palette → Open Unit Testing). They summarise:

Interface: All block parameters (Input, Output, InOut, Static, Temp) with types and default values
Boundary Values: Auto-generated min/max/zero/one values per parameter type (with a Generate from Boundaries action that seeds a parameterized case)
Dependencies: Called blocks, referenced DBs, referenced UDTs

Creating a New Test Suite

Click New Test Suite in the Test Suites view title bar (or run it from the Command Palette)
When you are connected to a TIA project with PLCs, pick the PLC, then the block under test — the suite name is auto-derived as <Block>_Tests and seeded from the block interface. (Only when no PLCs are available do you instead answer three free-text prompts in turn: the Suite name, the Block under test, and the PLC name.)
The new suite is written to <projectDir>/.tia-tests/{name}.json and appears in the Test Suites view
The Test Suite Editor opens with a starting test case wired up with arrange / act / assertions / watch so you can see how the pieces fit together. The block interface is extracted from the TIA project, so the inputs and assertions grids validate against the real parameter names. Always replace placeholder values with real names before pointing the runner at hardware.
Use the Visual mode (form-based master-detail) to add / remove / duplicate test cases through fields and grids — Name, Description, Cycles, Tags, Priority + Arrange-Inputs grid + Assertions grid + Watch list. Switch to Open as JSON Text anytime to edit the raw JSON. The SCL view shows the generated SCL test code and is editable too (see Managing Suites and Test Cases)
Click Save (Ctrl+S) to persist your changes

Renaming a suite from the JSON `name` field

When you edit the name field at the top of the suite JSON and press Save, the file is renamed on disk to match — {newName}.json — and the editor title updates in the same step. If a suite with the new name already exists in the same folder, Save aborts with an error and leaves both files untouched so you can pick a different name.

The blockName field is independent and continues to identify the TIA block under test; it is not affected by a name rename.

Test Suite JSON Schema

Every .tia-tests/*.json file follows the same shape. Top-level fields:

Field	Type	Required	Purpose
`name`	string	yes	Suite identifier (must match the file name without `.json`)
`blockName`	string	yes	Exact TIA block name (case-sensitive)
`plcName`	string	yes	PLC/CPU name in the TIA project (e.g. `PLC_1`)
`config`	object	yes	Connection + transport settings (see below)
`testCases`	array	yes	One entry per test case (see below)

config (all fields optional, defaults shown):

Field	Default	Purpose
`cycles`	`1`	Default PLC cycles per test case (overridable per case via `act.cycles`)
`cycleWaitMs`	`100`	Wait between sampled reads when `cycles > 1`
`timeoutMs`	`5000`	Hard timeout per test case
`instanceDbName`	`""`	Instance-DB to read/write; empty → `<blockName>_DB` (Siemens convention)
`preferFc`	`false`	Tolerate FC blocks (no Instance-DB); set when the target is a function instead of a function block
`transport`	`"plcSimApi"`	`"plcSimApi"` for direct PLCSIM Tag-API (the default), `"s7CommPlus"` to connect directly to a real S7 controller (see Connecting over S7 Native)
`s7IpAddress`	`192.168.0.1`	S7 connect target — used only when `preparationMode = "external"` (real PLC). For PLCSIM-backed modes (`userPreloaded`, `tiaTcpDownload`) the runner connects to `plcSimIp` instead, regardless of transport
`s7Port`	`102`	S7 ISO-on-TCP port. Default `102` matches every Siemens default — change only when the PLC was reconfigured to a non-standard TCP port
`s7User`, `s7PasswordKeyId`	`""` / null	Credentials reference (password lives in Windows Credential Manager). Required only when the PLC enforces user authentication — most projects leave this empty
`plcSimInstanceName`	`"TiaUnitTest"`	PLCSIM Advanced instance name. Used by both transports whenever `preparationMode != "external"`
`plcSimIp`	`"192.168.0.100"`	PLCSIM Virtual-Adapter IP. This is the actual S7 connect target for `s7CommPlus + userPreloaded/tiaTcpDownload` — set it to the IP you assigned to the Siemens PLCSIM Virtual Ethernet Adapter, not to the PLC project IP
`preparationMode`	`"userPreloaded"`	`userPreloaded` (default; assumes the PLCSIM instance is already running with your project loaded), `tiaTcpDownload` (compiles + downloads from the open TIA project — see prerequisites below), or `external` (real PLC; uses `s7IpAddress`)
`masterSecretPasswordKeyId`	null	Project master-secret password reference. Required for `tiaTcpDownload` only when the TIA project uses confidential PLC configuration protection. Password lives in Windows Credential Manager
`autoConnectS7`	`true`	Auto-connect S7 before run starts. Set to `false` only in advanced setups where another client (WinCC, custom HMI) already holds the session and the runner should read/write through it
`keepInstanceAfterRun`	`false`	Keep PLCSIM instance alive after the run. Has effect only in `tiaTcpDownload` mode — `userPreloaded` and `external` never manage the instance lifecycle

Each entry in testCases:

Field	Type	Required	Purpose
`name`	string	yes	PascalCase scenario name (e.g. `Start_FromIdle_Runs`)
`description`	string	no	One-sentence what-and-why
`arrange.inputs`	object	yes	`{ "MemberName": value, … }` — written to the DB before Act
`act.cycles`	int	no (defaults to `config.cycles`)	PLC cycles between Arrange and Assert; must be > 1 for `watch[]` to fire. Mutually exclusive with `act.steps` (the schema's `oneOf` rejects both on the same `act`)
`act.timeoutMs`	int	no (defaults to `config.timeoutMs`)	Per-case timeout. Covers the entire `steps` sequence end-to-end when `steps` is set
`act.steps`	array	no	Optional ordered sequence of `write` / `wait` / `assert` steps that runs between Arrange and the top-level `assertions`. See Multi-phase Steps below. Mutually exclusive with `act.cycles`
`assertions`	array	yes	`[ { "variable": "X", "operator": "equal", "expected": v, "tolerance": t? }, … ]`
`watch`	string[]	no	Variables sampled once per cycle during Act → time-series chart in the HTML reports
`tags`	string[]	no	Free-form labels
`priority`	string	no	`"low"` / `"normal"` (default) / `"high"` / `"critical"`
`requirements`	string[]	no	Traceability links
`order`	int	no	Stable sort key in the UI

Supported operator values: equal, notEqual, greaterThan, lessThan, greaterThanOrEqual, lessThanOrEqual, inRange (expects expected: [min, max]), inRangeExclusive, approximately, isTrue, isFalse, contains, startsWith, endsWith, matches, bitsSet, bitsClear, arrayEquals, allEqual, anyEqual, noneEqual, deepEquals. tolerance applies to inRange, approximately, arrayEquals, allEqual, anyEqual, noneEqual, and deepEquals (default 0.0001) — not to equal/notEqual; use approximately for fuzzy Real/LReal equality. expected is required by the schema for every assertion (set it to any value — false, true, 0 — for isTrue/isFalse, since the runner ignores it).

Safety — example values are placeholders. When the editor cannot infer real Input/Output names from the block interface, the seeded example uses safe defaults (Enable: false, Running: false) so a one-click Run cannot accidentally activate motors, valves, or other outputs on real hardware. Always review every arrange.inputs value and every assertion before pointing the runner at a real PLC.

Important — watch[] is the test case's diagnostic array, NOT the TIA Watch Table. Variables listed in watch are sampled once per PLC cycle during the Act phase (provided act.cycles > 1) and rendered as a time-series chart in the HTML reports. This is independent of TIA Portal's external Watch Table or any OPC UA subscription.

Complete example

A full minimal suite with one test case that exercises every block and uses watch[]:

{
  "name": "FB_Motor_Tests",
  "blockName": "FB_Motor",
  "plcName": "PLC_1",
  "config": {
    "cycles": 1,
    "timeoutMs": 5000,
    "instanceDbName": "FB_Motor_DB",
    "transport": "plcSimApi",
    "autoConnectS7": true,
    "preparationMode": "userPreloaded"
  },
  "testCases": [
    {
      "name": "Start_FromIdle_RunsWithinThreeCycles",
      "description": "Asserting Run goes high after StartCmd is set; watch shows trajectory.",
      "arrange": {
        "inputs": {
          "StartCmd": true,
          "Reset":    false,
          "Speed":    50.0
        }
      },
      "act": { "cycles": 5 },
      "assertions": [
        { "variable": "Run",       "operator": "equal",     "expected": true },
        { "variable": "Speed_Act", "operator": "inRange",   "expected": [49.5, 50.5] },
        { "variable": "Error",     "operator": "isFalse", "expected": false }
      ],
      "watch": ["Run", "Speed_Act", "StartCmd"],
      "tags": ["happy-path", "smoke"],
      "priority": "normal"
    }
  ]
}

After the run, the Test Results view shows pass/fail per assertion, and the HTML report renders a time-series chart for the three watch variables across the 5 cycles. To add more variables to watch on an existing case, append to the watch array and increase act.cycles to the smallest number that gives useful coverage (3-10 is typical).

Multi-phase Steps (`act.steps`)

When a single Arrange-then-Assert is not enough — for example reset → wait → start → wait → check — use act.steps instead of act.cycles. Three step types are available:

`type`	Required fields	Effect
`"write"`	`inputs` (object, same shape as `arrange.inputs`)	Writes the listed members additively on top of whatever is already in the DB. Members not listed keep their value.
`"wait"`	`ms` (int, `0`..`3 600 000`)	Async delay. The PLC keeps cycling. When `watch[]` is non-empty and `config.cycleWaitMs > 0`, watched variables are sampled once every `config.cycleWaitMs` ms throughout the wait window.
`"assert"`	`assertions` (array, same shape as the top-level `assertions`)	Per-step checkpoint. A failure aborts the case immediately and reports `Step <N> (assert): <reason>` (1-based).

Order of operations:

arrange.inputs is written once at case start (not replayed between steps).
Each entry in steps runs in declaration order.
After the last step, the top-level assertions[] runs as a final gate. Per-step asserts and top-level asserts are independent — both must pass. The schema requires minItems: 1 on the top-level assertions, so author at least one trivial top-level assertion even when relying mainly on per-step asserts.
watch[] continues to sample throughout every wait step (under the conditions above); the time-series chart annotates step boundaries.

Caps (enforced by the runner):

Limit	Value
Steps per case	`1024`
Inputs per `write` step	`256`
Assertions per `assert` step	`256`
`wait.ms`	`0`..`3 600 000` (1 hour)

Visual editor: the case detail view shows a Steps (optional) section with + Write, + Wait, + Assert buttons to append, per-step ↑ / ↓ buttons to reorder, and − to remove. The editor automatically omits cycles from the serialized JSON whenever the Steps section is non-empty (the schema rejects the cycles + steps combination, so the editor avoids ever producing it).

Backwards-compatibility: a case without steps runs the existing single-phase cycles-based path with no behavioural change. Older suites need no migration.

Complete example — timed motor start

StartCmd is set after a 500 ms reset window, then we wait 2 s for the FB to reach Run, plus a final speed check.

{
  "name": "MotorStart_ReachesRun",
  "description": "Reset, wait, start, wait, check Run.",
  "arrange": { "inputs": { "Enable": true } },
  "act": {
    "timeoutMs": 10000,
    "steps": [
      { "type": "write", "inputs": { "Reset": true } },
      { "type": "wait",  "ms": 500 },
      { "type": "write", "inputs": { "Reset": false, "StartCmd": true } },
      { "type": "wait",  "ms": 2000 },
      { "type": "assert", "assertions": [
          { "variable": "Run", "operator": "isTrue", "expected": true }
        ]
      }
    ]
  },
  "assertions": [
    { "variable": "Speed_Act", "operator": "greaterThan", "expected": 1000 }
  ],
  "watch": ["Run", "Speed_Act"]
}

If the per-step assert fails (for example Run did not go true within 2 s), the runner emits Step 5 (assert): variable Run is not true and aborts the case. If the per-step asserts all pass but the top-level Speed_Act > 1000 fails, the case fails with the top-level assertion message.

When in doubt, write a plain Arrange + cycles ≥ N + Assert case first. Convert it into a steps sequence only when the plain form cannot express the timing the test actually depends on.

Opening an Existing Suite

From the Test Suites view: Double-click the suite in the tree (it opens in the Test Suite Editor)
From Quick Open: Press Ctrl+P and type the suite file name under .tia-tests/

Where are test suites stored?

Test suites live in a hidden folder called .tia-tests directly next to your TIA project file (*.ap21, *.ap20, …). The AI assistant writes to the same folder. If you haven't chosen a test folder yet, the Test Suites view shows "Choose a test folder to load your test suites." — pick a folder first.

You pick the folder yourself: in the Test Suites view title bar, click Choose Test Folder and pick the folder your tests live in. When a folder is chosen, the Test Suites list follows that folder — switching projects doesn't change it; with no folder chosen, it follows the connected TIA project. The choice is remembered, so your tests show up again the next time you start the app, even without a TIA connection. It also helps you recover tests from an earlier setup: just pick the folder they're in. Clear Test Folder (shown only while a folder is chosen) clears the selection again; if the chosen folder no longer exists, the view shows "Selected test folder not found".

The AI can read, list, and edit these test-suite files directly through the chat interface.

Running Tests

Make sure PLCSIM Advanced V3.0+ is installed
Open the suite from the Test Suites view
Run it: use the per-suite inline Run Test Suite action in the Test Suites view, Run All Test Suites from the view title bar, or a right-click context-menu action
Watch the live progress in the Status Bar and the Test Results view
When the run is complete, status icons in the Test Suites view update and results are persisted to SQLite

The runner automatically:

Creates a PLCSIM Advanced instance (name configurable in the suite JSON)
Powers it on and compiles the current PLC project
Connects an S7 client to the instance
Writes the test inputs, waits the configured cycle count, reads the outputs, evaluates the assertions
Cleans up the S7 connection and unregisters the instance

Choosing a Transport

The transport decides how test reads and writes travel between AnyAutomation Studio and the PLC. You set it in the suite JSON via config.transport (see the schema above):

PLCSim Advanced (plcSimApi) — Direct access via the Siemens PLCSim API, faster and does not need an active S7 session. This is the default and the only transport that currently runs in-app.
S7 Native — Connects directly to a real S7 controller, the same kind of connection used by the PLC Online view. Pick this when you want to run a suite against live S7-1200 / S7-1500 hardware. See Connecting over S7 Native for the fields you fill in and the confirmation prompt before a live run. Included in Unit Testing (Pro+ plan), also during the free trial.

Each suite is targeted independently. For S7 Native, the suite owns its IP address, port, user name, and password — there is no shared lookup from the PLC Online view.

Right-click a suite in the Test Suites view → Connection Settings… opens a form that edits these values without touching JSON (connection type, instance name, IP, port, user, password, cycle wait, auto-connect — and it can save the values as defaults for new suites). You can still edit the values directly in the suite JSON via Open as JSON Text in the Test Suite Editor.

For PLCSim Advanced you can also set a Network Mode here: (leave unchanged), TCPIPSingleAdapter, TCPIPMultipleAdapter, or Softbus. Choose one if PLCSim has trouble starting on a machine with more than one network adapter; leaving it unchanged keeps your current PLCSim setting.

Connecting over OPC UA

A test suite can run against a PLC over OPC UA, alongside PLCSim Advanced. Open a suite's Connection Settings… (right-click the suite in the Test Suites view) and choose the connection type OPC UA.

For an OPC UA connection you provide:

Endpoint URL — the address of the PLC's OPC UA server, for example opc.tcp://192.168.0.5:4840.
Security Mode — None, Sign, or Sign & Encrypt. Pick the mode your server is set up to accept; Sign & Encrypt gives the highest protection.
Authentication — either Anonymous, or User Name & Password. When you choose User Name & Password, enter the credentials your PLC expects. The password is stored securely on your machine and is never written into the test file, so you can share or commit a suite without exposing it.
Namespace Index — leave this empty and Studio detects the right namespace on the controller automatically; this is the normal case. Set a number only for an unusual server where automatic detection does not find it.

The first time you connect to a server whose certificate AnyAutomation Studio does not yet trust, a prompt asks you to confirm the connection. Choose Connect Anyway to trust this server and continue; the prompt does not appear again for that server on the next run.

Make the PLC's variables reachable over OPC UA. For tests to read and write your variables, the data blocks they touch must be accessible through the PLC's OPC UA server. Enable OPC UA access for the relevant data blocks in your TIA Portal project and download it to the PLC before running the suite. Variables that are not exposed cannot be read or written and the affected test cases will fail.

Browsing the Address Space

In the Unit Testing workspace you can explore what an OPC UA server offers before you write or run tests. The address space is shown as a tree you can expand to walk through the available nodes and see which variables are reachable. Use it to confirm a variable's name and that it is exposed over OPC UA, so your test cases target the right items.

Authoring and Running over OPC UA

Once the connection is set, OPC UA suites behave exactly like PLCSim suites. You author test cases the same way, run them with the same Run Test Suite / Run All Test Suites actions, watch the same live progress, and get the same results and run history. The only difference is where the tests connect — and you do not need TIA Portal open to author or run them.

For pipelines and CI, a suite configured for OPC UA also runs in the generated pipeline. Because the OPC UA password is not stored in the test file, you provide the OPC UA password to the pipeline run so the build server can connect without anyone present.

Connecting over S7 Native

A test suite can also run directly against a real S7 controller over S7 Native. Open a suite's Connection Settings… (right-click the suite in the Test Suites view) and choose the connection type S7 Native. This connection type is included in Unit Testing (Pro+ plan), also during the free trial.

For an S7 Native connection you provide:

IP address — the controller's address, for example 192.168.0.10.
Port — the controller's TCP port. The default is 102, which fits most controllers. Change it only when your controller has been set to a different port.
User name (optional) — needed only when your controller requires a user sign-in. Most projects leave this empty.
Password — the password your controller expects. It is stored securely on your machine and is never written into the test file, so you can share or commit a suite without exposing it.

Use Save as default to apply these values as the default for new suites, so you don't have to enter them again each time.

Authoring and Running over S7 Native

Once the connection is set, you author test cases the same way you would for a PLCSim suite and run them with the same Run Test Suite / Run All Test Suites actions. The only difference is where the tests connect: an S7 Native suite connects directly to the controller you specified, writes the test inputs to the live controller, and checks the results there.

Because a run writes to live hardware, a prompt asks you to confirm before it starts. Confirm it only when the plant is in a safe test state, and the run begins. Cancel the prompt and nothing happens.

You can run against a safety (F) controller as well. The confirmation prompt warns you that the target may be a safety controller and that you must make sure the controller is safe to manipulate and the plant is in a safe test state before you continue. There is no automatic block: you decide whether to confirm or cancel.

Simulation Workspace

Planned for the in-app workspace. A dedicated Simulation view for managing PLCSim Advanced instances from inside AnyAutomation Studio is planned. The capabilities it would expose are described below.

A Simulation view for managing PLCSim Advanced instances would show:

API version, online-access mode (PLCSim / TCP/IP single / TCP/IP multi), strict motion timing toggle, and Runtime Manager Port.
A Virtual Adapter row showing the permanent status of the Siemens PLCSIM virtual Ethernet adapter (Ready / APIPA / Disabled / Not Installed / No IPv4), its IPv4 address, and a refresh button. If the adapter is stuck in the 169.254.x.y APIPA range, downloads will be unreliable — assign a static IPv4 address in Windows network settings.
Every registered PLCSim instance with inline action buttons: Power On, Run, Stop, Memory Reset, Power Off, Settings, Network, Delete. Each row shows the configured IP address; if the instance has more than one interface with an IP, they are listed as X1: 192.168.0.1, X2: 10.0.0.5.
A TIA PLC combobox per instance card: pick the TIA Portal PLC that backs the instance. Use this when the PLCSim instance was loaded with a snapshot compiled from a different TIA PLC than the one it is registered under (for example: an instance named PLC_2 actually holding the program from PLC_1). Without an explicit choice, the app matches by name and falls back to the only PLC in the project when there is one — which is enough for most setups but produces an empty tag tree when the names disagree. The choice is remembered per instance.
A New Instance button that prompts for a name and CPU type.
A Tag Browser pane that connects to any listed instance and shows all tags the running program exposes, filterable by name search, area (Input / Output / Marker / Data Block), and data type. Auto-refresh can be toggled for live value observation. Each writable tag has a pencil button that opens a type-aware write dialog (toggle for Bool, numeric input with type-specific range for integers and floats, one-character field for Char/WChar). String tags are read-only. Struct tags and array tags are listed with their inner members and elements as expandable rows so you can search, watch and edit individual fields like OUT[5] or DI10.Channel directly; Expand all and Collapse all toolbar buttons toggle the whole tree at once. Structured input/output tags whose layout comes from a project type definition (for example a PNPN block at %I0.0 declared as Array[0..63] of Byte) drill down into named member rows when the app is connected to the TIA Portal project that defines them — letting you write IN.PNPN[12] from the same dialog that handles the rest of the tree.
A Saved Instances section at the bottom listing every PLCSim Advanced instance persisted on the virtual SIMATIC memory card. The heading shows a counter with the total number of saved instances; the list shows up to three rows at a time and scrolls if there are more so the simulation area above stays visible. Click Load to re-register and resume one — PLCSim picks up the stored CPU type, I/O image and program automatically. Click Delete (with confirmation) to remove the persisted folder.

The view choice would be remembered between sessions, and switching keeps any open suite editors intact.

PLCSIM Preparation Mode

The preparation mode controls how the runner brings your project online before the test run. It is set in the suite JSON via config.preparationMode:

I load it myself (userPreloaded, recommended) — The runner expects a PLCSIM instance that you have already started and loaded manually via TIA Portal. It skips compile and download and connects directly to run the tests. Fastest option if you're iterating on the same project.
Automatic TCP download (tiaTcpDownload) — The runner compiles the project, starts a fresh PLCSIM instance and downloads via TCP over the PLCSIM Virtual Adapter. No manual TIA Portal interaction needed — just run.
Real PLC (no PLCSim) (external) — Skips the PLCSim lifecycle entirely and would prepare a run against live S7 hardware without any PLCSim instance. This preparation mode is not yet available for in-app runs. It is a separate, planned option and is not the same thing as the S7 Native connection: to run against a real controller today, choose the S7 Native connection type instead (see Connecting over S7 Native and Running Tests Against a Real PLC).

Before using Automatic TCP download: Four prerequisites must be met or the run will fail with a generic transport error: (1) the PLCSIM Virtual Adapter must be installed and have a static IPv4 address that matches the suite's plcSimIp; (2) the TIA Portal project must be open in TIA Portal — the runner triggers a compile against the open project before the download; (3) if the project enables Protect confidential PLC configuration data, you must supply the master-secret password (it lives in the Vault); (4) any existing PLCSIM instance with the same name is destroyed and recreated for every run, so other applications attached to that instance lose their session.

Running Tests Against a Real PLC

You can run tests directly against live S7-1200 / S7-1500 hardware using an S7 Native connection, without any PLCSim involvement:

Make sure the plant is in a safe test state — actuators isolated or interlocked, no production load on the PLC, operators informed.
In the suite's Connection Settings…, choose the S7 Native connection type and fill in the controller's IP address, port, and (only if the controller requires it) user name and password. See Connecting over S7 Native for the details.
Run the suite from the Test Suites view.
Before the tests reach the controller, a prompt asks you to confirm the run against live hardware. Confirm it once the plant is in a safe test state, and the run begins.

A few things to know about live-controller runs:

Wrong credentials: if the controller accepts the connection but rejects your sign-in, the run stops right away with a message telling you to check the user name and password, rather than failing later with a confusing "variable not found".
Batch runs: a suite set to an S7 Native connection is skipped during Run All Test Suites (the per-suite confirmation cannot be shown from a batch run) and is listed by name in a dismissible banner, so you run each one individually.
Controller must be running: a live-controller run checks the controller's operating state first and stops if it is not in RUN, so nothing is written while the program is not executing.

Safety (F) controllers. When you run interactively, you can target a safety (F) controller: the confirmation prompt warns you that the target may be a safety controller and that you must make sure the controller is safe to manipulate and the plant is in a safe test state, and you decide whether to confirm or cancel. AI-assisted runs are different: the assistant will not run tests against Safety logic on its own, so you always start a safety-controller run yourself.

Running Against a Protected Project

If your TIA project has "Protect confidential PLC configuration data" enabled, the automatic TCP download mode requires the project's master-secret password:

Set the preparation mode to Automatic TCP download (tiaTcpDownload) in the suite config
Supply the project's master-secret password when prompted
Optional: choose Remember password to store it in the Windows Credential Manager, so every future run picks it up automatically without asking again. Otherwise the password is used once and forgotten after the run
Run the suite from the Test Suites view

A hint "A password is stored for this user" appears next time if you saved the password. Leave the password field blank to keep the stored value. Clearing Remember password deletes the stored entry.

The password is never written to the suite file, to any log, or to any settings file. It lives only in Windows Credential Manager under the service name com.anyautomationstudio and is transferred from the UI to the runner through a user-bound DPAPI channel so a different Windows account cannot read it.

Managing Suites and Test Cases

Rename a suite by editing the JSON name field at the top of the suite and pressing Save — the file is renamed on disk to match (see Renaming a suite from the JSON name field). To delete a suite, remove its .json file from the .tia-tests/ folder.
Edit a test case in the Visual mode of the Test Suite Editor: each case row lets you rename it, edit its description, Duplicate it, remove it, and reorder its multi-phase steps with Move Up / Move Down.
Edit the generated SCL test code in the SCL view of the Test Suite Editor: rename test cases, change the input value assignments, edit, add or remove assertion checks, and add or delete whole test-case blocks. Changes flow back into the test cases automatically — the Visual and JSON views show them immediately, and case details the SCL text does not show (description, cycles, multi-phase steps, tolerance, watch list, tags, priority, owner, requirements) are kept. The runner block at the end is maintained automatically — edits to it have no effect. If the text no longer matches the test-block structure, an error message with the line number appears, and Save and Import to TIA are blocked until it is fixed. Switching to another editor tab discards SCL text that has not been applied yet; changes already applied are kept.
Connection settings are per suite. The transport, PLCSim instance name, IP address, cycle wait time, S7 Comm+ target (IP, port, user, password), and auto-connect behaviour all live in the suite file's config block. They are saved with the suite, so the next time you open it the same values are in effect. Passwords continue to live in the Windows Credential Manager — the suite file only stores a reference, never a plaintext password. (Right-click a suite → Connection Settings… opens a form to edit these without touching JSON — see Choosing a Transport.)
Before running, Studio checks that every variable name in your suite exactly matches the block interface, including case. If any mismatches are found, a warning dialog shows the suggested corrections — you can still continue the run after acknowledging it.
When a test case fails, its error message appears under the case name in the Test Results view (and on hover), and the detail grid marks the failing assertion with a red ✗ alongside its Expected and Actual values. (Jumping from a failed case straight to that variable in the Test Suite Editor is planned.)
If a suite file is modified externally — or renamed via the JSON name field above — the open editor automatically reloads, unless you have unsaved changes, in which case your edits are preserved.

Troubleshooting

Status icons don't update after a run: Make sure the suite file path is correct (absolute path under .tia-tests/). New unsaved suites get a unique path per click.
Run action unavailable: A TIA project must be open, and the active suite document must be valid JSON.
Test Explorer is empty: The working directory has no .tia-tests/ folder yet. Click New Test Suite to create the first one — the folder is created automatically.
PLCSIM errors: Check that PLCSIM Advanced V3.0+ is installed and licensed. The exact version is auto-detected at runtime.
Capture the TLS traffic for support (advanced): If support asks for a Wireshark capture of the S7 Comm+ handshake, set the SSLKEYLOGFILE environment variable to an absolute file path before starting the app — for example setx SSLKEYLOGFILE "C:\Temp\s7-keys.log" — then restart. Point Wireshark's TLS preference (Pre)-Master-Secret log filename at the same file. A warning is written to the app log on every connection while this is enabled so you don't forget to unset it. Leave the variable empty in normal operation.

AI-Authored Test Suites (Pro+)

AI Chat can draft, refine and (optionally) run full SCL unit-test suites for you. Requires a Pro+ license or higher.

Write Unit Tests skill (main chat)

Open AI Chat and pick the Write Unit Tests skill from the skill picker (🧪 icon), or simply ask: "write unit tests for FB_MotorControl"
The assistant reads the block interface, computes boundary values, plans test cases and summarises the plan in the chat — no giant JSON blobs
When it's ready to write the suite, an Approve / Deny prompt appears inline. Approve to let the assistant save the suite to .tia-tests/
If you asked the assistant to also run the suite, a second approval prompt appears for the run itself. Results come back as a pass/fail summary in the chat, and the assistant offers to refine any failing cases
The Test Suites view picks up the new suite automatically — open it, review it in the Visual mode or as JSON Text, and run it yourself whenever you want
You can also ask the assistant to "open that suite" after it's been created — the suite opens in the Test Suite Editor without you having to switch to the Test Suites view

Inline chat in the suite editor (Ctrl+I)

While a test suite is open in the JSON editor:

Place the cursor near the test case you want to change (or select a range)
Press Ctrl+I — a small prompt box appears above the editor
Type what you want, for example:
- "add an overflow test for Counter_Value"
- "make this test parametrized with five speed setpoints"
- "tighten the tolerance on all REAL assertions to 0.01"
The assistant streams the updated JSON and a diff overlay appears in the editor
Click Accept to apply the change (the file is saved and the Test Explorer reloads) or Reject to discard

The inline chat uses the same block interface the suite was opened with, so variable names and types are validated against the real block — mismatches are caught before you save.

Reviewing AI-authored test cases

The app stamps AI provenance on test cases it authors (stored internally), but it does not render a visible AI badge in the Test Explorer tree or the Visual editor today — only a Theory (N rows) badge appears for parameterized cases.

Review AI-authored test cases just like you would a colleague's pull request — read each assertion, sanity-check expected values, then run on the simulator before pointing it at real hardware.

Safety (F-CPU) protection

AI test authoring is gated for Safety blocks:

Creating tests — The assistant refuses to author tests for an F-CPU block without an explicit, in-conversation confirmation from you. The underlying tools also reject the write if the confirmation is missing
Running tests — The assistant cannot run tests against Safety blocks. Full stop, no override. Safety-block verification requires a certified methodology (TÜV/CE), not an AI-generated run. If you want to run a test against a Safety block, trigger the run yourself from the Test Explorer

Licensing

The Write Unit Tests skill and the unit-test MCP tools require Pro+ or higher. On a lower plan, the skill surfaces a license message before any AI call is made.

CI/CD Integration

The tia-test-runner command-line tool runs your SCL unit-test suites on a build server — no UI, no manual clicks. Point it at a TIA Portal project, and it discovers the suites in .tia-tests/, runs them, writes machine-readable reports, and returns an exit code your pipeline can act on. It opens TIA Portal headless (no window) so it runs cleanly on a build agent; pass --show-ui to watch it on screen while debugging. This is a Pro+ feature (Pro+ and above); the agent needs a CI licence file that you save from Studio (see Licensing for CI below).

The `tia-test-runner` command-line tool

The tool ships as tia-test-runner.exe and exposes seven verbs:

Verb	Purpose
`run`	Execute a unit-test run against a TIA Portal project and write reports
`compare`	Compare two previous runs and emit a diff report
`trend`	Export historical pass/fail trends to CSV
`flaky`	List a suite's flaky cases (unstable across recent runs) as a table or CSV
`validate`	Check test-suite definitions for schema errors without running them
`verify`	Verify the integrity manifest of a previous run (detect tampered or missing report files)
`version`	Print the runner, bridge, engine, build, .NET and operating-system versions

A typical run:

tia-test-runner run ^
  --project "C:\Projects\Plant.ap20" ^
  --plc PLC_1 ^
  --suite-filter * ^
  --out-dir reports

Key flags for run:

Flag	Meaning
`--project <PATH>`	TIA Portal project file (`.ap20`, `.ap21`, …). Required.
`--out-dir <PATH>`	Directory for reports and logs. Required.
`--plc <NAME>`	Restrict the run to a single PLC station.
`--suite-filter <PATTERN>`	Glob filter on suite names (`*`, `?`, literal).
`--tag <TAG>`	Run only cases carrying this tag (repeatable).
`--priority-min <LEVEL>`	Minimum priority (Low / Normal / High / Critical).
`--report-format <FORMAT>`	`junit`, `html`, or `cobertura` (repeatable). Defaults to JUnit + HTML.
`--rerun-affected`	Run only suites whose underlying block changed since the last run (needs `--plc`).
`--fail-fast`	Stop at the first failing case.
`--timeout-minutes <N>`	Per-run timeout. Defaults to 60.
`--attach`	Attach to a TIA Portal instance that is already running instead of opening the project file.
`--show-ui`	Open TIA Portal with its window visible instead of headless. Handy for watching a run while debugging locally.
`--quarantine`	Demote a failing case to a non-blocking skip when it has been flaky in recent runs, so a flaky failure does not break the build.
`--quarantine-window <N>`	How many recent runs to inspect for flakiness (default 10).
`--quarantine-min-flips <N>`	Minimum pass/fail changes within that window before a case counts as flaky (default 1).
`--retry-max <N>`	How many times to retry a suite after an infrastructure error such as a lost connection (`0` disables; default 3).
`--retry-delay-ms <MS>`	Delay between those retries.
`--config <PATH>`	Read defaults from a YAML configuration file (see below).

The run prints a summary line with passed / failed / errored / skipped / quarantined counts; when a run quarantines flaky cases, each demoted case is listed just before the summary so it is never hidden.

Exit codes (the pipeline contract):

Code	Meaning
`0`	All tests passed
`1`	One or more tests failed
`2`	Run error (could not start, lost the PLC connection, license not entitled, …)
`3`	Argument or configuration error
`4`	Timeout
`5`	Cancelled by the user (Ctrl+C)

After a run, --out-dir holds a JUnit XML file your CI test reporter can publish, a self-contained branded HTML report, and an integrity manifest. Reports are append-only: a second run into the same directory keeps the earlier run and adds an aggregate summary.

Flaky tests and quarantine

A flaky test is one that passes in some runs and fails in others without the block under test actually changing — usually a timing or environment effect. To see which cases are flaky for a suite over its recent history:

tia-test-runner flaky --suite-file "C:\Projects\Plant\.tia-tests\FB_Motor.json"

It prints a table of each unstable case with how many times it flipped between pass and fail and its recent result sequence. Add --window <N> to change how many recent runs are inspected (default 10), and --out-file <CSV> to also write the list as CSV.

Add --quarantine to a run to keep flaky failures from breaking the build: a case that fails the current run but has been flaky in recent history is demoted to a non-blocking skip and reported as quarantined (flaky) in the JUnit output and the run summary, while genuine, stable failures still fail the build. Tune it with --quarantine-window and --quarantine-min-flips. Quarantine never hides a demoted case — each one is listed before the run summary.

Report classnames

By default, JUnit classname values follow the pattern {PlcName}.{BlockName}, which most CI test viewers group into a readable tree. Override it with --report-classname-pattern using any combination of these placeholders:

{PlcName} — the PLC/CPU name
{BlockName} — the block under test
{SuiteName} — the test-suite name
{ProjectName} — the project name

Example: --report-classname-pattern "{ProjectName}/{PlcName}/{SuiteName}".

Capturing CI environment details

Pass --ci-env-capture to record a small, fixed set of CI variables into the run's provenance, so the reports show which pipeline, commit, and agent produced each run. Capture is opt-in — nothing is recorded unless you ask for it. The tool recognises GitHub Actions, Jenkins, Azure DevOps, and GitLab CI (and a generic CI marker); only a whitelisted set of identifiers (run/build IDs, commit SHA, branch/ref, workflow/job name, agent/runner name) is read, each clipped to a safe length.

YAML configuration file

To keep long command lines out of your pipeline, put the defaults in a YAML file and pass --config <PATH>. Command-line flags override the file. Example tia-tests.yml:

project: C:\Projects\Plant.ap20
plc: PLC_1
out_dir: reports
report_format:
  - junit
  - html
timeout_minutes: 45
ci_env_capture: true

Jenkins (declarative pipeline)

pipeline {
  agent { label 'tia-windows' }
  stages {
    stage('Checkout') {
      steps { checkout scm }
    }
    stage('Unit Tests') {
      steps {
        bat 'tia-test-runner run --project "%WORKSPACE%\\Plant.ap20" --plc PLC_1 --out-dir reports --ci-env-capture'
      }
    }
  }
  post {
    always {
      junit 'reports/**/junit.xml'
      archiveArtifacts artifacts: 'reports/**/*.html', allowEmptyArchive: true
    }
  }
}

GitHub Actions

name: TIA Unit Tests
on: [push]
jobs:
  unit-tests:
    runs-on: [self-hosted, windows, tia]
    steps:
      - uses: actions/checkout@v4
      - name: Run SCL unit tests
        run: tia-test-runner run --project "${{ github.workspace }}\Plant.ap20" --plc PLC_1 --out-dir reports --ci-env-capture
      - name: Publish test report
        if: always()
        uses: dorny/test-reporter@v1
        with:
          name: TIA Unit Tests
          path: reports/**/junit.xml
          reporter: java-junit
      - name: Upload HTML report
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: tia-html-report
          path: reports/**/*.html

Azure DevOps

pool:
  name: tia-windows
steps:
  - checkout: self
  - script: tia-test-runner run --project "$(Build.SourcesDirectory)\Plant.ap20" --plc PLC_1 --out-dir $(Build.ArtifactStagingDirectory)\reports --ci-env-capture
    displayName: Run SCL unit tests
  - task: PublishTestResults@2
    condition: always()
    inputs:
      testResultsFormat: JUnit
      testResultsFiles: '$(Build.ArtifactStagingDirectory)/reports/**/junit.xml'
      testRunTitle: TIA Unit Tests
  - publish: $(Build.ArtifactStagingDirectory)\reports
    artifact: tia-html-report
    condition: always()

GitLab CI

unit-tests:
  tags: [tia-windows]
  script:
    - tia-test-runner run --project "$CI_PROJECT_DIR\Plant.ap20" --plc PLC_1 --out-dir reports --ci-env-capture
  artifacts:
    when: always
    paths:
      - reports/
    reports:
      junit: reports/**/junit.xml

Generate a CI/CD pipeline from the app

Planned for the in-app workspace. An in-app generator for these files is planned. Until it ships, copy one of the templates above. The intended flow:

Instead of writing the configuration files by hand, you would let AnyAutomation Studio generate them for you. The generator reads the open project and produces two ready-to-commit files: a runner.yaml that holds the command-line defaults, and a pipeline file for the CI system you choose.

The intended flow:

Open a project.
Open the Test Suites view.
Run Generate Pipeline….
Pick your CI system (GitHub Actions, Jenkins, Azure DevOps, or GitLab CI) and adjust the prefilled fields — project path, PLC name, an optional suite filter, the report output folder, which report formats to write, an optional self-hosted runner label, optional report branding, and an optional Triggers section (push branches, pull-request branches, and a schedule).
Generate Preview to see both files side by side.
Write Files to save them. If any of the files already exist, a single prompt lists everything that would be replaced; cancelling writes nothing.

The runner.yaml is written to the project folder, and the pipeline file is written to the path its CI system expects: .github/workflows/tia-tests.yml for GitHub Actions, Jenkinsfile for Jenkins, azure-pipelines.yml for Azure DevOps, and .gitlab-ci.yml for GitLab CI. Commit both files. By default the generated pipeline runs on demand — start it manually from your CI system whenever you want to test. Fill in the Triggers section to also run it automatically: list branches to run on every push or pull request, or set a cron schedule for nightly runs. (For GitLab the schedule itself is set in GitLab's own pipeline-schedules screen; for Jenkins, push and pull-request runs come from your repository webhook.)

The pipeline must run on a self-hosted Windows runner. The tests drive TIA Portal and PLCSIM, so the machine that executes the pipeline needs TIA Portal installed, PLCSIM available, and an active Pro+ license or higher. Cloud-hosted runners cannot run the tests. Enter the label of your own Windows runner in the Self-hosted runner label field so the generated pipeline targets it; leave the field empty only if you have already wired up a self-hosted runner by other means.

Build Matrix

The generator dialog has a Build Matrix entry. Add one or more cells, and each cell runs your test suites as a separate, parallel job in the generated pipeline. Use it to test the same suites across several projects, or to split a large set of suites into parallel slices, so the whole CI run finishes faster and reports each cell on its own.

Per cell you provide:

Name (required) – a short label using letters, digits, _ or -. It identifies the cell in the pipeline and in the CI run report.
Project file path (optional) – leave it empty to use the connected project, or point the cell at a different project file.
Suite filter (optional) – run only some of the suites in that cell. Leave it empty to run them all.

Each cell writes its results to its own report folder, so cells never overwrite each other and you can compare them side by side.

The Build Matrix is available for GitHub Actions, Azure DevOps and GitLab CI. For Jenkins it is not available: if you add cells and then pick Jenkins, the dialog asks you to remove the cells or choose another target before you can continue.

Leaving the Build Matrix empty produces the normal single-run pipeline, exactly as described above.

Licensing for CI

Unit testing is a Pro+ feature (Pro+ and above), so the runner checks your licence before it does any work. A build agent is licensed with a CI licence file that you save from Studio and hand to the agent.

Save the file. In Studio open Generate CI Pipeline from the Unit Testing view, choose Save CI Licence File, and pick a location. You need to be signed in with an active Pro+ licence.

Hand it to the agent. Either put the file on the agent and point the runner at it:

tia-test-runner run --license-offline "C:\agent\anyautomation-ci.json" --project "C:\Projects\Plant.ap20" --out-dir reports

or store the file's contents as a masked pipeline variable named ANYAUTOMATION_CI_ENTITLEMENT, either exactly as saved or base64-encoded (some CI systems only mask single-line base64 values). The runner reads that variable when no path is given.

The --license-offline path must point to a local drive on the agent; a network-share path (\server...) is rejected. If the file lives on a share, copy it to the agent or use the variable instead.

Treat the file as a secret. It carries your entitlement, so anyone who obtains it can run the licensed verbs. Store it as a masked or secret variable, never echo it into a build log, and do not commit it to your repository.

The file stops working when your subscription ends, and in any case 90 days after you saved it. Save a fresh one from Studio when a pipeline reports an expired licence.

--license-key is no longer accepted and exits with code 3. Use the licence file instead.

These verbs do not need a licence file: version, verify and validate. verify deliberately stays open so you can re-check the integrity of an archived acceptance report years later, including after a subscription has ended, and validate is a plain schema check you can run in a pre-commit hook.

Proxy configuration

The runner honours the standard proxy environment variables by default — set HTTPS_PROXY, HTTP_PROXY, and NO_PROXY on the build agent and online license validation will route through your proxy. No extra flags are needed.

Supported environments

Environment	Supported?	Notes
Windows 11 (desktop)	Yes	Reference environment.
Windows Server 2022 with Desktop Experience	Yes	TIA Portal and PLCSIM Advanced require an interactive desktop session.
Windows Server Core	No	TIA Portal Openness needs the full desktop shell; the headless Server Core install does not provide it.
Docker / Windows container	No	TIA Portal and PLCSIM Advanced are not supported inside containers.

Troubleshooting CI runs

Symptom	Cause and fix
Exit code `3`, "`--project` is required"	A required argument is missing or the YAML config could not be read. Check `--project`, `--out-dir`, and `--config` paths.
Exit code `2`, licence message	The agent has no valid CI licence file. Save a fresh one from Studio and point the runner at it, or set the masked `ANYAUTOMATION_CI_ENTITLEMENT` variable.
Exit code `2`, "could not write to output directory"	`--out-dir` points at a protected system location. Use a path under the project directory or under the agent's local app-data area.
Exit code `4`, timeout	The run exceeded `--timeout-minutes`. Raise the limit, or split large suites; check that the PLC connection is reachable from the agent.
No test report appears in CI	The test-reporter step is not pointed at `junit.xml`, or it only runs on success. Publish reports with an `always()`/`when: always` condition and point it at `reports/**/junit.xml`.
Online validation hangs behind a corporate proxy	Set `HTTPS_PROXY` / `HTTP_PROXY` (and `NO_PROXY` for internal hosts) on the agent.

Privacy

No telemetry is collected by the CLI or the UI. The CI variables surfaced in reports are recorded only when you opt in with --ci-env-capture, and the project path is stored as an irreversible hash, never in plain text. HTML reports meet WCAG 2.1 AA, verified by automated checks; a manual screen-reader (NVDA) sign-off is part of the release end-to-end pass.

← Previous

CODESYS

Library Management