SCL Unit Testing for TIA Portal PLCs

A unit test in TIA Portal checks a single PLC block (FB/FC) in isolation: write defined inputs (Arrange), let the controller run a few cycles (Act) and check the outputs against expected values (Assert) — against PLCSIM Advanced or a real S7 PLC, automated and repeatable.

Integrated unit test framework for TIA blocks. Write, run and evaluate tests against a live PLCSIM Advanced instance without leaving the app.

Requirements:

Pro+ license or higher
PLCSIM Advanced V3.0+ installed (separate Siemens license)
A TIA project with at least one PLC open

Opening the Unit Testing Workspace

Click the Unit Testing icon (beaker) in the Activity Bar on the far left
The Side Bar then shows the Test Suites view, the tree of discovered .tia-tests suites. Its title bar shows Run All Test Suites / Stop Test Run, Re-run Failed Tests, Show Failed Only and Open Test Bench; New Test Suite, Refresh Test Suites, Choose Test Folder, Clear Test Folder, Generate CI Pipeline and Export Test Report sit in the title bar's … menu. Every suite and case row has an inline Run action on hover. Results of your runs appear in the Run tab of the Test Bench (see The Test Bench below).
Clicking a suite in the tree opens it as a Test Suite Editor in the editor area; clicking a test case opens its suite with that case already selected. The editor shows the same suite in three switchable views, Visual, JSON and SCL, chosen with the toggle at the right of its header; the active view is highlighted in the colour of your theme. The header also holds the suite's block, PLC and connection type, Validate, Run (the arrow next to it opens the run options, see Run options below), Save and a … menu with Generate from Boundaries, Import to TIA (SCL view), Open as JSON Text, Connection Settings and the contract and snapshot actions.
The Visual view is a two-part editor: the upper part lists every test case as one row (status, number, name, description, inputs, assertions, requirements, cycles, priority, tags and the last result); the lower part edits the selected case. Above the list, a filter box narrows the cases by any text, and the Status, Priority and Requirement dropdowns filter further; next to them, Add (test case, theory case or cases generated from boundary values), Duplicate, Move Up, Move Down and Remove act on the selected rows. Drag the divider between list and editor to give either part more room, or collapse the case editor with the chevron at the right of its first line (also Ctrl+Shift+J); both parts scroll on their own.
The case editor starts with the name, cycles, timeout and priority, followed by the description and the sections Inputs, Assertions, Steps, Watch and Metadata (plus Parameters for a theory case). Each section adds rows with the + icon in its header, every row shows the data type of its variable next to the name, and every row carries its remove icon at the right. Steps and Watch stay folded while empty, and Metadata shows its values in its header until you open it. The operator dropdown of an assertion is grouped (Comparison, Range, Boolean, Text, Bits, Aggregate, Snapshot); Array Equals and Deep Equals edit their expected value in a small JSON window, and a Snapshot assertion offers an Excludes window for the variables to leave out of the baseline.
Keyboard: in the case list, Ctrl+Shift+N adds a case, Ctrl+D duplicates, Alt+Up / Alt+Down move, Delete removes (with a confirmation when the case has assertions), Ctrl+Enter runs the selected cases (the whole suite when none is selected), Enter or F2 jumps to the name field and Escape returns to the list; Ctrl+1, Ctrl+2 and Ctrl+3 switch between Visual, JSON and SCL.

The Test Bench

The Test Bench is the workspace where you choose a block, look at its interface, run its tests and read the results. Open it in one of these ways:

Command Palette: Unit Testing: Open Test Bench
Open Test Bench in the title bar of the Test Suites view; the suite you selected in the tree is already chosen
Click the Unit Testing item in the status bar after a run; the Test Bench opens on the Run tab

Header. From left to right:

Control	What it does
PLC	Choose the PLC. The menu also offers Refresh PLC List while you are connected to TIA Portal.
Block	Choose the block under test; it shows the Siemens icon of the block type. Connected to TIA Portal, the list shows the blocks of your project; without a connection it shows the blocks your test suites use.
Suite	Choose the test suite of that block; the number in brackets is its number of test cases. When the block has exactly one suite, it is chosen for you.
+ (New Suite for This Block)	Creates a test suite for the chosen block without asking anything and opens it in the Test Suite Editor next to the Test Bench. The suite starts with one empty test case, so you add the inputs and assertions you need.
Connection	Grey Not connected, blue Source folder when you work offline from a folder of exported blocks, or green with the TIA Portal version and the name of the connected project. Click it for Connect to TIA Portal or Disconnect from TIA Portal, Choose Source Folder… (or Change Source Folder… and Clear Source Folder) and Open PLCSIM Dashboard.
Run	Runs the suite chosen in the header. The arrow next to it opens the run options: all suites, the suite or its failed cases, with coverage, mutation testing or a contract check, and the options for flaky cases and snapshot baselines (see Run options below). Stop appears while a run is in progress.
…	Open Suite in Editor, Connection Settings, Open Block XML, Show Coverage Heatmap, Show Mutation Markers, Export Test Report… and Generate CI Pipeline (opens the CI Pipeline page, see Generate a CI/CD pipeline from the app).

A thin bar under the header shows the progress of a run.

Block column (left). A card shows the block's name, type and language and where it is read from (TIA Portal, your source folder, or the exported files of the project). Below it are the tabs Interface, Boundaries, Dependencies and Address Space. As soon as you choose a block, Interface and Boundaries fill on their own, also without TIA Portal when a source folder is set. While a block is being read, a progress bar runs at the top of the Test Bench and the tabs name the current step, for example "Loading the interface of 'FB_Motor'…"; when the boundary values cannot be determined, the interface stays visible and Boundaries shows the reason. Dependencies take longer and are read only when you click Analyze Dependencies in the header of the Block column; Refresh Analysis next to it reads the block again.

Results column (right). The tabs Run, History, Quality and Traceability, described in the sections below. History lists your past runs and also compares two runs and shows trends; Quality shows coverage and mutation testing results; Traceability shows which test cases cover each requirement and their latest results.

Before a run. The Test Bench runs the saved suite. When the suite is open in the Test Suite Editor with unsaved changes, Studio asks whether to save it first (Save and Run). When the editor shows issues that block saving, the run does not start and a message names the suite. The same applies to every run from the Test Suite Editor and to coverage, mutation testing and contract checks.

Keyboard. With the Test Bench focused, Ctrl+; then S runs the suite chosen in the header, and Ctrl+; then Q stops a running test.

After a restart. The Test Bench opens again with the same PLC, block, suite and tabs, and shows the run it showed before.

Test Explorer

The Test Explorer shows the test suites of the TIA project you are currently connected to — every .tia-tests/*.json file appears as a suite node with its test cases as children. While no test folder is chosen and no project is connected, the view reads "Connect a TIA project or choose a test folder to load your test suites."; once a folder is chosen its suites appear automatically (a chosen folder with no suites shows "No test suites yet."; if the chosen folder no longer exists, it shows "Selected test folder not found: <path>").

Status icons (loaded from SQLite on startup):

✓ Pass — green checkmark, test passed in the last run
✗ Fail — red cross, test failed in the last run
⚠ Error — orange warning, test errored (exception, timeout, S7 error)
⊘ Skipped — grey slashed circle, test was filtered out of the last run
○ NotRun — grey empty circle, no persisted result yet

Interactions:

Action	Result
Click a suite (or select it and press Enter)	Opens the suite as a Test Suite Editor in the editor area
Click a test case	Opens its suite in the Test Suite Editor with that case selected
Point at a suite, then click Run Suite in its row	Runs this suite only
Point at a test case, then click Run Single in its row	Runs this test case only
Run All Test Suites (view title bar)	Runs all visible suites sequentially
Stop Test Run (view title bar, while a run is in progress)	Cancels the current run at the next safe point
New Test Suite / Refresh Test Suites (title-bar … menu)	Create a new suite, or rescan the `.tia-tests/` folder
Choose Test Folder / Clear Test Folder (title-bar … menu)	Pick which folder your test suites live in (when a folder is chosen the list follows it, otherwise the connected project), or clear the selection again

A filter box at the top of the Test Suites view narrows the tree by suite and case name, a Show Failed Only title-bar toggle hides everything that didn't fail, and a per-case Run Single action runs just one case.

Planned for the in-app workspace. A Run Affected Only action (re-run only suites whose underlying block changed) is planned. Today, single-suite, single-case, and all-suites runs are available as above; on the command line, tia-test-runner run --rerun-affected already runs only the suites whose underlying block changed (see CI/CD Integration).

Planned — Block-Change Badge. A small orange dot next to a suite whose underlying PLC block changed since its last run, with a "Block changed since last run" tooltip, is planned for the Test Suites view.

A test case that has been unstable across its recent runs (passing some times, failing others) carries a Flaky badge next to its name, with a tooltip showing how many times its result changed. This is computed from the run history, so it appears once a case has both passed and failed in recent runs. The badge clears on its own once the case settles back into a steady result, so a test you have fixed stops being flagged after a few consistent runs, and a case that now fails every time is shown as a real failure rather than flaky.

Test-Case Metadata

The case editor of the Visual view ends with a Metadata section (folded by default, below Watch; its header shows the current values). Click the section header to open it and edit the following fields per case; priority is also editable in the first line of the case editor, and requirements, priority and tags appear as columns of the case list:

Field	Purpose
Order	Numeric run order. Leave empty to keep the natural order in the suite
Tags	Labels (e.g. `safety`, `regression`) used for filters in HTML reports, entered as chips (see below)
Priority	One of Low / Normal / High / Critical. Drives report sort order
Owner	Free-text owner (team name, e-mail, etc.)
Requirements	Requirement IDs (e.g. `REQ-123`, `REQ-456`) for traceability, entered as chips with suggestions from your test folder (see below)

After Validate, every issue appears in a strip under the editor header as N issues; open it and click an issue to jump to the affected case and section. A case with issues also carries a warning mark in the status column of the case list, and Save and Run stay disabled until blocking errors are fixed. All edits are auto-saved with the rest of the suite. Suites created before this release load with empty metadata defaults — no migration is required.

Requirement IDs and tags

Tags and Requirements show each entry as a chip. Type into the box after the chips and press Enter or a comma to add an entry. Paste a list separated by commas or line breaks to add one chip per entry. To remove a chip, click its x. From the keyboard, Backspace in the empty box moves to the last chip, the arrow keys move between chips, and Backspace or Delete removes the chip that has the focus.

While you type a requirement ID, a list suggests the IDs that the saved test suites of your test folder already use, each with the number of test cases and the suites that name it, plus IDs you added to this suite but have not saved yet. This keeps one requirement spelled the same way in every suite. Suggestions do not include unsaved changes in other open suites.

A requirement ID is refused, with the reason under the box, when it is empty, contains control or invisible formatting characters, is longer than 256 characters, when the test case would name more than 50 IDs, or when the case already names it. A suite that already contains such an ID still opens: the problem appears in the issues strip, and Save and Run stay disabled until you fix it.

When two IDs differ only in upper and lower case, spacing or dash style (for example REQ-1 and req-1), a warning appears under the chips and in the issues strip. If the other spelling is used elsewhere in your test folder, Use REQ-1 replaces your chip with it. Studio never merges the two on its own: coverage and reports count them as different requirements until you choose one spelling.

Tags are suggested from the current suite. A tag longer than 64 characters, or more than 50 tags on one test case, shows a note, because reports shorten them.

The Requirement dropdown of the filter above the case list shows this suite's IDs with the number of test cases for each, then, under Other suites in the folder, the IDs that only other suites use. Choosing one of those shows "No case of this suite covers ..." with Clear Filter.

In AI Chat you can ask "which requirement IDs exist in my test folder?"; the assistant lists them, and when it adds requirements to test cases it follows the same rules and points out spelling variants.

Live Progress During a Run

While a test is running, you can follow it in several places:

Status bar: the Unit Testing item shows the progress, for example "Testing 4/36". When the run is done it shows the result, for example "36 passed" or "35 passed, 1 failed". Hover it for the full breakdown, click it to open the Run tab of the Test Bench.
Test Bench: a thin progress bar runs under the header, and the Run tab fills as each test case finishes. The first run after you start Studio opens the Test Bench on the Run tab; when you started the run from the Test Suite Editor, the Test Bench opens as a tab in the background so your editor stays in front.
Test Suites view: status icons update live per test case.
Stop Test Run: cancels the current run at the next safe point (the S7 connection and PLCSIM instance are always cleaned up).

Reading Results in the Run Tab

The Run tab of the Test Bench shows the latest run, wherever you started it (Test Bench, Test Suites view, Test Suite Editor, the CodeLens above a block or the AI assistant). A Run All shows every suite of the run together.

Header line: the suite name (or, for example, "Run All, 7 suites"), start time, duration, PLC, TIA Portal version, connection type and computer name. The list at the right switches between the runs of this session; a run opened from the run history is marked "(history)".
Counters: total, passed, failed, errors and skipped (with the number of flaky cases that were set aside); a counter at zero is shown without colour. Next to them: Rerun repeats the same run, Re-run Failed Tests repeats only its failed cases, Export Report… opens the export form for this run (see Exporting a Report), and Compare To… takes you to the History tab to pick the run to compare it with.
Filters: Failed only, the Suite, Block, Tag and Requirement lists, and a text box that searches case, suite, block and message.
Case list: status, case name (with Flaky and Quarantined marks), the suite (when the run covered several suites), duration and message. Double-click a case to open it in the Test Suite Editor. Column widths you set are remembered.
Assertions: every assertion of the selected case with Variable, Operator, Expected, Actual and Status. When a snapshot no longer matches its baseline, Approve Baseline on that row accepts the new result, and Approve Baselines (N) above the grid accepts all of them after a confirmation. Both work on the newest run of the session.
Timeline: a chart of how each watch variable changed cycle by cycle, refreshed live during the run. Collapse the section when you do not need it.
Sign-off line: below the header line, once the run is stored, whether the run is signed off, by whom and when, with Sign Off… or Revoke… (see Signing Off a Test Run). For a Run All it reads, for example, "3 of 7 runs signed off".

Runs from the history. A run opened from the History tab shows the values that were read during that run. The expected values come from the suite as it is now, which the header notes with "Expected values from the current suite". Values that older runs could not store readably show as "not recorded".

To collect watch data, add a "watch": ["Var1", "Var2"] array to a test case in the suite JSON and set "cycles" greater than 1. The sampling is capped at 100,000 cycles per test case. The same data is also rendered as a time-series chart in the HTML reports produced by the tia-test-runner command-line tool (see CI/CD Integration).

Test results are persisted to %LocalAppData%\AnyAutomation Studio\db\test_results.db: they survive app restarts and reappear in the Test Suites view automatically.

History

The History tab of the Test Bench lists every stored test run, newest first (up to the latest 200), whichever suite it ran. It has three modes, which you switch with Runs, Compare and Trend in its top line. The tia-test-runner command-line tool produces the same information as HTML and CSV reports (see CI/CD Integration).

Runs

Columns: by default Started, Duration, Result, Sign-off, Suite, Block, PLC, Host and TIA, after a status dot. The dot shows the result of the run: Passed, Failed (also when only one test case failed or had an error), Error (the run stopped early, for example because the connection was lost) or Cancelled; point at it to read the result. Sign-off shows whether the run is signed off (hover it to see by whom and when). The Result column shows small coloured counters: passed always, failed, errors and skipped only when there are any. When flaky failures were set aside, the skipped counter reads for example "2 (1 flaky)" with a dashed border.
Choosing columns: click Columns in the top line and tick the columns you want. Total, Passed, Failed, Errors, Skipped, Suite File and Run Id can be added; Reset Columns brings back the defaults. At least one column always stays visible. Your choice and the column widths are remembered, also for columns you hide and show again.
Filter box: narrows the list by suite, block, PLC or host.
Refresh: the refresh icon or F5 reloads the list; it also reloads on its own after every run.
Open a run: double-click a row or press Enter to show that run in the Run tab.
Select several runs: use Ctrl or Shift like in any list, for example to compare two runs or to export several reports at once.

Right-click a run (or press Shift+F10):

Action	Result
Open Results	Shows the run in the Run tab
Compare To…	Lets you pick the run to compare it with (see Compare below)
Export Report…	Opens the export form for this run, or for all selected runs (see Exporting a Report)
Sign Off…	Signs off this run, or the selected runs that can be signed off (see Signing Off a Test Run)
Revoke Sign-off…	Revokes the sign-off of this run with a reason
Delete Run…	Permanently removes the run from the history after a confirmation. Works on one run at a time; with several rows selected it is disabled. A signed-off run cannot be deleted until its sign-off is revoked. If the Run tab or a comparison showed that run, they are cleared

Delete on a single selected row does the same as Delete Run….

The project path of a run is never stored in plain text, only as an irreversible fingerprint, so runs can be matched across machines without revealing workspace paths.

Compare

Compare shows what changed between two runs.

Two selected runs: select two rows and click Compare. The older run becomes the baseline, the newer one the current run. Compare stays unavailable until two runs are selected.
Compare To…: in the Run tab or in the row menu of a run. The History tab opens with that run selected and the line "Choose the run to compare with …". Click, double-click or press Enter on the run you want as the baseline, and the comparison opens. Studio always waits for the run you pick next, even when another comparison was open before, and Cancel in that line returns to the run list without a comparison.

The comparison shows:

Baseline and Current side by side, each with its start time, its passed and failed counts and its duration, and Swap to reverse them.
Summary counters: Regressions, Fixed, New, Removed and Stable.
Filter: All, Regressions, Fixed, New or Removed narrows the case list.
Case list: each case with its change (for example Regression, Fixed or Still Failing), its message and the change in duration, with "N of M cases" below.

Trend

Trend shows how a suite develops over time.

Suite and Case: the suite list is built from your run history, so it is filled even when you have never opened the Test Suites view. When two PLCs have a suite with the same name, the PLC is shown in brackets. The case list comes from the chosen suite.
Time range: Last 30 Days, Last 90 Days, or your own From and To dates.
Pass Rate chart for the PLC of the chosen suite, Duration chart for the suite, and the Case History heatmap for the chosen case, one cell per run, coloured by result and marked with a symbol. Hover a cell to see when the run started.
The refresh icon in the top line loads the charts again.

When the Trend opens, it starts with the suite of the selected run, otherwise the suite chosen in the Test Bench header, otherwise the suite of the newest run.

Share at the right of the top line offers, depending on the mode:

Mode	Share offers
Runs	Export Report… for the selected runs (or the newest run), and Export CSV… with the rows and columns you currently see, including who signed off each run
Compare	Export Report… for the comparison
Trend	Export CSV… with the points of the trend

CSV files are written so that a spreadsheet program never runs a cell as a formula.

Exporting a Report

Export Report… opens a small form next to the button you clicked. You reach it from Share in the History tab, from the row menu of a run, from the export icon in the Run tab, from Export Test Report… in the Test Bench … menu, and from the Command Palette with Unit Testing: Export Test Report (which opens the History tab with the newest run selected).

Formats: HTML report and JUnit XML. A comparison is exported as HTML only.
Scope: This Run, Selected Runs (with the number of selected runs) or Comparison (available after you compared two runs). A scope that does not fit your selection is greyed out and tells you why.
Output: the Output folder, relative to the reports folder of your TIA project (by default report). The folder must lie inside that reports folder; otherwise the field says "Choose a folder inside the project's reports folder." and Export stays unavailable. When you export several runs, each run gets its own subfolder named after its start time, so no report overwrites another. The newest export always lands at the top of its folder: if the folder already holds the report of another run, that earlier report moves into its own subfolder under runs and stays verifiable there, and exporting the same run again simply replaces its report. If an export fails, the folder stays as it was.
Branding: open this section to set a Title, a Footer and an Accent color (a hex color such as #2563eb).

Click Export (or Export 3 Reports and so on), or press Ctrl+Enter; Escape closes the form. While you are not connected to a TIA project, Export is unavailable and the form says so, because reports are written into the project. The HTML report of the newest exported run opens in Studio as a page as soon as it is ready, and a message names the folder the files were written to and, when an earlier report was moved, where it went. Click Open Folder in that message to see the files in File Explorer. The form remembers your formats, folder and branding for the next export.

Quality: Coverage and Mutation

The Quality tab of the Test Bench shows how thoroughly the tests of a suite exercise their block. It belongs to the suite chosen in the Test Bench header; without a suite there, it shows the suite of your newest run. It has two sections, which you can fold and unfold; Studio remembers which ones are folded.

Coverage. The line under the heading names the run it belongs to, and whether that run measured coverage line by line. Up to five bars show what the tests reached: Requirement, Interface, Called Blocks, and after a run with coverage instrumentation also Statement and Branch. Each bar shows covered of total and a percentage, or n/a when the block has nothing of that kind. Open a bar to see what was not reached; for statements and branches, click a line to open the block at that line. The icons in the section heading are Refresh, Export Coverage… (writes a Cobertura file into the reports folder of your TIA project; the message names the folder, says where an earlier report in it was moved, and offers Open Folder) and Run with Coverage Instrumentation.

Mutation. Shows the latest mutation testing result of the suite (see Mutation Testing below), also when you ran the suite normally after the mutation run. The icons in the section heading are Refresh, Toggle Mutation Markers (ticked while the markers are shown in the editor, the same setting as Show Mutation Markers in the Test Bench … menu) and Run Mutation Testing.

Starting a run from here. Run with Coverage Instrumentation and Run Mutation Testing run the suite of the section. Like every run from the Test Bench, they ask to save the suite first when it has unsaved changes. They are greyed out, with the reason, when the suite does not run on PLCSIM Advanced or you are not connected to TIA Portal. While and after such a run, the Test Bench stays on the Quality tab and both sections update when the run ends.

Mutation Testing

Mutation testing answers a question coverage cannot: do your tests actually catch a fault? It makes a series of tiny changes to a copy of your block (for example, flipping a > to a >=, swapping an AND for an OR, or removing an output assignment), runs your existing test suite against each changed copy, and reports the changes your tests failed to notice. A change your suite caught is a "killed" mutant; a change that slipped through with every test still green is a "surviving" mutant, and that is a gap in your tests.

Your original block is never touched. Every change is made on a throwaway copy that Studio cleans up after each run.

Running mutation testing

Select a test suite in the Test Suites view.
Make sure you are connected to TIA Portal and your run targets a PLCSIM instance. Mutation testing needs a PLCSIM target, so a suite set up for a direct S7 connection cannot be used here.
Run Run Mutation Testing from the Command Palette.
Studio first runs the suite once against the unchanged block. The suite must pass first (mutation testing measures whether your passing tests are thorough, so it tells you to fix any failing cases before it continues). It then generates the mutants and runs the suite against each one.

You can also start it with Run Mutation Testing in the Mutation section of the Quality tab, or by choosing Mutation Testing in the run options of the Test Bench; the Test Bench then stays on the Quality tab.

The run can take a while because each mutant is compiled, loaded, and tested in turn. You can stop it at any time, and Studio still cleans up the temporary copies it created.

Reading the results

The Mutation section of the Quality tab of the Test Bench shows:

A mutation score (the percentage of mutants your tests caught, ignoring any that could not run), with the killed, survived, and stillborn counts.
A Surviving Mutants list, one row per gap: the line, the kind of change, and what was changed. Click a row to jump straight to that line in the block source.
Collapsible lists of the killed and stillborn mutants.

Surviving mutants are also highlighted in the editor: the affected lines get a red tint and a red marker in the gutter, with a tooltip describing the change no test caught. Use Show Mutation Markers in the Test Bench … menu, Toggle Mutation Markers in the Command Palette, or Toggle Mutation Markers in the Mutation section of the Quality tab to show or hide them.

A high score means your tests caught nearly every fault. A low score, or a long surviving list, tells you exactly where to add or strengthen test cases.

Settings

Maximum mutants per run sets how many changes Studio tries in one run. A higher number finds more gaps but takes longer.
Mutation operators lets you choose which kinds of change to apply: relational comparisons, logical operators, constants and arithmetic, and statement deletion. All are on by default.

Requirement Traceability

The Traceability tab of the Test Bench shows, for every requirement ID of your test folder, which test cases cover it and what their latest results are. It covers the whole test folder, whatever PLC, block or suite the Test Bench header shows. Open it from its tab next to Quality, or run Unit Testing: Show Requirement Traceability from the Command Palette. With a requirement chosen in the Requirement filter of the Test Suite Editor, Show in Traceability opens the tab with that requirement selected.

The requirement list. The upper table lists one row per requirement with its status, the number of test cases, one chip per result (for example 2 Passed, 1 Failed), the suites and the time of the latest evidence. Rows are sorted by status, the most urgent first:

Status	Meaning
Failed	At least one test case of the requirement failed or ended with an error in its latest run
Uncovered	Your imported requirement list names the requirement, but no test case does yet
Not Run	A test case of the requirement has not run in the recent runs of its suite
Outdated	A test case changed after its latest run, or a new snapshot baseline of its suite was approved; run it again
Unknown	The latest run was recorded before Studio could detect changes; run it again
Incomplete	A test case was skipped or quarantined as flaky
Passed	Every test case of the requirement passed in its latest run

Studio looks at the latest 20 runs of each suite. When you run only one test case or a selection, that run counts only for those cases; the other cases keep the result of their last full run. Changing the connection of a suite (PLCSIM instance, IP address, passwords) does not make results outdated.

The test cases of a requirement. Select a requirement to see its test cases in the Cases section below: suite, case, state, the start time of the run that produced the result (marked "partial run" when only some cases ran), whether that run is signed off, and the message. Double-click a case, or press Enter, to open that run in the Run tab with the case selected. Right-click a case for Show Run, Open in Suite Editor and Copy Requirement Id. Drag the divider to resize the sections, or press Tab until the divider is focused and use the Up and Down arrow keys (with Shift for larger steps, Home and End to move it all the way). Collapse the Cases section with its header.

Filter. Type into the filter box to search requirement IDs, titles, suites, cases and blocks. Status narrows the list to one status. Untraced cases lists the test cases that name no requirement at all, so you can find tests that still need a requirement ID. The counts at the right of the top line summarise the list, for example "12 requirements, 1 failed, 2 uncovered". Refresh (F5) reloads the tab; it also refreshes by itself after a run and when you save a suite.

Share. Share at the right of the top line offers:

Export CSV… writes the rows you currently see (one row per requirement and test case, with the sign-off of the run that produced each result) into a CSV file for your quality records. Cells that a spreadsheet program would run as a formula are written as plain text.
Export HTML Report writes a traceability report into the reports folder of your TIA project (the subfolder traceability of the output folder you last chose in Export Report, or report/traceability when you have not chosen one) and opens it as a page. It lists every requirement with its status and test cases, the untraced test cases, the problems of your requirement list and a legend of the states, with the title, footer and accent color you last used for a report. You can check later with tia-test-runner verify that nobody changed the files. The item is available while you are connected to the TIA project.

The tab, the selected requirement, the status filter and the Untraced cases toggle are kept when you restart Studio.

Importing a requirement list

Import the requirement list of your project (for example an export from your ALM or requirements tool, or a spreadsheet saved as CSV) to see which requirements have no test yet. Open Requirement List in the top line of the Traceability tab and choose Import Requirement List…, or run Unit Testing: Import Requirement List... from the Command Palette, and pick the file.

Files: CSV or text files up to 10 MB and 20,000 rows. UTF-8, UTF-16 (the "Unicode text" of spreadsheet programs) and ANSI files are read; columns may be separated by commas, semicolons or tabs. Save the list as UTF-8 to keep special characters exact.
Columns: the first row names the columns. The requirement ID column may be called id, requirement, requirement id, req id or req; an optional title column title, name or summary; an optional description column description, text or details. Other columns are ignored. A file with a single column of IDs and no header works too.
Before anything is written, Studio checks the file and shows what it found, for example "124 requirements found. 1 row is skipped.", together with any problems, and asks Import. Rows with an empty or invalid ID are skipped, a duplicated ID keeps its first row, and titles or descriptions that are too long are shortened. A file that cannot be imported at all (too large, no ID column, an unclosed quote) is refused with the reason.
The list is stored in the test folder next to your suites (.tia-tests/requirements.csv), so it travels with them in version control and the command-line runner finds it too. Importing again replaces it; your original file is never changed.

With a list, the Traceability tab adds a Title column, shows listed requirements without a test case as Uncovered, and marks requirement IDs of your suites that the list does not contain with "Not in requirement list". When an ID is spelled differently in the list and in the suites (for example REQ-1 and req-1), a note above the table points it out. Problems in the list are summarised above the table, and the IDs of the list are suggested when you type a requirement ID into a test case.

Open Requirement List opens the stored list in a text editor. If you open it in a spreadsheet program instead, cells that start with =, +, - or @ are calculated as formulas; Studio keeps such IDs, titles and descriptions exactly as written and warns about them. Remove Requirement List… moves the list to the recycle bin after a confirmation; requirements that no test case names then disappear from the tab.

In AI Chat you can ask "which requirements fail?" or "which requirements have no test yet?"; the assistant reads the same states as the tab. It can also check and import a requirement list file for you; it asks for your approval before it replaces the list, and before it checks a file outside your workspace. Removing the list is left to you.

Signing Off a Test Run

With a Pro+ or Enterprise plan (also during the trial), you can sign off a test run to record that you checked and accepted its result, for example before a factory acceptance test. The sign-off names your account and the time, and can carry a comment. You can sign off your own runs. Each run can be signed off once; to change a sign-off, revoke it and sign off again.

Which runs can be signed off. A run of a whole suite that finished. A run that was cancelled, a run of only some test cases, and a run whose suite was changed afterwards cannot be signed off; run the suite again and sign off the new run. Runs recorded before sign-off was available need a new run as well. When a run has failed, errored or quarantined test cases, a comment is required that explains why the result is accepted.

Sign off.

Open the run in the Run tab and click Sign Off… in its sign-off line, or right-click the run in the History tab and choose Sign Off…. You can also run Unit Testing: Sign Off Test Run… from the Command Palette, which opens the History tab with the selected or newest run.
A small form shows the runs with their results and the account you sign with ("Signing as …"). Enter a comment if you like (up to 1,000 characters, line breaks allowed).
Click Sign Off or press Ctrl+Enter.

To sign off several runs at once, select them in History (up to 50) and choose Sign Off… on one of them, or click Sign Off… in the Run tab after a Run All. Either all runs are signed off or none: if one run cannot be signed off, the form names the reason next to that run and nothing is recorded.

Revoke. Click Revoke… in the Run tab, choose Revoke Sign-off… on the run in History, or run Unit Testing: Revoke Test Run Sign-off…. Only the account that signed off the run can revoke it. Enter the reason (required) and click Revoke Sign-off. The sign-off and the reason stay in the history of the run, and the run can be signed off again.

What the states mean.

State	Meaning
Signed off	The run and its suite are unchanged since the sign-off
Outdated	The suite changed after the sign-off; run it again and sign off the new run
Signed off, suite deleted	The suite file was deleted after the sign-off
Broken	The stored run no longer matches what was signed off
Revoked	The sign-off was revoked; the run can be signed off again
Not signed off	Nobody has signed off the run

In the Run tab, hover the state to see who signed off, when, and every earlier sign-off and revocation of the run.

Signed-off runs are kept. A signed-off run cannot be deleted from History until you revoke its sign-off. Deleting a suite never deletes its runs; the confirmation tells you how many signed-off runs of the suite stay in History. When old runs are cleared from the result database, signed-off runs are kept.

In reports. The HTML report of a run shows its sign-off state as it was when you exported the report, with the account, the time, the comment and, for a revoked sign-off, the reason. The JUnit XML report carries the same details, so your CI system can show them. tia-test-runner verify prints the sign-off of a report and fails when its details were changed after the export.

Good to know. The sign-off is recorded on the computer that holds the run, in its local result database, for the account you are signed in with. It detects later changes to the run, the suite and the sign-off itself, but it is not an electronic signature. You need to be signed in with your account; if your licence was last checked more than 90 days ago, sign in again. If the clock of your computer shows a time before the end of the run or before its last sign-off, Studio refuses to sign off or revoke; check the system time. The AI assistant can tell you whether runs are signed off, but only you can sign off or revoke.

Block Analysis (before running tests)

When you open a suite, AnyAutomation Studio extracts the block interface from the TIA project so the editor knows the real parameter names and types, and so the inputs and assertions grids can validate against them.

You do not need TIA Portal open for this. If you are not connected to TIA Portal, Studio reads the block from a copy you exported earlier (export it from the project explorer). If the block has not been exported yet, Studio shows a message asking you to export it first or connect to TIA Portal, instead of leaving the interface empty.

Pick a Source Folder. Choose Source Folder… in the connection menu of the Test Bench header lets you tell Studio where your exported blocks are. Pick the folder that holds your export files, and the folder structure inside it no longer matters: whether the files sit directly in the folder or in sub-folders, Studio finds the right file for the block. This lets you analyze and run tests without TIA Portal open. A Source Folder you pick is remembered, so it is ready again right after a restart. The same menu then offers Change Source Folder… and Clear Source Folder. For the cleanest results, pick the folder you exported into, not a parent folder of several projects.

The Block column of the Test Bench has the tabs Interface, Boundaries and Dependencies. Choose a block in the header and Interface and Boundaries fill right away; Dependencies are read when you click Analyze Dependencies. They summarise:

Interface: All block parameters (Input, Output, InOut, Static, Temp) with their types; hover a type to see its default value. Structures, PLC data types and multi-instances start collapsed: click one to expand it, or use Expand All and Collapse All at the top of the tab.
Boundaries: Auto-generated min/max/zero/one values per parameter type (with a Generate Boundary Cases action that saves a parameterized case as a new suite)
Dependencies: Called blocks, referenced DBs, referenced UDTs

Creating a New Test Suite

Click New Test Suite in the Test Suites view title bar (or run it from the Command Palette)
When you are connected to a TIA project with PLCs, pick the PLC, then the block under test — the suite name is auto-derived as <Block>_Tests. (Only when no PLCs are available do you instead answer three free-text prompts in turn: the Suite name, the Block under test, and the PLC name.)
The new suite is written to <projectDir>/.tia-tests/{name}.json and appears in the Test Suites view
The Test Suite Editor opens with one empty test case, Case_1. Add the variables you need with Add Input and Add Assertion: the Variable field suggests every member of the block interface as you type, including members of structures, PLC data types and multi-instances. The block interface is read from the TIA project, so the inputs and assertions grids validate against the real parameter names. An input the case does not set keeps the value it has in the PLC when the case starts, so set every input your test depends on. To start from cases filled with boundary values instead, use Generate from Boundaries in the … menu.
Use the Visual mode (form-based master-detail) to add / remove / duplicate test cases through fields and grids — Name, Description, Cycles, Tags, Priority + Arrange-Inputs grid + Assertions grid + Watch list. Switch to Open as JSON Text anytime to edit the raw JSON. The SCL view shows the generated SCL test code and is editable too (see Managing Suites and Test Cases)
Click Save (Ctrl+S) to persist your changes

Renaming a suite from the JSON `name` field

When you edit the name field at the top of the suite JSON and press Save, the file is renamed on disk to match — {newName}.json — and the editor title updates in the same step. If a suite with the new name already exists in the same folder, Save aborts with an error and leaves both files untouched so you can pick a different name.

The blockName field is independent and continues to identify the TIA block under test; it is not affected by a name rename.

Test Suite JSON Schema

Every .tia-tests/*.json file follows the same shape. Top-level fields:

Field	Type	Required	Purpose
`name`	string	yes	Suite identifier (must match the file name without `.json`)
`blockName`	string	yes	Exact TIA block name (case-sensitive)
`plcName`	string	yes	PLC/CPU name in the TIA project (e.g. `PLC_1`)
`config`	object	yes	Connection + transport settings (see below)
`testCases`	array	yes	One entry per test case (see below)
`wiring`	object	no	Which data block variable the calling program passes to each input, written there by the test (see Wiring inputs to the calling program)

config (all fields optional, defaults shown):

Field	Default	Purpose
`cycles`	`1`	Default PLC cycles per test case (overridable per case via `act.cycles`)
`cycleWaitMs`	`100`	Wait between sampled reads when `cycles > 1`
`timeoutMs`	`5000`	Hard timeout per test case
`instanceDbName`	`""`	Instance-DB to read/write; empty → `<blockName>_DB` (Siemens convention)
`preferFc`	`false`	Tolerate FC blocks (no Instance-DB); set when the target is a function instead of a function block
`transport`	`"plcSimApi"`	`"plcSimApi"` for direct PLCSIM Tag-API (the default), `"s7CommPlus"` to connect directly to a real S7 controller (see Connecting over S7 Native)
`s7IpAddress`	`192.168.0.1`	S7 connect target — used only when `preparationMode = "external"` (real PLC). For PLCSIM-backed modes (`userPreloaded`, `tiaTcpDownload`) the runner connects to `plcSimIp` instead, regardless of transport
`s7Port`	`102`	S7 ISO-on-TCP port. Default `102` matches every Siemens default — change only when the PLC was reconfigured to a non-standard TCP port
`s7User`, `s7PasswordKeyId`	`""` / null	Credentials reference (password lives in Windows Credential Manager). Required only when the PLC enforces user authentication — most projects leave this empty
`plcSimInstanceName`	`"TiaUnitTest"`	PLCSIM Advanced instance name. Used by both transports whenever `preparationMode != "external"`
`plcSimIp`	`"192.168.0.100"`	PLCSIM Virtual-Adapter IP. This is the actual S7 connect target for `s7CommPlus + userPreloaded/tiaTcpDownload` — set it to the IP you assigned to the Siemens PLCSIM Virtual Ethernet Adapter, not to the PLC project IP
`preparationMode`	`"userPreloaded"`	`userPreloaded` (default; assumes the PLCSIM instance is already running with your project loaded), `tiaTcpDownload` (compiles + downloads from the open TIA project — see prerequisites below), or `external` (real PLC; uses `s7IpAddress`)
`masterSecretPasswordKeyId`	null	Project master-secret password reference. Required for `tiaTcpDownload` only when the TIA project uses confidential PLC configuration protection. Password lives in Windows Credential Manager
`autoConnectS7`	`true`	Auto-connect S7 before run starts. Set to `false` only in advanced setups where another client (WinCC, custom HMI) already holds the session and the runner should read/write through it
`keepInstanceAfterRun`	`false`	Keep PLCSIM instance alive after the run. Has effect only in `tiaTcpDownload` mode — `userPreloaded` and `external` never manage the instance lifecycle

Each entry in testCases:

Field	Type	Required	Purpose
`name`	string	yes	PascalCase scenario name (e.g. `Start_FromIdle_Runs`)
`description`	string	no	One-sentence what-and-why
`arrange.inputs`	object	yes	`{ "MemberName": value, … }` — written to the DB before Act
`act.cycles`	int	no (defaults to `config.cycles`)	PLC cycles between Arrange and Assert; must be > 1 for `watch[]` to fire. Mutually exclusive with `act.steps` (the schema's `oneOf` rejects both on the same `act`)
`act.timeoutMs`	int	no (defaults to `config.timeoutMs`)	Per-case timeout. Covers the entire `steps` sequence end-to-end when `steps` is set
`act.steps`	array	no	Optional ordered sequence of `write` / `wait` / `assert` steps that runs between Arrange and the top-level `assertions`. See Multi-phase Steps below. Mutually exclusive with `act.cycles`
`assertions`	array	yes	`[ { "variable": "X", "operator": "equal", "expected": v, "tolerance": t? }, … ]`
`watch`	string[]	no	Variables sampled once per cycle during Act → time-series chart in the HTML reports
`tags`	string[]	no	Free-form labels
`priority`	string	no	`"low"` / `"normal"` (default) / `"high"` / `"critical"`
`requirements`	string[]	no	Traceability links
`order`	int	no	Stable sort key in the UI

Supported operator values: equal, notEqual, greaterThan, lessThan, greaterThanOrEqual, lessThanOrEqual, inRange (expects expected: [min, max]), inRangeExclusive, approximately, isTrue, isFalse, contains, startsWith, endsWith, matches, bitsSet, bitsClear, arrayEquals, allEqual, anyEqual, noneEqual, deepEquals. tolerance applies to inRange, approximately, arrayEquals, allEqual, anyEqual, noneEqual, and deepEquals (default 0.0001) — not to equal/notEqual; use approximately for fuzzy Real/LReal equality. expected is required by the schema for every assertion (set it to any value — false, true, 0 — for isTrue/isFalse, since the runner ignores it).

Safety — example values are placeholders. When the editor cannot infer real Input/Output names from the block interface, the seeded example uses safe defaults (Enable: false, Running: false) so a one-click Run cannot accidentally activate motors, valves, or other outputs on real hardware. Always review every arrange.inputs value and every assertion before pointing the runner at a real PLC.

Important — watch[] is the test case's diagnostic array, NOT the TIA Watch Table. Variables listed in watch are sampled once per PLC cycle during the Act phase (provided act.cycles > 1) and rendered as a time-series chart in the HTML reports. This is independent of TIA Portal's external Watch Table or any OPC UA subscription.

Complete example

A full minimal suite with one test case that exercises every block and uses watch[]:

{
  "name": "FB_Motor_Tests",
  "blockName": "FB_Motor",
  "plcName": "PLC_1",
  "config": {
    "cycles": 1,
    "timeoutMs": 5000,
    "instanceDbName": "FB_Motor_DB",
    "transport": "plcSimApi",
    "autoConnectS7": true,
    "preparationMode": "userPreloaded"
  },
  "testCases": [
    {
      "name": "Start_FromIdle_RunsWithinThreeCycles",
      "description": "Asserting Run goes high after StartCmd is set; watch shows trajectory.",
      "arrange": {
        "inputs": {
          "StartCmd": true,
          "Reset":    false,
          "Speed":    50.0
        }
      },
      "act": { "cycles": 5 },
      "assertions": [
        { "variable": "Run",       "operator": "equal",     "expected": true },
        { "variable": "Speed_Act", "operator": "inRange",   "expected": [49.5, 50.5] },
        { "variable": "Error",     "operator": "isFalse", "expected": false }
      ],
      "watch": ["Run", "Speed_Act", "StartCmd"],
      "tags": ["happy-path", "smoke"],
      "priority": "normal"
    }
  ]
}

After the run, the Run tab of the Test Bench shows pass/fail per assertion, and the HTML report renders a time-series chart for the three watch variables across the 5 cycles. To add more variables to watch on an existing case, append to the watch array and increase act.cycles to the smallest number that gives useful coverage (3-10 is typical).

Multi-phase Steps (`act.steps`)

When a single Arrange-then-Assert is not enough — for example reset → wait → start → wait → check — use act.steps instead of act.cycles. Three step types are available:

`type`	Required fields	Effect
`"write"`	`inputs` (object, same shape as `arrange.inputs`)	Writes the listed members additively on top of whatever is already in the DB. Members not listed keep their value.
`"wait"`	`ms` (int, `0`..`3 600 000`)	Async delay. The PLC keeps cycling. When `watch[]` is non-empty and `config.cycleWaitMs > 0`, watched variables are sampled once every `config.cycleWaitMs` ms throughout the wait window.
`"assert"`	`assertions` (array, same shape as the top-level `assertions`)	Per-step checkpoint. A failure aborts the case immediately and reports `Step <N> (assert): <reason>` (1-based).

Order of operations:

arrange.inputs is written once at case start (not replayed between steps).
Each entry in steps runs in declaration order.
After the last step, the top-level assertions[] runs as a final gate. Per-step asserts and top-level asserts are independent — both must pass. The schema requires minItems: 1 on the top-level assertions, so author at least one trivial top-level assertion even when relying mainly on per-step asserts.
watch[] continues to sample throughout every wait step (under the conditions above); the time-series chart annotates step boundaries.

Caps (enforced by the runner):

Limit	Value
Steps per case	`1024`
Inputs per `write` step	`256`
Assertions per `assert` step	`256`
`wait.ms`	`0`..`3 600 000` (1 hour)

Visual editor: the case detail view shows a Steps (optional) section with + Write, + Wait, + Assert buttons to append, per-step ↑ / ↓ buttons to reorder, and − to remove. The editor automatically omits cycles from the serialized JSON whenever the Steps section is non-empty (the schema rejects the cycles + steps combination, so the editor avoids ever producing it).

Backwards-compatibility: a case without steps runs the existing single-phase cycles-based path with no behavioural change. Older suites need no migration.

Complete example — timed motor start

StartCmd is set after a 500 ms reset window, then we wait 2 s for the FB to reach Run, plus a final speed check.

{
  "name": "MotorStart_ReachesRun",
  "description": "Reset, wait, start, wait, check Run.",
  "arrange": { "inputs": { "Enable": true } },
  "act": {
    "timeoutMs": 10000,
    "steps": [
      { "type": "write", "inputs": { "Reset": true } },
      { "type": "wait",  "ms": 500 },
      { "type": "write", "inputs": { "Reset": false, "StartCmd": true } },
      { "type": "wait",  "ms": 2000 },
      { "type": "assert", "assertions": [
          { "variable": "Run", "operator": "isTrue", "expected": true }
        ]
      }
    ]
  },
  "assertions": [
    { "variable": "Speed_Act", "operator": "greaterThan", "expected": 1000 }
  ],
  "watch": ["Run", "Speed_Act"]
}

If the per-step assert fails (for example Run did not go true within 2 s), the runner emits Step 5 (assert): variable Run is not true and aborts the case. If the per-step asserts all pass but the top-level Speed_Act > 1000 fails, the case fails with the top-level assertion message.

When in doubt, write a plain Arrange + cycles ≥ N + Assert case first. Convert it into a steps sequence only when the plain form cannot express the timing the test actually depends on.

Opening an Existing Suite

From the Test Suites view: Click the suite in the tree (it opens in the Test Suite Editor); clicking one of its test cases opens the suite with that case selected
From Quick Open: Press Ctrl+P and type the suite file name under .tia-tests/

Where are test suites stored?

Test suites live in a hidden folder called .tia-tests directly next to your TIA project file (*.ap21, *.ap20, …). The AI assistant writes to the same folder. If you haven't chosen a test folder yet, the Test Suites view shows "Choose a test folder to load your test suites." — pick a folder first.

You pick the folder yourself: in the Test Suites view title bar, click Choose Test Folder and pick the folder your tests live in. When a folder is chosen, the Test Suites list follows that folder — switching projects doesn't change it; with no folder chosen, it follows the connected TIA project. The choice is remembered, so your tests show up again the next time you start the app, even without a TIA connection. It also helps you recover tests from an earlier setup: just pick the folder they're in. Clear Test Folder (shown only while a folder is chosen) clears the selection again; if the chosen folder no longer exists, the view shows "Selected test folder not found".

The AI can read, list, and edit these test-suite files directly through the chat interface.

Running Tests

Make sure PLCSIM Advanced V3.0+ is installed
Open the Test Bench and choose the PLC, the block and the suite in its header (or select the suite in the Test Suites view)
Run it: click Run in the Test Bench header, click Run Suite in a suite's row in the Test Suites view, Run All Test Suites from the view title bar, or a right-click context-menu action
Watch the live progress in the status bar and the Run tab of the Test Bench
When the run is complete, status icons in the Test Suites view update and results are persisted to SQLite

The runner automatically:

Creates a PLCSIM Advanced instance (name configurable in the suite JSON)
Powers it on and compiles the current PLC project
Connects an S7 client to the instance
Writes the test inputs, waits the configured cycle count, reads the outputs, evaluates the assertions
Cleans up the S7 connection and unregisters the instance

Run options

The arrow next to Run, in the Test Bench header and in the Test Suite Editor header, opens the run options. They come in four parts:

Scope: what to run. In the Test Bench: All Suites (every suite in the test folder of the project), Suite (the suite chosen in the header) or Failed Cases (the cases that failed in the last run). In the Test Suite Editor: Suite, Failed Cases or Selected Cases (the rows you selected in the case list; chosen for you when rows are selected). Each entry shows how many suites or cases it runs.
Mode: Standard, Coverage, Mutation Testing or Interface Contracts. Coverage and mutation testing need a PLCSIM Advanced target and a connection to TIA Portal; the contract check needs a connection to TIA Portal. These three modes run one suite.
Options (Standard runs only):
- Flaky confirm re-run: a case that fails but has passed and failed in recent runs is run once more; if it passes then, it counts as passed.
- Quarantine flaky cases: a case that fails but has been flaky in recent runs is reported as skipped (quarantined) instead of failed.
- Update snapshot baselines: replaces the stored snapshot baselines with the values of this run. Studio asks you to confirm first. Not available for suites that connect over S7 Native.
- Keep PLCSim instance after run: leaves the PLCSIM Advanced instance running after the run. Only for suites that download the project over TCP.
Target: where the suite runs, for example "PLCSim Advanced, PLC_SIM1 (192.168.0.5)". Change opens the connection settings of the suite.

An entry that is not available is greyed out; hover over it to see why. Start with the button at the bottom (for example Run Suite or Run 2 Cases) or press Ctrl+Enter; Escape closes without running. Tab moves between the parts, the arrow keys between the entries of a part.

What Run remembers. A plain click on Run runs the suite with your last choice of Flaky confirm re-run and Quarantine flaky cases; the same choice applies to runs from the Test Suites view, the Command Palette and the Run tab. Update snapshot baselines and Keep PLCSim instance after run apply only to the run you start from the options. When you last ran with Coverage or Mutation Testing from the options, that Run button reads Run (Coverage) or Run (Mutation Testing) and repeats the mode; choose Standard once to go back to normal runs. Hover over Run to see the current choice.

Choosing a Transport

The transport decides how test reads and writes travel between AnyAutomation Studio and the PLC. You set it in the suite JSON via config.transport (see the schema above):

PLCSim Advanced (plcSimApi): Direct access via the Siemens PLCSim API, faster and does not need an active S7 session. This is the default.
OPC UA: connects to the OPC UA server of a PLC, real or simulated. See Connecting over OPC UA.
S7 Native: Connects directly to a real S7 controller, the same kind of connection used by the PLC Online view. Pick this when you want to run a suite against live S7-1200 / S7-1500 hardware. See Connecting over S7 Native for the fields you fill in and the confirmation prompt before a live run. Included in Unit Testing (Pro+ plan), also during the free trial.

Each suite is targeted independently. For S7 Native, the suite owns its IP address, port, user name, and password — there is no shared lookup from the PLC Online view.

You choose the transport and everything else about the connection on the suite's Connection Settings page, described in the next section. You can still edit the values directly in the suite JSON via Open as JSON Text in the Test Suite Editor.

Connection Settings

Every suite has its own connection settings page. Open it in any of these ways:

click the connection type in the header of the Test Suite Editor, or choose Connection Settings in its … menu;
choose Connection Settings in the … menu of the Test Bench, for the suite chosen in its header;
right-click a suite in the Test Suites view and choose Connection Settings;
click Change next to the target in the run options;
run Unit Testing: Connection Settings from the Command Palette and pick the suite.

The page opens as a tab named Connection: followed by the suite name. It has four parts:

Transport: PLCSim Advanced, OPC UA or S7 Native. For PLCSim Advanced also the Preparation Mode: User Preloaded or TCP Download (see PLCSIM Preparation Mode); External (Real PLC) is not available yet.
The target of the chosen transport:
- PLCSim Advanced: Instance (suggests the instances PLCSIM Advanced knows), IP Address, Network Mode, Cycle Wait (ms), Auto-Connect and Keep Instance After Run. Choose a Network Mode (TCPIPSingleAdapter, TCPIPMultipleAdapter or Softbus) if PLCSim has trouble starting on a machine with more than one network adapter; (leave unchanged) keeps your current PLCSim setting.
- OPC UA: see Connecting over OPC UA.
- S7 Native: see Connecting over S7 Native.
Master Secret (TCP Download): only for TCP Download, see Running Against a Protected Project.
Isolation: Mode is Time-Based (the default) or Isolated (every test case starts from a known state). The Reset Mode (None, Defaults or Memory Reset) applies to isolated runs; Memory Reset needs PLCSim Advanced. When the suite contains something an isolated run does not support yet (stubs, multi-step cases, timeline watches, snapshot assertions), the page lists it under Isolation.

While the page is open, switching the transport keeps what you typed for the other transports, so you can switch back without typing it again. Saving stores only the settings of the chosen transport in the suite.

Passwords. The page never shows a password, only whether one is stored: Set (remembered), Set (this session only), Not available, enter it again or (not set). Click Change to enter a new password, then choose Remember (OS secret store) to keep it on this computer for later runs, or This session only to keep it until Studio closes. Clear removes the password when you save; Undo takes back Change or Clear. A password you do not change stays as it is when you save. If you change the server address, port or user of a connection, Studio asks for the password again. When the chosen transport no longer uses a stored password, the page shows Removed on save because the transport does not use it, and saving removes it from this computer. Passwords are never written into the suite file, so you can share or commit a suite without exposing them.

Test Connection checks the settings on the page before you save them. The result appears next to the button:

PLCSim Advanced needs a connection to TIA Portal. The check tells you whether the PLCSIM Advanced Runtime Manager is running and whether the instance exists. The run powers the instance on. means the instance is there but switched off; the run starts it. With TCP Download an instance that does not exist yet is fine (The run creates the instance.); with User Preloaded it is reported as a problem, because the run would start an empty instance.
OPC UA connects to the server once and reports Reachable (the server certificate is not checked). together with the namespace index it detected. An open connection in the Address Space tab stays as it is. Like a test run, the check accepts the server certificate without asking.
S7 Native has no check; each run asks you to confirm before it connects to the controller.

Saving. Save (or Ctrl+S) writes the settings into the suite. Save as Default also keeps them, without passwords, as the starting point for new suites: every suite you create afterwards, from the Test Bench, the Command Palette or the assistant, starts with this connection. Cancel discards your changes and closes the page. A field with a problem is marked with the reason, and Save stays disabled until it is fixed.

If the suite is also open in the Test Suite Editor with unsaved changes, the editor keeps them and saves them together with the new connection the next time you save it. If the suite file changes on disk while the page has unsaved changes, a bar offers Reload (take the settings from the file) or Keep My Changes.

Connecting over OPC UA

A test suite can run against a PLC over OPC UA, alongside PLCSim Advanced. Open the suite's Connection Settings (see Connection Settings) and choose OPC UA under Transport.

For an OPC UA connection you provide:

Endpoint URL — the address of the PLC's OPC UA server, for example opc.tcp://192.168.0.5:4840.
Security Mode — None, Sign, or Sign & Encrypt. Pick the mode your server is set up to accept; Sign & Encrypt gives the highest protection.
Authentication — either Anonymous, or User Name & Password. When you choose User Name & Password, enter the credentials your PLC expects. The password is stored securely on your machine and is never written into the test file, so you can share or commit a suite without exposing it.
Namespace Index — leave this empty and Studio detects the right namespace on the controller automatically; this is the normal case. Set a number only for an unusual server where automatic detection does not find it.

The first time you connect to a server whose certificate AnyAutomation Studio does not yet trust, a prompt asks you to confirm the connection. Choose Connect Anyway to trust this server and continue; the prompt does not appear again for that server on the next run.

Make the PLC's variables reachable over OPC UA. For tests to read and write your variables, the data blocks they touch must be accessible through the PLC's OPC UA server. Enable OPC UA access for the relevant data blocks in your TIA Portal project and download it to the PLC before running the suite. Variables that are not exposed cannot be read or written and the affected test cases will fail.

Browsing the Address Space

In the Address Space tab of the Test Bench's Block column you can explore what an OPC UA server offers before you write or run tests. The first line of the tab holds the server address. When the suite chosen in the header runs over OPC UA, its address is already filled in; otherwise the last address you used is.

Any server: type an address such as opc.tcp://192.168.0.5:4840. The second line lets you choose Message Security Mode (None, Sign, Sign and Encrypt) and Authentication (Anonymous, or Username and Password with User Name and Password). A password typed here is used for this connection only and is not saved.
The server of your OPC UA suite: when the address is the one of the suite in the header, the second line shows Suite settings instead; hover it to see the security and sign-in the suite uses. Studio connects with them and with the password saved for that suite, without asking. If no password is saved for it, or the server rejects the user name or password, Studio asks for the password right in this line; it is used for this session only and never saved.

Click Connect or press Enter in any field; Disconnect ends the connection. The address space is shown as a tree you can expand to walk through the available nodes and see which variables are reachable. Use it to confirm a variable's name and that it is exposed over OPC UA, so your test cases target the right items. Structured variables (PLC data types, structs and arrays) show the name of their data type, for example typMotor or Array of Bool, and open to their members, so you can walk down to the element a test case reads or writes. In a test case, address such an element by its path, for example DB_Motor.Setpoints[3] or DB_Motor.StartTimer.Q.

Authoring and Running over OPC UA

Once the connection is set, OPC UA suites behave exactly like PLCSim suites. You author test cases the same way, run them with the same Run Test Suite / Run All Test Suites actions, watch the same live progress, and get the same results and run history. The only difference is where the tests connect — and you do not need TIA Portal open to author or run them.

For pipelines and CI, a suite configured for OPC UA also runs in the generated pipeline. Because the OPC UA password is not stored in the test file, you provide the OPC UA password to the pipeline run so the build server can connect without anyone present.

Connecting over S7 Native

A test suite can also run directly against a real S7 controller over S7 Native. Open the suite's Connection Settings (see Connection Settings) and choose S7 Native under Transport. This connection type is included in Unit Testing (Pro+ plan), also during the free trial.

For an S7 Native connection you provide:

IP address — the controller's address, for example 192.168.0.10.
Port — the controller's TCP port. The default is 102, which fits most controllers. Change it only when your controller has been set to a different port.
User name (optional) — needed only when your controller requires a user sign-in. Most projects leave this empty.
Password — the password your controller expects. It is stored securely on your machine and is never written into the test file, so you can share or commit a suite without exposing it.

Use Save as Default on the page to make these values, without the password, the starting point for new suites, so you don't have to enter them again each time.

Authoring and Running over S7 Native

Once the connection is set, you author test cases the same way you would for a PLCSim suite and run them with the same Run Test Suite / Run All Test Suites actions. The only difference is where the tests connect: an S7 Native suite connects directly to the controller you specified, writes the test inputs to the live controller, and checks the results there.

Because a run writes to live hardware, a prompt asks you to confirm before it starts. Confirm it only when the plant is in a safe test state, and the run begins. Cancel the prompt and nothing happens.

You can run against a safety (F) controller as well. The confirmation prompt warns you that the target may be a safety controller and that you must make sure the controller is safe to manipulate and the plant is in a safe test state before you continue. There is no automatic block: you decide whether to confirm or cancel.

Simulation Workspace

Planned for the in-app workspace. A dedicated Simulation view for managing PLCSim Advanced instances from inside AnyAutomation Studio is planned. The capabilities it would expose are described below.

A Simulation view for managing PLCSim Advanced instances would show:

API version, online-access mode (PLCSim / TCP/IP single / TCP/IP multi), strict motion timing toggle, and Runtime Manager Port.
A Virtual Adapter row showing the permanent status of the Siemens PLCSIM virtual Ethernet adapter (Ready / APIPA / Disabled / Not Installed / No IPv4), its IPv4 address, and a refresh button. If the adapter is stuck in the 169.254.x.y APIPA range, downloads will be unreliable — assign a static IPv4 address in Windows network settings.
Every registered PLCSim instance with inline action buttons: Power On, Run, Stop, Memory Reset, Power Off, Settings, Network, Delete. Each row shows the configured IP address; if the instance has more than one interface with an IP, they are listed as X1: 192.168.0.1, X2: 10.0.0.5.
A TIA PLC combobox per instance card: pick the TIA Portal PLC that backs the instance. Use this when the PLCSim instance was loaded with a snapshot compiled from a different TIA PLC than the one it is registered under (for example: an instance named PLC_2 actually holding the program from PLC_1). Without an explicit choice, the app matches by name and falls back to the only PLC in the project when there is one — which is enough for most setups but produces an empty tag tree when the names disagree. The choice is remembered per instance.
A New Instance button that prompts for a name and CPU type.
A Tag Browser pane that connects to any listed instance and shows all tags the running program exposes, filterable by name search, area (Input / Output / Marker / Data Block), and data type. Auto-refresh can be toggled for live value observation. Each writable tag has a pencil button that opens a type-aware write dialog (toggle for Bool, numeric input with type-specific range for integers and floats, one-character field for Char/WChar). String tags are read-only. Struct tags and array tags are listed with their inner members and elements as expandable rows so you can search, watch and edit individual fields like OUT[5] or DI10.Channel directly; Expand all and Collapse all toolbar buttons toggle the whole tree at once. Structured input/output tags whose layout comes from a project type definition (for example a PNPN block at %I0.0 declared as Array[0..63] of Byte) drill down into named member rows when the app is connected to the TIA Portal project that defines them — letting you write IN.PNPN[12] from the same dialog that handles the rest of the tree.
A Saved Instances section at the bottom listing every PLCSim Advanced instance persisted on the virtual SIMATIC memory card. The heading shows a counter with the total number of saved instances; the list shows up to three rows at a time and scrolls if there are more so the simulation area above stays visible. Click Load to re-register and resume one — PLCSim picks up the stored CPU type, I/O image and program automatically. Click Delete (with confirmation) to remove the persisted folder.

The view choice would be remembered between sessions, and switching keeps any open suite editors intact.

PLCSIM Preparation Mode

The preparation mode controls how the runner brings your project online before the test run. You choose it on the suite's Connection Settings page, or in the suite JSON via config.preparationMode:

I load it myself (userPreloaded, recommended) — The runner expects a PLCSIM instance that you have already started and loaded manually via TIA Portal. It skips compile and download and connects directly to run the tests. Fastest option if you're iterating on the same project.
Automatic TCP download (tiaTcpDownload) — The runner compiles the project, starts a fresh PLCSIM instance and downloads via TCP over the PLCSIM Virtual Adapter. No manual TIA Portal interaction needed — just run.
Real PLC (no PLCSim) (external) — Skips the PLCSim lifecycle entirely and would prepare a run against live S7 hardware without any PLCSim instance. This preparation mode is not yet available for in-app runs. It is a separate, planned option and is not the same thing as the S7 Native connection: to run against a real controller today, choose the S7 Native connection type instead (see Connecting over S7 Native and Running Tests Against a Real PLC).

Before using Automatic TCP download: Four prerequisites must be met or the run will fail with a generic transport error: (1) the PLCSIM Virtual Adapter must be installed and have a static IPv4 address that matches the suite's plcSimIp; (2) the TIA Portal project must be open in TIA Portal — the runner triggers a compile against the open project before the download; (3) if the project enables Protect confidential PLC configuration data, you must supply the master-secret password (it lives in the Vault); (4) any existing PLCSIM instance with the same name is destroyed and recreated for every run, so other applications attached to that instance lose their session.

Running Tests Against a Real PLC

You can run tests directly against live S7-1200 / S7-1500 hardware using an S7 Native connection, without any PLCSim involvement:

Make sure the plant is in a safe test state — actuators isolated or interlocked, no production load on the PLC, operators informed.
On the suite's Connection Settings page, choose S7 Native under Transport and fill in the controller's IP address, port, and (only if the controller requires it) user name and password. See Connecting over S7 Native for the details.
Run the suite from the Test Suites view.
Before the tests reach the controller, a prompt asks you to confirm the run against live hardware. Confirm it once the plant is in a safe test state, and the run begins.

A few things to know about live-controller runs:

Wrong credentials: if the controller accepts the connection but rejects your sign-in, the run stops right away with a message telling you to check the user name and password, rather than failing later with a confusing "variable not found".
Batch runs: a suite set to an S7 Native connection is skipped during Run All Test Suites (the per-suite confirmation cannot be shown from a batch run) and is listed by name in a dismissible banner, so you run each one individually.
Controller must be running: a live-controller run checks the controller's operating state first and stops if it is not in RUN, so nothing is written while the program is not executing.

Safety (F) controllers. When you run interactively, you can target a safety (F) controller: the confirmation prompt warns you that the target may be a safety controller and that you must make sure the controller is safe to manipulate and the plant is in a safe test state, and you decide whether to confirm or cancel. AI-assisted runs are different: the assistant will not run tests against Safety logic on its own, so you always start a safety-controller run yourself.

Running Against a Protected Project

If your TIA project has "Protect confidential PLC configuration data" enabled, the automatic TCP download mode requires the project's master-secret password:

On the suite's Connection Settings page, choose PLCSim Advanced and the preparation mode TCP Download
Under Master Secret (TCP Download), click Change and enter the project's master-secret password
Choose Remember (OS secret store) so every later run picks it up without asking, or This session only to keep it until Studio closes, then click Save
Run the suite

The page then shows Set (remembered) or Set (this session only). Leave it as it is to keep the stored password; Clear removes it when you save. If no password is available when a run starts, Studio asks for it and keeps it for the current session.

The password is never written to the suite file, to any log, or to any settings file. It lives only in Windows Credential Manager under the service name com.anyautomationstudio and is transferred from the UI to the runner through a user-bound DPAPI channel so a different Windows account cannot read it.

Wiring inputs to the calling program

Many blocks are not called with fixed values but with variables of a data block, for example "idb_FB_MotorControl"(bStart := "dbMotorControl".Control.Start, ...). The calling program passes those variables to the block in every cycle. A value that a test writes straight into the block's own data would be replaced by the next cycle, and the test would report an error or a wrong result. The wiring tells the test where the calling program takes each input from, so the test writes its values there and the block receives them the same way it does in production.

Setting the wiring

Open the suite in the Test Suite Editor and select a test case. In the lower part, the INPUTS table of the case has a Written to column next to the value of every input. The header of the table names the instance DB of the block and how many inputs are wired.
Check the instance DB with Choose Instance DB... in the header of the table. It names the data block the calling program uses for this block instance; you can also type the name.
For every input the program passes, click Choose from DB... in its row, pick the data block and then the variable inside it. Variables of the same type as the input are listed first. You can also type the variable into the Written to field, for example "dbControl".Setpoints.rSpeedSetpoint, and an element of an array such as "UI_MSG".Messages[3].diMsgID works the same way. Clear wiring removes it again; an input without wiring is written into the instance DB as before.
Save the suite. The wiring belongs to the suite: the same input is written to the same variable in every test case, so you set it once and see it in every case.

What you should know

Inputs the program feeds with a fixed value, a calculation or a temporary variable cannot be driven by a test. Leave them unwired and out of your test cases; the value the program sets stays in effect.
Once the block is wired, its declared start values no longer reach it. Write every input a test case depends on in that case.
Wired values stay in the data block from one test case to the next. Inputs that react to a rising edge, such as a start command, need an explicit sequence in multi-phase steps: write FALSE, wait, write TRUE.
Before the run starts, Studio reads the current values of all wired variables and puts them back when the run has finished, so the calling program continues with the values it had before the test.
If a wired variable cannot be found on the controller, the run stops before the first test case and names the variable. Check the data block name, the variable path and, over OPC UA, that the data block is accessible for OPC UA.
Only outputs are still read from the instance DB. Wiring an output has no effect on the run.
The assistant can set the wiring for you: it looks up in the project how the block is called, wires every input accordingly and tells you which inputs the program fixes (see AI-Authored Test Suites).

Click Validate in the Test Suite Editor to check every variable name and every wiring against the block, including upper and lower case. Problems are listed above the case list, and Save and Run stay unavailable until they are fixed. A suite with an invalid wiring does not start.

Managing Suites and Test Cases

Rename a suite by editing the JSON name field at the top of the suite and pressing Save — the file is renamed on disk to match (see Renaming a suite from the JSON name field). To delete a suite, remove its .json file from the .tia-tests/ folder.
Edit a test case in the Visual mode of the Test Suite Editor: each case row lets you rename it, edit its description, Duplicate it, remove it, and reorder its multi-phase steps with Move Up / Move Down.
Edit the generated SCL test code in the SCL view of the Test Suite Editor: rename test cases, change the input value assignments, edit, add or remove assertion checks, and add or delete whole test-case blocks. Changes flow back into the test cases automatically — the Visual and JSON views show them immediately, and case details the SCL text does not show (description, cycles, multi-phase steps, tolerance, watch list, tags, priority, owner, requirements) are kept. The runner block at the end is maintained automatically — edits to it have no effect. If the text no longer matches the test-block structure, an error message with the line number appears, and Save and Import to TIA are blocked until it is fixed. Switching to another editor tab discards SCL text that has not been applied yet; changes already applied are kept.
Connection settings are per suite. The transport, PLCSim instance name, IP address, cycle wait time, S7 Comm+ target (IP, port, user, password), and auto-connect behaviour all live in the suite file's config block. They are saved with the suite, so the next time you open it the same values are in effect. Passwords continue to live in the Windows Credential Manager; the suite file only stores a reference, never a plaintext password. (The suite's Connection Settings page edits these without touching JSON, see Connection Settings.)
Click Validate in the Test Suite Editor to check every variable name and every wiring against the block, including upper and lower case. Problems are listed above the case list, and Save and Run stay unavailable until they are fixed.
When a test case fails, its message appears in the Message column of the Run tab, and the Assertions section marks the failing assertion next to its Expected and Actual values. Double-click the case to open it in the Test Suite Editor.
If a suite file is modified externally — or renamed via the JSON name field above — the open editor automatically reloads, unless you have unsaved changes, in which case your edits are preserved.

Troubleshooting

Status icons don't update after a run: Make sure the suite file path is correct (absolute path under .tia-tests/). New unsaved suites get a unique path per click.
Run action unavailable: A TIA project must be open, and the active suite document must be valid JSON.
Test Explorer is empty: The working directory has no .tia-tests/ folder yet. Click New Test Suite to create the first one — the folder is created automatically.
PLCSIM errors: Check that PLCSIM Advanced V3.0+ is installed and licensed. The exact version is auto-detected at runtime.
Capture the TLS traffic for support (advanced): If support asks for a Wireshark capture of the S7 Comm+ handshake, set the SSLKEYLOGFILE environment variable to an absolute file path before starting the app — for example setx SSLKEYLOGFILE "C:\Temp\s7-keys.log" — then restart. Point Wireshark's TLS preference (Pre)-Master-Secret log filename at the same file. A warning is written to the app log on every connection while this is enabled so you don't forget to unset it. Leave the variable empty in normal operation.

AI-Authored Test Suites (Pro+)

AI Chat can draft, refine and (optionally) run full SCL unit-test suites for you. Requires a Pro+ license or higher.

Write Unit Tests skill (main chat)

Open AI Chat and pick the Write Unit Tests skill from the skill picker (🧪 icon), or simply ask: "write unit tests for FB_MotorControl"
The assistant reads the block interface, computes boundary values, plans test cases and summarises the plan in the chat — no giant JSON blobs
When it's ready to write the suite, a confirmation card appears in the chat. Click Allow to let the assistant save the suite to .tia-tests/, or choose Allow in this Session from the card to stop being asked for the rest of the chat session
If you asked the assistant to also run the suite, a second card asks before the run itself. Results come back as a pass/fail summary in the chat, and the assistant offers to refine any failing cases
The Test Suites view picks up the new suite automatically — open it, review it in the Visual mode or as JSON Text, and run it yourself whenever you want
You can also ask the assistant to "open that suite" after it's been created — the suite opens in the Test Suite Editor without you having to switch to the Test Suites view

Letting the assistant edit an open suite

When a suite is open in the Test Suite Editor, you can ask the assistant in AI Chat to change it, for example "add an overflow test for Counter_Value" or "tighten the tolerance on all REAL assertions to 0.01".

The assistant reads the open suite and stages its change. The editor shows a bar: "AI edits are staged for review. Keep them to apply, or Undo to discard."
Click JSON in the editor to compare the suite before and after the change; the comparison opens at the first change.
If the assistant makes several changes in a row, they add up and all show in the same comparison.
Click Keep to apply all staged changes (save the suite as usual afterwards), or Undo to discard all of them.

The assistant checks variable names and types against the real block, so mismatches are caught before you keep a change. Avoid typing in the suite while a change is staged: Keep applies the assistant's version of the suite.

Choosing which assistant tools may act. In AI Chat, Open Customizations, Tool Approvals lists the unit test tools as the group unit-tests. For each tool that writes or runs (for example creating or deleting a suite, editing an open suite or running a suite) you can choose to be asked every time, allow it always or deny it always; a denied tool stays blocked even when the chat approves tools automatically. Tools that only read never ask.

Reviewing AI-authored test cases

The app stamps AI provenance on test cases it authors (stored internally), but it does not render a visible AI badge in the Test Explorer tree or the Visual editor today — only a Theory (N rows) badge appears for parameterized cases.

Review AI-authored test cases just like you would a colleague's pull request — read each assertion, sanity-check expected values, then run on the simulator before pointing it at real hardware.

Safety (F-CPU) protection

AI test authoring is gated for Safety blocks:

Creating tests — The assistant refuses to author tests for an F-CPU block without an explicit, in-conversation confirmation from you. The underlying tools also reject the write if the confirmation is missing
Running tests — The assistant cannot run tests against Safety blocks. Full stop, no override. Safety-block verification requires a certified methodology (TÜV/CE), not an AI-generated run. If you want to run a test against a Safety block, trigger the run yourself from the Test Explorer

Licensing

The Write Unit Tests skill and the unit test tools in the AI chat require Pro+ or higher. On a lower plan, the skill surfaces a license message before any AI call is made.

CI/CD Integration

The tia-test-runner command-line tool runs your SCL unit-test suites on a build server — no UI, no manual clicks. Point it at a TIA Portal project, and it discovers the suites in .tia-tests/, runs them, writes machine-readable reports, and returns an exit code your pipeline can act on. It opens TIA Portal headless (no window) so it runs cleanly on a build agent; pass --show-ui to watch it on screen while debugging. This is a Pro+ feature (Pro+ and above); the agent needs a CI licence file that you save from Studio (see Licensing for CI below).

The `tia-test-runner` command-line tool

The tool ships as tia-test-runner.exe and exposes these verbs:

Verb	Purpose
`run`	Execute a unit-test run against a TIA Portal project and write reports
`compare`	Compare two previous runs and emit a diff report
`trend`	Export historical pass/fail trends to CSV
`flaky`	List a suite's flaky cases (unstable across recent runs) as a table or CSV
`validate`	Check test-suite definitions for schema errors without running them
`verify`	Verify the integrity manifest of a previous run (detect tampered or missing report files); it also prints the sign-off of the run and reports when the sign-off details were changed
`traceability`	Check requirement traceability and fail the build on failed or untested requirements (see Requirement traceability gate below)
`version`	Print the runner, bridge, engine, build, .NET and operating-system versions

Options for every verb:

Option	Meaning
`--log-level <LEVEL>`	How much the runner logs: Verbose, Debug, Information, Warning (the default), Error or Fatal. Log lines go to the error output, so the normal output holds only the results.
`--log-file <PATH>`	Write the log file to this path instead of the daily log file under `%LocalAppData%\AnyAutomation\Logs`.
`--lang <LANGUAGE>`	Language of the help headings: `en` (the default), `de` or `fr`. The results are always in English.
`--ansi`, `--no-ansi`	Force or turn off coloured output.

A typical run:

tia-test-runner run ^
  --project "C:\Projects\Plant.ap20" ^
  --plc PLC_1 ^
  --suite-filter * ^
  --out-dir reports

Key flags for run:

Flag	Meaning
`--project <PATH>`	TIA Portal project file (`.ap20`, `.ap21`, …). Required.
`--out-dir <PATH>`	Directory for reports and logs. Required.
`--plc <NAME>`	Restrict the run to a single PLC station.
`--suite-filter <PATTERN>`	Glob filter on suite names (`*`, `?`, literal).
`--tag <TAG>`	Run only cases carrying this tag (repeatable).
`--priority-min <LEVEL>`	Minimum priority (Low / Normal / High / Critical).
`--report-format <FORMAT>`	`junit`, `html`, or `cobertura` (repeatable). Defaults to JUnit + HTML.
`--rerun-affected`	Run only suites whose underlying block changed since the last run (needs `--plc`).
`--fail-fast`	Stop at the first failing case.
`--timeout-minutes <N>`	Per-run timeout. Defaults to 60.
`--attach`	Attach to a TIA Portal instance that is already running instead of opening the project file.
`--show-ui`	Open TIA Portal with its window visible instead of headless. Handy for watching a run while debugging locally.
`--quarantine`	Demote a failing case to a non-blocking skip when it has been flaky in recent runs, so a flaky failure does not break the build.
`--quarantine-window <N>`	How many recent runs to inspect for flakiness (default 10).
`--quarantine-min-flips <N>`	Minimum pass/fail changes within that window before a case counts as flaky (default 1).
`--retry-max <N>`	How many times to retry a suite after an infrastructure error such as a lost connection (`0` disables; default 3).
`--retry-delay-ms <MS>`	Delay between those retries.
`--config <PATH>`	Read defaults from a YAML configuration file (see below).

The run prints a summary line with passed / failed / errored / skipped / quarantined counts; when a run quarantines flaky cases, each demoted case is listed just before the summary so it is never hidden.

Exit codes (the pipeline contract):

Code	Meaning
`0`	All tests passed
`1`	One or more tests failed
`2`	Run error (could not start, lost the PLC connection, license not entitled, …)
`3`	Argument or configuration error
`4`	Timeout
`5`	Cancelled by the user (Ctrl+C)

After a run, --out-dir holds a JUnit XML file your CI test reporter can publish, a self-contained branded HTML report, and an integrity manifest. Reports are append-only: the newest run is always at the top of --out-dir, so the JUnit file, the HTML report and the manifest there belong to it (with several suites in one call, the last suite). Each earlier run moves into its own subfolder under runs with its own manifest, and as soon as the directory holds more than one run, an aggregate summary counts them all.

Requirement traceability gate

tia-test-runner traceability checks requirement traceability on the build agent and fails the pipeline when requirements failed or have no test. It reads the test results the run verb recorded on the same agent and the requirement list of the test folder, if there is one.

tia-test-runner traceability ^
  --project "C:\Projects\Plant.ap20" ^
  --runs-from reports ^
  --out-dir reports\traceability ^
  --fail-on failed,uncovered

Flag	Meaning
`--project <PATH>`	TIA Portal project file or its folder. Required.
`--runs-from <DIR>`	Use only the runs whose reports lie in this folder, normally the `--out-dir` of the `run` step before it. Recommended: without it, a result recorded by another pipeline or branch on the same agent could count.
`--fail-on <LIST>`	Which requirement states fail the build: any of `failed`, `uncovered`, `notRun`, `outdated`, `incomplete`, or `none`. Default `failed,uncovered`. `outdated` also covers requirements whose latest run was recorded before changes could be detected.
`--requirements <PATH>`	A requirement list to use instead of the one in the test folder, for example a list exported from your ALM tool during the build.
`--suite-filter <PATTERN>`	Check only the suites whose file name matches the pattern, the same suites `run --suite-filter` runs (for example `Conveyor*`). Requirements that only the other suites name are left out; requirements of your list that no suite names still count as untested.
`--run-window <N>`	How many of the latest runs of each suite to look at (1 to 100, default 20).
`--format <text\|json>`	Output as a table (default) or as JSON. With `json`, the standard output holds only the JSON document, so a script can read it directly.
`--json-file <PATH>`	Also write the JSON document to a file.
`--out-dir <DIR>`	Write the HTML traceability report and its integrity manifest into this folder. Use a folder of its own, such as `reports\traceability`, not the folder of a run report.
`--brand-title`, `--brand-footer`, `--brand-color`, `--brand-logo`	Branding of the HTML report, as for `run`.

The table lists each requirement with its status, number of test cases, latest evidence and title, followed by a summary such as "12 requirements: 1 failed, 2 uncovered, 0 not verified". Besides the exit codes of run, the verb returns:

Code	Meaning
`1`	A requirement failed (when `failed` is in `--fail-on`)
`3`	Input problem: for example an error in the requirement list, a suite file that cannot be read, a `--suite-filter` that matches no suite, or no run reports in the `--runs-from` folder
`9`	A requirement has no test case (when `uncovered` is in `--fail-on`)
`10`	A requirement is not verified: not run, outdated or incomplete (when chosen in `--fail-on`)

Warnings about the requirement list are printed but never fail the build. Like run, the verb needs the CI licence file.

Flaky tests and quarantine

A flaky test is one that passes in some runs and fails in others without the block under test actually changing — usually a timing or environment effect. To see which cases are flaky for a suite over its recent history:

tia-test-runner flaky --suite-file "C:\Projects\Plant\.tia-tests\FB_Motor.json"

It prints a table of each unstable case with how many times it flipped between pass and fail and its recent result sequence. Add --window <N> to change how many recent runs are inspected (default 10), and --out-file <CSV> to also write the list as CSV.

Add --quarantine to a run to keep flaky failures from breaking the build: a case that fails the current run but has been flaky in recent history is demoted to a non-blocking skip and reported as quarantined (flaky) in the JUnit output and the run summary, while genuine, stable failures still fail the build. Tune it with --quarantine-window and --quarantine-min-flips. In Studio, quarantine is the option Quarantine flaky cases of Run (see Run options). Quarantine never hides a demoted case: each one is listed before the run summary.

Report classnames

By default, JUnit classname values follow the pattern {PlcName}.{BlockName}, which most CI test viewers group into a readable tree. Override it with --report-classname-pattern using any combination of these placeholders:

{PlcName} — the PLC/CPU name
{BlockName} — the block under test
{SuiteName} — the test-suite name
{ProjectName} — the project name

Example: --report-classname-pattern "{ProjectName}/{PlcName}/{SuiteName}".

Capturing CI environment details

Pass --ci-env-capture to record a small, fixed set of CI variables into the run's provenance, so the reports show which pipeline, commit, and agent produced each run. Capture is opt-in — nothing is recorded unless you ask for it. The tool recognises GitHub Actions, Jenkins, Azure DevOps, and GitLab CI (and a generic CI marker); only a whitelisted set of identifiers (run/build IDs, commit SHA, branch/ref, workflow/job name, agent/runner name) is read, each clipped to a safe length.

YAML configuration file

To keep long command lines out of your pipeline, put the defaults in a YAML file and pass --config <PATH>. Command-line flags override the file. Example tia-tests.yml:

project: C:\Projects\Plant.ap20
plc: PLC_1
out_dir: reports
report_format:
  - junit
  - html
timeout_minutes: 45
ci_env_capture: true

Jenkins (declarative pipeline)

pipeline {
  agent { label 'tia-windows' }
  stages {
    stage('Checkout') {
      steps { checkout scm }
    }
    stage('Unit Tests') {
      steps {
        bat 'tia-test-runner run --project "%WORKSPACE%\\Plant.ap20" --plc PLC_1 --out-dir reports --ci-env-capture'
      }
    }
  }
  post {
    always {
      junit 'reports/**/junit.xml'
      archiveArtifacts artifacts: 'reports/**/*.html', allowEmptyArchive: true
    }
  }
}

GitHub Actions

name: TIA Unit Tests
on: [push]
jobs:
  unit-tests:
    runs-on: [self-hosted, windows, tia]
    steps:
      - uses: actions/checkout@v4
      - name: Run SCL unit tests
        run: tia-test-runner run --project "${{ github.workspace }}\Plant.ap20" --plc PLC_1 --out-dir reports --ci-env-capture
      - name: Publish test report
        if: always()
        uses: dorny/test-reporter@v1
        with:
          name: TIA Unit Tests
          path: reports/**/junit.xml
          reporter: java-junit
      - name: Upload HTML report
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: tia-html-report
          path: reports/**/*.html

Azure DevOps

pool:
  name: tia-windows
steps:
  - checkout: self
  - script: tia-test-runner run --project "$(Build.SourcesDirectory)\Plant.ap20" --plc PLC_1 --out-dir $(Build.ArtifactStagingDirectory)\reports --ci-env-capture
    displayName: Run SCL unit tests
  - task: PublishTestResults@2
    condition: always()
    inputs:
      testResultsFormat: JUnit
      testResultsFiles: '$(Build.ArtifactStagingDirectory)/reports/**/junit.xml'
      testRunTitle: TIA Unit Tests
  - publish: $(Build.ArtifactStagingDirectory)\reports
    artifact: tia-html-report
    condition: always()

GitLab CI

unit-tests:
  tags: [tia-windows]
  script:
    - tia-test-runner run --project "$CI_PROJECT_DIR\Plant.ap20" --plc PLC_1 --out-dir reports --ci-env-capture
  artifacts:
    when: always
    paths:
      - reports/
    reports:
      junit: reports/**/junit.xml

Generate a CI/CD pipeline from the app

Instead of writing the configuration files by hand, let AnyAutomation Studio generate them. Studio produces two ready-to-commit files for a TIA project: a runner.yaml that holds the command-line defaults, and a pipeline file for the CI system you choose. You set them up on the CI Pipeline page, which shows the finished files while you fill in the form.

Open the page. Run Generate CI Pipeline from the Command Palette, from the … menu of the Test Suites view, or from the … menu of the Test Bench. The page opens for the TIA project you are connected to. Without a connection, Studio asks you to choose the project file first. Each project has its own page, titled CI Pipeline: <project>; running the command again brings you back to it.

Fill in the form. The left side of the page has these sections:

Section	What you set
Project	The project file (shown, not editable), the PLC name (while you are connected to the project, the field suggests its PLCs and fills in the first one when it is empty), an optional Suite filter, Cycle wait (ms) and Capture CI environment.
Reports	HTML report and JUnit XML (at least one), the Report output folder inside the project folder (empty means `tia-test-results`), the Classname pattern, and under Branding the Title, Footer and Accent color of the HTML report.
CI Target	The CI system: GitHub Actions, GitLab CI, Azure DevOps or Jenkins, and an optional Agent label that picks the build agent (see the note on build agents below).
Triggers	Optional Push branches, Pull request branches (comma-separated) and a Schedule (cron). Leave all three empty to start the pipeline by hand.
Build Matrix	Optional parallel cells, see Build Matrix below.
Coverage	Publish coverage and three optional minimum percentages, which you can fill in while Publish coverage is on. Fail on failed or uncovered requirements adds a step after the test run that checks requirement traceability against this run, keeps the traceability report with the other reports and fails the build; under Fail on choose which requirement states fail it (Failed and Uncovered by default). With a suite filter, the check covers the same suites as the test run.
CI Runner	Whether the command-line runner is installed on this computer, and Save CI Licence File (see Licensing for CI).

A field with a problem is marked and explains what to change, for example "Enter the PLC name." or "A cell with this name already exists."; the line at the bottom of the page repeats the first problem.

Watch the files. The right side of the page shows the generated files. Click a file name above the text, or use the Left and Right arrow keys, to switch between runner.yaml and the pipeline file. The files update about half a second after you stop typing. While a field has a problem, the preview shows "Complete the highlighted fields to preview the files." instead. Drag the divider between form and preview to give either side more room.

Write the files.

Preview as Untitled Files opens both files as unsaved editors, so you can copy them or save them somewhere else.
Write Files saves both files into the project folder. If any of them already exists, one question lists everything that would be replaced; Cancel writes nothing.

Both buttons are available only while the form has no problem. A field may not contain ${{ or $(, because your CI system would evaluate them, and the Schedule (cron) needs five fields (minute, hour, day of month, month, day of week); the message under the field tells you what to change. The runner.yaml is written to the project folder, and the pipeline file to the path its CI system expects: .github/workflows/tia-tests.yml for GitHub Actions, Jenkinsfile for Jenkins, azure-pipelines.yml for Azure DevOps, and .gitlab-ci.yml for GitLab CI. Commit both files.

Your settings stay. You can close the page at any time without losing anything: Studio remembers the form for each project and shows it again the next time you open the page, also after a restart. A page that was open when Studio closed opens again with it. When you are connected to a different project than the one of the page, a line at the top reads "This page belongs to <project file>."; the page keeps writing into its own project folder.

When the pipeline runs. By default the generated pipeline runs on demand: start it manually from your CI system whenever you want to test. Fill in Triggers to also run it automatically on every push or pull request to the listed branches, or on a cron schedule for nightly runs. (For GitLab the schedule itself is set in GitLab's own pipeline-schedules screen; for Jenkins, push and pull-request runs come from your repository webhook.)

The pipeline must run on a self-hosted Windows build agent. The tests drive TIA Portal and PLCSIM, so the machine that executes the pipeline needs TIA Portal installed, PLCSIM available, and an active Pro+ license or higher. Cloud-hosted runners cannot run the tests. The generated pipeline therefore asks for a self-hosted Windows agent: on GitHub Actions a runner with the labels self-hosted and Windows, on Azure DevOps a Windows agent of the pool Default, on GitLab CI a runner with the tag windows, on Jenkins an agent with the label windows. Enter an Agent label on the page to use your own label, pool or tag instead.

The agent needs bash. The test step of the generated GitHub Actions, GitLab CI, Azure DevOps and Jenkins pipelines runs in bash, so a Windows build agent needs Git Bash installed and on the PATH; a GitLab runner can keep its PowerShell shell. A pipeline file you generated with an earlier version keeps working as it is until you generate it again.

Build Matrix

Add one or more cells with Add Cell in the header of the Build Matrix section, and each cell runs your test suites as a separate, parallel job in the generated pipeline. Use it to test the same suites across several projects, or to split a large set of suites into parallel slices, so the whole CI run finishes faster and reports each cell on its own.

Each cell is a row in the table:

Name (required) – a short label using letters, digits, _ or -, unique within the matrix. A new cell starts with a free name such as cell1.
Project file (optional) – leave it empty to use the project of the page, or point the cell at a different project file.
Suite filter (optional) – run only some of the suites in that cell. Leave it empty to run them all.

Remove a cell with the remove icon at the end of its row. Each cell writes its results to its own report folder, so cells never overwrite each other and you can compare them side by side. The Build Matrix is available for all four CI systems. Leaving it empty produces the normal single-run pipeline.

Licensing for CI

Unit testing is a Pro+ feature (Pro+ and above), so the runner checks your licence before it does any work. A build agent is licensed with a CI licence file that you save from Studio and hand to the agent.

Save the file. In Studio run Generate CI Pipeline, click Save CI Licence File in the CI Runner section of the CI Pipeline page, and pick a location. You need to be signed in with an active Pro+ licence.

Hand it to the agent. Either put the file on the agent and point the runner at it:

tia-test-runner run --license-offline "C:\agent\anyautomation-ci.json" --project "C:\Projects\Plant.ap20" --out-dir reports

or store the file's contents as a masked pipeline variable named ANYAUTOMATION_CI_ENTITLEMENT, either exactly as saved or base64-encoded (some CI systems only mask single-line base64 values). The runner reads that variable when no path is given.

The --license-offline path must point to a local drive on the agent; a network-share path (\server...) is rejected. If the file lives on a share, copy it to the agent or use the variable instead.

Treat the file as a secret. It carries your entitlement, so anyone who obtains it can run the licensed verbs. Store it as a masked or secret variable, never echo it into a build log, and do not commit it to your repository.

The file stops working when your subscription ends, and in any case 90 days after you saved it. Save a fresh one from Studio when a pipeline reports an expired licence.

--license-key is no longer accepted and exits with code 3. Use the licence file instead.

These verbs do not need a licence file: version, verify and validate. verify deliberately stays open so you can re-check the integrity of an archived acceptance report years later, including after a subscription has ended, and validate is a plain schema check you can run in a pre-commit hook.

Proxy configuration

The runner honours the standard proxy environment variables by default — set HTTPS_PROXY, HTTP_PROXY, and NO_PROXY on the build agent and online license validation will route through your proxy. No extra flags are needed.

Supported environments

Environment	Supported?	Notes
Windows 11 (desktop)	Yes	Reference environment.
Windows Server 2022 with Desktop Experience	Yes	TIA Portal and PLCSIM Advanced require an interactive desktop session.
Windows Server Core	No	TIA Portal Openness needs the full desktop shell; the headless Server Core install does not provide it.
Docker / Windows container	No	TIA Portal and PLCSIM Advanced are not supported inside containers.

Troubleshooting CI runs

Symptom	Cause and fix
Exit code `3`, "`--project` is required"	A required argument is missing or the YAML config could not be read. Check `--project`, `--out-dir`, and `--config` paths.
Exit code `2`, licence message	The agent has no valid CI licence file. Save a fresh one from Studio and point the runner at it, or set the masked `ANYAUTOMATION_CI_ENTITLEMENT` variable.
Exit code `2`, "could not write to output directory"	`--out-dir` points at a protected system location. Use a path under the project directory or under the agent's local app-data area.
Exit code `4`, timeout	The run exceeded `--timeout-minutes`. Raise the limit, or split large suites; check that the PLC connection is reachable from the agent.
No test report appears in CI	The test-reporter step is not pointed at `junit.xml`, or it only runs on success. Publish reports with an `always()`/`when: always` condition and point it at `reports/**/junit.xml`.
Online validation hangs behind a corporate proxy	Set `HTTPS_PROXY` / `HTTP_PROXY` (and `NO_PROXY` for internal hosts) on the agent.

Privacy

No telemetry is collected by the CLI or the UI. The CI variables surfaced in reports are recorded only when you opt in with --ci-env-capture, and the project path is stored as an irreversible hash, never in plain text. HTML reports meet WCAG 2.1 AA, verified by automated checks; a manual screen-reader (NVDA) sign-off is part of the release end-to-end pass.

← Previous

CODESYS

Library Management