capabilities · testing

Multi-vendor test infrastructure

2ot treats verification as a compiler capability: shared suites, runtime-equivalence checks and target-specific runners live alongside the backends they validate.

One test layer across multiple targets

2ot contains shared IEC test harness infrastructure and dedicated suites for multi-vendor behavior. This keeps target comparison in one place instead of scattering validation logic across individual integrations.

Runtime equivalence

The test suite includes runtime-equivalence checks designed to compare behavior across supported targets. That gives the compiler a concrete regression surface for changes that must preserve semantics as code moves between environments.

Vendor-specific runners

Siemens TIA Portal has dedicated runner support, while Beckhoff TwinCAT includes runner and TcUnit integration. Vendor-specific mechanics stay behind the 2ot test layer instead of becoming separate public products.

AGENTS4PLC

Benchmark rows enter the same vendor-neutral test model

2ot includes an Agents4PLC import adapter that converts benchmark rows, traces and validation properties into the common test document model before target-specific rendering. That keeps benchmark-specific structure out of the downstream runners.

The benchmark corpus is loaded from local/external dataset files. No public pass-rate is claimed here unless a result artifact is explicitly published.

5 benchmark suites
v1 easy · v1 medium · v2 high-fidelity · v2 medium · v2 hard
4 target profiles
Siemens TIA Portal · Beckhoff TwinCAT · CODESYS · RuSTy
Evaluation path: Agents4PLC JSONL → vendor-neutral test documents → target transpilation / validation. A quick smoke path and a full-suite evaluation harness are both present in the source tree.

What the test layer covers

Current 2ot testing surfaces
AreaPurpose
multi-vendorCross-target verification.
runtime equivalenceBehavioral comparison across runtimes.
Siemens TIA PortalTarget-specific execution support.
Beckhoff TwinCATBeckhoff TwinCAT test integration and TcUnit support.
AGENT EVALUATION

Measure separate claims separately

Agent-generated PLC code should not collapse generation, compilation, vendor acceptance and requirement satisfaction into one success metric. 2ot's testing direction is to keep those evidence sources distinct.

generation result
2ot compiler result
target-vendor result
formal/property result

Built for compiler evolution

As new passes, backends and integrations land, the shared test infrastructure gives 2ot a place to prove both target-specific behavior and cross-vendor consistency.