# Ansible-Test: IEC 62443-3-3 SL2 Compliance Validation Ansible playbooks reimagined as a **test framework** for industrial control system (ICS/OT) security compliance. Every task is a test that collects evidence, never aborts on failure, produces structured JSON, and renders into human-readable reports via Go templates or Python. --- ## Table of Contents 1. [Architecture](#architecture) 2. [Ansible Control Node — QEMU-Bootable](#ansible-control-node--qemu-bootable-alpine-linux) 3. [Quick Start](#quick-start) 4. [How It Works](#how-it-works) 5. [The Test Pattern (The "Secret Sauce")](#the-test-pattern-the-secret-sauce) 6. [JSON Output Schema](#json-output-schema) 7. [IEC 62443-3-3 SL2 Coverage](#iec-62443-3-3-sl2-coverage) 8. [Adding a New Test](#adding-a-new-test) 9. [Rendering Reports](#rendering-reports) 10. [File Reference](#file-reference) 11. [Design Decisions & Tradeoffs](#design-decisions--tradeoffs) 12. [Comparison to Alternatives](#comparison-to-alternatives) 13. [Roadmap](#roadmap) --- ## Architecture ``` playbooks/ ├── site.yml # Entry point: orchestrates all FR suites ├── suites/ │ ├── fr1_auth.yml # FR1: Identification & Authentication Control │ ├── fr2_use_control.yml # FR2: Use Control (audit, sudo, sessions) │ └── fr5_data_flow.yml # FR5: Restricted Data Flow (firewall, services) ├── library/ │ └── report.yml # Aggregates test_results[] → JSON → disk reports/ ├── render.go # Go renderer (text/template on external .gohtml) ├── render_report.py # Python renderer (terminal + markdown modes) ├── report.gohtml # Go template file for terminal-box-drawing report ├── go.mod # Go module definition └── sample-output.json # Example output for offline rendering tests inventory.ini # Ansible inventory (target host list) run.sh # End-to-end wrapper: ansible → find json → render README.md # This file ``` ### Data Flow ``` ansible-playbook site.yml │ ├── FR1 suite ──┐ ├── FR2 suite ──┤ Each task appends to test_results[] ├── FR5 suite ──┘ via set_fact, wrapped in ignore_errors │ ▼ library/report.yml - Assembles __report dict with summary, by_category, failures - Prints summary to console - Writes JSON to reports/-.json │ ▼ run.sh → python3 reports/render_report.py reports/*.json → go run reports/render.go reports/*.json reports/report.gohtml ``` --- ## Ansible Control Node — QEMU-Bootable (Alpine Linux) A minimal, portable Ansible control node that boots on QEMU `pc-q35-10.0`. Designed to run on Windows hosts, providing Ansible connectivity to Windows, Cisco, VMware, MSSQL, and Linux targets. No systemd — OpenRC + BusyBox. ### Build & Launch ```bash # Prerequisites: Docker, QEMU (qemu-full), passwordless sudo for mount ./scripts/build-qemu.sh # Dockerfile → qcow2 disk (2GB) ./scripts/run-qemu.sh # Boot VM (KVM, 1GB RAM, 2 vCPUs) # Access the VM (password: ansible) sshpass -p ansible ssh -p 2222 ansible@localhost ``` ### Architecture ``` Dockerfile # Complete rootfs definition │ ▼ scripts/build-qemu.sh # Docker export → ext4 → qcow2 │ ├── output/ansible-node.qcow2 (bootable disk, ext4) ├── output/vmlinuz-virt (kernel, direct -kernel boot) └── output/initramfs-virt (initramfs) │ ▼ scripts/run-qemu.sh # QEMU pc-q35-10.0, KVM, virtio │ ▼ ┌─────────────────────────────────────────────┐ │ Alpine Linux 3.20 | Kernel 6.6 (virt) │ │ OpenRC | BusyBox | no systemd │ │ Console: ttyS0 xterm-256color │ │ Keyboard: PS/2 atkbd │ │ Docker 26.1 daemon (running) │ │ SSH: ansible@ (password: ansible) │ └─────────────────────────────────────────────┘ ``` ### Target Integrations | Target | Ansible Collection | Protocol | Python | |---|---|---|---| | **Windows** | `ansible.windows` 2.3 | WinRM + Kerberos | `pywinrm` 0.5 | | **Cisco ASA** | `cisco.asa` 4.0 | SSH/CLI | `paramiko` 4.0 | | **Cisco IOS/Catalyst** | `cisco.ios` 5.3 | SSH/CLI | `netmiko` 4.7 | | **Cisco NX-OS** | `cisco.nxos` 5.3 | SSH/NX-API | `ncclient` 0.7 | | **VMware vSphere** | `community.vmware` 4.3 | SOAP API | `pyvmomi` 9.1 | | **MSSQL** | `microsoft.ad` 1.5 | TDS | `pymssql` 2.3 | | **Linux** | built-in SSH | SSH | — | ### Key Design Decisions | Decision | Rationale | |---|---| | Dockerfile as build system | Dependency resolution, layer caching. Same file usable as `docker run` or QEMU boot | | `root=/dev/vda` (not LABEL) | Alpine's `nlplug-findfs` can't resolve labels on virtio-block; device path works | | `modules=virtio_blk,ext4` on cmdline | `CONFIG_VIRTIO_BLK=m` — module not auto-detected by initramfs | | No `apk del` purge step | Purging `util-linux-dev` cascades to removing `mount`, `umount`, `blkid` | | Direct `-kernel` boot (no extlinux) | QEMU loads kernel/initrd directly; no bootloader needed in the disk image | --- ## Quick Start ### Prerequisites - Ansible ≥ 2.9 (core modules only: `shell`, `stat`, `copy`, `set_fact`, `debug`) - Python ≥ 3.6 (for the Python report renderer) - Go ≥ 1.21 (optional, for the Go template renderer) - Target host: Linux with systemd (Debian/Ubuntu or RHEL/Rocky) ### Run Against Localhost ```bash # Edit inventory.ini to add your target, or test locally: echo "localhost ansible_connection=local" > inventory.ini # Run with sudo (many checks need root): ansible-playbook -i inventory.ini playbooks/site.yml --limit localhost -K # Or use the wrapper script: KEEP_SUDO=1 ./run.sh --limit localhost -K ``` ### Render a Report ```bash # Terminal format (default): python3 reports/render_report.py reports/localhost-2026-07-07.json # Markdown format (for GitHub/GitLab wikis): python3 reports/render_report.py reports/localhost-2026-07-07.json --format md > REPORT.md # Go template (if Go is installed): (cd reports && go run render.go ../sample-output.json report.gohtml) ``` --- ## How It Works ### The Test Pattern ("The Secret Sauce") Every test follows a rigid 3-step **Gather → Evaluate → Record** structure wrapped in an Ansible `block` with `ignore_errors: yes`: ```yaml # ── SR 1.5: Password minimum length ─────────────────────────── - block: # ← 1) Wrap everything - name: "Gather: Check pwquality minlen" # ← 2) Gather: read system state ansible.builtin.shell: | grep -E '^\s*minlen\s*=' /etc/security/pwquality.conf 2>/dev/null | tail -1 || echo "NOT SET" register: _minlen # store raw output changed_when: false # never report as "changed" - name: "Evaluate: IAC-05" # ← 3) Evaluate: judge pass/fail ansible.builtin.set_fact: test_results: "{{ test_results + [{ # append to shared list 'test_id': 'IAC-05', 'category': 'FR1 - Identification and Authentication Control', 'requirement': 'SR 1.5 — Authenticator Strength', 'description': 'Password minimum length shall be ≥ 14 characters', 'passed': ( _minlen.stdout | regex_search('minlen\\s*=\\s*([0-9]+)') | regex_replace('minlen\\s*=\\s*', '') | int >= 14 ), 'expected': 'minlen >= 14 in /etc/security/pwquality.conf', 'actual': _minlen.stdout | trim, 'severity': 'high', 'remediation': 'Set minlen=14 in /etc/security/pwquality.conf' }] }}" ignore_errors: yes # ← 4) NEVER abort the run ``` **Why this works**: `ignore_errors: yes` prevents Ansible from stopping. The `set_fact` always executes (it never fails). The `block` ensures the gather and evaluate steps share a scope, so `register`ed variables are available. ### Why `block` + `ignore_errors` Instead of `failed_when` Using `failed_when: false` on individual tasks is tempting, but: | Approach | Behavior | |---|---| | `failed_when: false` on shell task | Shell always "succeeds"; but if the shell returns non-zero, Ansible still marks it red in output | | `block` + `ignore_errors: yes` | Failures are captured, marked orange, but execution continues to the next block. Registered vars are still set. This is the **cleanest** pattern. | --- ## JSON Output Schema Each test produces one entry in `test_results[]`. The complete report looks like: ```jsonc { "meta": { "standard": "IEC 62443-3-3", // Standard being validated "security_level": "SL2", // Target security level "target": "ics-gateway-01", // inventory_hostname "timestamp": "2026-07-07T16:42:00+02:00", // ISO 8601 "executed_by": "auditor" // ansible_user_id }, "summary": { "total": 15, // Total test count "passed": 10, // passed == true "failed": 4, // passed == false "skipped": 1 // passed == "review" or "skipped" }, "by_category": [ // Results grouped by FR category ["FR1 - ...", [{...}, {...}]], ["FR2 — Use Control", [{...}]] ], "by_severity": { // Drill-down by criticality "critical": [{...}], "high": [{...}], "medium": [{...}], "low": [{...}] }, "failures": [{...}], // Only tests where passed == false "results": [ // All results, ordered by execution { "test_id": "IAC-05", // Unique identifier (category-number) "category": "FR1 - ...", // Which IEC 62443 FR section "requirement": "SR 1.5 — ...", // Specific SR (System Requirement) "description": "Password...", // Human-readable check "passed": false, // bool | "review" | "skipped" "expected": "minlen >= 14...", // What compliance looks like "actual": "minlen = 8", // What we found "severity": "high", // critical | high | medium | low "remediation": "Set minlen=14..." // How to fix } ] } ``` ### `passed` Field Semantics | Value | Meaning | Icon | |---|---|---| | `true` | Automated check passed | ✅ | | `false` | Automated check failed | ❌ | | `"review"` | Requires human assessment | 🔍 | | `"skipped"` | Not applicable to this target | ⏭️ | ### `severity` Field Semantics | Value | Meaning | Examples | |---|---|---| | `critical` | Allows immediate compromise | Empty passwords, world-writable sudoers | | `high` | Defeats a core SL2 control | No firewall, no auditd, weak password policy | | `medium` | Weakens a control | Password aging not set, some unused accounts | | `low` | Best-practice gap | Extra listening ports, stale accounts | --- ## IEC 62443-3-3 SL2 Coverage IEC 62443-3-3 defines 7 Foundational Requirements (FRs), each with System Requirements (SRs) at escalating Security Levels (SL1–SL4). SL2 adds to SL1. ### Mapping of Implemented Tests | Test ID | FR | SR | Description | Severity | |---|---|---|---|---| | IAC-01 | FR1 | SR 1.1 | No duplicate UIDs in /etc/passwd | high | | IAC-02 | FR1 | SR 1.3 | No default/unnecessary system accounts | medium | | IAC-03 | FR1 | SR 1.3 | No human accounts that have never logged in | low | | IAC-04 | FR1 | SR 1.4 | No empty or trivially-weak password hashes | critical | | IAC-05 | FR1 | SR 1.5 | Password min length ≥ 14 (pwquality) | high | | IAC-06 | FR1 | SR 1.5 | ≥ 3 character classes required (pwquality) | medium | | IAC-07 | FR1 | SR 1.7 | PASS_MAX_DAYS ≤ 90 | medium | | IAC-08 | FR1 | SR 1.7 | PASS_MIN_DAYS ≥ 1 | low | | IAC-09 | FR1 | SR 1.11 | Account lockout after ≤ 5 failures (pam_faillock) | high | | IAC-10 | FR1 | SR 1.6 | Password history ≥ 5 (pam_pwhistory) | medium | | UC-01 | FR2 | SR 2.1 | No unrestricted NOPASSWD sudo | high | | UC-02 | FR2 | SR 2.1 | /etc/sudoers owned root:root, mode 0440 | critical | | UC-03 | FR2 | SR 2.5 | Shell idle timeout ≤ 900s (TMOUT) | medium | | UC-04 | FR2 | SR 2.4 | Audit rules immutable (-e 2) | high | | UC-05 | FR2 | SR 2.8 | auditd service active | high | | UC-06 | FR2 | SR 2.8 | ≥ 4 critical syscall types audited | medium | | RDF-01 | FR5 | SR 5.1 | Host-based firewall with rules | critical | | RDF-02 | FR5 | SR 5.1 | Default INPUT policy is DROP | high | | RDF-03 | FR5 | SR 5.3 | No insecure legacy services (telnet, rsh, ftp) | critical | | RDF-04 | FR5 | SR 5.3 | Listening TCP ports documented | low | ### Not Yet Implemented (FR3, FR4, FR6, FR7) | FR | Key SL2 Controls | Suggested Checks | |---|---|---| | FR3 — System Integrity | Malware protection, file integrity, secure boot | AIDE/IMA, ClamAV service, /proc/sys/kernel/secure_boot | | FR4 — Data Confidentiality | Encryption at rest/transit | TLS version on listening ports, LUKS/dm-crypt, SSH ciphers | | FR6 — Timely Response | Audit log forwarding, alerting | rsyslog remote config, auditd dispatcher | | FR7 — Resource Availability | DoS protection, backup | Disk quota, CPU limits, backup cron/schedule | --- ## Adding a New Test ### Step-by-Step 1. **Choose an FR category file** (or create a new `suites/frN_*.yml`). 2. **Copy the test template**: ```yaml # ── SR X.Y: Short description ─────────────────────────────── - block: - name: "Gather: What to check" ansible.builtin.shell: | your-check-command-here register: _myvar changed_when: false - name: "Evaluate: XXX-NN" ansible.builtin.set_fact: test_results: "{{ test_results + [{ 'test_id': 'XXX-NN', 'category': 'FRN — Category Name', 'requirement': 'SR X.Y — Requirement Name', 'description': 'One-line description of the check', 'passed': ( _myvar.stdout | trim | length > 0 ), # boolean expression 'expected': 'What compliance looks like', 'actual': _myvar.stdout | trim | default('NOT FOUND', true), 'severity': 'high', # critical|high|medium|low 'remediation': 'Command or steps to fix' }] }}" ignore_errors: yes ``` 3. **Register the suite** in `playbooks/site.yml`: ```yaml - name: "Suite: FRN — Category Name" block: - ansible.builtin.include_tasks: suites/frN_category.yml ignore_errors: yes ``` 4. **Add to the coverage table** in this README. ### Test ID Naming Convention - **Prefix**: 3-letter abbreviation of the FR (e.g., `IAC`, `UC`, `RDF`, `SYS`, `CONF`, `RESP`, `AVAIL`) - **Number**: Sequential within that FR, padded to 2 digits (`01`, `02`, ...) - **Examples**: `IAC-01`, `UC-12`, `RDF-05` ### Tips for Writing Robust Checks | Tip | Why | |---|---| | Always use `changed_when: false` on gather tasks | Tests must never report as "changed" | | Use `| trim` on shell output | Shell often returns trailing newlines | | Use `| default('NOT FOUND', true)` for missing files | Prevents undefined-variable errors | | Parse numbers with `| int` before comparison | String `"8"` ≠ integer 8 in Jinja2 | | Use `| regex_search(...)` for structured config | More robust than exact-match grep | | Test an `or` of `pam_faillock` and `pam_tally2` | Different distros use different PAM modules | --- ## Rendering Reports ### Python Renderer (`reports/render_report.py`) No dependencies beyond Python 3 stdlib. Two output formats: ```bash # Terminal box-drawing (default): python3 reports/render_report.py reports/localhost-2026-07-07.json # Markdown (for wiki, PR comment, etc.): python3 reports/render_report.py reports/localhost-2026-07-07.json --format md ``` **Terminal output** uses Unicode box-drawing characters (╔═╗║╚╝), groups results by FR category, and shows a failure-detail section at the bottom. **Markdown output** produces a GitHub-flavored table plus per-failure sections with remediation instructions. ### Go Renderer (`reports/render.go`) Requires Go ≥ 1.21. Uses `text/template` with an external template file: ```bash cd reports go run render.go sample-output.json report.gohtml ``` The template (`report.gohtml`) is a standalone file — you can customize it without recompiling. Template functions: | Function | Purpose | |---|---| | `passIcon` | Maps `passed` value to ✅/❌/🔍 | | `severityIcon` | Maps severity string to 🔴/🟠/🟡/🟢 | | `title` | Capitalizes first letter of each word | | `divf` | Float division for compliance rate | ### Extending: Custom Go Templates Copy `report.gohtml` to `my-report.gohtml`, modify it, and run: ```bash go run reports/render.go reports/*.json my-report.gohtml ``` The template receives the full JSON document as `.` (a `map[string]any`). Access fields with `(index . "key")` since it's untyped. --- ## File Reference ### `playbooks/site.yml` Entry-point playbook. Runs against `hosts: all`. Pre-tasks create the local `reports/` directory. Each suite is included as a named task wrapped in a `block` with `ignore_errors: yes`. The final task includes `library/report.yml` to aggregate results. Variables set here: - `report_dir: "./reports"` — where JSON output lands ### `playbooks/suites/fr1_auth.yml` **FR1 — Identification and Authentication Control**. 10 tests covering: - Duplicate UIDs (SR 1.1) - Default/unused accounts (SR 1.3) - Empty password hashes (SR 1.4) - Password complexity via libpwquality (SR 1.5) - Password history via pam_pwhistory (SR 1.6) - Password aging via login.defs (SR 1.7) - Account lockout via pam_faillock (SR 1.11) Uses: `/etc/passwd`, `/etc/shadow`, `/etc/security/pwquality.conf`, `/etc/login.defs`, `/etc/pam.d/common-auth`, `/etc/pam.d/common-password`, `lastlog` ### `playbooks/suites/fr2_use_control.yml` **FR2 — Use Control**. 6 tests covering: - Unrestricted sudo access (SR 2.1) - sudoers file permissions (SR 2.1) - Shell idle timeout via TMOUT (SR 2.5) - auditd immutable mode (SR 2.4) - auditd service status (SR 2.8) - Audited syscall coverage (SR 2.8) Uses: `/etc/sudoers`, `/etc/sudoers.d/`, `/etc/profile`, `/etc/bash.bashrc`, `/etc/audit/audit.rules`, `systemctl` ### `playbooks/suites/fr5_data_flow.yml` **FR5 — Restricted Data Flow**. 4 tests covering: - Host-based firewall with active rules (SR 5.1) - Default DROP inbound policy (SR 5.1) - Insecure legacy services disabled (SR 5.3) - Listening port inventory for review (SR 5.3) Uses: `iptables`/`nft`, `systemctl`, `ss` ### `playbooks/library/report.yml` **Report aggregation**. Included last by `site.yml`. Builds a `__report` dict from the `test_results[]` list, prints a summary box to the console, and writes `reports/-.json` to the local machine. Key computed fields: - `summary.total/passed/failed/skipped` — basic counts - `by_category` — results grouped by FR category string - `by_severity` — results grouped into critical/high/medium/low buckets - `failures` — only tests where `passed == false` ### `reports/render_report.py` Python-based report renderer. Two modes: - `terminal` (default) — Unicode box-drawing, grouped by category, failure details - `md` — GitHub-flavored Markdown tables Functions: - `pass_icon(v)` → ✅/❌/🔍/⏭️ based on bool/string - `severity_icon(s)` → 🔴/🟠/🟡/🟢 - `render_terminal(report)` — writes to stdout - `render_markdown(report)` — writes to stdout ### `reports/render.go` Go-based report renderer. Uses `text/template` and `golang.org/x/text/cases`. Loads an external `.gohtml` template file so templates can be customized without rebuilding. Registers template functions for pass/severity icons and arithmetic. ### `reports/report.gohtml` Go template producing a terminal box-drawing report. Iterates over `results`, renders a summary header, a per-result listing with icons, and a failure-detail section. Uses template functions registered by `render.go`. ### `reports/sample-output.json` A hand-crafted example JSON report for testing the renderers without running Ansible. Contains 15 results (10 passed, 4 failed, 1 review) across FR1 and FR2. Useful for CI validation of the report rendering pipeline. ### `inventory.ini` Standard Ansible inventory. Structure: ```ini [all] localhost ansible_connection=local [ics_assets] # Add real targets here ``` ### `run.sh` End-to-end wrapper that: 1. Runs `ansible-playbook` with the given arguments 2. Finds the latest JSON report in `reports/` 3. Renders it via `render_report.py` (or `render.go` if available) 4. Falls back to a Python one-liner summary if neither is available --- ## Design Decisions & Tradeoffs ### Why Ansible and Not a Dedicated Compliance Scanner? | Decision | Rationale | |---|---| | **Ansible** | Already deployed in most OT environments. No new agent, no new approval. | | **Shell-based checks** | `shell` module is the most flexible. We trade idempotence for expressiveness. `changed_when: false` keeps it clean. | | **Inline `set_fact`** vs custom module | Custom Ansible modules require Python on the control node. Inline facts work everywhere. Less code to maintain. | | **`test_results[]` list** vs file-per-test | A single growing list is simpler than per-file concatenation. At 100+ tests the memory footprint is negligible (~50KB). | | **`ignore_errors` at block level** vs task level | Block-level isolates failures cleanly. A failed gather still reaches evaluate. | | **JSON as canonical output** | Machine-readable, schema-validatable, ingestible by SIEM/SOAR/Jira. | | **Go templates for rendering** | `text/template` is standard, fast, and supports external template files for customization without recompilation. | ### Known Limitations 1. **Shell-heavy**: Tests depend on shell commands. Different distros may need different check commands (e.g., `apt` vs `rpm` package verification). Mitigate with `when: ansible_os_family == 'Debian'` variants. 2. **No built-in Windows support**: Currently Linux-only. Windows checks would need `win_shell`/`win_regedit` modules and a separate suite. 3. **No diff/drift detection**: Each run is independent. To detect drift between runs, diff two JSON reports externally (`jd`, `diff`, or a time-series DB). 4. **No immediate pass/fail exit code**: `ansible-playbook` always exits 0 unless a task fails without `ignore_errors`. For CI integration, parse the JSON summary and use `--format json` output. 5. **Scalability at 500+ targets**: Running against large inventories is fine (Ansible's strength), but the JSON file-per-host can be unwieldy. Consider post-processing into a single aggregated report. --- ## Comparison to Alternatives | Tool | Type | Pros | Cons | |---|---|---|---| | **Inspec** | Ruby DSL, Chef ecosystem | Rich compliance profiles, CIS/STIG built-in | Ruby runtime; less common in OT | | **Goss** | YAML config, Go binary | Fast, simple, YAML-based | No native IEC mapping; limited to local checks | | **OpenSCAP** | XML-based, SCAP standard | NIST/STIG aligned, XCCDF/OVAL | Heavy, complex, US-govt focused | | **Lynis** | Shell script, single binary | Broad Linux coverage, professional reports | Non-extensible output format | | **This project** | Ansible + JSON + Go/Python templates | Zero new agents, customizable, IEC 62443 mapped | Requires Ansible; shell-dependent checks | --- ## Roadmap - [ ] **FR3 — System Integrity**: File integrity (AIDE/IMA), malware scanner status, secure boot, /tmp noexec - [ ] **FR4 — Data Confidentiality**: TLS version/cipher audit on listening ports, disk encryption, SSH hardening - [ ] **FR6 — Timely Response**: rsyslog remote forwarding, auditd dispatcher, journald persistent storage - [ ] **FR7 — Resource Availability**: Disk quotas, CPU/memory limits, backup schedule verification - [ ] **Aggregated multi-host report**: Single HTML/PDF report across all inventory hosts - [ ] **GitLab CI / GitHub Actions integration**: Pipeline stage that runs playbook + posts markdown report as PR comment - [ ] **Custom Ansible module**: Replace shell-based checks with a native `iec62443_test` module for cleaner code - [ ] **CIS Benchmark mapping**: Dual-map tests to both IEC 62443-3-3 and CIS distribution benchmarks