ansible-testing/README.md

# Ansible-Test: IEC 62443-3-3 SL2 Compliance Validation

Ansible playbooks reimagined as a **test framework** for industrial control system
(ICS/OT) security compliance. Every task is a test that collects evidence, never
aborts on failure, produces structured JSON, and renders into human-readable
reports via Go templates or Python.

Supports automated testing across **Linux, Windows Server, Windows Clients,
MS SQL Server, VMware vSphere, Hyper-V, Cisco IOS switches, and Cisco ASA
firewalls** — from a single containerised Ansible control node.

---

## Table of Contents

1. [Architecture](#architecture)
2. [Quick Start](#quick-start)
3. [Platform-Specific Examples](#platform-specific-examples)
4. [The Test Pattern](#the-test-pattern)
5. [Human-in-the-Loop Tests](#human-in-the-loop-tests)
6. [Adding a New Test](#adding-a-new-test)
7. [JSON Output Schema](#json-output-schema)
8. [IEC 62443-3-3 SL2 Coverage](#iec-62443-3-3-sl2-coverage)
9. [Rendering Reports](#rendering-reports)
10. [Ansible Control Node — Container & QEMU VM](#ansible-control-node--container--qemu-vm)
11. [File Reference](#file-reference)
12. [Design Decisions & Tradeoffs](#design-decisions--tradeoffs)
13. [Comparison to Alternatives](#comparison-to-alternatives)
14. [Roadmap](#roadmap)

---

## Architecture

```
playbooks/
├── site.yml                      # Entry point for Linux targets (all FR suites)
├── suites/                       # Included task-list suites for Linux
│   ├── fr1_auth.yml              # FR1: Identification & Authentication (10 tests)
│   ├── fr2_use_control.yml       # FR2: Use Control — audit, sudo, sessions (6 tests)
│   └── fr5_data_flow.yml         # FR5: Restricted Data Flow — firewall, services (4 tests)
├── library/
│   └── report.yml                # Aggregates test_results[] → JSON → disk
├── examples/                     # ◄ Standalone playbooks — one per platform type
│   ├── linux_vm.yml              #   Linux: SSH hardening, sysctl, sudo logging
│   ├── windows_server.yml        #   Windows Server: WinRM, security policy, audit
│   ├── windows_client.yml        #   Windows Client: BitLocker, screen lock, USB
│   ├── mssql_server.yml          #   MS SQL Server: auth mode, sa, xp_cmdshell, audit
│   ├── vmware_vsphere.yml        #   VMware ESXi: lockdown, NTP, SSH/Shell services
│   ├── hyperv_cluster.yml        #   Hyper-V: Gen2, vSwitch isolation, integration svc
│   ├── cisco_switch.yml          #   Cisco IOS: SSH v2, no SNMPv1, banner, NTP
│   └── cisco_firewall.yml        #   Cisco ASA: no Telnet, IKEv2, syslog, AAA, ACL
└── templates/                    # ◄ Copy-and-fill templates for writing new tests
    ├── test_automated.yml        #   Shell-based gather → evaluate pattern
    ├── test_file_check.yml       #   File permission check via ansible.builtin.stat
    ├── test_service_check.yml    #   Service state check via service_facts
    └── test_hitl.yml             #   Human-in-the-Loop with ansible.builtin.pause

reports/
├── render.go                     # Go renderer (text/template + external .gohtml)
├── render_report.py              # Python renderer (terminal + markdown output)
├── report.gohtml                 # Go template for terminal box-drawing report
├── go.mod                        # Go module definition
└── sample-output.json            # Example output for offline renderer tests
inventory.ini                     # Ansible inventory — all platform groups defined
run.sh                            # End-to-end wrapper: ansible → find JSON → render
```

### Data Flow

```
  ansible-playbook <playbook>.yml
         │
         ├── Gather tasks ──┐
         │                  ├─ collect system state (shell, stat, ios_command, etc.)
         ├── Evaluate tasks─┘
         │                    judge pass/fail, append to test_results[]
         │
         ▼
  library/report.yml
    - Assembles __report dict with summary, by_category, by_severity, failures
    - Prints summary box to console
    - Writes JSON to reports/<hostname>-<date>.json
         │
         ▼
  run.sh / render manually:
    python3 reports/render_report.py reports/<hostname>-<date>.json
    go run reports/render.go reports/<hostname>-<date>.json reports/report.gohtml
```

### Target Integrations

| Platform | Ansible Collection | Connection | Python library |
|---|---|---|---|
| **Linux** | built-in | SSH | — |
| **Windows Server / Client** | `ansible.windows` | WinRM + NTLM/Kerberos | `pywinrm` |
| **MS SQL Server** | `ansible.windows` | WinRM → PowerShell `Invoke-Sqlcmd` | `pywinrm` |
| **VMware vSphere ESXi** | `community.vmware` | vSphere SOAP API (delegate_to: localhost) | `pyvmomi` |
| **Hyper-V** | `ansible.windows` | WinRM → PowerShell Hyper-V cmdlets | `pywinrm` |
| **Cisco IOS / IOS-XE** | `cisco.ios` | SSH via `network_cli` | `paramiko`, `netmiko` |
| **Cisco ASA** | `cisco.asa` | SSH via `network_cli` | `paramiko` |
| **Cisco NX-OS** | `cisco.nxos` | SSH / NX-API via `network_cli` | `ncclient` |

---

## Quick Start

### Prerequisites

- Ansible ≥ 2.9 with the collections listed above (pre-installed in the Docker image)
- Python ≥ 3.6 (for the Python report renderer)
- Go ≥ 1.21 (optional, for the Go renderer)
- For Windows / VMware / Cisco targets: the Python libraries listed above

### Step 1: Configure Inventory

Edit `inventory.ini`. Each platform group has the required connection variables
already set — just uncomment the hosts:

```ini
[linux_vms]
linux-vm-01.example.com  ansible_user=auditor

[windows_servers]
win-srv-01.example.com

[cisco_switches]
sw-core-01.example.com
```

Store passwords in Ansible Vault:
```bash
ansible-vault encrypt_string 'MyPassword' --name ansible_password
```

### Step 2: Run a Playbook

```bash
# Linux targets — all FR suites via the main entry point:
ansible-playbook -i inventory.ini playbooks/site.yml --limit linux_vms -K

# Or use the wrapper script (finds & renders the report automatically):
./run.sh --limit linux_vms -K

# Platform-specific examples:
ansible-playbook -i inventory.ini playbooks/examples/windows_server.yml
ansible-playbook -i inventory.ini playbooks/examples/cisco_switch.yml
ansible-playbook -i inventory.ini playbooks/examples/vmware_vsphere.yml

# Local self-test (localhost is pre-configured in inventory.ini):
ansible-playbook -i inventory.ini playbooks/site.yml --limit localhost -K
```

### Step 3: Render the Report

```bash
# Terminal box-drawing format (default):
python3 reports/render_report.py reports/localhost-2026-08-14.json

# Markdown (for GitHub / GitLab wikis, PR comments):
python3 reports/render_report.py reports/localhost-2026-08-14.json --format md

# Go template renderer:
cd reports && go run render.go ../reports/localhost-2026-08-14.json report.gohtml
```

---

## Platform-Specific Examples

The `playbooks/examples/` directory contains a complete, runnable playbook for
each supported platform. Each file:

- Has a header comment with the required `inventory.ini` group vars and any
  prerequisites (WinRM setup, SQLPS module, vCenter permissions, etc.)
- Follows the identical `block` → gather → evaluate → `ignore_errors` pattern
- Uses the most Ansible-native module available for each check (e.g.
  `win_security_policy` instead of `win_shell` for Windows password policy)
- Ends with `include_tasks: ../library/report.yml` to produce a JSON report
- Includes at least one Human-in-the-Loop test where automated checks cannot
  cover the full control

### Linux VMs — `playbooks/examples/linux_vm.yml`

```bash
ansible-playbook -i inventory.ini playbooks/examples/linux_vm.yml -K
```

Checks beyond the core FR suites: SSH root login and password auth settings,
sudo session logging (`Defaults log_input,log_output`), kernel IP forwarding
and ICMP redirect sysctl values, core dump disabled.

### Windows Server — `playbooks/examples/windows_server.yml`

```bash
ansible-playbook -i inventory.ini playbooks/examples/windows_server.yml
```

Uses `ansible.windows.win_security_policy` for password and lockout policy
(no PowerShell shell-out needed), `win_audit_policy_system` for audit subcategories,
`win_service_info` for Telnet/FTP service state, and `win_shell` with
`ConvertTo-Json` for Firewall profile states parsed via Ansible's `from_json` filter.

### Windows Client — `playbooks/examples/windows_client.yml`

```bash
ansible-playbook -i inventory.ini playbooks/examples/windows_client.yml
```

Checks: domain membership, BitLocker status on the OS drive, screen lock timeout
via registry (`win_reg_stat`), Public firewall profile. Includes a HITL check
for USB storage port controls with registry evidence displayed.

### MS SQL Server — `playbooks/examples/mssql_server.yml`

```bash
ansible-playbook -i inventory.ini playbooks/examples/mssql_server.yml
# Add per-host: mssql_instance=NAMED_INSTANCE  (default: MSSQLSERVER)
```

Connects via WinRM to the Windows host, then runs `Invoke-Sqlcmd` via `win_shell`
to query SQL Server internals. Checks: Windows-only authentication mode, SA account
disabled, `xp_cmdshell` disabled, login audit level. HITL check for sysadmin
role membership against an authorised list.

**Prerequisite on target:** `Install-Module SqlServer -Force -AllowClobber`

### VMware vSphere — `playbooks/examples/vmware_vsphere.yml`

```bash
ansible-playbook -i inventory.ini playbooks/examples/vmware_vsphere.yml
```

All tasks use `delegate_to: localhost` — no SSH to ESXi hosts. The inventory
host is the ESXi FQDN; `vcenter_hostname/username/password` are group vars
pointing at vCenter. Uses `community.vmware` info modules:
`vmware_host_lockdown_info`, `vmware_host_ntp_info`, `vmware_host_service_info`,
`vmware_host_config_info`. HITL check for vSwitch isolation using
`vmware_vswitch_info`.

**Required vCenter read-only permissions:**
Host → Configuration → Security Profile, Advanced Settings; Global → Settings.

### Hyper-V Cluster — `playbooks/examples/hyperv_cluster.yml`

```bash
ansible-playbook -i inventory.ini playbooks/examples/hyperv_cluster.yml
```

Connects via WinRM and runs PowerShell Hyper-V cmdlets via `win_shell`. Checks:
all VMs use Generation 2 with Secure Boot enabled, External vSwitch
`AllowManagementOS` exposure (flagged as review), Hyper-V VMMS Admin event log
active, integration services enabled on all running VMs. HITL check for
physical NIC segregation between ICS and management networks.

### Cisco Switch (IOS / IOS-XE) — `playbooks/examples/cisco_switch.yml`

```bash
ansible-playbook -i inventory.ini playbooks/examples/cisco_switch.yml
```

Uses `cisco.ios.ios_command` to run `show` commands and parses output with
`regex_search` / `regex_findall`. Uses `ios_facts` (gathered automatically via
`gather_facts: yes`) for interface data. Checks: SSH v2 / no Telnet on VTY,
no SNMPv1/v2c community strings, login banner, NTP synchronised. HITL check
for port VLAN assignment and physical port labelling.

### Cisco Firewall (ASA) — `playbooks/examples/cisco_firewall.yml`

```bash
ansible-playbook -i inventory.ini playbooks/examples/cisco_firewall.yml
```

Uses `cisco.asa.asa_command` with `ansible_become=yes` (enable mode). Checks:
no Telnet management access, SSH access configured, IKEv1 disabled (IKEv2 only),
remote syslog server configured, AAA authentication for SSH/enable, inbound
ACLs applied on zone interfaces. HITL check for ACL rule review
(no broad `permit ip any any`).

---

## The Test Pattern

Every test follows a rigid **Gather → Evaluate → Record** structure inside an
Ansible `block` with `ignore_errors: yes`. This ensures the run never aborts
regardless of what is found on the target.

```yaml
# ── SR 1.5: Password minimum length ────────────────────────────────────────

- block:
    - name: "Gather: Check pwquality minlen"
      ansible.builtin.shell: |
        grep -E '^\s*minlen\s*=' /etc/security/pwquality.conf 2>/dev/null | tail -1 || echo "NOT SET"
      register: _minlen
      changed_when: false                    # gather tasks must never say "changed"

    - name: "Evaluate: IAC-05"
      ansible.builtin.set_fact:
        test_results: "{{ test_results + [{
          'test_id':     'IAC-05',
          'category':    'FR1 — Identification and Authentication Control',
          'requirement': 'SR 1.5 — Authenticator Strength',
          'description': 'Password minimum length shall be ≥ 14 characters',
          'passed': (
            (_minlen.stdout | regex_search('minlen\\s*=\\s*(\\d+)', '\\1')
             | default(['0'], true) | first | int) >= 14
          ),
          'expected':    'minlen >= 14 in /etc/security/pwquality.conf',
          'actual':      _minlen.stdout | trim,
          'severity':    'high',
          'remediation': 'Set minlen=14 in /etc/security/pwquality.conf'
        }] }}"
  ignore_errors: yes                         # NEVER abort the run
```

### Why `block` + `ignore_errors` Instead of `failed_when`

| Approach | Behaviour |
|---|---|
| `failed_when: false` on shell task | Shell always "succeeds"; non-zero exit still shows red in Ansible output |
| `block` + `ignore_errors: yes` | Failures are captured and marked orange; execution continues; `register`ed variables remain available in the evaluate task |

### Tips for Robust Checks

| Tip | Why |
|---|---|
| `changed_when: false` on all gather tasks | Tests must never report as "changed" |
| `\| trim` on shell output | Shell often returns trailing newlines |
| `\| default('NOT SET', true)` | Prevents undefined-variable errors when files or keys are absent |
| Guard numeric comparisons | `'' \| int` = 0 in Jinja2 — a missing value can silently pass a `≤ 90` check; always verify the value exists and is non-zero first |
| `\| regex_search(pattern, '\\1')` | Use capture group syntax to extract a number cleanly, avoiding chained `regex_replace` calls that crash on `None` |
| `ConvertTo-Json` on Windows | Return structured data from `win_shell` and parse with Ansible's `from_json` filter instead of regex |
| Platform branching | Use `when: ansible_os_family == 'Debian'` variants for distro-specific commands |

---

## Human-in-the-Loop Tests

Some IEC 62443 SL2 controls cannot be verified automatically — physical access
controls, policy document review, proprietary vendor interfaces, or checks where
the output must be interpreted by a qualified reviewer.

**HITL tests use `ansible.builtin.pause` with `delegate_to: localhost`**, which
prompts the reviewer on the Ansible control node even when running against remote
targets. For multi-host runs the prompt fires once per host, so each target gets
an independent verdict.

```yaml
- block:
    - name: "Gather: [HITL] Collect evidence"
      ansible.builtin.shell: your-gather-command
      register: _evidence
      changed_when: false

    - name: "Display: [HITL] TEST_ID — evidence"
      ansible.builtin.debug:
        msg: |
          ══════════════════════════════════════════════════════════════
           MANUAL REVIEW REQUIRED  ·  TEST_ID  ·  {{ inventory_hostname }}
          ══════════════════════════════════════════════════════════════
           {{ _evidence.stdout | indent(1) }}
          ══════════════════════════════════════════════════════════════

    - name: "Prompt: TEST_ID — verdict"
      ansible.builtin.pause:
        prompt: "Enter verdict [pass / fail / skip]:"
      register: _verdict
      delegate_to: localhost               # ← always prompts on the control node

    - name: "Prompt: TEST_ID — notes on failure"
      ansible.builtin.pause:
        prompt: "Describe the finding:"
      register: _notes
      delegate_to: localhost
      when: _verdict.user_input | lower | trim in ['fail', 'f']

    - name: "Evaluate: TEST_ID"
      ansible.builtin.set_fact:
        test_results: "{{ test_results + [{
          'test_id':  'TEST_ID',
          'passed': (
            'skipped' if (_verdict.user_input | lower | trim in ['skip', 's', ''])
            else (_verdict.user_input | lower | trim in ['pass', 'p'])
          ),
          'reviewer': ansible_user_id,
          'notes':    (_notes.user_input | trim) if _notes is defined else ''
        }] }}"
  ignore_errors: yes
```

The `reviewer` and `notes` fields are written into the JSON report alongside
the automated evidence, creating an auditable record of who reviewed what and
when. See `playbooks/templates/test_hitl.yml` for the full copy-and-fill template.

**Note:** Playbooks containing HITL tests require an interactive terminal and
cannot be run unattended in a CI pipeline. Keep HITL tests in separate playbooks
or use Ansible tags to separate them from automated checks.

---

## Adding a New Test

### Step 1: Choose a Template

The `playbooks/templates/` directory contains four ready-to-use templates.
Copy the one that matches your check type:

| Template | Use when |
|---|---|
| `playbooks/templates/test_automated.yml` | Running a shell command and evaluating its output |
| `playbooks/templates/test_file_check.yml` | Checking file ownership, permissions, or existence (`ansible.builtin.stat`) |
| `playbooks/templates/test_service_check.yml` | Checking service running/enabled state (`service_facts`) |
| `playbooks/templates/test_hitl.yml` | Control requires human reviewer input |

Every template has detailed comments explaining the `passed` expression patterns
relevant to that check type.

### Step 2: Fill in the Placeholders

Replace all UPPERCASE placeholders: `TEST_ID`, `FR_NUMBER`, `CATEGORY_NAME`,
`REQUIREMENT`, `DESCRIPTION`, `SEVERITY`, `REMEDIATION`, and the gather command.

### Step 3: Add to a Suite or Example

For Linux targets, append the block to the appropriate `suites/frN_*.yml` file
and ensure that suite is included in `site.yml`:

```yaml
- name: "Suite: FRN — Category Name"
  block:
    - ansible.builtin.include_tasks: suites/frN_category.yml
  ignore_errors: yes
```

For other platforms, append the block to the relevant `examples/` playbook.

### Test ID Naming Convention

- **Prefix**: Platform abbreviation + category (e.g., `IAC`, `UC`, `RDF` for
  Linux; `WIN-IAC`, `SQL-UC`, `FW-RDF`, `SW-RDF`, `VMW-IAC` for platform examples)
- **Number**: Sequential within prefix, zero-padded (`01`, `02`, ...)
- **HITL suffix**: Append `-HITL` for human-in-the-loop tests (e.g., `WIN-CLI-HITL-01`)

---

## JSON Output Schema

Each test produces one entry in `test_results[]`. The complete report schema:

```jsonc
{
  "meta": {
    "standard":       "IEC 62443-3-3",
    "security_level": "SL2",
    "target":         "ics-gateway-01",          // inventory_hostname
    "timestamp":      "2026-08-14T09:00:00Z",    // ISO 8601
    "executed_by":    "auditor"                  // ansible_user_id
  },
  "summary": {
    "total":   20,
    "passed":  14,
    "failed":   4,
    "review":   1,   // passed == "review" (manual review items)
    "skipped":  1    // passed == "skipped" (HITL skipped or not applicable)
  },
  "by_category": [                         // [["FR1 — ...", [{...}]], ...]
    ["FR1 — Identification and Authentication Control", [{...}, {...}]],
    ["FR2 — Use Control", [{...}]]
  ],
  "by_severity": {
    "critical": [{...}],
    "high":     [{...}],
    "medium":   [{...}],
    "low":      [{...}]
  },
  "failures": [{...}],                     // Only tests where passed == false
  "results": [
    {
      "test_id":     "IAC-05",
      "category":    "FR1 — ...",
      "requirement": "SR 1.5 — Authenticator Strength",
      "description": "Password minimum length shall be >= 14 characters",
      "passed":      false,                // bool | "review" | "skipped"
      "expected":    "minlen >= 14",
      "actual":      "minlen = 8",
      "severity":    "high",              // critical | high | medium | low
      "remediation": "Set minlen=14 in /etc/security/pwquality.conf"
      // HITL tests also carry: "reviewer", "notes"
    }
  ]
}
```

### `passed` Field Semantics

| Value | Meaning | Report icon |
|---|---|---|
| `true` | Automated check passed | ✅ |
| `false` | Automated check failed | ❌ |
| `"review"` | Listed for human assessment (port inventory, ACL review) | 🔍 |
| `"skipped"` | Reviewer entered `skip`; not applicable to this target | ⏭️ |

### `severity` Field Semantics

| Value | Meaning |
|---|---|
| `critical` | Allows immediate compromise (empty passwords, world-writable sudoers) |
| `high` | Defeats a core SL2 control (no firewall, no auditd, weak password policy) |
| `medium` | Weakens a control (password aging not set, no session timeout) |
| `low` | Best-practice gap (extra listening ports, stale accounts) |

---

## IEC 62443-3-3 SL2 Coverage

### Core Linux Suites (`playbooks/suites/`)

| Test ID | FR | SR | Description | Severity |
|---|---|---|---|---|
| IAC-01 | FR1 | SR 1.1 | No duplicate UIDs in /etc/passwd | high |
| IAC-02 | FR1 | SR 1.3 | No default/unnecessary system accounts | medium |
| IAC-03 | FR1 | SR 1.3 | No human accounts that have never logged in | low |
| IAC-04 | FR1 | SR 1.4 | No empty or trivially-weak password hashes | critical |
| IAC-05 | FR1 | SR 1.5 | Password min length >= 14 (pwquality) | high |
| IAC-06 | FR1 | SR 1.5 | >= 3 character classes required (pwquality) | medium |
| IAC-07 | FR1 | SR 1.7 | PASS_MAX_DAYS <= 90 and is set | medium |
| IAC-08 | FR1 | SR 1.7 | PASS_MIN_DAYS >= 1 | low |
| IAC-09 | FR1 | SR 1.11 | Account lockout after <= 5 failures (pam_faillock) | high |
| IAC-10 | FR1 | SR 1.6 | Password history >= 5 (pam_pwhistory) | medium |
| UC-01 | FR2 | SR 2.1 | No unrestricted NOPASSWD sudo | high |
| UC-02 | FR2 | SR 2.1 | /etc/sudoers owned root:root, mode 0440 | critical |
| UC-03 | FR2 | SR 2.5 | Shell idle timeout <= 900s (TMOUT) | medium |
| UC-04 | FR2 | SR 2.4 | Audit rules immutable (-e 2) | high |
| UC-05 | FR2 | SR 2.8 | auditd service active and enabled | high |
| UC-06 | FR2 | SR 2.8 | >= 4 critical syscall types audited | medium |
| RDF-01 | FR5 | SR 5.1 | Host-based firewall with active rules | critical |
| RDF-02 | FR5 | SR 5.1 | Default INPUT policy is DROP | high |
| RDF-03 | FR5 | SR 5.3 | No insecure legacy services (telnet, rsh, ftp) | critical |
| RDF-04 | FR5 | SR 5.3 | Listening TCP ports — review | low |

### Additional Checks in Platform Examples

| Platform | Example file | FR areas covered |
|---|---|---|
| Linux VM | `examples/linux_vm.yml` | FR1 (SSH hardening), FR2 (sudo logging), FR3 (sysctl, core dumps) |
| Windows Server | `examples/windows_server.yml` | FR1 (password/lockout policy), FR2 (audit policy), FR5 (firewall, Telnet) |
| Windows Client | `examples/windows_client.yml` | FR1 (domain join), FR3 (BitLocker), FR2 (screen lock), FR5 (firewall) |
| MS SQL Server | `examples/mssql_server.yml` | FR1 (auth mode, SA account, role review), FR2 (xp_cmdshell, audit) |
| VMware vSphere | `examples/vmware_vsphere.yml` | FR1 (lockdown, account lockout), FR2 (NTP), FR5 (SSH/Shell, vSwitch) |
| Hyper-V | `examples/hyperv_cluster.yml` | FR3 (SecureBoot, integration svc), FR2 (event log), FR5 (vSwitch/NIC) |
| Cisco Switch | `examples/cisco_switch.yml` | FR1 (SNMP, banner), FR2 (NTP), FR5 (SSH/Telnet, port shutdown) |
| Cisco Firewall | `examples/cisco_firewall.yml` | FR1 (IKEv2, AAA), FR2 (syslog), FR5 (Telnet, ACLs) |

### Not Yet Implemented as Dedicated Suites

| FR | Key SL2 Controls | Suggested Checks |
|---|---|---|
| FR3 — System Integrity | File integrity monitoring | AIDE/IMA service, `/proc/sys/kernel/kexec_load_disabled` |
| FR4 — Data Confidentiality | Encryption at rest/transit | TLS cipher audit on listening ports, LUKS/dm-crypt, SSH cipher suite |
| FR6 — Timely Response | Log forwarding, alerting | `rsyslog` remote config, auditd dispatcher, journald persistence |
| FR7 — Resource Availability | DoS protection, backup | Disk quota, systemd resource limits, backup schedule |

---

## Rendering Reports

### Python Renderer (`reports/render_report.py`)

Zero dependencies beyond Python 3 stdlib. Two output formats:

```bash
# Terminal box-drawing (default):
python3 reports/render_report.py reports/hostname-2026-08-14.json

# Markdown for GitHub / GitLab:
python3 reports/render_report.py reports/hostname-2026-08-14.json --format md > REPORT.md
```

Terminal output groups results by FR category with a failure-detail section.
Markdown output produces GFM tables plus per-failure sections with remediation.

### Go Renderer (`reports/render.go`)

Requires Go >= 1.21. Uses `text/template` with an external `.gohtml` file:

```bash
cd reports
go run render.go ../reports/hostname-2026-08-14.json report.gohtml
```

The template file is standalone — customise it without recompiling. Registered
template functions:

| Function | Purpose |
|---|---|
| `passIcon` | Maps `passed` value to ✅ PASS / ❌ FAIL / ❓ MANUAL |
| `severityIcon` | Maps severity to 🔴/🟠/🟡/🟢 |
| `title` | Capitalises first letter of each word |
| `divf` | Float division for compliance rate percentage |
| `add`, `sub` | Integer arithmetic |

Custom templates:
```bash
go run reports/render.go reports/*.json my-custom.gohtml
```

---

## Ansible Control Node — Container & QEMU VM

Two independent deployment artifacts. The container image runs anywhere Docker
is available; the QEMU VM provides a self-contained appliance for environments
without existing Docker infrastructure.

### The Two Artifacts

| Artifact | Defined by | Built with | Result |
|---|---|---|---|
| **Alpine Docker Host** | `Dockerfile.alpine-host` | `./scripts/build-qemu.sh` | `output/ansible-node.qcow2` (~470 MB) |
| **Ansible Control Node** | `Dockerfile.ansible` | `./scripts/build-ansible.sh` | `ansible-node` Docker image (~793 MB) |

The container image includes: Ansible 2.17, 10 collections (`ansible.windows`,
`cisco.asa`, `cisco.ios`, `cisco.nxos`, `community.vmware`, `community.general`,
`community.crypto`, `ansible.netcommon`, `ansible.utils`, `microsoft.sql`),
and all required Python libraries (`pywinrm`, `pyvmomi`, `pymssql`, `paramiko`,
`netmiko`, `ncclient`).

### Build & Launch

```bash
# Prerequisites: Docker, QEMU (qemu-full), passwordless sudo for mount

# 1. Build the Ansible container image
./scripts/build-ansible.sh
# Push to a registry:
REGISTRY=my-registry ./scripts/build-ansible.sh --push

# 2. Build the VM disk (optional — only needed for QEMU deployment)
./scripts/build-qemu.sh          # Alpine + Docker + SSH → qcow2

# 3. Boot the VM
./scripts/run-qemu.sh            # QEMU pc-q35-10.0, KVM, 1 GB, 2 vCPUs
```

### Interacting with the VM

```
QEMU VM boots in ~6 seconds
    │
    ├── ttyS0 (serial console) ──► TTY menu
    │   ╔══════════════════════════════════════╗
    │   ║  1) Run all tests                   ║
    │   ║  2) Run FR1 — Auth                  ║
    │   ║  3) Run FR2 — Use Control           ║
    │   ║  4) Run FR5 — Data Flow             ║
    │   ║  5) Download reports (tar.gz)       ║
    │   ║  6) View latest report              ║
    │   ║  7) Shell (Ansible container)       ║
    │   ║  8) Shell (Docker host)             ║
    │   ║  0) Shutdown                        ║
    │   ╚══════════════════════════════════════╝
    │
    ├── :8080 ──► Web UI (browser dashboard)
    │   • Run buttons per playbook
    │   • Live output streaming
    │   • Report downloads (JSON / Markdown / PDF)
    │
    └── :22 ──► SSH (ansible / ansible)
```

### Deployment Targets for the Ansible Container

| Environment | How |
|---|---|
| QEMU VM (Windows host) | `docker run ansible-node` inside the Alpine Docker Host |
| Docker Swarm | `docker stack deploy` with the `ansible-node` image |
| Kubernetes | `kubectl create job` with the `ansible-node` image |
| CI/CD pipeline | `docker run --rm -v ...` in GitHub Actions / GitLab CI |

---

## File Reference

| Path | Purpose |
|---|---|
| `playbooks/site.yml` | Entry-point for Linux targets; orchestrates FR1, FR2, FR5 suites |
| `playbooks/suites/fr1_auth.yml` | 10 FR1 authentication tests for Linux |
| `playbooks/suites/fr2_use_control.yml` | 6 FR2 use-control tests for Linux |
| `playbooks/suites/fr5_data_flow.yml` | 4 FR5 data-flow tests for Linux |
| `playbooks/library/report.yml` | Aggregates `test_results[]` → JSON file on localhost |
| `playbooks/examples/linux_vm.yml` | Standalone example: Linux SSH + kernel hardening |
| `playbooks/examples/windows_server.yml` | Standalone example: Windows Server via WinRM |
| `playbooks/examples/windows_client.yml` | Standalone example: Windows Client / HMI workstation |
| `playbooks/examples/mssql_server.yml` | Standalone example: SQL Server via WinRM + Invoke-Sqlcmd |
| `playbooks/examples/vmware_vsphere.yml` | Standalone example: ESXi via vSphere API |
| `playbooks/examples/hyperv_cluster.yml` | Standalone example: Hyper-V via WinRM |
| `playbooks/examples/cisco_switch.yml` | Standalone example: Cisco IOS via network_cli |
| `playbooks/examples/cisco_firewall.yml` | Standalone example: Cisco ASA via network_cli |
| `playbooks/templates/test_automated.yml` | Copy-and-fill template: shell gather → evaluate |
| `playbooks/templates/test_file_check.yml` | Copy-and-fill template: `ansible.builtin.stat` |
| `playbooks/templates/test_service_check.yml` | Copy-and-fill template: `service_facts` |
| `playbooks/templates/test_hitl.yml` | Copy-and-fill template: `pause` + `delegate_to: localhost` |
| `reports/render_report.py` | Python renderer: terminal box-drawing and Markdown output |
| `reports/render.go` | Go renderer: `text/template` on external `.gohtml` |
| `reports/report.gohtml` | Go template producing the terminal box-drawing report |
| `reports/go.mod` | Go module (requires `golang.org/x/text`) |
| `reports/sample-output.json` | Hand-crafted example report for offline renderer testing |
| `inventory.ini` | Ansible inventory with all platform groups and connection vars |
| `run.sh` | End-to-end wrapper: run ansible → find latest JSON → render |
| `Dockerfile.ansible` | Ansible control node image (all collections + Python libs) |
| `Dockerfile.alpine-host` | Alpine VM image (Docker daemon + TTY menu + web UI) |
| `scripts/build-ansible.sh` | Builds `ansible-node` Docker image |
| `scripts/build-qemu.sh` | Converts `Dockerfile.alpine-host` → bootable qcow2 |
| `scripts/run-qemu.sh` | Launches the Alpine VM in QEMU |
| `scripts/tty-menu.sh` | Serial console menu (launched by `agetty -l` on ttyS0) |
| `webui/container-app.py` | Web UI inside the Ansible container (JSON/Markdown/PDF export) |
| `webui/app.py` | Web UI for the Alpine Docker Host VM |
| `scripts/entrypoint.sh` | Container entrypoint: web UI if no args, else `ansible-playbook` |

---

## Design Decisions & Tradeoffs

| Decision | Rationale |
|---|---|
| **Ansible** over dedicated scanners | Already deployed in most OT environments. No new agent, no new approval process. |
| **Shell-based checks** for Linux | `shell` module is the most flexible. `changed_when: false` keeps runs clean. |
| **Native modules** for Windows/Cisco | `win_security_policy`, `win_service_info`, `ios_command`, `vmware_host_lockdown_info` — typed return values avoid brittle text parsing. |
| **`delegate_to: localhost` for VMware** | vSphere API is consumed from the control node; no SSH to ESXi. |
| **`pause` for HITL** | Ansible-native, no custom tooling. `delegate_to: localhost` ensures the prompt always reaches the operator regardless of the remote target. |
| **Inline `set_fact`** vs custom module | Custom modules require Python on the control node. Inline facts work everywhere and are easier to audit. |
| **`test_results[]` list** vs file-per-test | A single growing list is simpler than per-file concatenation. At 100+ tests the memory footprint is negligible. |
| **JSON as canonical output** | Machine-readable, schema-validatable, ingestible by SIEM/SOAR/Jira/ServiceNow. |
| **Go templates for rendering** | `text/template` supports external template files so reports can be restyled without modifying Go code. |

### Known Limitations

1. **Shell-heavy for Linux**: Linux checks depend on shell commands. Different
   distros may use different paths or tools. Mitigate with
   `when: ansible_os_family == 'Debian'` variants.

2. **No diff / drift detection**: Each run is independent. To detect configuration
   drift between runs, diff two JSON reports externally (`jd`, `diff`,
   or a time-series database).

3. **No CI exit code**: `ansible-playbook` exits 0 unless a task fails without
   `ignore_errors`. For pipeline gates, parse `summary.failed` from the JSON
   report and exit non-zero if `> 0`.

4. **HITL tests block automation**: Any playbook containing HITL tests requires
   an interactive terminal. Keep HITL tests in separate playbooks or use Ansible
   tags to separate them from fully-automated runs.

5. **Scalability at 500+ targets**: Multi-host runs work well, but one JSON file
   per host can be unwieldy. Consider post-processing into a single aggregated
   report.

---

## Comparison to Alternatives

| Tool | Type | Pros | Cons |
|---|---|---|---|
| **Inspec** | Ruby DSL, Chef ecosystem | Rich compliance profiles, CIS/STIG built-in | Ruby runtime; less common in OT |
| **Goss** | YAML config, Go binary | Fast, simple | No native IEC mapping; local checks only |
| **OpenSCAP** | XML/SCAP standard | NIST/STIG aligned, XCCDF/OVAL | Heavy, complex, US-govt focused, Linux-only |
| **Lynis** | Shell script | Broad Linux coverage | Non-extensible output; Linux-only |
| **This project** | Ansible + JSON + Go/Python | Zero new agents; multi-platform; IEC 62443 mapped; HITL support | Requires Ansible; shell-dependent Linux checks |

---

## Roadmap

- [x] FR1, FR2, FR5 core suites for Linux
- [x] Multi-platform examples: Windows, MSSQL, VMware, Hyper-V, Cisco IOS, Cisco ASA
- [x] Human-in-the-Loop test pattern with `ansible.builtin.pause`
- [x] Four copy-and-fill test templates (shell, stat, service_facts, HITL)
- [ ] **FR3 suite**: File integrity (AIDE/IMA), malware scanner status, secure boot, `/tmp noexec`
- [ ] **FR4 suite**: TLS version/cipher audit, disk encryption (LUKS), SSH cipher hardening
- [ ] **FR6 suite**: rsyslog remote forwarding, auditd dispatcher, journald persistent storage
- [ ] **FR7 suite**: Disk quotas, CPU/memory limits, backup schedule verification
- [ ] **Aggregated multi-host report**: Single HTML/PDF across all inventory hosts
- [ ] **CI/CD integration**: GitHub Actions / GitLab CI pipeline with Markdown report posted as PR comment
- [ ] **CIS Benchmark dual-mapping**: Each test maps to both IEC 62443-3-3 SR and CIS Benchmark control