97 lines
3.9 KiB
Markdown
97 lines
3.9 KiB
Markdown
# Test Automation Template
|
|
|
|
This repository is a GitLab CI template for repeatable testing of virtual and physical HLC environments. A pipeline selects an environment, runs enabled testing methodologies as Kubernetes pods on `testserv`, normalizes each tool's output, and publishes JSON, Markdown, HTML, and PDF artifacts.
|
|
|
|
## Dependencies
|
|
|
|
```mermaid
|
|
flowchart TD
|
|
U[Tester starts GitLab pipeline] --> P[GitLab CI]
|
|
A[assets.yml<br/>project and targets] --> P
|
|
V[Environment-scoped GitLab variables<br/>credentials and keys] --> P
|
|
P --> R[GitLab Kubernetes Runner]
|
|
R --> K[k3s namespace: test-automation]
|
|
K --> C[(tool-cache PVC<br/>downloaded databases)]
|
|
K --> AN[Ansible methodology pod]
|
|
K --> Z[ZAP methodology pod]
|
|
AN --> T[Linux, Windows, SQL,<br/>VMware, Hyper-V, Cisco]
|
|
Z --> W[Web applications]
|
|
AN --> N[Normalized JSON schema]
|
|
Z --> N
|
|
N --> O[Markdown, HTML, PDF]
|
|
O --> G[GitLab pipeline artifacts]
|
|
```
|
|
|
|
Required infrastructure:
|
|
|
|
- GitLab project and Kubernetes-executor runner on `testserv`.
|
|
- k3s namespace, RBAC, and cache PVC from `platform/kubernetes/`.
|
|
- Runner configuration from `platform/gitlab-runner/values.example.yml`.
|
|
- Container registry containing the Ansible image.
|
|
- Network access from k3s pods to the selected test environment.
|
|
- Environment-scoped GitLab CI/CD variables for credentials.
|
|
|
|
## Start A Test
|
|
|
|
1. Define project metadata and targets in `assets.yml`.
|
|
2. In GitLab, create protected and masked CI/CD variables with an environment scope matching `all.vars.test_project.environment`.
|
|
3. Start a pipeline and set:
|
|
|
|
| Variable | Purpose |
|
|
| --- | --- |
|
|
| `TARGET_ENVIRONMENT` | GitLab environment scope; must match `assets.yml` |
|
|
| `RUN_ANSIBLE=true` | Enable infrastructure tests |
|
|
| `RUN_ZAP=true` | Enable web tests after the ZAP methodology is implemented |
|
|
| `ANSIBLE_LIMIT` | Optional Ansible host/group limit; defaults to `all` |
|
|
|
|
The pipeline validates that `TARGET_ENVIRONMENT` matches `assets.yml`. A mismatch stops before any testing begins.
|
|
|
|
## Secrets
|
|
|
|
The Ansible job requires `ANSIBLE_SECRET_VARS` as an environment-scoped GitLab **File** variable containing Ansible variables. SSH keys may be supplied as `ANSIBLE_PRIVATE_KEY_FILE`, also as a File variable.
|
|
|
|
Validation, normalization, and report jobs do not declare a GitLab environment and therefore do not receive environment-scoped credentials. See [docs/secrets.md](docs/secrets.md) for variable examples and rotation guidance.
|
|
|
|
## Pipeline Flow
|
|
|
|
1. `validate`: validate `assets.yml`, compile adapters, and test the report contract.
|
|
2. `platform`: manually verify that the persistent tool cache is mounted.
|
|
3. `test`: run enabled methodology pods against selected assets.
|
|
4. `normalize`: convert native tool output to `schemas/test-report.schema.json`.
|
|
5. `report`: create Markdown, HTML, and PDF reports.
|
|
|
|
Generated files are written below `artifacts/`:
|
|
|
|
```text
|
|
artifacts/
|
|
├── raw/<methodology>/
|
|
├── normalized/
|
|
└── rendered/
|
|
```
|
|
|
|
GitLab artifacts are the authoritative test evidence. The Kubernetes PVC stores only replaceable tool databases and caches.
|
|
|
|
## Methodologies
|
|
|
|
- [Ansible](methodologies/ansible/README.md): active infrastructure and platform checks.
|
|
- [OWASP ZAP](methodologies/zap/README.md): planned web application testing with ASVS mappings.
|
|
|
|
Shared pipeline code stays at the repository root. Methodology-specific images, runners, adapters, fixtures, and references stay under `methodologies/<name>/`.
|
|
|
|
## Platform Bootstrap
|
|
|
|
Apply the Kubernetes resources once with an administrator context:
|
|
|
|
```bash
|
|
export KUBECONFIG=/etc/rancher/k3s/admin/kubeconfig.yaml
|
|
kubectl apply -k platform/kubernetes
|
|
```
|
|
|
|
Build and push the Ansible image before enabling `RUN_ANSIBLE`:
|
|
|
|
```bash
|
|
REGISTRY=<registry>/<project> ./methodologies/ansible/scripts/build-image.sh --push
|
|
```
|
|
|
|
Set the CI image reference in `.gitlab-ci.yml` or publish it as `$CI_REGISTRY_IMAGE/ansible:latest`.
|