Testing and debugging

This page explains how to validate evil changes and where to look when something goes wrong.

For the full raw command matrix, the repo also ships tests/testing.md. This page is the longer, tutorial-style version.

---

The test layers

evil has four useful testing layers.

1. Pure tests

These cover things like:

• canvas math
• output layout
• focus stack behavior
• move/resize helpers
• binding parsing

These are good when the change is mostly logic.

2. Headless runtime tests

These cover:

• runtime truth
• snapshots
• Lua session behavior
• config validation
• CLI behavior

These are very important because headless is the first semantic proving ground for a lot of policy work.

3. Nested live smoke tests

These cover:

• real Wayland clients on the winit backend
• live hook behavior
• layer-shell
• XWayland probes
• IPC
• screenshot flow
• output-management slices

This is the main live proving ground.

4. Manual tty checks

These are still important because the standalone backend is not yet fully covered by automated harnesses.

---

Full validation pass

A good high-confidence pass mirrors what CI runs (see .forgejo/workflows/ci.yml):

cargo fmt --check
cargo clippy --all-targets --all-features -- -D warnings
cargo test                       # default features (lua + winit)
cargo test --all-features        # adds the udev/tty backend
cargo test --no-default-features # pure-core only
cargo run --bin evil -- --check-config --config examples/tty-baseline.lua

--all-features matters: a number of tty regression tests are gated behind #[cfg(all(feature = "lua", feature = "udev"))] and would otherwise be silently compiled away. The Forgejo CI workflow runs all three feature combinations so this gap can no longer regress unnoticed.

---

Development ladder

For comparable runtime work, the project tries to follow this order:

1. headless semantics first when possible
2. nested live verification next
3. tty trials after the nested path looks believable

This matters because it keeps debugging more controlled.

---

Example-config validation

To validate the shipped examples:

cargo run --bin evil -- --check-config --config examples/tty-baseline.lua
cargo run --bin evil -- --check-config --config examples/tiling.lua

---

Live smoke tests

Run these with:

EVIL_RUN_LIVE_TESTS=1 cargo test --test live_smoke

They only run when:

• a display is available
• EVIL_RUN_LIVE_TESTS=1 is set

---

Where live test artifacts go

Artifacts are kept under:

target/test-artifacts/live-smoke/

Useful things you may find there:

• compositor stdout/stderr logs
• structured compositor events JSONL
• IPC request/response traces
• screenshots
• per-test summary JSON

This is the first place to look after a live failure.

---

Helpful env vars

EVIL_EVENT_LOG

Path for a structured JSONL event log. The log grows without bound — there is no built-in rotation. For long-running sessions, configure external rotation:

# Example logrotate snippet (/etc/logrotate.d/evil)
/tmp/evil-tty-logs/events.jsonl {
    weekly
    rotate 8
    missingok
    notifempty
    copytruncate
}

The TTY launcher (scripts/start-tty.sh) defaults to /tmp/evil-tty-logs/ when EVIL_TTY_LOG_DIR is unset.

EVIL_IPC_TRACE_DIR

Capture raw IPC request/response JSONL.

RUST_LOG

The compositor uses tracing-subscriber and respects RUST_LOG. All compositor lifecycle, render, DRM, XWayland, and tty messages now flow through tracing rather than println!/eprintln!, so something like RUST_LOG=evil=debug will widen the log without rebuilding. (User-facing CLI output from --check-config and IPC responses still goes to plain stdout.)

These are useful both for tests and for manual debugging.

---

What to test for common change types

If you changed geometry or layout math

Run:

• pure tests
• headless runtime tests if snapshot behavior changed

If you changed hook payloads or Lua API shape

Run:

• lua_session
• lua_config
• headless runtime tests
• any relevant live parity tests

If you changed live compositor behavior

Run:

• the normal test suite
• live smoke tests

If you changed tty behavior

Run:

• normal suite
• whatever tty-specific automated checks exist
• manual spare-VT validation

---

Current tty support target and release gates

The project's first non-experimental tty target is intentionally narrow:

• canonical launcher: scripts/start-tty.sh
• canonical config: baseline config adapted for udev backend
• single-seat seatd / libseat flow on a spare login VT
• single-output baseline first
• required baseline behavior: startup, focus, modifier move/resize, VT switch away/back, clean quit, and usable diagnostics

The project should keep calling tty experimental until these gates are satisfied:

Contract / docs gates

• the repo docs point to one canonical launcher/config/log path
• the supported target is documented as single-seat, spare-VT, and single-output baseline first

Automated gates

• scripts/tty-virt-smoke.sh reliably proves the supported automated baseline
• tty failures are diagnosable from captured logs and artifacts

Manual gate

• at least one real-hardware spare-VT validation pass is recorded separately

---

Useful tty evidence now includes:

• launcher log lines like tty startup phase: ...
• launcher log lines like tty session state: ...
• the final launcher exit line (compositor exited cleanly vs a non-zero exit status)
• tty harness IPC snapshots and summary JSON under target/test-artifacts/tty-virt/

---

Automated tty harness

The canonical automated tty proof path is:

scripts/tty-virt-smoke.sh --dry-run

A real run still needs the harness prerequisites and root because it uses vkms, openvt, and VT switching.

The harness now checks a narrow but deliberate baseline:

• lifecycle startup markers and clean shutdown markers
• repo-owned terminal probe spawn (Super+Return under the harness)
• click / modifier-drag move / modifier-drag resize / close evidence
• repo-owned panel (layer-shell mapping/lifecycle evidence; tty render path does not yet visually render layer-shell surfaces) + Wayland probe + X11 probe evidence
• VT switch away/back evidence
• runtime snapshots captured through compositor IPC

---

Manual spare-VT checklist (still a pending gate)

This checklist is ready to use, but the tty plan still keeps the real-hardware gate pending until an actual run is recorded.

1. switch to a spare login VT
2. launch scripts/start-tty.sh
3. verify the startup log reaches tty startup phase: ready
4. spawn a terminal or terminal-role probe with Super+Return
5. verify click-to-focus and modifier move/resize
6. verify the panel and basic probe spawn
7. switch away and back with the configured VT shortcut
8. quit with Ctrl+Alt+Backspace
9. inspect the tty log for clean shutdown markers and any tty runtime failures

---

Good debugging habit

When a behavior is surprising, ask:

• is the issue in pure logic?
• is the issue only in headless?
• is the issue only in live nested?
• is the issue only in tty?
• is the issue in the config/policy or the runtime?

That question alone often cuts the search space in half.