Reporters: JSON and JUnit XML output formats

charles commented

2026-04-11 10:55:40 +00:00

Owner

User story

As a CI maintainer, I want machine-readable test reports in JSON and JUnit XML, so that my CI system can parse pass/fail counts, post comments on PRs, and surface per-test details on the dashboard.

Acceptance criteria

Reporter trait

A Reporter trait abstracts the three output modes (Console, Json, Junit), picked from RunConfig.format: OutputFormat.
Reporters implement start_run, start_test, end_test, end_run.

JSON

--format json writes a single JSON object to stdout at the end of the run:

{
  "tests": [
    {"name": "...", "tags": [...], "status": "passed|failed|skipped",
     "duration_ms": 123, "error": null | { "type": "Assertion", ... }}
  ],
  "passed": 8, "failed": 1, "skipped": 1, "filtered": 0,
  "duration_ms": 21200,
  "started_at": "2026-04-11T10:00:00Z",
  "finished_at": "2026-04-11T10:00:21Z"
}

Stable key order (alphabetical or as listed) so output diffs cleanly.

JUnit XML

--format junit writes a <testsuites> document with one <testsuite> per run.
Each test becomes a <testcase> with <failure> (assertion errors), <error> (other errors), or <skipped> children as appropriate.
Failure messages include the full assertion diff (expected/got) in the message attribute and <failure> body.
Includes time (seconds) per testcase and time aggregate per testsuite.

General

In non-console formats, the per-test progress lines printed by the console reporter are suppressed; only the final structured document is written.
No ANSI color codes leak into JSON or XML even without --no-color.
Self-tests parse the output back and verify round-trip.

References

Spec §5.2, §5.3

## User story As a **CI maintainer**, I want machine-readable test reports in JSON and JUnit XML, so that my CI system can parse pass/fail counts, post comments on PRs, and surface per-test details on the dashboard. ## Acceptance criteria ### Reporter trait - [ ] A `Reporter` trait abstracts the three output modes (Console, Json, Junit), picked from `RunConfig.format: OutputFormat`. - [ ] Reporters implement `start_run`, `start_test`, `end_test`, `end_run`. ### JSON - [ ] `--format json` writes a single JSON object to stdout at the end of the run: ```json { "tests": [ {"name": "...", "tags": [...], "status": "passed|failed|skipped", "duration_ms": 123, "error": null | { "type": "Assertion", ... }} ], "passed": 8, "failed": 1, "skipped": 1, "filtered": 0, "duration_ms": 21200, "started_at": "2026-04-11T10:00:00Z", "finished_at": "2026-04-11T10:00:21Z" } ``` - [ ] Stable key order (alphabetical or as listed) so output diffs cleanly. ### JUnit XML - [ ] `--format junit` writes a `<testsuites>` document with one `<testsuite>` per run. - [ ] Each test becomes a `<testcase>` with `<failure>` (assertion errors), `<error>` (other errors), or `<skipped>` children as appropriate. - [ ] Failure messages include the full assertion diff (expected/got) in the `message` attribute and `<failure>` body. - [ ] Includes `time` (seconds) per testcase and `time` aggregate per testsuite. ### General - [ ] In non-console formats, the per-test progress lines printed by the console reporter are **suppressed**; only the final structured document is written. - [ ] No ANSI color codes leak into JSON or XML even without `--no-color`. - [ ] Self-tests parse the output back and verify round-trip. ## References - Spec §5.2, §5.3

charles added this to the v0.1.0 milestone

2026-04-11 10:55:40 +00:00

charles added the

area:reporting

type:user-story

labels

2026-04-11 10:55:40 +00:00

charles referenced this issue

2026-04-11 11:01:11 +00:00

Screenshot capture on test failure (GNOME, focused-window) #18

charles referenced this issue

2026-04-11 13:01:45 +00:00