TestRunner: registration and sequential execution

charles commented

2026-04-11 10:55:38 +00:00

Owner

User story

As a test author, I want to declare tests with closures and run them all with one call, so that I can build a suite without per-test boilerplate.

Acceptance criteria

Registration

TestRunner::new(harness: ProcessHarness) takes ownership of a built harness.
runner.test(name, |client| async move { ... }) registers a TestCase and returns &mut TestCase for chaining (.tag("backend").parallel(), etc.).
The closure receives Arc<RpcClient>, returns impl Future<Output = TestResult>, and is 'static + Send.

Run

runner.run() -> Result<TestReport, TestError> (async):
1. Starts the harness (waits for health).
2. Connects the RpcClient.
3. Executes each registered test sequentially.
4. Prints result lines as they complete.
5. Stops the harness.
6. Returns the TestReport.
If the harness fails to start or the client fails to connect, the runner prints the error with the stderr tail and returns early; no tests run.

Per-test outcome

Ok(()) → passed.
Err(TestError::Skip(_)) → skipped (with reason recorded).
Anything else → failed.
Each test runs with a default per-test timeout (RunConfig.timeout, default 60 s); exceeding it produces TestError::Timeout { event: "test", duration }.

Report

TestReport { tests: Vec<TestCaseResult>, passed, failed, skipped, filtered, duration }.
TestCaseResult { name, tags, status, duration, error: Option<TestError> }.
Per-test timing is collected.

Out of scope

Hooks (separate issue), filtering / fail-fast (separate issue), parallel mode (separate issue).

References

Spec §4.1, §4.2, §4.4, §6 (Test execution flow), §3.3 (TestRunner)

## User story As a **test author**, I want to declare tests with closures and run them all with one call, so that I can build a suite without per-test boilerplate. ## Acceptance criteria ### Registration - [ ] `TestRunner::new(harness: ProcessHarness)` takes ownership of a built harness. - [ ] `runner.test(name, |client| async move { ... })` registers a `TestCase` and returns `&mut TestCase` for chaining (`.tag("backend").parallel()`, etc.). - [ ] The closure receives `Arc<RpcClient>`, returns `impl Future<Output = TestResult>`, and is `'static + Send`. ### Run - [ ] `runner.run() -> Result<TestReport, TestError>` (async): 1. Starts the harness (waits for health). 2. Connects the `RpcClient`. 3. Executes each registered test sequentially. 4. Prints result lines as they complete. 5. Stops the harness. 6. Returns the `TestReport`. - [ ] If the harness fails to start or the client fails to connect, the runner prints the error with the stderr tail and returns early; no tests run. ### Per-test outcome - [ ] `Ok(())` → passed. - [ ] `Err(TestError::Skip(_))` → skipped (with reason recorded). - [ ] Anything else → failed. - [ ] Each test runs with a default per-test timeout (`RunConfig.timeout`, default 60 s); exceeding it produces `TestError::Timeout { event: "test", duration }`. ### Report - [ ] `TestReport { tests: Vec<TestCaseResult>, passed, failed, skipped, filtered, duration }`. - [ ] `TestCaseResult { name, tags, status, duration, error: Option<TestError> }`. - [ ] Per-test timing is collected. ## Out of scope - Hooks (separate issue), filtering / fail-fast (separate issue), parallel mode (separate issue). ## References - Spec §4.1, §4.2, §4.4, §6 (Test execution flow), §3.3 (TestRunner)

charles added this to the v0.1.0 milestone

2026-04-11 10:55:38 +00:00

charles added the

area:runner

type:user-story

labels

2026-04-11 10:55:38 +00:00

charles referenced this issue from a commit

2026-04-11 12:40:53 +00:00

feat(runner): registration + sequential execution (closes #9)

charles referenced this issue from a pull request that will close it,

2026-04-11 12:41:20 +00:00