Self-tests with in-process mock WebSocket JSON-RPC server #16

New issue

Closed

opened 2026-04-11 10:55:40 +00:00 by charles · 0 comments

charles commented

2026-04-11 10:55:40 +00:00

Owner

User story

As a framework maintainer, I want to run the framework against itself without depending on a real application binary, so that CI is fast, hermetic, and doesn't drift when external apps change.

Acceptance criteria

Mock server

tests/mock_server.rs (or src/testing/mock.rs behind #[cfg(any(test, feature = "testing"))]) exposes a MockServer that:
- Listens on an ephemeral port (port 0, real port read from the bound socket).
- Speaks JSON-RPC 2.0 over WebSocket.
- Supports on(method, |params| -> Result<Value, RpcError>) to register handlers.
- Supports emit(method, params) to push notifications to all connected clients.
- Supports configurable per-method delay before responding.
- Tracks the last subscribe filter per client.
- Has a clean shutdown() method.

Self-tests (`tests/framework_tests.rs`)

Harness: spawning a dummy binary that exits immediately produces a useful error; spawning the mock-server-as-binary works end-to-end.
Client: every public method on RpcClient has at least one test:
- call happy path and JSON-RPC error path
- call_timeout actually times out
- subscribe filters work (replace semantics)
- wait_for matches a buffered notification
- wait_for matches a fresh notification
- wait_for times out cleanly
- wait_for_any returns the matched method
- collect_events collects exactly the events emitted in the window
- call_and_wait works without race when the event is emitted before the call response
- Notification buffer overflow drops oldest
Assertions: each macro has a passing and a failing case.
Runner: every hook (before_all, before_each, after_each, after_all), failure paths, fail-fast, filtering, parallel mode, skip via Skip(...) and skip_if.
Reporters: round-trip JSON and JUnit (parse the generated output and verify counts).

Performance

Full self-test suite runs under 30 seconds on a developer laptop.
Tests use ephemeral ports — no hardcoded port collisions in CI.

References

Spec §1 (Crate structure mentions tests/framework_tests.rs); §11 (future extension: WebSocket server mock — being delivered now since v0.1 needs it)

## User story As a **framework maintainer**, I want to run the framework against itself without depending on a real application binary, so that CI is fast, hermetic, and doesn't drift when external apps change. ## Acceptance criteria ### Mock server - [ ] `tests/mock_server.rs` (or `src/testing/mock.rs` behind `#[cfg(any(test, feature = "testing"))]`) exposes a `MockServer` that: - Listens on an ephemeral port (port 0, real port read from the bound socket). - Speaks JSON-RPC 2.0 over WebSocket. - Supports `on(method, |params| -> Result<Value, RpcError>)` to register handlers. - Supports `emit(method, params)` to push notifications to all connected clients. - Supports configurable per-method delay before responding. - Tracks the last subscribe filter per client. - Has a clean `shutdown()` method. ### Self-tests (`tests/framework_tests.rs`) - [ ] **Harness**: spawning a dummy binary that exits immediately produces a useful error; spawning the mock-server-as-binary works end-to-end. - [ ] **Client**: every public method on `RpcClient` has at least one test: - `call` happy path and JSON-RPC error path - `call_timeout` actually times out - `subscribe` filters work (replace semantics) - `wait_for` matches a buffered notification - `wait_for` matches a fresh notification - `wait_for` times out cleanly - `wait_for_any` returns the matched method - `collect_events` collects exactly the events emitted in the window - `call_and_wait` works without race when the event is emitted before the call response - Notification buffer overflow drops oldest - [ ] **Assertions**: each macro has a passing and a failing case. - [ ] **Runner**: every hook (before_all, before_each, after_each, after_all), failure paths, fail-fast, filtering, parallel mode, skip via `Skip(...)` and `skip_if`. - [ ] **Reporters**: round-trip JSON and JUnit (parse the generated output and verify counts). ### Performance - [ ] Full self-test suite runs under 30 seconds on a developer laptop. - [ ] Tests use ephemeral ports — no hardcoded port collisions in CI. ## References - Spec §1 (Crate structure mentions `tests/framework_tests.rs`); §11 (future extension: WebSocket server mock — being delivered now since v0.1 needs it)

charles added this to the v0.1.0 milestone

2026-04-11 10:55:40 +00:00

charles added the

area:meta

type:user-story

labels

2026-04-11 10:55:40 +00:00

charles referenced this issue

2026-04-11 12:41:20 +00:00

feat(runner): TestRunner registration + sequential execution (#9) #27

charles referenced this issue from a commit

2026-04-11 12:58:08 +00:00