Phase 1 / Etch / Test-runner + test identity#44
Merged
Conversation
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
M1.0.15 E1. Separate `test_symbols` namespace so `test "Foo"` no longer collides with `component Foo` (E0101, contextual "duplicate test 'X'" for intra-namespace dups). `TestDecl` keeps its `@tag`/`@skip`/`@only` annotation range (was discarded); applicability rides `validateAnnotations` with a new `.test_` target, args validated (reusing E0502, no new code). Test bodies are type-checked as a sync context (`await` -> E0901); the `in_test_body` flag gates the test-scoped builtins landing in E3/E4. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
M1.0.15 E2. `test_runner.zig`: iterate a type-checked program's `test` blocks in declaration order, each in a fresh World + Interpreter (full isolation), honouring `@skip`/`@only`, into a `RunReport` (per-test name/status/duration/message/span + passed/failed/skipped aggregate; owns its strings). `interp.runTestBody` runs one body sync-to-completion and converts a failed assert / uncaught throw / runtime failure into a `.fail` outcome (span + message), not an aggregated count. The `assert` statement now raises `fail(.AssertFailed, span)` with its literal message (`RuntimeErrorKind.AssertFailed`, `pending_message`, owned `test_msg_buf` for E4 formatted messages). Runner + report types exported from root.zig. Absorbs the E1-review nit: drop the now-dead `SymbolKind.test_` variant and fix the stale buildExports comment. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
M1.0.15 E3. `Value.world_handle` + `ResolvedType.test_world`: `test_world()`
(test-body-gated builtin) returns the mono-world handle; `spawn_with([C {…}])`
spawns immediately and returns a live Entity (shares the observer-firing
primitive `world.spawnWithObservers`, factored out of applyWithObservers's
.spawn arm), `emit(T {…})` mirrors the emit statement (shared enqueueEvent),
`tick(n)` advances n ticks. `world.emit(...)` parses via a new `.kw_emit`
postfix arm (emit is a keyword). `stepOnce`'s head event-clear is suppressed
for a test's first tick so `emit; tick(1)` delivers the event (§32); the store
is cleared at the end of tick(). Test bodies get unrestricted ECS access.
Also fixes two latent defects the E3 test wire-in exposed: test_runner.zig's
inline tests were never collected (missing from root.zig's comptime import
guard, §13), and `std.time.Timer` does not exist in Zig 0.16 — per-test
duration now uses `std.Io.Clock` with a threaded `io`.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
M1.0.15 E3 review fix. A `test` body outlives the rule/guard/timer/observer/ hook bodies that `world.tick(n)` drives; those bodies' per-body resets of the shared value stores (collections/closures/structs/optionals/run-strings) freed heap-backed values the test's locals still held — a use-after-free for a `.string_run` (concat) held across a tick (the M1.0.14 class), and stale/OOB handles for collections/structs/closures. Route every per-body reset through a single `resetBodyStores()` gated by `suppress_body_store_resets`, which `runTestBody` holds for the whole body: the driven bodies accumulate into the shared stores (test-scale, bounded) and the test's own raw end-defers free everything at once. No production-path change (flag inactive outside a test). Two regression tests (collection + string_run held across tick(1)) abort under testing.allocator with the guard neutered, pass with it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
M1.0.15 E4. Adds the §19.4 assertion builtins (assert_eq/assert_neq/
assert_approx (tol 1e-6)/assert_some/assert_none) + panic/todo/unreachable
(global) and the test-scoped tick_until(pred: () -> bool, timeout) -> bool,
dispatched via synthBuiltinCall (types) + evalBuiltinCall (interp) which folds
the E3 test_world. The `measure { block }` expression (new kw_measure token,
ExprKind.measure_expr, E0910 MeasureOutsideTest, interp eval via std.Io.Clock)
returns wall-clock Duration, test-body only. Duration ordering added to
binaryCompare (the checker already typed it). Assert failures format both
values into test_msg_buf.
Includes 5 fixes from an adversarial review of this diff: harvestError clears
the stale pending_message; test_world/tick_until are gated by a runtime
in_test_body flag (checker/interp parity); assert_eq/neq compare strings by
bytes (eventValueEql) and reject non-comparable aggregate operands at
type-check; a throwing tick_until predicate stops the loop and surfaces.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
M1.0.15 E4 review fix. assert_approx only synth'd its args, so assert_approx(1, 2) type-checked clean then failed loud at runtime — asymmetric with the rest of the assert family, which is fail-loud at type-check. synthBuiltinCall now requires a, b, and the optional tolerance to be float (isFloatType), emitting type_mismatch otherwise; the interp float check stays as a defensive belt. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
M1.0.15 E5. tools/etch_test — a thin CLI shim over weld_etch.test_runner (etch_cook pattern: arg parsing + I/O + report printing only): parse -> type-check -> run each .etch file, print per-test ✓/✗/skipped lines with durations + aggregates, clean nonzero exit on any diagnostic/failure. build.zig adds the etch_test exe + a `test-etch` step (Zig acceptance driver + the shim over the green fixtures in one invocation); the driver also joins `zig build test`, and CI runs `zig build test-etch` per leg. Acceptance corpus tests/etch/test_runner/: .etch fixtures (green/failing/only/isolation/world/ timing) embedded + inline diagnostic cases in driver_test.zig — the 13 acceptance tests under testing.allocator. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
§3.6.1 drift audit: the kw_test token comment still said "parse + validate only, no execution" — tests now run end-to-end via the M1.0.15 runner. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Milestone M1.0.15 — Etch test-runner + test identity. Brief:
briefs/M1.0.15-test-runner.md.test "X" { }blocks now execute end-to-end (parse + validate + register existed since M1.0.8; zero execution surface until now), closing the M1.0.15 line of the C1.6 gap plan. Delivered gate-by-gate E1→E5.Closing notes
What worked
compile()is the full type-registration point, so "fresh World + Interpreter per test" fell out cleanly; no gap inbindToWorld.spawn_withshares the observer-firing spawn path (spawnWithObserversfactored from the deferred.spawnflush);emitmethod sharesenqueueEvent; assert string comparison reuseseventValueEql; E0101 reused for duplicate tests (no new code).assert_neqfalse-PASS on strings) before they shipped.What deviated from the original spec / brief plan
suppress_body_store_resets(E3 UAF) andin_test_bodyruntime flag (E4 test-scoped-builtin gate).observers.zig/world.zigspawnWithObservers) required by the brief's own "reuse the same application primitive" trap — one spawn path, no flush behavior change.test-etchruns the shim over the green fixtures in ONE invocation (clean un-interleaved ✓ output) rather than one Run step per fixture.What to flag explicitly in review
emit; tick(1):stepOnceclears the event store at its head, sotick/tick_untilset a one-shotsuppress_event_clearto let a pre-tick event reach the first tick (§32). Non-test paths never set it.assert_eq/assert_neqare intentionally MORE capable than==(which rejects strings at type-check): they compare strings by bytes and reject aggregates — a test-only Eq surface, soassert_neqcannot silently false-PASS.hunt:semanticsadversarial agent stalled (never returned); the other three lenses + inline/driver tests cover measure/tick_until/Duration/asserts.Final measurements — no perf gate this milestone.
zig build testgreen debug + ReleaseSafe;zig build test-etchprintstotal: 10 passed / 0 failed / 2 skipped(exit 0);measurearound a 5000-iter interp loop reports ~5 ms wall-clock. New diagnostic: E0910 (sole new code).Residual risks / tech debt (intentional)
tick(n)/tick_untilinside a test grows (per-body store resets suppressed) — bounded by the test body's end (freed en bloc); test-scale, acked at the E3 STOP.test_world()is mono-world (spec'd v0.6 restriction, not debt); multi-world, async test bodies, theweldCLI proper, and filtering flags are out per §32.Validation (Step 4)
driver_test.zig+ inline suites).zig build test-etchprints ✓/skip lines + aggregate (exit 0);etch_test failing.etchprints the ✗ line +file:line+ exit 1.zig build,zig build test,zig build test-etch,zig fmt --check,zig build lintall green.CLAUDE.mdupdated (§3.4 current-state table + Tags row + open-decisions bullet) — commitdocs(claude-md): update for M1.0.15..lematch is the Zig<=enum tag).kw_testcomment refreshed; no residualSymbolKind.test_(removed E2);.test_matches areAnnotTarget.test_.Status: CLOSED,Closed: 2026-07-05; final commitdocs(brief): close M1.0.15.Merge + tag (
v0.10.15-test-runner) + close decision are Guy's.🤖 Generated with Claude Code