Add memory benchmarking and arena trimming to reduce waste#254
Merged
Conversation
…ession detection Measures parse-only and parse+walk memory across Wallace, csstree, and postcss, broken down into heap vs external (ArrayBuffer) so arena overhead is visible separately. Also reports arena stats (node count, capacity, waste%, growths) to diagnose over-allocation. Run: node --expose-gc benchmark/memory.ts Save baseline: pnpm run benchmark:memory:baseline Compare branch vs baseline: pnpm run benchmark:memory Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TADcY37pEm5JGh8YJ7UuVQ
Adds CSSDataArena.trim() which shrinks the backing ArrayBuffer to count * BYTES_PER_NODE, eliminating the 50-65% of wasted pre-allocated capacity that was visible as external memory on the parse result. Before: Bootstrap parse = 3336 KB external (106k-node pre-alloc for 37k actual) After: Bootstrap parse = 1174 KB external (37k * 32 bytes, zero waste) The NODES_PER_KB heuristic is still useful for avoiding arena growths during parsing; it just no longer determines the final allocation size. Also fixes the memory benchmark's measurement loop to hold the parse result in a module-level sink across the post-operation GC. A local `void result` is insufficient because V8's JIT can shorten the variable's lifetime before the GC call runs, causing the arena buffer to be collected and the delta to read near-zero. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TADcY37pEm5JGh8YJ7UuVQ
Measured actual nodes/KB across real-world files: bootstrap 137 | bootstrap.min 166 | tailwind 157 | tailwind.min 195 | small 198 210 * 1.1 = 231 nodes/KB effective ceiling, ~16% above the observed max of 198. All tested files still parse with zero arena growths. Effect on peak arena allocation during parsing (before trim): Bootstrap: 3336 KB → 1976 KB (-41%) Tailwind: 43350 KB → 25677 KB (-41%) Retained memory after trim is unchanged (always exact node_count * 32 bytes). The constants now only govern transient peak during the parse call itself. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TADcY37pEm5JGh8YJ7UuVQ
Bundle ReportChanges will increase total bundle size by 1.06kB (0.56%) ⬆️. This is within the configured threshold ✅ Detailed changes
Affected Assets, Files, and Routes:view changes for bundle: @projectwallace/css-parser-esmAssets Changed:
Files in
Files in
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #254 +/- ##
==========================================
+ Coverage 93.20% 93.22% +0.01%
==========================================
Files 17 17
Lines 3062 3071 +9
Branches 853 854 +1
==========================================
+ Hits 2854 2863 +9
Misses 208 208 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Fixes oxlint "Do not construct dirname" error in benchmark/memory.ts. Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds comprehensive memory diagnostics and optimizes arena memory usage by implementing a trim operation that releases unused pre-allocated capacity after parsing completes.
Key Changes
Memory Benchmarking Tool (
benchmark/memory.ts): New diagnostic script that measures and compares memory usage across Wallace, css-tree, and PostCSS parsers. Supports baseline comparison to detect regressions with configurable thresholds.Arena Trimming (
src/arena.ts):trim()method to shrink the arena buffer to exactly match the number of live nodes, releasing wasted pre-allocated capacityparse()to automatically optimize memory after parsing completesNODES_PER_KBfrom 325 to 210 based on real-world measurements across Bootstrap, Tailwind, and custom CSS filesCAPACITY_BUFFERfrom 1.2 to 1.1 for tighter pre-allocation (effective ceiling: 231 nodes/KB)Test Updates (
src/arena.test.ts):trim()method covering normal operation, data preservation, and edge casesBuild Configuration: Added
benchmark:memorynpm script and updated.gitignoreto exclude benchmark resultsImplementation Details
The trim operation safely shrinks the ArrayBuffer by copying live data to a new, smaller buffer. It's a no-op when capacity already equals count, making it safe to call multiple times. The memory benchmarking tool uses Node's
--expose-gcflag to force garbage collection between measurements, ensuring accurate heap snapshots and enabling regression detection against saved baselines.https://claude.ai/code/session_01TADcY37pEm5JGh8YJ7UuVQ