Skip to content
This repository was archived by the owner on Jun 16, 2026. It is now read-only.

⚡ Bolt: [performance improvement] Optimize dynamic SQL generation string formatting#320

Open
bashandbone wants to merge 1 commit into
mainfrom
bolt-d1-sql-optimization-5632950206202982987
Open

⚡ Bolt: [performance improvement] Optimize dynamic SQL generation string formatting#320
bashandbone wants to merge 1 commit into
mainfrom
bolt-d1-sql-optimization-5632950206202982987

Conversation

@bashandbone

@bashandbone bashandbone commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

💡 What:
Replaced intermediate Vec<String> accumulations and format! / .join(", ") calls with direct string generation using String::with_capacity and std::fmt::Write in build_upsert_stmt and build_delete_stmt within crates/flow/src/targets/d1.rs.

🎯 Why:
To construct dynamic SQL efficiently without triggering multiple O(n) heap allocations during string fragment generation. D1 adapter methods execute dynamically heavily and allocations impact end-to-end traversal/export time heavily.

📊 Impact:
Significantly reduces transient heap allocation counts, garbage collection sweeps, and memory copying overhead for every target synchronization query executed. Eliminates temporary memory arrays.

🔬 Measurement:
Ensure the tests from cargo test -p thread-flow --test d1_target_tests --test d1_minimal_tests pass successfully with zero performance regressions or feature changes.


PR created automatically by Jules for task 5632950206202982987 started by @bashandbone

Summary by Sourcery

Optimize dynamic SQL generation and apply minor API and documentation cleanups across rule and AST components.

Enhancements:

  • Streamline D1 upsert and delete SQL construction to avoid intermediate string allocations and reduce heap churn.
  • Simplify rule-engine variable checking function signatures by removing unnecessary lifetime annotations.
  • Tidy up string handling and formatting in AST engine and rule-engine utilities for improved clarity and consistency.

Documentation:

  • Extend internal Bolt performance notes with guidance on efficient dynamic SQL string construction in database adapters.

- Pre-allocate strings using `String::with_capacity`
- Replace `vec![]` allocations, format macros, and joins with `std::fmt::Write` and manual string concatenation in `build_upsert_stmt` and `build_delete_stmt`
- Minor cleanups and formatting using nightly Rust fmt

Co-authored-by: bashandbone <89049923+bashandbone@users.noreply.github.com>
@google-labs-jules

Copy link
Copy Markdown
Contributor

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.


For security, I will only act on instructions from the user who triggered this task.

@sourcery-ai

sourcery-ai Bot commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Reviewer's Guide

Optimizes dynamic SQL generation in the D1 export context to reduce allocations, performs minor signature cleanups in the rule-engine, and applies small style/formatting adjustments across a few modules, plus documents the performance pattern in the Bolt guide.

File-Level Changes

Change Details Files
Optimize INSERT/UPSERT SQL generation to avoid intermediate string allocations and vectors in D1ExportContext::build_upsert_stmt.
  • Replace Vec column/placeholder/update clause accumulation with direct writes into a preallocated String using String::with_capacity and std::fmt::Write.
  • Pre-size params Vec based on key/value schema lengths to minimize reallocations.
  • Manually generate the VALUES placeholder list and ON CONFLICT UPDATE clause using in-place string concatenation and simple boolean flags for comma handling.
crates/flow/src/targets/d1.rs
Optimize DELETE SQL generation in D1ExportContext::build_delete_stmt to build the WHERE clause directly into a preallocated String.
  • Replace where_clauses Vec and format!/join pattern with a single String::with_capacity and write!-based appends.
  • Use a boolean flag to handle "AND" separator insertion while iterating key fields.
  • Pre-size the params Vec based on key_fields_schema length.
crates/flow/src/targets/d1.rs
Simplify lifetimes in rule-engine variable checking helpers by using non-lifetime-borrowed references for constraints and transform parameters.
  • Update function signatures in check_var.rs to accept &RapidMap and &Option without explicit lifetime parameters.
  • Propagate signature changes through helper functions check_vars_in_rewriter, check_vars, check_var_in_constraints, and check_var_in_transform.
crates/rule-engine/src/check_var.rs
Apply small readability and style cleanups in several modules.
  • Reformat a from_utf8 unwrap_or_else call onto a single line in tree_sitter::ContentExt implementation.
  • Expand a test assert_eq! over multiple lines for clarity in tree-sitter tests.
  • Format the Rule::Pattern defined_vars() mapping chain across multiple lines for readability.
  • Inline a referent_rule::Registration::read() lock unwrap chain into a single line.
crates/ast-engine/src/tree_sitter/mod.rs
crates/rule-engine/src/rule/mod.rs
crates/rule-engine/src/rule/referent_rule.rs
Document the dynamic SQL string-generation performance pattern in the Bolt engineering notes.
  • Add a new section describing why Vec + format!/join is inefficient for highly called SQL generators.
  • Recommend using String::with_capacity and write!-based appends for dynamic string building in performance-critical paths.
.jules/bolt.md

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've reviewed your changes and they look great!


Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant