Skip to content

improvement(media-gen): retire vision block, add hosted key for fal ai for image/video gen, search visibility in cmd-k#4684

Open
icecrasher321 wants to merge 2 commits into
stagingfrom
feat/hosted-fal
Open

improvement(media-gen): retire vision block, add hosted key for fal ai for image/video gen, search visibility in cmd-k#4684
icecrasher321 wants to merge 2 commits into
stagingfrom
feat/hosted-fal

Conversation

@icecrasher321
Copy link
Copy Markdown
Collaborator

@icecrasher321 icecrasher321 commented May 20, 2026

Summary

  • Retire vision block since Agent just supports files now
  • Add hosted key support for image/video gen
  • Can mark subblocks to have their dropdown options be search visible

Type of Change

  • New feature

Testing

Tested manually

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel
Copy link
Copy Markdown

vercel Bot commented May 20, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped May 20, 2026 11:34pm

Request Review

@cursor
Copy link
Copy Markdown

cursor Bot commented May 20, 2026

PR Summary

Medium Risk
Medium risk because it changes hosted-key injection/billing behavior and adds new Fal.ai cost-tracking network calls that affect pricing, rate limiting, and tool responses.

Overview
Adds hosted-key support for Fal.ai image/video generation with cost tracking: the image/video proxy routes optionally fetch Fal.ai billing/price estimates and return __falaiCostDollars/__falaiBilling, while the Fal.ai tools apply a markup-based custom pricing model and send useHostedCostTracking when running on hosted keys.

Updates the hosted-key injector to skip injection when a tool’s hosting predicate is false or when the user already supplied an API key, and adds Fal.ai to BYOK provider IDs/UI.

Improves Cmd-K search by introducing commandSearchable subblocks and indexing marked dropdown/combobox options into each block’s searchValue; also hardens the add-block toolbar event typing/validation and refreshes search data when provider models change.

Retires Vision from surfaced UI/docs by removing the vision docs/integration entry and hiding vision_v2 from the toolbar, plus minor Azure DevOps docs output clarifications.

Reviewed by Cursor Bugbot for commit 3c7fd57. Configure here.

@icecrasher321 icecrasher321 changed the title improvement(media-gen): retire vision block, add hosted key for fal ai for image/video gen improvement(media-gen): retire vision block, add hosted key for fal ai for image/video gen, search visibility in cmd-k May 20, 2026
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 20, 2026

Greptile Summary

This PR retires the Vision block (hidden from toolbar), adds Fal.ai hosted-key support for image and video generators with markup-based billing, and enables provider dropdown options to appear as Cmd-K search entries via a new commandSearchable flag on SubBlockConfig.

  • Hosted-key billing for Fal.ai: getFalAICostMetadata polls the billing-events API (2 attempts, 500 ms apart) then falls back to a historical-estimate endpoint, and finally to a hard-coded floor — it never throws, so generated content is never lost due to cost-lookup failure. The markup multiplier (1.5×) is applied in each tool's getCost function.
  • Dual apiKey subblocks: both image and video generator blocks now carry two apiKey subblocks sharing the same id — one with hideWhenHosted: true scoped to the falai provider, and one for all other providers — so the block UI hides the key field in hosted mode only when Fal.ai is selected.
  • commandSearchable search indexing: provider labels and IDs from any subblock marked commandSearchable are folded into the block's searchValue at store-initialization time, letting users find blocks by provider name (e.g., "Fal.ai", "OpenAI") in Cmd-K.

Confidence Score: 5/5

Safe to merge — the cost-tracking chain is robust end-to-end and no generated content can be lost due to pricing-lookup failures.

The three-tier fallback in getFalAICostMetadata (billing events → historical estimate → floor) ensures the billing path never propagates an exception to callers. The hosted-key injection correctly checks the enabled predicate, respects user-provided and BYOK keys, and gates cost tracking via __usingHostedKey. Dual apiKey subblock IDs are an intentional, mutually-exclusive conditional pattern consistent with how other blocks handle provider-scoped fields.

The fallback-floor cost heuristic in falai-pricing.ts and the defensive throws in image/generate.ts and video/falai.ts are worth a second look if Fal.ai adds new endpoint families in the future.

Important Files Changed

Filename Overview
apps/sim/lib/tools/falai-pricing.ts New file implementing Fal.ai cost metadata lookup with billing-events API, historical-estimate fallback, and a hard-coded floor — function never throws, ensuring robust downstream usage.
apps/sim/tools/image/generate.ts Adds hosting config with enabled predicate (falai-only), cost markup logic, and forwards __falaiCostDollars/__falaiBilling from API response to getCost.
apps/sim/tools/video/falai.ts Adds identical hosted-key config as image tool (no enabled predicate needed since this tool is Fal.ai-only); forwards cost fields from response.
apps/sim/blocks/blocks/image_generator.ts Adds two mutually-exclusive apiKey subblocks (falai with hideWhenHosted, non-falai without), commandSearchable on provider dropdown, and relaxes required-key validation for falai provider.
apps/sim/blocks/blocks/video_generator.ts Same pattern as image_generator: duplicate apiKey subblocks, commandSearchable, default provider switched to falai for V2 and V3 blocks.
apps/sim/app/api/tools/image/route.ts Conditionally calls getFalAICostMetadata when useHostedCostTracking is set; result attached to response as __falaiCostDollars/__falaiBilling.
apps/sim/app/api/tools/video/route.ts Same cost-tracking integration as image route; boolean flag is explicitly coerced with === true to prevent truthy undefined.
apps/sim/stores/modals/search/store.ts Adds buildCommandSearchableOptionSearchValue to fold commandSearchable dropdown options into each block's searchValue at initialization time.
apps/sim/blocks/blocks/vision.ts Vision block retired by flipping hideFromToolbar to true; no functional logic removed.
apps/sim/tools/index.ts injectHostedKeyIfNeeded now respects the optional enabled predicate and short-circuits if the user has already supplied a key, preventing double-injection.

Sequence Diagram

sequenceDiagram
    participant Block as Image/Video Block
    participant Index as tools/index.ts
    participant Route as API Route
    participant FalAI as fal.ai
    participant Pricing as falai-pricing.ts
    participant Billing as Billing System

    Block->>Index: "execute(params, provider=falai)"
    Index->>Index: injectHostedKeyIfNeeded
    Index->>Route: POST api/tools/image or video
    Route->>FalAI: generate(model, prompt)
    FalAI-->>Route: result + requestId
    Route->>Pricing: getFalAICostMetadata
    Pricing->>FalAI: GET billing-events (2 attempts)
    alt billing event found
        FalAI-->>Pricing: cost_estimate_nano_usd
        Pricing-->>Route: "source=billing_events"
    else no event
        Pricing->>FalAI: POST pricing/estimate
        alt estimate ok
            FalAI-->>Pricing: total_cost
            Pricing-->>Route: "source=historical_estimate"
        else estimate fails
            Pricing-->>Route: "source=fallback_floor"
        end
    end
    Route-->>Index: response with costDollars
    Index->>Billing: getCost with 1.5x markup
    Billing-->>Index: billing recorded
Loading

Reviews (2): Last reviewed commit: "address comments" | Re-trigger Greptile

Comment thread apps/sim/app/api/tools/image/route.ts
Comment thread apps/sim/app/api/tools/video/route.ts
Comment thread apps/sim/lib/tools/falai-pricing.ts Outdated
Comment thread apps/sim/app/api/tools/image/route.ts
@icecrasher321
Copy link
Copy Markdown
Collaborator Author

@greptile

@icecrasher321
Copy link
Copy Markdown
Collaborator Author

bugbot run

Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 3c7fd57. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant