feat(copilot): add copilot_chat_messages table with dual-write rollout

[waleedlatif1]

Summary

• Introduces dedicated copilot_chat_messages table to replace the JSONB messages array on copilot_chats. The JSONB column is preserved indefinitely as the source of truth during rollout — no breaking schema change.
• Migration 0212_cynical_jack_murdock.sql runs an inline jsonb_array_elements ... ON CONFLICT DO NOTHING backfill. bun run db:migrate does everything; no scripts required.
• Follows existing migration precedent (0091_backfill_user_stats.sql, 0192_invitation_unification.sql): multi-statement with gen_random_uuid(), FK with CASCADE, inline INSERT ... SELECT ... ON CONFLICT DO NOTHING.
• Dual-write helpers (lib/copilot/chat/messages-dual-write.ts) wired into every JSONB-mutation callsite (post, terminal-state, inbox, fork, update-messages, superuser-import). Best-effort — errors logged, never thrown.
• Catch-up sweep on server boot closes the rolling-deploy gap. Bounded 7-day window, idempotent.

Indexes

• UNIQUE (chat_id, message_id) — idempotent appends, dedup streaming retries
• (chat_id, created_at, id) WHERE deleted_at IS NULL — transcript load, last-N
• (chat_id, stream_id) WHERE stream_id IS NOT NULL — stream replay/resume

Type of Change

• New feature

Testing

Tested manually. Lint passes.

Checklist

• Code follows project style guidelines
• Self-reviewed my changes
• Tests added/updated and passing
• No new warnings introduced
• I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
docs	Ready	Preview, Comment	May 23, 2026 6:52pm

[cursor]

PR Summary

Medium Risk
Introduces a new persistence path and background backfill for copilot chat messages, touching multiple write callsites and adding a new table/migration; issues could cause partial data duplication or missing rows in the new table (legacy JSONB remains canonical).

Overview
Adds a new copilot_chat_messages table (with indexes + FK) and a migration that backfills from copilot_chats.messages JSONB via jsonb_array_elements with idempotent ON CONFLICT behavior.

Wires best-effort dual-write into existing chat message mutation paths: snapshot saves now replaceCopilotChatMessages, while streaming/append flows appendCopilotChatMessages (including mothership fork/import and inbox execution) and propagate model/streamId metadata.

Adds a non-blocking, on-boot 7-day catch-up sweep to close rolling-deploy gaps, plus updates/tests to cover the new dual-write helpers and adjusted DB .returning() behavior in update-messages route tests.

^{Reviewed by Cursor Bugbot for commit b7d172b. Configure here.}

[greptile-apps]

Greptile Summary

This PR introduces a dedicated copilot_chat_messages table to normalize the JSONB messages array currently stored on copilot_chats, using a dual-write rollout strategy where the existing JSONB column remains the source of truth throughout the transition.

• Schema & migration: 0212_cynical_jack_murdock.sql creates the table with appropriate indexes (unique on chat_id/message_id, partial indexes for transcript loads and stream resume) and runs an inline backfill from the existing JSONB column.
• Dual-write helpers: messages-dual-write.ts exposes appendCopilotChatMessages and replaceCopilotChatMessages, both fully best-effort (errors are caught and logged), wired into all JSONB-mutation callsites (post.ts, terminal-state.ts, inbox/executor.ts, fork, update-messages, import-workflow).
• Catch-up sweep: messages-catchup.ts runs once per server boot to close the rolling-deploy gap, bounded to 7 days of activity and idempotent via ON CONFLICT DO NOTHING.

Confidence Score: 5/5

Safe to merge — the dual-write is fully best-effort, the JSONB column remains the source of truth, and the migration backfill is idempotent.

All callsites correctly gate on the returned row before writing to the normalized table, preventing FK-violation noise. The catch-up sweep is fire-and-forget and bounded to 7 days. The transaction patterns in terminal-state.ts and replaceCopilotChatMessages are sound. No breaking schema changes, and all previously flagged gaps have been addressed.

No files require special attention.

Important Files Changed

Filename	Overview
apps/sim/lib/copilot/chat/messages-dual-write.ts	New dual-write helpers; appendCopilotChatMessages uses ON CONFLICT DO UPDATE to keep streaming content fresh, replaceCopilotChatMessages runs a transactional delete+upsert for full-snapshot saves. Both are best-effort with proper error handling.
apps/sim/lib/copilot/chat/messages-catchup.ts	Fire-and-forget boot-time sweep that backfills the last 7 days of JSONB messages into the normalized table via ON CONFLICT DO NOTHING. Idempotent and bounded; runs in the background so boot is not blocked.
packages/db/migrations/0212_cynical_jack_murdock.sql	Creates copilot_chat_messages with appropriate NOT NULL constraints, FK with CASCADE, three targeted indexes, and an inline INSERT ... SELECT ... ON CONFLICT DO NOTHING backfill over the full copilot_chats history.
packages/db/schema.ts	New copilotChatMessages table definition mirrors the SQL migration exactly, with unique index and two partial indexes defined inline.
apps/sim/lib/copilot/chat/terminal-state.ts	Adds model to the in-transaction SELECT projection, captures chatModel via closure, and appends the assistant message to the normalized table after the transaction commits. The appendedAssistantMessage closure variable pattern is correct.
apps/sim/lib/copilot/chat/post.ts	Adds model to the RETURNING clause and dual-writes the user message after the JSONB update succeeds, gated on the returned row. Uses streamId: userMessageId to group the user message with its turn.
apps/sim/lib/mothership/inbox/executor.ts	Adds .returning({ model }) to the JSONB update and dual-writes both user and assistant messages, gated on the returned row to avoid FK violations on missing chats.
apps/sim/app/api/copilot/chat/update-messages/route.ts	Adds .returning({ model }) and gates replaceCopilotChatMessages on the returned row, preventing spurious FK-violation logs when the target chat doesn't exist.
apps/sim/app/api/mothership/chats/[chatId]/fork/route.ts	Dual-writes forked messages immediately after the new chat row is inserted, passing the parent chat's model. Best-effort; does not block the fork response.
apps/sim/app/api/superuser/import-workflow/route.ts	Adds .returning({ id }) to the chat INSERT and dual-writes imported messages, guarded by existence and non-empty message array checks.
apps/sim/instrumentation-node.ts	Registers the catch-up sweep as a fire-and-forget task on server boot, with error catching so a sweep failure never affects startup.
apps/sim/lib/copilot/chat/messages-dual-write.test.ts	New test suite covering append (no-op on empty, row shape, ordering, options, conflict target, error-swallowing) and replace (empty snapshot, delete+upsert, model propagation, error-swallowing).

_{Reviews (3): Last reviewed commit: "fix(copilot): thread chatModel through p..." | Re-trigger Greptile}

[waleedlatif1]

@greptile

[waleedlatif1]

@cursor review

[waleedlatif1]

@greptile

[waleedlatif1]

@cursor review

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit b7d172b. Configure here.}