[comp] Production Deploy by github-actions[bot] · Pull Request #3183 · trycompai/comp

github-actions · 2026-06-17T22:27:26Z

This is an automated pull request to release the candidate branch into production, which will trigger a deployment.
It was created by the [Production PR] action.

Summary by cubic

Adds Separation of Environments checks for AWS, Azure, and GCP using a shared, token‑exact classifier. Raises the pass bar to require production plus at least one non‑production environment and tightens coverage/verification behavior.

New Features
- AWS: evaluates non‑default, available VPCs by Environment/Name tag; surfaces per‑region read failures with guidance.
- Azure: two‑tier check (subscriptions → strong, resource groups → logical); no cross‑tier union; records scope/coverage in evidence.
- GCP: evaluates active projects (paged); classifies by environment/env labels then name; honors project_ids scope.
Bug Fixes
- Shared classifier: treats non-prod/nonprod/not-prod as non‑production and pre-prod as staging; token‑exact across any separator; env‑tag keys prioritized (environment > env > stage > tier).
- Pass criteria: requires production plus a non‑production environment across AWS, Azure, and GCP (prevents dev+staging false passes).
- Verdicts and coverage: Azure marks subscription‑name read failures and resource‑group pagination caps as “could not verify” (evidence includes coverage gaps); AWS suppresses regional “could not verify” when separation is already confirmed; GCP propagates discovery truncation/read gaps as “could not verify.”

^{Written for commit 4619574. Summary will update on new commits.}

The Separation of Environments evidence task had no check for ANY provider, so its integration picker was empty ("can't connect to any cloud provider"). Add a heuristic GCP check (→ separationOfEnvironments): it lists all accessible projects and classifies each into an environment (by `environment`/`env` label, else a name/id token matched with word boundaries so "product"/"developer" do NOT false-match "prod"/"dev"), and passes when ≥2 distinct environments are detected. Separation is an architectural property with no single API field, so the check is evidence-first: when it cannot confirm separation (single env, unclassified, or no projects) it emits actionable guidance (label projects, or upload a diagram) rather than a silent pass. It evaluates the whole project footprint, ignoring the project_ids scoping variable. Tagged service 'iam' (governance) to satisfy the manifest-service-tags invariant. Runs on both the manual and scheduled paths like the other GCP checks. Adds 11 tests (classifier word-boundary precision + check behavior). GCP coverage: 6 → 7 task templates (8 → 9 checks). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

… + shared env classifier Brings AWS and Azure to parity with GCP on the Separation of Environments task (previously empty for ALL providers). Also extracts a shared, token-exact environment classifier and fixes an underscore bug the review caught. - New manifests/environment-classification.ts: TOKEN-split (not substring) classification so "production"/"product" and "dev"/"developer" never collide AND any separator works (-, _, ., /). Fixes a latent bug where _-separated names (myapp_prod) classified to null under the previous \b-regex. Reads only env-key tag/label values (never arbitrary tags) so a stray tag can't fabricate an environment. - GCP env-separation refactored onto the shared classifier (gets the underscore fix). - AWS aws-environment-separation (service ec2-vpc): classifies non-default, available VPCs by Environment/Name tag across regions; passes on >=2 distinct environments. Account-per-environment is invisible from one connection (no Organizations access), so a single-account result fails (severity LOW) with guidance to connect each env account or upload a diagram. PASS wording scopes the claim to within-account network labeling, NOT cross-account isolation. - Azure azure-environment-separation (service policy): two-tier, evaluated PER TIER (never unioned) — subscriptions (real boundary) first, then resource groups (logical, disclosed as such). Footprint-wide (lists all enabled subs). All branches fail-with-guidance, never silent pass; read failures surface as "could not verify"; the task accepts manual evidence. +25 tests (shared classifier incl. underscore/preprod/product, AWS pure evaluators, Azure run() incl. the no-cross-tier-union safety case). 371 package tests pass; typecheck + build green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…n scan cap Addresses two review findings: - P1 (envTagValues ordering): values were returned in tag-map insertion order instead of ENV_TAG_KEYS priority order, so a less-authoritative key (stage=dev) could outrank a more-authoritative one (environment=prod) and misclassify. Now iterates the env keys in priority order via a lowercased lookup map, so `environment` always wins over `stage`/`tier`. +test. - P2 (Azure RG scan cap): the resource-group tier silently bounded to 50 subscriptions. Now the cap is surfaced as explicit coverage info — the fail verdict flips to "could not verify", names "only N of M enabled subscriptions were scanned", carries enabledSubscriptions vs subscriptionsScannedForResourceGroups in evidence, and the remediation tells the user to reduce scope. (A >=2-env PASS is not invalidated by truncation — scanning more subscriptions can only add environments — so the scanned count is recorded in pass evidence for transparency rather than failing a valid pass.) +test (60 enabled subscriptions). 373 package tests pass; typecheck + build green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…ironments feat(integrations): Separation of Environments checks for GCP, AWS & Azure

vercel · 2026-06-17T22:27:33Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
app (staging)	Ready	Preview, Comment	Jun 18, 2026 12:05am
comp-framework-editor (staging)	Ready	Preview, Comment	Jun 18, 2026 12:05am

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
portal (staging)	Skipped		Jun 18, 2026 12:05am

cubic-dev-ai

3 issues found across 14 files

Confidence score: 3/5

In packages/integration-platform/src/manifests/azure/checks/environment-separation.ts, the check ignores configured subscription scope and scans all enabled subscriptions, which can generate out-of-scope findings and silently broaden customer coverage beyond opt-in expectations — enforce the configured subscription filter before merging.
In packages/integration-platform/src/manifests/gcp/checks/environment-separation.ts, the pass condition is weaker than the control intent, so environments can pass without confirming both production and non-production separation; that risks false compliance results — require production plus at least one non-production environment for a pass.
In packages/integration-platform/src/manifests/gcp/checks/environment-separation.ts, the pagination cap can evaluate only part of the footprint but still return a normal verdict, which may hide incomplete verification from users — treat capped discovery as "could not verify" (or propagate truncation) before merging.

_{Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic}

…od bar, truncation Fixes the 3 issues cubic flagged on the release PR (all verified real against the code), plus a related consistency fix: 1. Azure ignored the configured subscription scope and scanned ALL enabled subscriptions, breaking the opt-in scan contract. Now scopes via resolveAzureSubscriptionIds (selection → legacy → first enabled; bounds the fan-out and surfaces over-limit/empty scope as its own finding). Both the per-subscription name read and the resource-group list touch ONLY in-scope subscriptions. 2. Pass condition was too weak (>=2 of any environments could pass on dev + staging without confirming production). New shared confirmsEnvironmentSeparation requires PRODUCTION plus at least one NON-PRODUCTION environment; applied to GCP, AWS and Azure. 3. GCP pagination cap evaluated a partial footprint but returned a normal verdict. Discovery truncation now propagates: a non-confirmed verdict under truncation becomes "could not verify" (evidence.discoveryTruncated). A confirmed pass still stands (scanning more projects can only ADD environments, never remove the prod/non-prod already found). 4. (Consistency, same class as #1) GCP now honors the project_ids opt-in scope — fetches the selected projects directly instead of always listing all. All branches fail-with-guidance, never silent pass; read failures → "could not verify"; the task accepts manual evidence. Adversarially re-verified (no new false-pass, no regressions). 382 package tests pass; typecheck + build green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

fix(integrations): env-separation review fixes — subscription scope, prod+non-prod pass bar, truncation

tofikwest · 2026-06-17T23:10:15Z

@cubic-dev-ai review it

cubic-dev-ai · 2026-06-17T23:10:22Z

@cubic-dev-ai review it

@tofikwest I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

3 issues found across 14 files

Confidence score: 3/5

packages/integration-platform/src/manifests/environment-classification.ts can misclassify labels like non-prod as production because of token-level prod matching, which can flip environment-separation checks to incorrect pass/fail outcomes; tighten matching rules (e.g., exact/word-boundary semantics) and add a non-prod regression test before merging.
packages/integration-platform/src/manifests/azure/checks/environment-separation.ts does not carry subscription read failures into the final "could not verify" result, so the check may return a confident non-separation verdict with incomplete data; propagate read errors into the final decision path so partial coverage cannot produce a definitive fail.
packages/integration-platform/src/manifests/aws/checks/environment-separation.ts always emits a fail on region read errors even when gathered VPC evidence already proves separation, which can produce conflicting outcomes in one run; gate the read-failure fail to non-pass states (or short-circuit after fail) to keep results consistent.

_{Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic}

…verdicts Addresses three correctness issues flagged in review of the Separation of Environments checks (cubic-dev-ai bot): 1. Shared classifier read negated labels as production. "non-prod" tokenized to ["non","prod"] and matched the bare "prod" token, classifying as production. That both hid real separation (prod + "non-prod" collapsed to production-only -> false fail) and fabricated production (only non-prod envs -> false pass). A production token immediately preceded by "non"/"not" (or a joined "nonprod") now classifies as non-production, and "pre-prod" classifies as staging (matching the already-handled joined "preprod"). 2. Azure check swallowed Tier-1 subscription read failures: only resource-group read failures drove the "could not verify" vs "could not confirm" wording, so an unreadable subscription name could still yield a confident verdict. A read failure in either tier now marks coverage incomplete. 3. AWS check emitted a region-read "could not verify" fail even when the VPCs it did read already confirmed separation -- a contradictory pass+fail in one run. Extracted a pure buildEnvironmentSeparationOutcomes() that suppresses the region-failure finding once separation is confirmed (reading more regions can only add environments, never un-confirm a pass). Adds regression tests for all three (393 pass). No DB/API changes; the shared classifier's new 'non-production' value only flows into display strings and the production-vs-non-production check, verified safe across both repos. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…c-review fix(integrations): env-separation classifier negation + read-failure verdicts

tofikwest · 2026-06-17T23:46:58Z

@cubic-dev-ai review it

cubic-dev-ai · 2026-06-17T23:47:06Z

@cubic-dev-ai review it

@tofikwest I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

1 issue found across 14 files

Confidence score: 3/5

In packages/integration-platform/src/manifests/azure/checks/environment-separation.ts, the resource-group scan can stop at a pagination cap and still return a definitive environment-separation result, so partial Azure data could produce false pass/fail outcomes for users. Treat capped pagination as "coverage unverified" (or fail closed) and gate/adjust verdict emission before merging.

_{Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic}

fix(integration-platform): fail closed on capped azure env pagination

tofikwest · 2026-06-18T00:02:13Z

@cubic-dev-ai review it

cubic-dev-ai · 2026-06-18T00:02:21Z

@cubic-dev-ai review it

@tofikwest I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

No issues found across 16 files

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{Re-trigger cubic}

claudfuen · 2026-06-18T00:11:41Z

🎉 This PR is included in version 3.86.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

github-actions Bot and others added 5 commits June 17, 2026 18:40

chore: merge release v3.85.0 back to main [skip ci]

8ba64a3

Merge pull request #3182 from trycompai/tofik/cloud-separation-of-env…

af897d8

…ironments feat(integrations): Separation of Environments checks for GCP, AWS & Azure

github-actions Bot added prod-deploy automated-pr labels Jun 17, 2026

vercel Bot deployed to staging – comp-framework-editor June 17, 2026 22:27 View deployment

vercel Bot deployed to staging – app June 17, 2026 22:29 View deployment

cubic-dev-ai Bot reviewed Jun 17, 2026

View reviewed changes

tofikwest and others added 2 commits June 17, 2026 18:54

Merge pull request #3184 from trycompai/tofik/fix-env-separation-review

c8f5229

fix(integrations): env-separation review fixes — subscription scope, prod+non-prod pass bar, truncation

vercel Bot temporarily deployed to staging – portal June 17, 2026 23:06 Inactive

vercel Bot deployed to staging – comp-framework-editor June 17, 2026 23:07 View deployment

vercel Bot deployed to staging – app June 17, 2026 23:10 View deployment

cubic-dev-ai Bot reviewed Jun 17, 2026

View reviewed changes

tofikwest and others added 2 commits June 17, 2026 19:36

Merge pull request #3185 from trycompai/tofik/fix-env-separation-cubi…

30dfff9

…c-review fix(integrations): env-separation classifier negation + read-failure verdicts

vercel Bot temporarily deployed to staging – portal June 17, 2026 23:42 Inactive

vercel Bot deployed to staging – comp-framework-editor June 17, 2026 23:43 View deployment

vercel Bot deployed to staging – app June 17, 2026 23:46 View deployment

cubic-dev-ai Bot reviewed Jun 17, 2026

View reviewed changes

Comment thread packages/integration-platform/src/manifests/azure/checks/environment-separation.ts Outdated

tofikwest and others added 2 commits June 17, 2026 19:59

fix(integration-platform): fail closed on capped azure env pagination

ee53b72

Merge pull request #3186 from trycompai/tofik/fix-azure-env-pagination

4619574

fix(integration-platform): fail closed on capped azure env pagination

vercel Bot temporarily deployed to staging – portal June 18, 2026 00:01 Inactive

vercel Bot deployed to staging – comp-framework-editor June 18, 2026 00:03 View deployment

vercel Bot deployed to staging – app June 18, 2026 00:05 View deployment

cubic-dev-ai Bot reviewed Jun 18, 2026

View reviewed changes

tofikwest merged commit 89d2442 into release Jun 18, 2026
14 checks passed

claudfuen added the released label Jun 18, 2026

Conversation

github-actions Bot commented Jun 17, 2026 • edited by cubic-dev-ai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by cubic

Uh oh!

vercel Bot commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tofikwest commented Jun 17, 2026

Uh oh!

cubic-dev-ai Bot commented Jun 17, 2026

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tofikwest commented Jun 17, 2026

Uh oh!

cubic-dev-ai Bot commented Jun 17, 2026

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

tofikwest commented Jun 18, 2026

Uh oh!

cubic-dev-ai Bot commented Jun 18, 2026

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

claudfuen commented Jun 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

github-actions Bot commented Jun 17, 2026 •

edited by cubic-dev-ai Bot

Loading

vercel Bot commented Jun 17, 2026 •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading