Skip to content

feat(cli): node selection for dataform compile#2212

Open
alandrebraga wants to merge 1 commit into
dataform-co:mainfrom
alandrebraga:feat/compile-node-selection
Open

feat(cli): node selection for dataform compile#2212
alandrebraga wants to merge 1 commit into
dataform-co:mainfrom
alandrebraga:feat/compile-node-selection

Conversation

@alandrebraga

Copy link
Copy Markdown

Summary

Adds node selection to dataform compile, mirroring dbt compile --select. compile now accepts the same selection flags as run/build--actions, --tags, --include-deps, --include-dependents — and filters the printed graph to just the selected action(s):

dataform compile --actions my_model --json
dataform compile --tags daily --include-deps --json

Resolves #2203.

How it works

This is output filtering, not partial compilation. The whole project must
still compile (ref() resolution and the dependency graph require every action
registered), so we compile fully and then prune the result before printing —
reusing the existing prune() that run/build already use, plus the shared
yargs option definitions. No proto changes.

  • A clean graph is pruned only when a selector (--actions/--tags) is present.
  • On compile errors, the full graph + errors print unchanged (graph-level errors kept as-is).
  • A selector matching nothing emits an empty graph and exits 0, matching prune's run-side behavior.
  • --include-deps / --include-dependents keep their existing validation (only valid alongside --actions/--tags).

Tests

New compile node selection suite in cli/index_compile_test.ts over an
upstream -> midstream -> downstream project, covering: full graph (no selector),
--actions, --include-deps, --include-dependents, --tags, empty-match
exits 0, and the no-selector validation error. //cli:index_compile_test passes.

Credit

Implementation by @ihistand (Ivan Histand), proposed in SQLAnvil/sqlanvil#27 in
response to this issue. Thank you!

@alandrebraga alandrebraga requested a review from a team as a code owner June 21, 2026 02:51
@alandrebraga alandrebraga requested review from andrzej-grudzien and removed request for a team June 21, 2026 02:51
@google-cla

google-cla Bot commented Jun 21, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@alandrebraga alandrebraga force-pushed the feat/compile-node-selection branch from 949c973 to 9fa4e64 Compare June 21, 2026 02:55
Adds --actions/--tags/--include-deps/--include-dependents to `dataform
compile`, filtering the printed graph via the existing prune() used by
run/build. This is output filtering only; the full project still compiles
(ref() resolution needs every action registered).

Resolves dataform-co#2203

Co-Authored-By: Ivan Histand <490193+ihistand@users.noreply.github.com>
@alandrebraga alandrebraga force-pushed the feat/compile-node-selection branch from 9fa4e64 to 2d1b912 Compare June 21, 2026 03:02
@kolina

kolina commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

/gcbrun

@kolina kolina left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it makes sense to add some prefix to these options for compile to make it more clear that these options are different from not compiling them. I'd suggest output-{...} format, feel free to suggest something else if you want.

@kolina kolina removed the request for review from andrzej-grudzien June 25, 2026 23:55
@alandrebraga

alandrebraga commented Jun 28, 2026

Copy link
Copy Markdown
Author

I think it makes sense to add some prefix to these options for compile to make it more clear that these options are different from not compiling them. I'd suggest output-{...} format, feel free to suggest something else if you want.

Thanks for the review! Just to check I understood correctly: the concern is that for compile these flags work as filters rather than changing what actually gets compiled, so the unprefixed names could lead to confusion, is that it?

This version of the solution leans towards keeping parity with commands like run and build, and from a DX perspective I personally like that more. Do you think a clear distinction between filtering vs. compiling outweighs that consistency?

If we go the prefix way, which do you think is better --output-* or --filter-*? "Filter" reads a bit closer to the actual behavior for me, but I'm happy with either.

And with a prefix, would --include-deps / --include-dependents become --output-include-deps / --output-include-dependents?

@kolina

kolina commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Sorry for the late reply from my side.

Thanks for the review! Just to check I understood correctly: the concern is that for compile these flags work as filters rather than changing what actually gets compiled, so the unprefixed names could lead to confusion, is that it?

Yes, it may lead to confusion. Also if we decide to implement actual filtering of compiled nodes, it may be hard to pick filters for them in the future.

This version of the solution leans towards keeping parity with commands like run and build, and from a DX perspective I personally like that more. Do you think a clear distinction between filtering vs. compiling outweighs that consistency?

In my opinion, yes.

If we go the prefix way, which do you think is better --output-* or --filter-*? "Filter" reads a bit closer to the actual behavior for me, but I'm happy with either.

output- may be more clear to me, but I don't have a strong opinion here.

And with a prefix, would --include-deps / --include-dependents become --output-include-deps / --output-include-dependents?

Yeah, I think so.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Node selection for dataform compile (compile/output a single model)

2 participants