feat(cli): node selection for dataform compile#2212
Conversation
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
949c973 to
9fa4e64
Compare
Adds --actions/--tags/--include-deps/--include-dependents to `dataform compile`, filtering the printed graph via the existing prune() used by run/build. This is output filtering only; the full project still compiles (ref() resolution needs every action registered). Resolves dataform-co#2203 Co-Authored-By: Ivan Histand <490193+ihistand@users.noreply.github.com>
9fa4e64 to
2d1b912
Compare
|
/gcbrun |
kolina
left a comment
There was a problem hiding this comment.
I think it makes sense to add some prefix to these options for compile to make it more clear that these options are different from not compiling them. I'd suggest output-{...} format, feel free to suggest something else if you want.
Thanks for the review! Just to check I understood correctly: the concern is that for compile these flags work as filters rather than changing what actually gets compiled, so the unprefixed names could lead to confusion, is that it? This version of the solution leans towards keeping parity with commands like run and build, and from a DX perspective I personally like that more. Do you think a clear distinction between filtering vs. compiling outweighs that consistency? If we go the prefix way, which do you think is better --output-* or --filter-*? "Filter" reads a bit closer to the actual behavior for me, but I'm happy with either. And with a prefix, would --include-deps / --include-dependents become --output-include-deps / --output-include-dependents? |
|
Sorry for the late reply from my side.
Yes, it may lead to confusion. Also if we decide to implement actual filtering of compiled nodes, it may be hard to pick filters for them in the future.
In my opinion, yes.
Yeah, I think so. |
Summary
Adds node selection to
dataform compile, mirroringdbt compile --select.compilenow accepts the same selection flags asrun/build—--actions,--tags,--include-deps,--include-dependents— and filters the printed graph to just the selected action(s):Resolves #2203.
How it works
This is output filtering, not partial compilation. The whole project must
still compile (
ref()resolution and the dependency graph require every actionregistered), so we compile fully and then prune the result before printing —
reusing the existing
prune()thatrun/buildalready use, plus the sharedyargs option definitions. No proto changes.
--actions/--tags) is present.prune's run-side behavior.--include-deps/--include-dependentskeep their existing validation (only valid alongside--actions/--tags).Tests
New
compile node selectionsuite incli/index_compile_test.tsover anupstream -> midstream -> downstreamproject, covering: full graph (no selector),--actions,--include-deps,--include-dependents,--tags, empty-matchexits 0, and the no-selector validation error.
//cli:index_compile_testpasses.Credit
Implementation by @ihistand (Ivan Histand), proposed in SQLAnvil/sqlanvil#27 in
response to this issue. Thank you!