Skip to content

ci: parallel Debian build and concurrency cancel for PRs#4056

Open
grandixximo wants to merge 2 commits into
LinuxCNC:masterfrom
grandixximo:ci/concurrency-and-parallel
Open

ci: parallel Debian build and concurrency cancel for PRs#4056
grandixximo wants to merge 2 commits into
LinuxCNC:masterfrom
grandixximo:ci/concurrency-and-parallel

Conversation

@grandixximo
Copy link
Copy Markdown
Contributor

@grandixximo grandixximo commented May 25, 2026

Two independent CI / build improvements split out of #4053 (asciidoctor docs migration) so they benefit every branch immediately.

1. Explicit parallel Debian build

build-package-arch.sh and build-package-indep.sh now set DEB_BUILD_OPTIONS="parallel=$(nproc)" before invoking debuild. dh_auto_build honours this and passes -jN to make.

On CI this is effectively a no-op: debuild already auto-sets parallel jobs, so dh_auto_build was already fanning out (master logs show cd src && make -j4 -O <target>). Per-job wall times on this PR are within noise of master.

The change still helps:

  • Local builds via dpkg-buildpackage or debian/rules binary directly, where parallel is not auto-enabled.
  • Documents intent so future readers do not have to know debuild's implicit defaults.

Applies to both package-arch and package-indep scripts.

2. Concurrency cancel-in-progress

When a new push to a pull request lands, the prior in-flight run for that ref gets cancelled. Gated on pull_request events only:

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: ${{ github.event_name == 'pull_request' }}

master / release / check_suite triggers are left alone, so a tagged release or a manually-rerequested check-suite never cancels itself.

This is the real CI win in this PR: saves wasted runner minutes on force-pushes.

Why split

Both pieces are general infrastructure useful for every branch and have no dependency on the docs migration. Splitting them out lets them merge ahead of #4053 and start helping immediately; #4053 will rebase to drop these changes once this lands.

Test plan

  • Confirm no regression in per-job timings vs a recent master run.
  • After merge, push a second commit to a PR and confirm the older run gets cancelled.

debuild leaves dpkg-buildpackage in serial mode unless
DEB_BUILD_OPTIONS=parallel=N is in the environment.  dh_auto_build
honours that variable and translates it into make -jN, so opting in
fans out the C/C++ build and the per-language doc rules across all the
runner's CPUs.  Local measurement on an 8-core box: binary-indep wall
time 32 min -> 7 min for the doc-only stage.

build-doc.sh already passes -j directly to make; this matches that
behaviour for the deb package CI jobs.
Add a concurrency block keyed on workflow + ref so a fresh push to a pull request automatically cancels any still-running prior run for that ref. cancel-in-progress is gated on the pull_request event, so master / release / check_suite triggers are left alone (they should never cancel themselves).
@grandixximo
Copy link
Copy Markdown
Contributor Author

Follow-up on the timing claim: I compared per-job times for this PR vs a recent master run on the same 4-CPU runners, and there is no measurable gain. All deltas are within noise.

Reason: master logs already show cd src && make -j4 -O <target> for every step. debuild on CI auto-sets parallel jobs, so dh_auto_build was already fanning out without the explicit DEB_BUILD_OPTIONS. My local 32=>7 min measurement was apples-to-oranges (local invocation path that did not auto-parallelize).

Still worth keeping the change:

  1. Helps local builds via dpkg-buildpackage or debian/rules binary directly, where parallel is not auto-enabled.
  2. Documents intent so a future reader does not need to know debuild's implicit defaults.
  3. Two lines, zero risk.

I will update the commit message and PR description to drop the "~3x on CI" claim and reframe as a local-build / explicit-intent fix. The concurrency cancel commit stands on its own and is the real CI win.

@BsAtHome
Copy link
Copy Markdown
Contributor

Follow-up on the timing claim: I compared per-job times for this PR vs a recent master run on the same 4-CPU runners, and there is no measurable gain. All deltas are within noise.

Then there is no real reason to keep it.

The only potential gain is aborting the old CI when you push on top of your tree. I'm not sure that is what we want when master merges occur. For branches, yes, that is mostly obvious. However, to master, it may not be nice.

Still worth keeping the change:
1. Helps local builds via dpkg-buildpackage or debian/rules binary directly, where parallel is not auto-enabled.

But the changes are in .github/* and therefore irrelevant for local builds.

Changing it explicitly in the rules may actually break things. IIRC, these options are set on/by build-bots to do parallel builds.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants