ci: parallel Debian build and concurrency cancel for PRs#4056
ci: parallel Debian build and concurrency cancel for PRs#4056grandixximo wants to merge 2 commits into
Conversation
debuild leaves dpkg-buildpackage in serial mode unless DEB_BUILD_OPTIONS=parallel=N is in the environment. dh_auto_build honours that variable and translates it into make -jN, so opting in fans out the C/C++ build and the per-language doc rules across all the runner's CPUs. Local measurement on an 8-core box: binary-indep wall time 32 min -> 7 min for the doc-only stage. build-doc.sh already passes -j directly to make; this matches that behaviour for the deb package CI jobs.
Add a concurrency block keyed on workflow + ref so a fresh push to a pull request automatically cancels any still-running prior run for that ref. cancel-in-progress is gated on the pull_request event, so master / release / check_suite triggers are left alone (they should never cancel themselves).
|
Follow-up on the timing claim: I compared per-job times for this PR vs a recent master run on the same 4-CPU runners, and there is no measurable gain. All deltas are within noise. Reason: master logs already show Still worth keeping the change:
I will update the commit message and PR description to drop the "~3x on CI" claim and reframe as a local-build / explicit-intent fix. The concurrency cancel commit stands on its own and is the real CI win. |
Then there is no real reason to keep it. The only potential gain is aborting the old CI when you push on top of your tree. I'm not sure that is what we want when master merges occur. For branches, yes, that is mostly obvious. However, to master, it may not be nice.
But the changes are in Changing it explicitly in the rules may actually break things. IIRC, these options are set on/by build-bots to do parallel builds. |
Two independent CI / build improvements split out of #4053 (asciidoctor docs migration) so they benefit every branch immediately.
1. Explicit parallel Debian build
build-package-arch.shandbuild-package-indep.shnow setDEB_BUILD_OPTIONS="parallel=$(nproc)"before invokingdebuild.dh_auto_buildhonours this and passes-jNtomake.On CI this is effectively a no-op:
debuildalready auto-sets parallel jobs, sodh_auto_buildwas already fanning out (master logs showcd src && make -j4 -O <target>). Per-job wall times on this PR are within noise of master.The change still helps:
dpkg-buildpackageordebian/rules binarydirectly, where parallel is not auto-enabled.debuild's implicit defaults.Applies to both
package-archandpackage-indepscripts.2. Concurrency cancel-in-progress
When a new push to a pull request lands, the prior in-flight run for that ref gets cancelled. Gated on
pull_requestevents only:master/release/check_suitetriggers are left alone, so a tagged release or a manually-rerequested check-suite never cancels itself.This is the real CI win in this PR: saves wasted runner minutes on force-pushes.
Why split
Both pieces are general infrastructure useful for every branch and have no dependency on the docs migration. Splitting them out lets them merge ahead of #4053 and start helping immediately; #4053 will rebase to drop these changes once this lands.
Test plan