[IsaacLab CI] Run only necessary tests as pre-merge, run full test as post merge#6296
[IsaacLab CI] Run only necessary tests as pre-merge, run full test as post merge#6296mataylor-nvidia wants to merge 6 commits into
Conversation
Greptile SummaryThis PR replaces manually maintained test-node-ID TOML files and the
Confidence Score: 5/5Safe to merge; testmon integration is internally consistent, the workspace volume-mount correctly persists data between container and host, and the smoke-hook fallback provides a minimum coverage floor for install-ci cold-starts. The core mechanics (cache restore → select/collect → write-back via volume mount → cache save) are correctly wired.
Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
PR[Pull Request / Push event] --> Changes{Changed files\ndetected?}
Changes -->|No relevant files| Skip[Skip CI]
Changes -->|Relevant files| PyOnly{All relevant\nfiles .py?}
PyOnly -->|Yes - premerge=true| RestoreCache[Restore Testmon Cache]
PyOnly -->|No - static files changed| RestoreCache
RestoreCache --> DataExists{.testmondata\nexists?}
DataExists -->|Yes + premerge=true| ForceSelect["--testmon-forceselect\nRun affected tests only"]
DataExists -->|No + premerge=true| Fallback["::warning:: Missing data\n--testmon-noselect\nRun full suite"]
DataExists -->|premerge=false| NoSelect["--testmon-noselect\nRun full suite\n(collect mode)"]
ForceSelect --> SmokeRestore[Smoke tests restored\nvia conftest hook]
Fallback --> RunAll
SmokeRestore --> RunAffected[Run affected + smoke tests]
NoSelect --> RunAll[Run full test suite]
RunAffected --> SaveCache[Save updated Testmon cache]
RunAll --> SaveCache
SaveCache --> Results[Check test results]
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
PR[Pull Request / Push event] --> Changes{Changed files\ndetected?}
Changes -->|No relevant files| Skip[Skip CI]
Changes -->|Relevant files| PyOnly{All relevant\nfiles .py?}
PyOnly -->|Yes - premerge=true| RestoreCache[Restore Testmon Cache]
PyOnly -->|No - static files changed| RestoreCache
RestoreCache --> DataExists{.testmondata\nexists?}
DataExists -->|Yes + premerge=true| ForceSelect["--testmon-forceselect\nRun affected tests only"]
DataExists -->|No + premerge=true| Fallback["::warning:: Missing data\n--testmon-noselect\nRun full suite"]
DataExists -->|premerge=false| NoSelect["--testmon-noselect\nRun full suite\n(collect mode)"]
ForceSelect --> SmokeRestore[Smoke tests restored\nvia conftest hook]
Fallback --> RunAll
SmokeRestore --> RunAffected[Run affected + smoke tests]
NoSelect --> RunAll[Run full test suite]
RunAffected --> SaveCache[Save updated Testmon cache]
RunAll --> SaveCache
SaveCache --> Results[Check test results]
Reviews (2): Last reviewed commit: "fix" | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| --testmon-data-dir "$TESTMON_DATA_DIR" | ||
| --results-dir "${{ github.workspace }}/results" | ||
| -- --tb=short -sv) | ||
| -- --tb=short -sv --testmon) |
There was a problem hiding this comment.
Can we use full expand form of --tb and --sv args?
There was a problem hiding this comment.
sure I will replace this
There was a problem hiding this comment.
no long form for --tb traceback
| description: 'Comma-separated list of specific test files to include' | ||
| default: '' | ||
| required: false | ||
| test-node-ids-file: |
There was a problem hiding this comment.
As I am not that fluent in this part of the github workflow, why did we remove this? Is this the old system that we used to detect the changes and it's replaced by testmon?
There was a problem hiding this comment.
I added this in the previous pull request to only run only specific subset of rendering tests on develop, now I am just reverting it because everything should be run post-merge.
2 participants
Description
This PR reduces pre-merge CI time by using
pytest-testmoncoverage data to select tests affected by Python source changes.Read about how testmon works: https://www.testmon.org/
Pre-merge CI now:
Post-merge CI runs the full test suite and refreshes the Testmon dependency database for future pull requests.
The previous manually maintained test-subset configuration and exact node-ID selection plumbing have been removed.
Dependencies:
pytest-testmonas a CI-only dependency.Fixes # (OMPE-99210)
Type of change
Checklist
pre-commitchecks with./isaaclab.sh --formatconfig/extension.tomlfileCONTRIBUTORS.mdor my name already exists there