Skip to content

Change "greater than" to "greater than or equal to" in results validation to handle problem subtests#59

Open
malucius-rh wants to merge 1 commit into
mainfrom
allow_zero_as_error
Open

Change "greater than" to "greater than or equal to" in results validation to handle problem subtests#59
malucius-rh wants to merge 1 commit into
mainfrom
allow_zero_as_error

Conversation

@malucius-rh

Copy link
Copy Markdown

Description

This allows the validator to consider a zero result for a subtest as "valid but error", allowing the "good" results to clear validation and be saved off. One example is if a system doesn't support the CPU_avx512 test a zero is returned, which is a valid "something went wrong here but everyone else is okay" result.

Before/After Comparison

Before:
If one of the subtests returns a zero reflecting an error in one of the subtests, the validation rejects the entire run
Sample:
CPU_fma,144077.57151551355,2026-06-08T15:37:26Z,2026-06-08T15:40:29Z
CPU_avx,118863.51315704771,2026-06-08T15:37:26Z,2026-06-08T15:40:29Z
CPU_avx512,0,2026-06-08T15:37:26Z,2026-06-08T15:40:29ZCPU_fma,144077.57151551355,2026-06-08T15:37:26Z,2026-06-08T15:40:29Z
CPU_avx,118863.51315704771,2026-06-08T15:37:26Z,2026-06-08T15:40:29Z
CPU_avx512,0,2026-06-08T15:37:26Z,2026-06-08T15:40:29Z
m_CPU_enc_SHA,219491544665.70303,2026-06-08T15:37:26Z,2026-06-08T15:40:29Z
m_CPU_enc_AES,141409149289.58951,2026-06-08T15:37:26Z,2026-06-08T15:40:29Z
m_CPU_enc_SHA,219491544665.70303,2026-06-08T15:37:26Z,2026-06-08T15:40:29Z
m_CPU_enc_AES,141409149289.58951,2026-06-08T15:37:26Z,2026-06-08T15:40:29Z

1 validation error for list[Passmark_Results]
13.Operations
Input should be greater than 0 [type=greater_than, input_value=0.0, input_type=float]

After:
After changing the greater_than to greater_equal the validation returns:
Results verified
updating: results_passmark_throughput-performance.tar (deflated 96%)

and the full results are supported.

Clerical Stuff

This closes #58
Relates to JIRA: RPOPC-1276

This allows the validator to consider a zero result for a subtest as "valid but error", allowing the "good" results to clear validation and be saved off.
@malucius-rh malucius-rh requested a review from dvalinrh June 10, 2026 12:26
@malucius-rh malucius-rh self-assigned this Jun 10, 2026
@malucius-rh malucius-rh added the bug Something isn't working label Jun 10, 2026
@github-actions

Copy link
Copy Markdown

This relates to RPOPC-1276

@coderabbitai

coderabbitai Bot commented Jun 10, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Enterprise

Run ID: b791dd03-5ad5-4345-acd4-b98b084f9d08

📥 Commits

Reviewing files that changed from the base of the PR and between 374ccc3 and db6fea9.

📒 Files selected for processing (1)
  • passmark/results_schema.py

📝 Walkthrough

Summary by CodeRabbit

  • Bug Fixes
    • Updated validation to accept zero values for Operations field, where previously only positive values were permitted.

Walkthrough

The Operations field validation in Passmark_Results is relaxed from requiring values strictly greater than 0 to accepting values greater than or equal to 0. This permits zero values to represent unsupported subtests while maintaining rejection of infinite and NaN values.

Changes

Operations Field Validation Update

Layer / File(s) Summary
Operations field constraint update
passmark/results_schema.py
Passmark_Results.Operations field constraint relaxed from gt=0 to ge=0, allowing zero values for unsupported subtests while still rejecting infinite/NaN values.

🎯 1 (Trivial) | ⏱️ ~2 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately describes the main change: relaxing the Operations field validation from 'greater than 0' to 'greater than or equal to 0' to handle problem subtests.
Description check ✅ Passed The description covers all required template sections: summary of changes, before/after comparison with examples, and clerical information linking to issue #58 and JIRA ticket RPOPC-1276.
Linked Issues check ✅ Passed The code change directly addresses issue #58 by modifying the Operations field constraint from gt=0 to ge=0, enabling zero results to be treated as valid errors for unsupported subtests.
Out of Scope Changes check ✅ Passed The change is narrowly scoped to the specific requirement: only the Operations field constraint in Passmark_Results was modified; all other validation behavior remains unchanged.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@dvalinrh dvalinrh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Validation doesn't handle "0 as 'valid but error'" in results

2 participants