fix(schemas): normalize confidence from 0-100 scale before Pydantic validation#93
Open
Shrotriya-lalit wants to merge 1 commit into
Open
Conversation
…alidation LLMFinding and MetaAnalyzerFinding both hard-fail with le=1.0 when Ollama (or other local models) return confidence as an integer on a 0-100 scale. Add a mode="before" field_validator that: converts to float, divides by 100 if the value exceeds 1.0, then clamps to [0.0, 1.0]. This also handles negative values and values above 100 gracefully rather than crashing the meta-analyzer for the entire file. Closes NVIDIA#89 Signed-off-by: Lalit Shrotriya <shrotriya.lalit@outlook.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
LLMFindingandMetaAnalyzerFindingboth declareconfidence: float = Field(ge=0.0, le=1.0). When a local model (e.g. Ollama-hosted Llama or Mistral) returns confidence as an integer on a 0–100 scale ("confidence": 85), Pydantic raises aValidationErrorat thele=1.0boundary and crashes the meta-analyzer for the entire file — all findings for that file are silently dropped.Root Cause
The constraint is evaluated after type coercion but before any user logic, so there is no opportunity to rescale at the model level.
Fix
Add a
mode="before"@field_validatoronconfidencein both schemas:This also clamps negative values and values above 100 (e.g.
110) gracefully instead of crashing.Tests
0.85(already valid) passes through unchanged85(0-100 integer) is rescaled to0.85100→1.0,0→0.0-5→0.0,110→1.0ValueErrorLLMFindingandMetaAnalyzerFindingvalidatedCloses #89
Checklist
make lintpassesgit commit -s)