Skip to content

fix(score): raise ValueError when create_score receives None name or value#1733

Open
RudraDudhat2509 wants to merge 1 commit into
langfuse:mainfrom
RudraDudhat2509:fix/create-score-silent-validation-failure
Open

fix(score): raise ValueError when create_score receives None name or value#1733
RudraDudhat2509 wants to merge 1 commit into
langfuse:mainfrom
RudraDudhat2509:fix/create-score-silent-validation-failure

Conversation

@RudraDudhat2509

@RudraDudhat2509 RudraDudhat2509 commented Jun 30, 2026

Copy link
Copy Markdown

What does this PR do?

Fixes langfuse/langfuse#14615

create_score() catches a pydantic ValidationError inside the broad except Exception block when name or value is None. The error gets logged at ERROR level but the function still returns None, so the caller has no way to tell the score wasn't created.

This catches ValidationError separately and re-raises it as ValueError so the failure is visible at the call site.

Type of change

  • Bug fix

Verification

List the main commands you ran:

uv run --frozen pytest tests/unit/test_create_score_validation.py -v
uv run --frozen ruff check langfuse/_client/client.py tests/unit/test_create_score_validation.py
uv run --frozen ruff format --check langfuse/_client/client.py tests/unit/test_create_score_validation.py
uv run --frozen mypy langfuse/_client/client.py --no-error-summary
uv run --frozen pytest -n auto --dist worksteal tests/unit

Full unit suite passes aside from 3 pre-existing failures unrelated to this change (Windows path separator issue in test_serializer.py::test_path, flaky atexit tests under parallel execution in test_prompt_atexit.py). Confirmed these fail identically on main before this change too.

Checklist

  • I self-reviewed the diff using code_review.md.
  • I added or updated tests for behavior changes.
  • I updated docs, examples, or .env.template if needed.
  • I did not hand-edit generated files; if generated files changed, I used the upstream regeneration path.
  • I did not commit secrets or credentials.

Greptile Summary

This PR fixes a silent failure in create_score() where passing None for name or value caused a pydantic ValidationError to be caught by the broad except Exception block, logged, and then swallowed — leaving the caller with no indication the score was never created. The fix intercepts ValidationError before the catch-all and re-raises it as ValueError.

  • A new except ValidationError clause is added ahead of except Exception in create_score, converting the pydantic error into a caller-visible ValueError.
  • Two unit tests verify that None name and None value each raise ValueError with the expected message.

Confidence Score: 4/5

Safe to merge; the core fix is correct and well-tested, with one minor behavioral gap when tracing is disabled.

The fix correctly intercepts pydantic ValidationError before the broad except Exception swallows it, and the import is properly placed at module scope. The one gap is that the early-return guard on _tracing_enabled means invalid inputs are not validated at all when tracing is off, so the same None call would raise in a production environment but silently succeed in a tracing-disabled test or staging environment.

The early-return at lines 1917–1918 of langfuse/_client/client.py is worth a second look — consider whether argument validation should be hoisted above it.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[create_score called] --> B{_tracing_enabled?}
    B -- No --> C[return None\nno validation]
    B -- Yes --> D[ScoreBody pydantic constructor]
    D -- ValidationError\ne.g. name=None --> E[NEW: except ValidationError]
    E --> F[raise ValueError\nInvalid score parameters]
    D -- success --> G[build event dict]
    G --> H[add_score_task]
    H -- Exception --> I[log error, swallow]
    H -- success --> J[score enqueued]
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A[create_score called] --> B{_tracing_enabled?}
    B -- No --> C[return None\nno validation]
    B -- Yes --> D[ScoreBody pydantic constructor]
    D -- ValidationError\ne.g. name=None --> E[NEW: except ValidationError]
    E --> F[raise ValueError\nInvalid score parameters]
    D -- success --> G[build event dict]
    G --> H[add_score_task]
    H -- Exception --> I[log error, swallow]
    H -- success --> J[score enqueued]
Loading
Prompt To Fix All With AI
Fix the following 1 code review issue. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 1
langfuse/_client/client.py:1917-1918
**Validation skipped when tracing is disabled**

When `_tracing_enabled` is `False`, the function returns on line 1918 before entering the `try` block, so `create_score(name=None, value=None)` silently returns `None` instead of raising `ValueError`. This means the same call raises in production (where tracing is on) but succeeds silently in environments where tracing has been disabled — making it easy to ship a bug without test coverage catching it.

Reviews (1): Last reviewed commit: "fix(score): raise ValueError when create..." | Re-trigger Greptile

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@CLAassistant

CLAassistant commented Jun 30, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@RudraDudhat2509 RudraDudhat2509 force-pushed the fix/create-score-silent-validation-failure branch from 999a531 to 416f699 Compare June 30, 2026 08:28
@RudraDudhat2509

Copy link
Copy Markdown
Author
Screenshot 2026-06-30 134756

for the teams reference

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(sdk-python): create_score silently swallows ValidationError when name or value is None

2 participants