Fix #291: accept EVENTSENT/EVENTRAISED confirmation for entity calls on classic (Azure Storage) backend#294
Merged
Conversation
…on classic (Azure Storage) backend On the classic backend (Azure Storage via the Durable Functions gRPC shim), a callEntity round-trips as a classic send-event rather than the DTS entity protocol: the request confirmation replays as an EVENTSENT history event and the result as an EVENTRAISED (name = requestId, input = a DTFx ResponseMessage JSON). Previously handleEventSent required the pending action to be a sendEvent and threw NonDeterminismError for a sendEntityMessage action, and handleEventRaised never routed the entity result. Net effect: entities only replayed on DTS, never on Azure Storage. This mirrors the Java/Python/.NET SDKs (all of which handle this) with a two-half, core-only fix: - Request half: handleEventSent now treats an EVENTSENT confirming a sendEntityMessage action (call/signal/lock/unlock) as valid and removes the pending action instead of throwing. Genuinely-wrong action types (e.g. a ScheduleTask confirmed via EVENTSENT) still throw. - Response half: handleEventRaised routes an EVENTRAISED whose name matches a pending entity call (by requestId) into the callEntity task, decoding the DTFx ResponseMessage wrapper (result / exceptionType / failureDetails) to resolve or reject with EntityOperationFailedException. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Fixes entity-call replay determinism for the classic (Azure Storage / DurableTask.Core) backend by tolerating the legacy EVENTSENT confirmation shape for entity message actions and by correlating entity call responses that arrive as EVENTRAISED (name = requestId) containing a DTFx ResponseMessage JSON wrapper. This brings the JS SDK behavior in line with other language SDKs while keeping DTS behavior unchanged.
Changes:
- Route classic-backend entity call responses delivered via
EVENTRAISED(name = requestId) to pendingcallEntitytasks, decoding the DTFxResponseMessagewrapper. - Accept
EVENTSENTconfirmations forsendEntityMessageactions (call/signal/lock/unlock) and clear the pending action instead of throwingNonDeterminismError. - Add unit tests covering the classic
EVENTSENT/EVENTRAISEDentity-call path, including success and failure variants.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
| packages/durabletask-js/src/worker/orchestration-executor.ts | Adds classic-backend entity response routing + ResponseMessage decoding; relaxes EVENTSENT validation to accept entity-message confirmations. |
| packages/durabletask-js/test/orchestration_executor.spec.ts | Adds regression tests for classic backend entity confirmation (EVENTSENT) and result delivery (EVENTRAISED) including failure decoding. |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
…ResponseMessage Autofix commit fd8f39a added `stackTrace: decoded.stackTrace` at the EntityOperationFailedException call site in handleEntityResponseFromEventRaised but never updated decodeEntityResponseMessage to return it, breaking the build: error TS2339: Property 'stackTrace' does not exist on type '{ isFailure: true; errorType: string; errorMessage: string; }'. Over gRPC the classic (Azure Storage) backend uses DurableTask.Core native entities, whose ResponseMessage.FailureDetails carries a real StackTrace. The wire JSON is {"result":null,"failureDetails":{"ErrorType":..,"ErrorMessage":..,"StackTrace":..}}. Extract failureDetails.StackTrace in the decoder and return it in the failure arm (return type widened to include an optional stackTrace) so the call site type-checks and the trace propagates into EntityOperationFailedException.failureDetails.stackTrace. This intentionally goes one step beyond the Java SDK (which passes null for the stack trace); documented in a JSDoc note so it isn't "fixed" back for parity. Tests: the failureDetails-variant test now includes a StackTrace and asserts it propagates; the exceptionType-only (WebJobs-variant) test asserts stackTrace stays undefined. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
kaibocai
approved these changes
Jul 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #291.
Problem
callEntityis not replay-deterministic on the classic (Azure Storage / DurableTask.Core) backend. The worker throws aNonDeterminismErrorwhenever an entity call is confirmed via anEVENTSENThistory event, so entities only work on the DTS backend.Root cause
On the classic backend, entity operations do not use the DTS entity protocol. The Durable Functions gRPC shim (
ProtobufUtils.ToHistoryEventProto) has noEntityOperationCalledcase, so acallEntityround-trips as a classic send-event:EVENTSENThistory event.EVENTRAISEDevent whosename == requestIdand whose input is a DTFxResponseMessageJSON.But
handleEventSentinorchestration-executor.tspopped the pending action by id and required it to be asendEventaction — when it was asendEntityMessageit threw:And
handleEventRaisedonly correlated external events by name, so it never routed the entity result back to the pendingcallEntitytask. Only the DTSENTITYOPERATIONCALLED/ENTITYOPERATIONCOMPLETEDpath worked.Fix (two halves — core SDK only, surgical)
Half 1 — request confirmation (
handleEventSent): When the pending action popped for anEVENTSENTis asendEntityMessage(call / signal / lock / unlock), treat it as a valid confirmation and remove the pending action instead of throwing. Genuinely-wrong action types (e.g. aScheduleTask/createTimerconfirmed viaEVENTSENT) still throwNonDeterminismError— the existing guard is preserved.Half 2 — response routing (
handleEventRaised): When anEVENTRAISED's (lowercased) name matches a pending entity call (keyed byrequestId), route it tohandleEntityResponseFromEventRaised, which decodes the DTFxResponseMessagewrapper and resolves/rejects thecallEntitytask. On DTS this branch never fires (responses arrive asENTITYOPERATIONCOMPLETED, which removes the pending call first), so there is no double-handling and no behavior change for DTS or ordinary external events.The correlation needs no new map:
pendingEntityCallsis already keyed byrequestId, and both JSnewGuid()and .NETGuid.ToString()emit lowercase GUIDs, so the lowercasedEVENTRAISEDname matches directly.ResponseMessage decoding (the flagged-uncertain part)
Mirrors the Java SDK's
handleEntityResponseFromEventRaised(authoritative; handles both DTFx variants). Wrapper shape:result— serialized operation result (double-encoded string; may be null/absent).exceptionType— present ⇒ failure. Misleading key name: it carries the error message (the C# property isErrorMessagebut a[DataMember(Name = "exceptionType")]annotation overrides the JSON key). Omitted when null.failureDetails— optional structured{ ErrorType, ErrorMessage, StackTrace, ... }(PascalCase).Decode logic:
JSON.parsethe raw input. If it isn't an object or has noresultkey ⇒ fall back to treating the parsed value directly as the result (matches Java's fallback for a possible future raw-result format).exceptionTypeorfailureDetailsis present ⇒ fail withEntityOperationFailedException(errorTypefromfailureDetails.ErrorTypeelse"unknown";errorMessagefromfailureDetails.ErrorMessageelse theexceptionTypetext).resultstring andJSON.parseit once (null⇒undefined), with a defensive fallback to the raw string.Cross-SDK reference
All three other gRPC SDKs already handle this; this ports their semantics:
TaskOrchestrationExecutor.java):handleEventSentis a type-agnostic no-op that just removes the pending action; the result arrives viaEVENTRAISEDand is decoded byhandleEntityResponseFromEventRaised.durabletask/worker.py): theeventSenthandler tolerates asendEntityMessageconfirmation and records correlation state so the laterEVENTRAISEDresult is routed.durabletask-dotnet): converts bothEVENTSENTandENTITYOPERATIONCALLEDinto the sameEventSentEventbefore replay, so no type mismatch arises.Tests
Added to
test/orchestration_executor.spec.ts(new describe block next to the existing EventSent Handler tests):EVENTSENTconfirming acallEntityCALL action removes the pending action and does not throw.signalEntitySIGNAL action.callEntityconfirmed viaEVENTSENT, result delivered viaEVENTRAISED(ResponseMessage {result}) resolves the task with the decoded value.failureDetailsResponseMessagerejects withEntityOperationFailedException(correcterrorType/errorMessage).exceptionType-onlyResponseMessagerejects witherrorType = "unknown"and the message fromexceptionType.The existing test asserting that an
EVENTSENTconfirming a non-entity, non-sendEventaction (aScheduleTask) still throwsNonDeterminismErroris preserved and passing.Validation
npm run build:core— ✅npm run lint— ✅test:unit, 60 suites / 1075 tests) — ✅ no regressionsNotes / follow-up
EVENTRAISEDsolockEntitiesfully works on the classic backend.signalEntity/lock/unlock confirmations no longer crash (Half 1), but wiring the lock-grant response on classic is a separate change (Java handles it via anAutoCloseabledata-type branch). Filed as a follow-up rather than bundled here to keep this fix surgical.