feat(openai-agents): stream tool-call argument deltas to the UI#378
Closed
danielmillerp wants to merge 1 commit into
Closed
feat(openai-agents): stream tool-call argument deltas to the UI#378danielmillerp wants to merge 1 commit into
danielmillerp wants to merge 1 commit into
Conversation
The TemporalStreamingModel already parsed the Responses API function_call_arguments deltas but only accumulated them — tool calls reached the UI all-at-once via the hooks layer. Now, as the model generates a tool call, the model layer opens a ToolRequestContent streaming context per call (keyed by output_index) and pushes ToolRequestDelta chunks as the arguments arrive, closing on item-done (mirrors the existing text-delta path). on_tool_start no longer emits a duplicate ToolRequestContent (the model layer streams it live now); on_tool_end still emits the ToolResponseContent result. The returned ModelResponse/output_items/usage assembly is unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
Author
|
Closing as a duplicate of #355 (@vkalmathscale, |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Stream OpenAI Agents SDK tool-call arguments live as the model generates them, so the UI shows the tool name + arguments building incrementally instead of appearing all-at-once.
Why / finding
TemporalStreamingModel.get_responsealready parsed the Responses APIresponse.function_call_arguments.deltaevents but only accumulated them — tool calls only surfaced later (all-at-once) via the hooks layer. This wires the deltas into the same agentex streaming path the text deltas already use.Change
models/temporal_streaming_model.py:function_callitem announce (ResponseOutputItemAddedEvent): open astreaming_task_message_contextwith an initialToolRequestContent(name + call_id, empty args), stashed peroutput_index.ResponseFunctionCallArgumentsDeltaEvent: push aStreamTaskMessageDelta(ToolRequestDelta(arguments_delta=...))(name/call_id pulled from the per-call record — the delta event carries neither).hooks/hooks.py:on_tool_startno longer emits a duplicateToolRequestContent(the model layer now streams it live — emitting again would double-render).on_tool_end'sToolResponseContent(the result) is unchanged.Invariant
The returned
ModelResponse/output_items/ usage / response_id assembly is untouched — streaming is a side-effect only.Validation
Existing plugin tests pass (
test_streaming_model.py+test_convert_tools.py: 43 passed); ruff clean; modules import.🤖 Generated with Claude Code
Greptile Summary
This PR wires tool-call argument deltas from the OpenAI Responses API stream directly to the UI, so the tool name and arguments build incrementally instead of appearing all at once. It also removes the now-redundant
ToolRequestContentemission from theon_tool_starthook to avoid double-rendering.temporal_streaming_model.py: On afunction_callResponseOutputItemAddedEvent, astreaming_task_message_contextis opened with an initialToolRequestContent(arguments={})and stored in a newtool_call_contextsdict keyed byoutput_index. EachResponseFunctionCallArgumentsDeltaEventpushes aToolRequestDeltathrough that context, andResponseOutputItemDoneEvent(or post-loop/exception-path cleanup) closes it.hooks/hooks.py:on_tool_startno longer callsstream_lifecycle_contentwith aToolRequestContent; it now only emits a debug log.on_tool_endis unchanged.Confidence Score: 4/5
The streaming side-effect is well-guarded and does not touch the ModelResponse assembly path, making the core output unchanged and the risk of regression low.
The new tool-call streaming follows the established text/reasoning context pattern closely, cleanup is handled in three places (item-done, post-loop, error path), and the on_tool_start hook removal is correct. The only rough edge is the locals() dance in the exception handler, which is functional but harder to maintain than initializing the dict before the try block.
temporal_streaming_model.py exception handler (lines 1123–1129) — the locals() pattern should be reviewed, but it does not affect correctness.
Important Files Changed
Sequence Diagram
sequenceDiagram participant OAI as OpenAI Responses API participant TSM as TemporalStreamingModel participant Ctx as streaming_task_message_context participant UI as Agentex UI participant Hook as TemporalStreamingHooks OAI->>TSM: ResponseOutputItemAddedEvent (function_call) TSM->>Ctx: "__aenter__() with ToolRequestContent(arguments={})" Ctx->>UI: "initial ToolRequestContent streamed" loop For each argument delta OAI->>TSM: ResponseFunctionCallArgumentsDeltaEvent(delta) TSM->>Ctx: "stream_update(StreamTaskMessageDelta(ToolRequestDelta))" Ctx->>UI: incremental argument delta end OAI->>TSM: ResponseFunctionCallArgumentsDoneEvent OAI->>TSM: ResponseOutputItemDoneEvent (function_call) TSM->>Ctx: close() Note over Hook: on_tool_start() only logs now Note over Hook: no longer emits ToolRequestContent Hook->>UI: "on_tool_end: stream_lifecycle_content(ToolResponseContent)"Prompt To Fix All With AI
Reviews (1): Last reviewed commit: "feat(openai-agents): stream tool-call ar..." | Re-trigger Greptile