slurm sglang-disagg: allow co-located proxy (xP+yD==nnodes)#148
Open
mkuznet1 wants to merge 1 commit into
Open
slurm sglang-disagg: allow co-located proxy (xP+yD==nnodes)#148mkuznet1 wants to merge 1 commit into
mkuznet1 wants to merge 1 commit into
Conversation
The sglang-disagg launcher hard-required a dedicated proxy node (prefill+decode+1==nnodes), unlike the vllm launcher which leaves topology to the model script. Relax the custom-split validation to also accept a co-located proxy/router on the first prefill node (prefill+decode==nnodes). xP/yD and the exported topology env are unchanged; the proxy is started by the model run.sh, so both layouts are valid.
mkuznet1
requested review from
Cemberk,
Rohan138,
coketaste,
gargrahul and
leconcio
as code owners
There was a problem hiding this comment.
Pull request overview
This PR updates the SLURM launcher’s SGLang-disaggregated custom-split validation to accept both (a) a dedicated proxy node and (b) a co-located proxy topology, aligning with the vLLM-disagg-style layout flexibility.
Changes:
- Relax custom split validation to allow either
prefill+decode==nnodes(co-located proxy) orprefill+decode+1==nnodes(dedicated proxy node). - Update the validation failure message to describe both valid configurations.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+883
to
+887
| if (prefill_nodes + decode_nodes != nnodes | ||
| and prefill_nodes + decode_nodes + 1 != nnodes): | ||
| raise ValueError( | ||
| f"Custom split validation failed: " | ||
| f"prefill_nodes ({prefill_nodes}) + decode_nodes ({decode_nodes}) + 1 proxy " | ||
| f"must equal nnodes ({nnodes}), but got {prefill_nodes + decode_nodes + 1}" | ||
| f"Custom split validation failed: prefill_nodes ({prefill_nodes}) + " | ||
| f"decode_nodes ({decode_nodes}) must equal nnodes ({nnodes}) for a " |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request updates the validation logic for the SGLang Disaggregated deployment command in
slurm.pyto support both co-located and dedicated proxy node topologies. The new logic allows for more flexible node allocation, mirroring the layout used by vllm-disagg.Validation logic improvements:
Updated the custom split validation to accept either a dedicated proxy node (
prefill_nodes + decode_nodes + 1 == nnodes) or a co-located proxy/router on the first prefill node (prefill_nodes + decode_nodes == nnodes). This change enables both topologies to be valid, reflecting how the proxy is actually launched by the model script, not the launcher.Improved error messaging to clearly describe the two valid configurations for node allocation, aiding in debugging and user understanding.outer on the first prefill node (prefill+decode==nnodes). xP/yD and the exported topology env are unchanged; the proxy is started by the model run.sh, so both layouts are valid.