You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
For get, test, and export, we should allow running resources concurrently. Have a setting to specify the number or perhaps based on CPU cores/threads by default. set is more complex that we'll have to think about later.
We want to expose settings at the configuration level, and also consider how to expose in a resource manifest how the resource itself determines concurrency. We want to avoid overloading the system.
Follow-up from the 10/8 working group discussion on concurrency:
RFC likely needed. Concurrency touches configuration document processing, settings, definitions, and resource manifests. It most likely needs an RFC, and certainly one if set is included.
get/test vs. set symmetry (open). If get/test/export can run concurrently but set stays serial, the behavior of the node graph becomes harder to predict and to visualize. One view is that the language, not a setting, should define what runs in parallel. Another is that an asymmetric design is acceptable if it is clean, since set is the harder case. Not resolved.
Marking resources. We considered having each resource declare everything it touches so the engine can detect conflicts. That is probably not feasible. The alternative is that the configuration author decides and tests it. Neither is settled.
Unsupported directives. Where concurrency isn't supported, the directive is ignored and everything runs serially.
Rollout. Mark the feature experimental, and start opt-in because of the added complexity and unexpected behavior.
Prototype option. Try a parallel group resource implemented only by the engine. Resources outside the group stay serial. Resources inside it run in parallel (serial A, then the group, then serial B). This avoids adding directives to many resources and is easier to reason about in the document. If it proves valuable, pull it into the engine through directives and deprecate the group.
Activity
theJasonHelmick commentedon Oct 8, 2026
We want to expose settings at the configuration level, and also consider how to expose in a resource manifest how the resource itself determines concurrency. We want to avoid overloading the system.
Follow-up from the 10/8 working group discussion on concurrency:
setis included.get/testvs.setsymmetry (open). Ifget/test/exportcan run concurrently butsetstays serial, the behavior of the node graph becomes harder to predict and to visualize. One view is that the language, not a setting, should define what runs in parallel. Another is that an asymmetric design is acceptable if it is clean, sincesetis the harder case. Not resolved.