[Question] In what order durable function process sub-orchestrator and activity function

[quytrieu37]

I have some questions about the theory of the order durable function process sub-orchestrator
I use orchestrator to call multies sub orchestrator in parallel (task 1, 2, 3), each sub orchestrator have it's next step and it call multies recursive sub orchestrator (1 call 4,5; 2 call 6,7),...
I try to limit the scale out instance in function app to one and it work like expected 1,2,3 -> 4,5,6,7
But IF possible under extreme circumstances there only have one instance with one worker config maxConcurrentOrchestratorFunctions is 1 (so only one orchestrator can run at same time), does it process one by one and in order 1,2,3 -> 4,5,6,7 (FIFO) or it'll process 1 > 4,5 -> 2 -> 6,7?

Sorry for my bad english

[cgillum]

The exact behavior depends on which storage provider you're using, but generally speaking, when you schedule sub-orchestrations and activities in parallel, there's no guarantee about the order they run in.

Assuming you're using the Azure Storage provider, if you limit concurrency to just one worker and one concurrent orchestrator function like you described, they might run in the order they are scheduled, but it's not guaranteed. This is because we use the process's thread pool to schedule the parallel sub-orchestrations (or activities), so we can't guarantee what order in which the thread pool will add messages to the Azure Storage queue.

The behavior might be different with something like the MSSQL storage provider because it adds all messages to the database in a single database transaction, so the order in which they appear in the database will always be consistent. However, it's best to not rely on this behavior since there are other factors that could possibly cause parallel orchestrations/activities to run in arbitrary order.

[quytrieu37]

Hi @cgillum, Thank you for reply me
Sorry but your diagram can't achieve the desired outcome. In my diagram, all step is an independent one sub-orchestrator.

In the normal behavior, the step 1,3,2 will run in parallel and when each step complete it'll trigger the chill step run,
EX: Step 1 finish and it'll trigger step 4,5 run and do not wait for steps 3,2 to complete (they are 3 separate flow), Step 3 complete will trigger step 8 run and don't care about step 1,2.

But I wonder that if there only 1 sub-orchestrator run at a time, can it run 1,3,2 completed first then run 4,6,7,8,5 and finally 10,9.

[cgillum]

Maybe it would help if you explain in more detail what you're trying to accomplish. Why are you concerned about whether one sub-orchestrator runs at a time? I'm having a hard time understanding your requirements.

[quytrieu37]

hi @cgillum , sorry about this inconvenient
I'm try to running somthing like a gitlab pipeline, the child step don't need the wait for irrelevant steps complete, it just need wait for the parent of it completed.
As I comment above
In the normal behavior, the step 1,3,2 will run in parallel and when each step complete it'll trigger the child step run.
EX: Step 1 finish and it'll trigger step 4,5 run and do not wait for steps 3,2 to complete (they are 3 separate flow), Step 3 complete will trigger step 8 run and don't care about steps 1,2.

And I though if there only have one orchestrator run at a time (just for confirm it'll run in the right way), the step 4 and 5 will run before 2 and 3 start, even if it's don't affect the result,
but I need it'll start step 2,3 before step 4,5 start.

[quytrieu37]

The root orchestrator'll CallSubOrchestratorAsync the step 1,2,3 at a time, when step 1 is done, it CallSubOrchestratorAsync step 4,5 , step 2 CallSubOrchestratorAsync step 6,7 start, and step 3 call step 8 to start and so on.
So I guess it'll run like I expected (cause step 4,5 will call after step 2,3 (which is call from main orchestrator)).
But I don't know if there is any case of it will run the step 4,5 before step 2,3 start so I need a confirmation on this.

[quytrieu37]

Hi @cgillum , sorry for this inconvenience but do you understand what I am trying to describe here?
Let me know if there are any concerns about this, thank you so much!

[cgillum]

And I though if there only have one orchestrator run at a time (just for confirm it'll run in the right way), the step 4 and 5 will run before 2 and 3 start, even if it's don't affect the result,
but I need it'll start step 2,3 before step 4,5 start.

If you run one orchestration at a time by setting concurrency to 1 (and having only one app instance), then I expect that most of the time the order will be 1, 2, 3, 4, 5, 6, 7, 8, 10, 9. In other words, the order will be effectively "breadth first".

[quytrieu37]

Hi @cgillum, thank you for supporting me so enthusiastically!
Just for clarify, Step 1,2,3 will enqueue at one time on the main orchestrator, the child step will enqueue (by recursive call) once the direct father is completed.
In other words, steps 4 and 5 always enqueue after 2 and 3.
I'm just afraid that the recursive call will make step 4 run immediately instead of step 2 or 3 if there can be only one orchestrator (and one app instance)

[cgillum]

In other words, steps 4 and 5 always enqueue after 2 and 3.

Yes, this is the expectation. I'll try to explain why:

1. The main orchestrator schedules tasks 1, 2, 3 and then awaits. When the orchestration awaits, messages for starting 1, 2, 3 are all put on the internal queue ([1, 2, 3]).
2. Message 1 is on the front of the queue, so it will be processed first. Sub-orchestration 1 runs, schedules 4 & 5, and then awaits, adding two more messages to the end of the queue. The queue is now [2, 3, 4, 5].
3. Because 2 is on the top of the queue, it gets executed next. Sub-orchestration schedules 6 & 7 and then awaits, adding those messages to the queue. Now the queue looks like [3, 4, 5, 6, 7]
4. This process repeats until the queue is empty and all orchestrations have completed.

Does that help explain the expected behavior?

[quytrieu37]

Perfect, that's what i'm looking for, have a good day and thank you so much