Correctly show concurrent containers in Runtime Control tool

[fmauch]

Hi there,

I noticed that when having a concurrent container I still seem to need define a starting state (and can only define one).
When I then run the state machine and execution is blocked by one of the concurrent states (e.g. because it is waiting for some input, or a larger action is executed), only the state that has the transition from the entry point is shown in the runtime control display.

I think it would be beneficial to have a view that can also model concurrency, or even show behaviours hierarchically like the smach viewer does.

[pschillinger]

Thank you for the feedback! Yes, I agree that it would be helpful to see which states are active in parallel, and yes, it is not possible right now as you describe. This is mostly due to early design decisions we had to make when designing the system primarily for low-bandwidth usage between the robot and the UI. It is in my top 3 for design changes I would make for a major version update, but this will most likely not happen in the next weeks. Please let me know if this has high priority for you, then we can try to find an intermediate solution.

[fmauch]

Without opening a totally new issue: I don't really see the necessity to decouple the viewer and the execution in bandwidth concerns in all use-cases. As for us, we used SMACH mostly with the state machine and the viewer running on the same pc, as we often only use one controller pc together with sensors and a robotic arm.

Regarding your question: I'm currently working on a project, where we use a lot of parallel executions, so it definitely is a point for me, if some "hacky" intermediate solution with minimial effort could be used, I'm open for that. Again, state machine and viewer will be running on the same machine.

Edit:
Actually, as far as I investigated, it could be possible to simply create a second viewer similar to the smach viewer. The behaviour's structure is published and can be requested from the mirror. The only problem is that not all transitions are published somewhere, right? So when I enter a container that won't be published, but only the first state inside the container (Where we are at the concurrency problem again).
So when we add a debug topic for entering and leaving states, e.g. inside EventState._event_execute (not in on_enter / on_exit, as they might get overwritten), we could use the structure from the mirror and track the currently active states, which we then in turn could colorize or something like that.
As the current smach-viewer isn't that much code, I could imagine creating something similar quite fast.

[pschillinger]

I like the idea of having a second, simple debugging visualization tool as an intermediate solution for this. I will check where it makes sense to publish some debugging data for having sufficient information for such a viewer.

[fmauch]

I've created a very simple viewer using the libraries from rqt_graph and already existing topics. Apart from states inside concurrencies this works pretty well which leaves the original issue. I'll try to formulate a minimal example during the weekend.

[fmauch]

Alright, so here's a minimal example:
There's a series of log and wait states, in between there's a concurrent container. One of the concurrent lines contains another state machine container, which is the "main line" of the concurrent container. The following screenshot shows the structure of the whole behavior. The currently active state is highlighted in green, the screenshot was taken right after starting the behavior, so obviously the active state is PrintMessage.

The behavior was started with autonomy level 'no'. so what I would expect is that with the next transition I land inside ConcurrentContainer/StateMachineContainer/logstate1. When I enable the transition out of Print_Message, ConcurrentContainer/StateMachineContainer/logstate1 indeed gets highlighted shortly, but next I land inside Wait_After_Logging

The output for the whole behavior on /flexbe/debug/current_state is

data: /Print_Message > done
---
data: /Wait_After_Logging > done
---
data: /logEnd > done
---

The topic /flexbe/behavior_update (which is used for visualization, btw) states:

data: /Print_Message
---
data: /ConcurrentContainer/StateMachineContainer/logstate1
---
data: /Wait_After_Logging
---
data: /logEnd
---

The log output of all logstates is printed in stdout, so they actually get executed as expected. So what I would expect / What I am missing here:

• Execution notices of
  • ConcurrentContainer/logstate
  • ConcurrentContainer/logstate2
  • ConcurrentContainer/StateMachineContainer/logstate2
• Triggerable transitions from
• ConcurrentContainer/StateMachineContainer/logstate1
• ConcurrentContainer/StateMachineContainer/logstate2

I don't quite get this mirror thing, but as far as I have looked into the code, this has something to do with that.

[dcconner]

@pschillinger Any further discussion of this?

[dcconner]

Please see FlexBE/flexbe_app#88 for a discussion of new ros2-pre-release version that hopefully will address the issues raised here.