Re-design publish/subscribe interface, prepare for multiple SSR instances

[mgeier]

This PR is supposed to keep the previous functionality mostly unchanged. For now, the only visible new feature is the introduction of the --follow flag which should facilitate the future feature that one SSR instance can connect to another SSR instance. Since the current network interface doesn't support that, it doesn't really do anything (except making it impossible to load a scene).

I extended the internal APIs to allow network clients to subscribe to specific groups of messages (but the existing network client doesn't support that and most likely never will) and to allow 3D source/reference positions and rotations (but the existing renderers and the GUI are still 2D).

The new API is defined in src/api.h. I recommend to build the Doxygen documentation with make doc and read the pages about the ssr::api namespace.

This is just one step towards multiple network interfaces and 3D support, there will be more PRs coming!

Ideas (some of which are implemented in this PR):

• There is only one scene (including transport state) that's shared between all SSR instances.
• Each SSR instance has exactly one renderer and one reproduction setup, but there might be multiple SSR instances with different renderers connected via network interface(s).
• An SSR instance can either be a "leader" (which has central control over the scene, including JACK transport) or a "follower" (which is supposed to have a connection to a "leader" at some point).
• There can be at most one "leader" in the network, but many "followers".
• When starting an SSR instance, it can be decided if this instance is the "leader", using the --no-follow option (which is the default). Alternatively, it can also immediately start to "follow" another SSR instance (to be implemented). Finally, it can use --follow and simply wait for the "leader" to connect (to be implemented).
• A network "client" (such as a GUI) can connect to either a "leader" or a "follower". It will be able to get information about the renderer of the SSR instance it is directly connected to (but no other renderers in the network). If it is connected to a "follower", it will immediately get information about its renderer, but it will only get information about the scene (and be able to control sources) as soon as the "follower" is connected to the "leader".
• Control messages are grouped in "bundles". During one bundle, the whole SSR instance is locked with a mutex. For now, there is only one central mutex (I even removed locking from the renderer). Network interfaces can choose if they implement bundles or not, the legacy network interface doesn't. The idea is that future network interfaces will send all changes from one "bundle" in a single network message (to save network overhead).

Some collateral changes in this PR:

• The trackers now affect the "reference offset" instead of the "reference" directly.
  Reason: The "reference offset" is a property of the renderer, not the scene.

Some Questions:

• I changed source IDs from being integers to strings (except in the legacy network protocol), any objections?
• The problem with strings is that I don't know if the API functions should use const std::string& or std::string or std::string_view or const char* or ...?
• In general, I'm open for suggestions about changing certain data types in the API
• Some names could probably be improved, I'm open for suggestions!
• I've switched to C++17, anyone against that?

Future Work (not part of this PR):

• Implement dynamic ASDF scenes
• Implement OSC interface, see Networking with osc #122
• Implement WebSocket interface, see https://github.com/SoundScapeRenderer/ssr/wiki/SSR-IP-OSC-interface
• Implement browser-based GUI (and deprecate the built-in GUI?), see https://github.com/SoundScapeRenderer/ssr/wiki/Browser-GUI
• Extend renderers to 3D
• (subscribable) interface to propagate error/warning messages back to clients?
• Provide a way for a client to "take over" one source (or multiple sources), blocking other clients (and a future scene player) from moving that source.
• Implement a special kind of SSR instance that is able to be the "leader" and connect to multiple "followers" that are already running (= nodes in a rendering cluster). It will probably have a special kind of renderer that doesn't do any actual rendering but just combines the information from the rendering nodes in order to provide it to a connected GUI. This special kind of renderer will also have to be able to read a dynamic ASDF scene (to be implemented) and provide the JACK ports of the sound sources to the rendering nodes.
• Make JACK connections work between sound sources and SSR instances. Make this work even if they run on different computers (and are e.g. connected via MADI, or probably NetJACK?)
• Implement "partial" reproduction setups (if multiple rendering nodes are used for a single loudspeaker setup)

[dvzrv]

There can be at most one "leader" in the network, but many "followers".

Do you mean subnet by that or network of SSRs?

A network "client" (such as a GUI) can connect to either a "leader" or a "follower". It will be able to get information about the renderer of the SSR instance it is directly connected to (but no other ??renderers in the network). If it is connected to a "follower", it will immediately get information about its renderer, but it will only get information about the scene (and be able to control sources) as soon as the "follower" is connected to the "leader".

Would this also translate to the Qt GUI? That could be interesting (eventually).

Control messages are grouped in "bundles". During one bundle, the whole SSR instance is locked with a mutex. For now, there is only one central mutex (I even removed locking from the renderer). Network interfaces can choose if they implement bundles or not, the legacy network interface doesn't. The idea is that future network interfaces will send all changes from one "bundle" in a single network message (to save network overhead).

Is this thought of as "OSC bundles"? The OSC interface I designed doesn't support this either though ;-)

The problem with strings is that I don't know if the API functions should use const std::string& or std::string or std::string_view or const char* or ...?

I would probably opt for const std::string& or std::string

[dvzrv]

Would be great to be able to use this from a system package as well.
Currently there's no release of this header-only library though.

Regarding submodules: Please keep in mind, that if you do a git tag, github will not include submodules in your tarball!

[mgeier]

Currently there's no release of this header-only library

Yes, it's not very well known. But I liked it because it is very small. Most other libraries I found had a million other features that I don't need.

If you know a better option, just let me know, we can easily switch. I deliberately didn't use GML types in any API functions.

github will not include submodules in your tarball!

I know. But Github's tarball is quite useless anyway, because it doesn't have the ./configure in it. We ship our separate tarballs anyway.

[dvzrv]

Will have a look at the API more closely after work (hopefully). It looks thoroughly documented and more clear than the previous Publisher/Subscriber implementation though! Congrats :)

[mgeier]

Thanks for the review!

There can be at most one "leader" in the network, but many "followers".

Do you mean subnet by that or network of SSRs?

Oh, sorry, I didn't see the ambiguity ...

I meant "the network of interconnected SSR instances".

It doesn't matter if there are multiple "leaders" available on the physical (or virtual) network, but they cannot connect to each other.

Would this also translate to the Qt GUI? That could be interesting (eventually).

I'm not sure if I understand. Each SSR instance can have it's own (optional) Qt GUI.

Or do you mean a stand-alone Qt GUI without renderer?

This was our plan many years ago, but this should be obsolete once we have a usable browser-based GUI.

Is this thought of as "OSC bundles"?

That's where I took the name from.
And the OSC interface could of course use OSC bundles to implement those bundles.
But it doesn't have to. It can completely ignore the bundle_start and bundle_stop events (it doesn't even have to subscribe to BundleEvents) and simply send each event as a separate message over the network.

The OSC interface I designed doesn't support this either though ;-)

That's not a problem, it's completely optional.

The WebSocket interface I'm currently working on (in the websocket branch) does support bundles and only sends one WebSocket message per bundle.

It would also be quite natural for the XML-based interface to put multiple events into one network message. But this wasn't done before and I didn't want to change it. So it ignores the bundle events.

I would probably opt for const std::string& or std::string

OK, cool, I'm currently using const std::string& almost everywhere.