Reloading of units

[KillingSpark]

Rustysd needs a control interface command that triggers a complete reload of all units, which integrates changed / new units into the running system.

Adding and activating new units should be done similarly to the initial startup by walking the dependency graph, but ignoring unchanged units. Changed units should be killed and restarted with the new configuration

Currently there is only support for adding new units one by one which is fine for manual enabling of services but not for automatically adding a bunch of services with dependencies between them.

[cdbattags]

IMO would be interesting to use Blake3 hashes for checking changed units but maybe that's taking things too far?

[KillingSpark]

Using hashes could lead to false positives, I think. Just because the file hash changed doesn't mean the unit changes in a meaningful way. Comments/Formatting can change without the unit having to be restarted

[pwFoo]

Would be a nice feature, but complexity should be in mind. I like the small size and simple usage of rustysd...

update
simple commands to enable / disable, start / stop / restart and add / remove and "add all new units" would be a nice first step and could work for me if it's easier to do.

[KillingSpark]

Those exist already, with the exception of the last one. And adding the last one shouldn't be too tough

[KillingSpark]

The control interface is somewhat undocumented as of yet, this will need to change but there are some docs in here

There is rsdctl in src/bin/ which can convert cli args into jsonrpc and send it to rustysd.

[pwFoo]

Thanks.
For first steps commands would be fine! I also see a shutdown cmd? Looks like poweroff? And is there a reboot cmd possible to trigger a system / rustysd reboot?

So it is used like rsdctl <CMD> <UNIT>? I tried it and the usage help shows me :, but there is no service listening?

I build the binaries and have:

-rwxr-xr-x 1 root root 3,0M Jan 31 20:06 rsdctl
-rwxr-xr-x 1 root root 4,4M Jan 31 20:06 rustysd

You write about the binary size of 2MB? How to build smaller binaries?

[MggMuggins]

You mention walking the tree and ignoring unchanged units and restarting those that have changed. When a unit is restarted, are all of its dependents restarted as well?

[KillingSpark]

@MggMuggins That is an interesting question. If the socket files of a service change then potentially everything needs to be restarted. I will need to think about this some more

[KillingSpark]

@pwFoo Shutdown just shuts down rustysd (by killing all services), it's not a 'poweroff' command. The rsdctl works like this: 'rsdctl '. The argument can be a unit name for example.

As for building smaller binaries look at the build script for the docker container. It involves stripping the binaries of unneeded symbols

[KillingSpark]

I am not sure how systemd handles this in their daemon-reload. From their doc:

Use systemctl daemon-reload or an equivalent command to reload unit configuration while the unit is already loaded. In this case all configuration settings are flushed out and replaced with the new configuration (which however might not be in effect immediately), however all runtime state is saved/restored.

Theres also a "ExecReload=" on services but nothing specific for reloading in sockets.

Some questions that have to be answered before implementing this:

1. Does this mean, the services don't get restarted even if their config changed?
2. Are socket units closed and reopened if their config changes?
3. (And depending on the answers to those above, do we want that behavior or do we accept differences from systemd in favor of a possible better design?)

[pwFoo]

Some time ago I changed a systemd unit and get notified during restart that daemon-reload is needed to reload the changed configuration. But I think it reloads the files and NOT restart / reload the units...?
I'm not sure about, but I think so...

a rustysd poweroff / reboot handling would be nice, but I don't know how it's done in systemd or busybox init...

[KillingSpark]

with systemd poweroff/reboot/suspend/hibernate is handled by systemd-logind which presumably tells systemd to shutdown all services and then shutdown linux

[pwFoo]

@pwFoo Shutdown just shuts down rustysd (by killing all services), it's not a 'poweroff' command. The rsdctl works like this: 'rsdctl '. The argument can be a unit name for example.
But compared to systemctl I would try to use rsdctl the same way...

rsdctl status <service>
rsdctl restart <service>
...

rsdctl : [...] is required? And how to start rustysd web service? Or could it be possible to use a local socket to rustysd?

Would be interesting how to "install" / enable a unitfile by command in the near future.
Systemd unit files are sometimes in sub directories. During testing I could be run into a problem with Wants / Required dependencies and not existing directory inside of unitfiles? Is it possible?
If rustysd doesn't take care about creating that sub directories it would be a nice new feature to just execute an enable cmd to auto create needed sub directories with the symlink / file?

[KillingSpark]

Rustysd automatically listens on a unix socket and on a tcp socket for connections that send a jsonrpc call. rsdctl is just a convenient tool to send these commands, use whatever you like :)

for example this should work: echo '{"method": "restart", "params": "test.service"}' | socat - TCP-CONNECT:0.0.0.0:8080

As for subdirs: rustysd scans the given unitdirs recursively for units so subdirs should be detected. I am not sure how that plays together with the dependencies? Rustysd currently does not manage unit files like systemctl does. Rustysd just takes the dirs it is given as is. If you want to take the 'link it in to enable' way it would probably be best to extend rsdctl to do so

[KillingSpark]

So there are some settings that can change while a unit is active (Like before/after/requires/...). But there are some that can't really like the ExecStart/ExecPrestart settings. For sockets there also exist settings in both categories. This seems very complicated. For your usecase it might even be easier to not use an overlay but to spawn a second instance of rustysd for the units that appear after mounting all drives.

[pwFoo]

Maybe I move the mount part to /init and exec to rustysd with prepared drives. I'll do some tests. Thanks!

[pwFoo]

Those exist already, with the exception of the last one. And adding the last one shouldn't be too tough

So just reload all and add new units is missing? I would suggest to add it as simple as possible.
Rescan unitfiles directory, but do not restart anything because of running services and maybe complex dependencies for running services?

[KillingSpark]

Adding new units is supported. You just move the unit into the directory and say rsdctl <addr> enable my_new_unit.service. I think I will first add the rescan and adding all new units and ignore changes to other units until later. Maybe a warning should be generated though. Something like "All new units have been added but rustysd detected changed units. For those changes to take effect please restart rustysd"

[KillingSpark]

@pwFoo I just added loading of new units. rsdctl reload should make rustysd load all newly appeared units in the unitdirs rustysd scans

[pwFoo]

Hi @KillingSpark,
rsdctl /notifications/control.socket reload seems to work. enable, restart and stop are known methods, but looks like start / disable are unknown:

Write cmd: {"jsonrpc":"2.0","method":"disable"}
Wait for response
Got response
{
  "error": {
    "code": -32601,
    "message": "Unknown method: disable"
  },
  "jsonrpc": "2.0"
}

Write cmd: {"jsonrpc":"2.0","method":"start"}
Wait for response
Got response
{
  "error": {
    "code": -32601,
    "message": "Unknown method: start"
  },
  "jsonrpc": "2.0"
}

[KillingSpark]

You might wanna have look at the doc/ControlInterface.md that I wrote earlier today. Disable is still missing (stop works though!), and to start units you can use restart :)