Skip to content

Implement basic varlink API and rhc-server daemon#366

Open
mjcr99 wants to merge 8 commits intomainfrom
macano/implement-rhc-server-scaffolding
Open

Implement basic varlink API and rhc-server daemon#366
mjcr99 wants to merge 8 commits intomainfrom
macano/implement-rhc-server-scaffolding

Conversation

@mjcr99
Copy link
Contributor

@mjcr99 mjcr99 commented Mar 3, 2026

Description

Hi team,

This PR implements a preliminary version of the varlink API we will need to communicate with the rhc-server daemon, which is also implemented as a first approach, providing a very basic functionality to test the communication.

Testing

Created a custom package from the branch, using RHEL 10.1, following dependencies are valid to have a working environment:

sudo dnf update -y
sudo dnf install python3 pip go bash-completion nano rpm-build tree -y
python3 -m pip install meson ninja go-vendor-tools
sudo dnf install https://dl.fedoraproject.org/pub/epel/epel-release-latest-10.noarch.rpm -y
sudo dnf install mock -y

Locate in the repository folder and run:

make srpm
mock -r fedora-43-x86_64 ~/rpmbuild/SRPMS/rhc-*.src.rpm

Install custom package;

sudo dnf install /var/lib/mock/fedora-43-x86_64/result/rhc-0.3.8-1.fc43.x86_64.rpm

We can now, enable the socket

[vagrant@lv-rhel101-1 rhc]$ sudo systemctl enable --now rhc-server.socket
Created symlink '/etc/systemd/system/sockets.target.wants/rhc-server.socket' → '/usr/lib/systemd/system/rhc-server.socket'.

The socket now shows as active, but the rhc-server daemon is not:

[vagrant@lv-rhel101-1 rhc]$ sudo systemctl status rhc-server.socket 
● rhc-server.socket - rhc server socket
     Loaded: loaded (/usr/lib/systemd/system/rhc-server.socket; enabled; preset: disabled)
     Active: active (listening) since Thu 2026-03-05 17:02:26 UTC; 1min 28s ago
 Invocation: f178d69c82594686b376a67eb73dfaa9
   Triggers: ● rhc-server.service
       Docs: https://github.com/RedHatInsights/rhc
     Listen: /run/rhc/com.redhat.rhc (Stream)
     CGroup: /system.slice/rhc-server.socket

Mar 05 17:02:26 lv-rhel101-1 systemd[1]: Listening on rhc-server.socket - rhc server socket.

[vagrant@lv-rhel101-1 rhc]$ sudo systemctl status rhc-server
○ rhc-server.service - rhc server
     Loaded: loaded (/usr/lib/systemd/system/rhc-server.service; disabled; preset: disabled)
     Active: inactive (dead)
TriggeredBy: ● rhc-server.socket
       Docs: https://github.com/RedHatInsights/rhc

Mar 05 16:20:28 lv-rhel101-1 rhc-server[20233]: 2026/03/05 16:20:28 INFO PID lock acquired pid=20233 pidFile=/run/rhc/rhc-server.pid
Mar 05 16:20:28 lv-rhel101-1 rhc-server[20233]: 2026/03/05 16:20:28 INFO Using systemd socket activation address=/run/rhc/com.redha>
Mar 05 16:20:28 lv-rhel101-1 rhc-server[20233]: 2026/03/05 16:20:28 INFO rhc-server starting version=0.3.8
Mar 05 16:20:28 lv-rhel101-1 rhc-server[20233]: 2026/03/05 16:20:28 INFO Listening on socket address=/run/rhc/com.redhat.rhc
Mar 05 17:01:04 lv-rhel101-1 systemd[1]: Stopping rhc-server.service - rhc server...
Mar 05 17:01:04 lv-rhel101-1 rhc-server[20233]: 2026/03/05 17:01:04 INFO Received signal, shutting down gracefully signal=terminated
Mar 05 17:01:04 lv-rhel101-1 rhc-server[20233]: 2026/03/05 17:01:04 INFO rhc-server stopped
Mar 05 17:01:04 lv-rhel101-1 rhc-server[20233]: 2026/03/05 17:01:04 INFO PID lock released
Mar 05 17:01:04 lv-rhel101-1 systemd[1]: rhc-server.service: Deactivated successfully.
Mar 05 17:01:04 lv-rhel101-1 systemd[1]: Stopped rhc-server.service - rhc server.

Then, we can send a request to the socket, the rhc-server will automatically start, we will get a response and the server will keep now listening for other calls:

[vagrant@lv-rhel101-1 rhc]$ varlinkctl call unix:/run/rhc/com.redhat.rhc com.redhat.rhc.internal.Test '{"input": "Helloo!"}'
{
        "output" : "Echo from rhc-server: Helloo!"
}

[vagrant@lv-rhel101-1 rhc]$ sudo systemctl status rhc-server
● rhc-server.service - rhc server
     Loaded: loaded (/usr/lib/systemd/system/rhc-server.service; disabled; preset: disabled)
     Active: active (running) since Thu 2026-03-05 17:04:35 UTC; 1min 18s ago
 Invocation: 7ea42d18d74b4384be2c632b37d7cd1c
TriggeredBy: ● rhc-server.socket
       Docs: https://github.com/RedHatInsights/rhc
   Main PID: 20611 (rhc-server)
      Tasks: 6 (limit: 16770)
     Memory: 3M (peak: 3.7M)
        CPU: 12ms
     CGroup: /system.slice/rhc-server.service
             └─20611 /usr/libexec/rhc/rhc-server

Mar 05 17:04:35 lv-rhel101-1 systemd[1]: Started rhc-server.service - rhc server.
Mar 05 17:04:35 lv-rhel101-1 rhc-server[20611]: 2026/03/05 17:04:35 INFO PID lock acquired pid=20611 pidFile=/run/rhc/rhc-server.pid
Mar 05 17:04:35 lv-rhel101-1 rhc-server[20611]: 2026/03/05 17:04:35 INFO Using systemd socket activation address=/run/rhc/com.redha>
Mar 05 17:04:35 lv-rhel101-1 rhc-server[20611]: 2026/03/05 17:04:35 INFO rhc-server starting version=0.3.8
Mar 05 17:04:35 lv-rhel101-1 rhc-server[20611]: 2026/03/05 17:04:35 INFO Listening on socket address=/run/rhc/com.redhat.rhc

[vagrant@lv-rhel101-1 rhc]$ varlinkctl call unix:/run/rhc/com.redhat.rhc com.redhat.rhc.internal.Test '{"input": "Hello again!"}'
{
        "output" : "Echo from rhc-server: Hello again!"
}

The service can be safely stopped, and the socket remains:

[vagrant@lv-rhel101-1 rhc]$ sudo systemctl stop rhc-server
Stopping 'rhc-server.service', but its triggering units are still active:
rhc-server.socket
[vagrant@lv-rhel101-1 rhc]$ ls /run/rhc
com.redhat.rhc

The socket can be reconnected, and the service will automatically boot as well.

[vagrant@lv-rhel101-1 rhc]$ sudo varlinkctl call unix:/run/rhc/com.redhat.rhc com.redhat.rhc.internal.Test '{"input": "Hello again after stopping service!"}'
{
        "output" : "Echo from rhc-server: Hello again after stopping service!"
}

@mjcr99 mjcr99 force-pushed the macano/implement-rhc-server-scaffolding branch 4 times, most recently from 207c4af to 7671637 Compare March 3, 2026 16:18
@mjcr99 mjcr99 requested a review from m-horky March 3, 2026 17:10
Copy link
Contributor

@pkoprda pkoprda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR, I have left some comments below.

@mjcr99 mjcr99 force-pushed the macano/implement-rhc-server-scaffolding branch from 0ac7e54 to 860dc2b Compare March 4, 2026 10:10
@jirihnidek
Copy link
Contributor

Hi @mjcr99 ,
Thanks for the PR. Please modify Makefile and please add this line to build target:

go build -ldflags "-X main.Version=${VERSION}" -o rhc-server ./cmd/rhc-server

Then the rhc-server will be build, when make is run.

Thanks in advance

@mjcr99 mjcr99 force-pushed the macano/implement-rhc-server-scaffolding branch 2 times, most recently from 28eb80d to 83e397c Compare March 4, 2026 12:55
Copy link
Contributor

@jirihnidek jirihnidek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR. I have few comments and request in comments.

I have another request here: could you please add rhc-server.service file and could you please modify rhc.spec file, because your PR contains some code that is not possible test without these changes.

Copy link
Contributor

@pkoprda pkoprda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some comments below.

Please also fix the typo in the commit message d367b32 to feat(varlink api): added preliminary varlink API implementation (preliminar -> preliminary) and remove the dot so it would fit the conventional commit format.

@mjcr99 mjcr99 force-pushed the macano/implement-rhc-server-scaffolding branch from 83e397c to 54a40d6 Compare March 5, 2026 16:32
@mjcr99 mjcr99 requested review from jirihnidek, m-horky and pkoprda March 5, 2026 16:48
Copy link
Contributor

@pkoprda pkoprda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you also write some unit tests?

Copy link
Contributor

@jirihnidek jirihnidek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your update. I can confirm that it works as expected and the service is started automatically, when rhc-server.socket is enabled and call the Varlink method using varlinkctl.

However, when I stop the rhc-server.service using systemctl stop rhc-server.service, then something deletes the /run/rhc directory including socket file and then it is not possible to call varlink method again, because the socket service does not have the socket file and it cannot trigger the rhc-server.service. Maybe, there is something wrong in service configuration or some function in rhc-server has side-effect and it deletes this directory.

@mjcr99 mjcr99 force-pushed the macano/implement-rhc-server-scaffolding branch from 54a40d6 to 6285684 Compare March 10, 2026 12:42
@mjcr99 mjcr99 requested review from jirihnidek and pkoprda March 10, 2026 12:42
@mjcr99 mjcr99 force-pushed the macano/implement-rhc-server-scaffolding branch 5 times, most recently from bc7ea71 to 6df2545 Compare March 11, 2026 11:10
* Card ID: CCT-1753

This initial implementation support a test call that returns the same
received message.

Signed-off-by: mjcr99 <macano@redhat.com>
Assisted-by: Claude Code
mjcr99 added 4 commits March 11, 2026 12:34
* Card ID: CCT-1753

Add a minimal backend implementation that handles varlink Test method
requests following the pattern established in go-varlink/example.

The Test method accepts an input string and returns it prefixed with
"Echo from rhc-server:" to demonstrate bidirectional communication.

Signed-off-by: mjcr99 <macano@redhat.com>
Assisted-by: Claude Code
* Card ID: CCT-1753

Implement the rhc-server daemon that exposes varlink API over a Unix
socket for internal RHC communication.

Key features:
- Uses govarlink.NewRegistry() to handle varlink requests
- Systemd socket activation support with fallback to manual socket creation
- Unix socket at /run/rhc/com.redhat.rhc with 0666 permissions
- Graceful shutdown on SIGINT/SIGTERM signals
- Integrates with internal/rhc-server backend for request handling

The server follows the architecture pattern from go-varlink/example,
registering the internal API handler and serving requests through
the varlink protocol.

Signed-off-by: mjcr99 <macano@redhat.com>
Assisted-by: Claude Code
* Card ID: CCT-1753

It also updates the .gitignore file to avoid adding rhc-server file and
updates the clean target to clean the rhc-server binary as well.

Signed-off-by: mjcr99 <macano@redhat.com>
* Card ID: CCT-1753

Add a PID file-based locking mechanism to ensure only one instance of
rhc-server can run at a time. This addresses the issue where starting
rhc-server multiple times could lead to socket conflicts and communication
failures.

The PID lock prevents the race condition where:
- First rhc-server instance creates socket A
- Second rhc-server instance removes socket A and creates socket B
- Second instance terminates and removes socket B
- First instance becomes unreachable

With this change, the second instance will fail immediately with:
"another instance of rhc-server is already running"

The implementation is robust against crashes: flock() is automatically
released by the kernel when the process terminates (even with kill -9),
allowing new instances to start cleanly after a crash.

Key changes:

1. PID Lock Implementation:
- Introduce acquirePIDLock() function that creates and locks
/run/rhc/rhc-server.pid using flock(LOCK_EX|LOCK_NB)
- Non-blocking lock ensures immediate failure if another instance
is already running
- Returns cleanup function for proper lock release and PID file removal
- Lock is automatically released by kernel if process crashes or is killed

2. Improve systemd socket activation:
- Iterate through all listeners from activation.Listeners() instead of
blindly using the first one
- Verify listener.Addr().Network() == "unix" to ensure we get the
correct socket type
- Return descriptive error if systemd provides listeners but none
are unix sockets
- This makes the code more robust for future scenarios where multiple
listeners of different types might be provided

3. Socket removal safety:
- Add comments explaining that removing existing socket is now safe
 because we hold the PID lock
- If socket exists, it means previous instance terminated abnormally
- No risk of removing socket from running instance

4. Permission changes:
- Change socket permissions from 0666 to 0660 for better security

Signed-off-by: mjcr99 <macano@redhat.com>
Assisted-by: Claude Code
@mjcr99 mjcr99 force-pushed the macano/implement-rhc-server-scaffolding branch from 6df2545 to 6749c23 Compare March 11, 2026 11:34
ReadWritePaths=/run/rhc
RuntimeDirectory=rhc
RuntimeDirectoryMode=0755
RuntimeDirectoryPreserve=yes
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

mjcr99 added 3 commits March 11, 2026 16:01
…erver

* Card ID: CCT-1753
* Card ID: CCT-1754
Introduce systemd service and socket units to enable rhc-server to run as
a system service with socket activation support. Update the RPM spec file
to build, install, and manage the rhc-server daemon.

New systemd units:
1. rhc-server.socket:
- Defines the Unix socket at /run/rhc/com.redhat.rhc
- Enables socket activation (service starts on-demand when client connects)
- Socket mode 0660

2. rhc-server.service:
- Executes /usr/libexec/rhc/rhc-server binary
- Requires and orders after rhc-server.socket for socket activation
- RuntimeDirectory directive ensures /run/rhc is created with correct
permissions (0755)
- ReadWritePaths grants write access to /run/rhc for PID lock
- Logs to systemd journal for centralized log management
- RuntimeDirectoryPreserve ensures /run/rhc folder is not deleted if the
service is stopped

RPM packaging changes (rhc.spec):

Build phase:
- Compile rhc-server binary alongside rhc client

Install phase:
- Install rhc-server to /usr/libexec/rhc/ (daemon, not user-facing)
- Install systemd units to /usr/lib/systemd/system/
- Improved rhc binary installation from wildcard to explicit path
to avoid unintended files

Scriptlets:
- %post: Enable rhc-server.socket after package installation
- %preun: Stop socket and service before package removal
- %postun: Restart service after package update
- %files: Declare rhc-server binary and systemd units as package files

Socket activation benefits:
- Service starts automatically when client connects (lazy loading)
- Systemd handles socket lifecycle and file descriptor passing
- Compatible with rhc-server PID lock for instance management

Deployment workflow:
1. Install RPM package
2. systemctl enable --now rhc-server.socket
3. Service auto-starts when rhc client connects to the socket
4. PID lock prevents multiple instances from running concurrently

Signed-off-by: mjcr99 <macano@redhat.com>
Assisted-by: Claude Code
…ming conventions

* Card ID: CCT-1753

Reorganize rhc-server code structure and rename varlink interface files
to follow the fully qualified naming convention used in varlink ecosystems.

Also moves backend implementation from internal/rhc-server to cmd/rhc-server.
No functional changes - this is purely a structural refactoring.

Signed-off-by: mjcr99 <macano@redhat.com>
Assisted-by: Claude Code
Add comprehensive unit tests for the Backend struct and its Test method
in cmd/rhc-server. Tests cover various input scenarios.

Signed-off-by: mjcr99 <macano@redhat.com>
Assisted-by: Claude Code
@mjcr99 mjcr99 force-pushed the macano/implement-rhc-server-scaffolding branch from 6749c23 to a3fa20e Compare March 11, 2026 15:02
Copy link
Contributor

@pkoprda pkoprda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good from my side, I will let @jirihnidek and @m-horky finalize the review :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants