Browse compose files on Remote Server via Hawser Agent possible?

[ptC7H12]

Hello,
I've installed a remote Server and connected it to my dockhand instance via Hawser edge mode.
I can see the Server and it's stacks, but can't browse its filesystem to add the compose files to the untracked stacks.

Is it possible? Do I have to create a local compose and deploy this on the remote Server?

Thank you and kind regards..

[jarrah31]

I too would like the ability to import compose files from stacks accessed via Hawser Standard hosts.
I've deployed Hawser as docker containers and mapped a volume as an external stack directory.
Perhaps Locations could list the different Environments which in turn would show the folder structure of the remote Hawser client?

[t0g3pii]

+1 on this one.
I have many Hawser Edge Hosts connected to my Dockhand.
Either see+edit the Stack on the Remote Server, or see it once on the Remote Server and let it auto import to dockhand (and handle the rest from there)

[BadFlo]

+1 from me also.
Arcane-edge-agent enable you to access remote stacks compose files if you want to compare.

[Nirzak]

+1. Will like to have this feature

[Dampfwalze]

+1 I too want to show my support for this!

When I tried Dockhand, I really liked the model that I could keep my compose files, config files and mount points together. Sadly, with the current way of how remote environments work, this model falls apart.

[WhimsicalWalrus]

+1 as well!

Forgot to add this:

I got "around" it with a quick hack, it works...
On your Dockhand Host VM, mount the remote folder locally using SSHFS

Bash
sudo apt install sshfs -y
sudo mkdir -p /mnt/remote-host
sudo sshfs -o allow_other root@<REMOTE_IP>:/usr/docker /mnt/remote-host (or whatever account/location you use/want)

Update Dockhand's compose and map that new folder into the Dockhand container.

volumes:

• /mnt/remote-host:/usr/remote-host

In the UI click the folder icon on your untracked stack and just browse to the mount. Once the YAML opens in the editor, make sure your volume paths are absolute (e.g., /usr/docker/app/data) so the remote host knows where its own files are when it actually runs the containers.

[thutex]

to me this is more of a bug: dockhand is intended to manage my stacks, but it can't track stacks for remote hosts,
meaning it can't fully/correctly manage my stacks.

and i see there's a few issues open about it, where it's stated that "this is on the roadmap":
#143
#584

[yannduran]

It's not really a bug as it's not behaving incorrectly per se, it's just functionality that's missing. It's basically a feature request.

Thanks for the links to those issues. I've upvoted them both.

[hiagocosta]

do we have any news about it ? would be very good to browse hawser host and get from there.
my workaround is use a private github repo with the stacks and synchronize with a script on the host

[yannduran]

Tedious to have to do something like that, but very clever workaround!

[hiagocosta]

the worst part is to change it on GH every time you need to modify something on the compose. if the Dockhand could edit the GH compose files like on local host yaml file, we just will need to use only one platform instead of two changes

sorry for any errors on my english

[yannduran]

+1 on this. I'm managing stacks on a remote NAS via Hawser Standard mode, and the inability to browse the remote filesystem makes it impossible to track existing stacks. The only workaround of recreating stacks from scratch in the managing instance of DockHand violates DRY principles and risks conflicts with already-running containers.

Being able to browse and import compose files from remote Hawser hosts would make multi-host management actually practical for those of us running existing infrastructure.

Great work on this project btw. I'm a convert from Portainer (which drives me nuts sometimes).

[hiagocosta]

hahah, me too, changed from portainer to dockhand made me learning better how docker and containers work

[Dampfwalze]

Came here too because I couldn't see Portainer anymore! It was just lacking everywhere! Once you start actually using compose, the cracks appear rapidly!

[Dampfwalze]

After more usage with Dockhand, I started to like the idea that the one source of truth is on the managing node and once I fixed my messup, it also correctly syncs depending files (using the git integration). That is actually a really nice workflow. The thing that I am mostly missing is that I cannot adopt stacks from remote nodes. Currently the only way to do that is to tear everything down on the remote node and rebuild everything from the manager node. It should be possible to adopt those stacks, so that they are moved to the manager node and be integrated in the system.

This still means that you cannot change your stacks on the remote node outside of Dockhand, because Dockhand will override the changes again. To allow changes on both sides, Dockhand must either implement aggressive syncing mechanisms, and/or be able to merge conflicting files. Both of which are hard to solve without much benefit. It would be easier to let the user decide which node should be the only source of truth. So for every stack, the user should be able to decide that. And it should also be possible to later change you mind and move a stack from one node to the other.

[radoeka]

Like @hiagocosta I use a local git repository ([forgejo)[https://forgejo.org]) for this purpose.
Each host is connected with the git repository.
Make a change on the compose file, run it.
If ok, commit to the repository -> a webhook will automatically sync on Dockhand.
I've a small script that syncs with the repository on all the hosts.
It is easy to work with.

Dockhand can automatically sync with git repository, but I don't use it.

Further I use NFS so containers can be stopped on 1 host and then moved to another.

[radoeka]

@rxunique repo-first for me has a positive effect, as it now easy to change the compose files, knowing that the working one is always present in the Git repository.

The fun thing with the repo-first is that you can have many checkouts, on the workstation, on the remote node - on local file system, as well as on a NFS shared file system. This makes it easy to move containers around over various nodes (without the need for Docker swarm).

Because the compose and related files can be retrieved from the Git repository they can also be worked on with VScode. I like it this way a lot.

[hiagocosta]

@radoeka do you use your volume as NFS file system?

If yes, you mount in the host and start the container or use NFS driver on docker?

[rxunique]

@radoeka

On NFS, yes, I would totally switch to your setup, like you said, move without swarm.

On local file without NFS, can dockhand move bind mounts (maybe also volumes) across VMs with couple clicks? Correct me if I'm wrong, I don't think it or docker can do it, which is the reason of my current setup of manual rsync when needed and wishing for dockhand to have local compose

[radoeka]

@hiagocosta mounted on the host. Like so:

srv/docker/swarm $ ls -1a

bin
conf
data
stacks
.volumes

I called the NFS mounted directory swarm, but it is not Docker swarm (but may be later).
The stacks are stored in directory stacks and are managed with Git.
The volumes are in the .volumes directory with the same directory structure as the stacks directory structure.

For apps that I want to be always available independent of NFS, I have a local dock directory docks:

/srv/docker/docks $ ls -1a

dock101
.dockhandler
.volumes

All the above directories are local, and can be used for critical applications such as Pihole, that must be available even if the node that provides NFS is down are unavailable.

• dock101 (on node 01) has all the stacks from Git. On node02 it is dock102, on node03 it is dock103 and so on.
• .dockhandler is for Hawser handling the Dockhand files.
• .volumes is for local stores volumes.

For now I mainly focus on the situation that the node providing the NFS service breaks. Is it in that situation possible to continue the other services on other nodes and in the meantime replace the (bad) NFS node? There maybe some downtime, but still working on it though. However, this setup makes it already possible to start a container (stacks) easily on various nodes. Traefik-kop takes care that the URL remains functional automatically. Quite nice :)

The NFS directory is synced to another node, that can then function as NFS node. For now synced with rsync, but I'm looking making it possible on and with ZFS.

Something to investigate as well, is it possible to move Dockhand from its current node c103 where is uses the socket /var/run/docker.sock to for example c102, in which case it has no access anymore to the socket on cl103. I think a solution for this could to also use Hawser on the node where Dockhand is running...

Hope this helps.

[yannduran]

In my experience, repo first is cumbersome

You're absolutely right! That's exactly why I don't use it anymore. I'm retired, so my containers aren't in production as such. Only locally (well, on a NAS). But it's exhausting!

Testing every little change requires the whole cycle to be performed:

a) edit code
b) commit to git
c) push to repo
d) redeploy the updated code (that'd be ameliorated by using web hooks, but I wasn't doing that)

It also means you're constantly downing the whole stack every time you want to test a small change/fix.

so much simpler if compose & .env is in a single SSH or VScode session

These days that's exactly what I do. I edit & save in vscode. I keep an ssh session in the vscode terminal, I can then deploy the changes straight away by downing the stack in the terminal, then redeploying the stack from the terminal, using docker compose up -d again.

If I know I'm only testing stuff out, I can remove just the container, then up it on its own without affecting every container in the stack. DockHand automatically creates its own stack of one container, and once I'm happy it's working properly I can remove the test container (the test stack disappears) or remove the created stack (the container gets removed). Then to get the container back into its original stack I down the original stack from the terminal, then redeploy it using docker compose up -d again.

If anyone has a better process for doing this kind of testing, please let me know.

[radoeka]

See Dockhand manual for a description of the stack architecture.

[KaKi87]

@jotka 🙏