Need help, not running - Portainer issues

[edenon]

NNN---@NNN-TEST:/netprobe_lite$ docker compose up
WARN[0000] /home/NNN---/netprobe_lite/compose.yml: version is obsolete
[+] Running 5/0
✔ Container netprobe-presentation Created 0.0s
✔ Container netprobe-grafana Created 0.0s
✔ Container netprobe-probe Created 0.0s
✔ Container netprobe-redis Created 0.1s
✔ Container netprobe-prometheus Created 0.0s
Attaching to netprobe-grafana, netprobe-presentation, netprobe-probe, netprobe-prometheus, netprobe-redis
netprobe-prometheus | ts=2024-05-03T02:41:39.181Z caller=main.go:573 level=info msg="No time or size retention was set so using the default time retention" duration=15d
netprobe-prometheus | ts=2024-05-03T02:41:39.181Z caller=main.go:617 level=info msg="Starting Prometheus Server" mode=server version="(version=2.51.2, branch=HEAD, revision=b4c0ab52c3e9b940ab803581ddae9b3d9a452337)"
netprobe-prometheus | ts=2024-05-03T02:41:39.181Z caller=main.go:622 level=info build_context="(go=go1.22.2, platform=linux/amd64, user=root@b63f02a423d9, date=20240410-14:05:54, tags=netgo,builtinassets,stringlabels)"
netprobe-prometheus | ts=2024-05-03T02:41:39.181Z caller=main.go:623 level=info host_details="(Linux 5.4.0-177-generic #197-Ubuntu SMP Thu Mar 28 22:45:47 UTC 2024 x86_64 b856a8d8dbca (none))"
netprobe-prometheus | ts=2024-05-03T02:41:39.181Z caller=main.go:624 level=info fd_limits="(soft=1048576, hard=1048576)"
netprobe-prometheus | ts=2024-05-03T02:41:39.181Z caller=main.go:625 level=info vm_limits="(soft=unlimited, hard=unlimited)"
netprobe-prometheus | ts=2024-05-03T02:41:39.184Z caller=web.go:568 level=info component=web msg="Start listening for connections" address=0.0.0.0:9090
netprobe-prometheus | ts=2024-05-03T02:41:39.185Z caller=main.go:1129 level=info msg="Starting TSDB ..."
netprobe-prometheus | ts=2024-05-03T02:41:39.186Z caller=dir_locker.go:77 level=warn component=tsdb msg="A lockfile from a previous execution already existed. It was replaced" file=/prometheus/lock
netprobe-prometheus | ts=2024-05-03T02:41:39.192Z caller=tls_config.go:313 level=info component=web msg="Listening on" address=[::]:9090
netprobe-prometheus | ts=2024-05-03T02:41:39.192Z caller=tls_config.go:316 level=info component=web msg="TLS is disabled." http2=false address=[::]:9090
netprobe-prometheus | ts=2024-05-03T02:41:39.194Z caller=head.go:616 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any"
netprobe-prometheus | ts=2024-05-03T02:41:39.194Z caller=head.go:698 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=2.95µs
netprobe-prometheus | ts=2024-05-03T02:41:39.194Z caller=head.go:706 level=info component=tsdb msg="Replaying WAL, this may take a while"
netprobe-prometheus | ts=2024-05-03T02:41:39.225Z caller=head.go:778 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=3
netprobe-prometheus | ts=2024-05-03T02:41:39.226Z caller=head.go:778 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=3
netprobe-prometheus | ts=2024-05-03T02:41:39.227Z caller=head.go:778 level=info component=tsdb msg="WAL segment loaded" segment=2 maxSegment=3
netprobe-prometheus | ts=2024-05-03T02:41:39.227Z caller=head.go:778 level=info component=tsdb msg="WAL segment loaded" segment=3 maxSegment=3
netprobe-prometheus | ts=2024-05-03T02:41:39.227Z caller=head.go:815 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=95.788µs wal_replay_duration=32.827096ms wbl_replay_duration=161ns total_replay_duration=32.974262ms
netprobe-prometheus | ts=2024-05-03T02:41:39.230Z caller=main.go:1150 level=info fs_type=EXT4_SUPER_MAGIC
netprobe-prometheus | ts=2024-05-03T02:41:39.230Z caller=main.go:1153 level=info msg="TSDB started"
netprobe-prometheus | ts=2024-05-03T02:41:39.230Z caller=main.go:1335 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml
netprobe-prometheus | ts=2024-05-03T02:41:39.233Z caller=main.go:1372 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=2.440331ms db_storage=1.321µs remote_storage=1.21µs web_handler=592ns query_engine=1.12µs scrape=2.11521ms scrape_sd=38.265µs notify=795ns notify_sd=1.333µs rules=1.361µs tracing=6.411µs
netprobe-prometheus | ts=2024-05-03T02:41:39.233Z caller=main.go:1114 level=info msg="Server is ready to receive web requests."
netprobe-prometheus | ts=2024-05-03T02:41:39.233Z caller=manager.go:163 level=info component="rule manager" msg="Starting rule manager..."
netprobe-redis | 1:C 03 May 2024 02:41:39.248 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see jemalloc/jemalloc#1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
netprobe-redis | 1:C 03 May 2024 02:41:39.248 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
netprobe-redis | 1:C 03 May 2024 02:41:39.248 * Redis version=7.2.4, bits=64, commit=00000000, modified=0, pid=1, just started
netprobe-redis | 1:C 03 May 2024 02:41:39.248 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
netprobe-redis | 1:M 03 May 2024 02:41:39.249 * monotonic clock: POSIX clock_gettime
netprobe-redis | 1:M 03 May 2024 02:41:39.250 * Running mode=standalone, port=6379.
netprobe-redis | 1:M 03 May 2024 02:41:39.251 * Server initialized
netprobe-redis | 1:M 03 May 2024 02:41:39.251 * Ready to accept connections tcp
netprobe-probe | /netprobe_lite/helpers/network_helper.py:25: SyntaxWarning: invalid escape sequence '|'
netprobe-probe | ping = subprocess.getoutput(f"ping -n -i 0.1 -c {count} {site} | grep 'rtt|loss'")
Error response from daemon: driver failed programming external connectivity on endpoint netprobe-grafana (588de0b3fe6d13d712538a22086b31bbbe5bd9f4c0ce4753f92b700e8aafff26): Bind for 0.0.0.0:3001 failed: port is already allocated
NNN---@NNN-TEST:/netprobe_lite$ sudo vim compose.yml
NNN---@NNN-TEST:~/netprobe_lite$ docker compose up
WARN[0000] /home/NNN---/netprobe_lite/compose.yml: version is obsolete
[+] Running 5/0
✔ Container netprobe-probe Running 0.0s
✔ Container netprobe-redis Running 0.0s
✔ Container netprobe-prometheus Running 0.0s
✔ Container netprobe-grafana Recreated 0.0s
✔ Container netprobe-presentation Running 0.0s
Attaching to netprobe-grafana, netprobe-presentation, netprobe-probe, netprobe-prometheus, netprobe-redis
Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "/home/NNN---/netprobe_lite/config/grafana/dashboards/netprobe.json" to rootfs at "/var/lib/grafana/dashboards/netprobe.json": mount /home/NNN---/netprobe_lite/config/grafana/dashboards/netprobe.json:/var/lib/grafana/dashboards/netprobe.json (via /proc/self/fd/6), flags: 0x5000: not a directory: unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type

[j64bit]

Hi,

many thanks for the great docker file. I think I see the same or similar issue - which is blocking it to start up.
I use Portianer to create a stack.

failed to deploy a stack: Network netprobe_light_custom_network Creating Network netprobe_light_custom_network Created Volume "netprobe_light_prometheus_data" Creating Volume "netprobe_light_prometheus_data" Created Volume "netprobe_light_grafana_data" Creating Volume "netprobe_light_grafana_data" Created Container netprobe-presentation Creating Container netprobe-redis Creating Container netprobe-grafana Creating Container netprobe-probe Creating Container netprobe-prometheus Creating Container netprobe-redis Created Container netprobe-probe Created Container netprobe-prometheus Created Container netprobe-presentation Created Container netprobe-grafana Created Container netprobe-redis Starting Container netprobe-presentation Starting Container netprobe-probe Starting Container netprobe-prometheus Starting Container netprobe-grafana Starting Container netprobe-redis Started Container netprobe-presentation Started Container netprobe-probe Started Container netprobe-grafana Started Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "/data/compose/45/config/prometheus/prometheus.yml" to rootfs at "/etc/prometheus/prometheus.yml": mount /data/compose/45/config/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml (via /proc/self/fd/6), flags: 0x5000: not a directory: unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type

[edenon]

Non docker or portainer works. Regards El vie, 3 de may de 2024, 05:58, j64bit ***@***.***> escribió:

…

Hi, many thanks for the great docker file. I think I see the same or similar issue - which is blocking it to start up. I use Portianer to create a stack. failed to deploy a stack: Network netprobe_light_custom_network Creating Network netprobe_light_custom_network Created Volume "netprobe_light_prometheus_data" Creating Volume "netprobe_light_prometheus_data" Created Volume "netprobe_light_grafana_data" Creating Volume "netprobe_light_grafana_data" Created Container netprobe-presentation Creating Container netprobe-redis Creating Container netprobe-grafana Creating Container netprobe-probe Creating Container netprobe-prometheus Creating Container netprobe-redis Created Container netprobe-probe Created Container netprobe-prometheus Created Container netprobe-presentation Created Container netprobe-grafana Created Container netprobe-redis Starting Container netprobe-presentation Starting Container netprobe-probe Starting Container netprobe-prometheus Starting Container netprobe-grafana Starting Container netprobe-redis Started Container netprobe-presentation Started Container netprobe-probe Started Container netprobe-grafana Started Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "/data/compose/45/config/prometheus/prometheus.yml" to rootfs at "/etc/prometheus/prometheus.yml": mount /data/compose/45/config/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml (via /proc/self/fd/6), flags: 0x5000: not a directory: unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type — Reply to this email directly, view it on GitHub <#29 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BCYO37KZQTBNYNONZPXCHY3ZANGSDAVCNFSM6AAAAABHEYWO3CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOJSGU4TEMJVGU> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[edenon]

Portainer works for you?

[plaintextpackets]

I haven't tested it with portainer not sure if anyone else has

[edenon]

j64bit mention it but do not understand if he made it work.
I'm still trying getting error:
WARN[0000] /home/NNN---/netprobe_lite/compose.yml: version is obsolete
[+] Running 5/0
✔ Container netprobe-probe Running 0.0s
✔ Container netprobe-redis Running 0.0s
✔ Container netprobe-prometheus Running 0.0s
✔ Container netprobe-grafana Recreated 0.0s
✔ Container netprobe-presentation Running 0.0s
Attaching to netprobe-grafana, netprobe-presentation, netprobe-probe, netprobe-prometheus, netprobe-redis
Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "/home/NNN---/netprobe_lite/config/grafana/dashboards/netprobe.json" to rootfs at "/var/lib/grafana/dashboards/netprobe.json": mount /home/NNN---/netprobe_lite/config/grafana/dashboards/netprobe.json:/var/lib/grafana/dashboards/netprobe.json (via /proc/self/fd/6), flags: 0x5000: not a directory: unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type

Any idea about this?

[j64bit]

Just to clarify: it does NOT work for me. See my error message above.

[sduensin]

Using tag v1.2.1 I get the same error as @j64bit under Portainer.

[fiveangle]

One can map the parent directly as a separate volume, then map the "file" but this merely appeases docker-compose so it will start the stack, but the "file" map will again be a directory unless it already exists everywhere as a file.

I even tried using the bind volume type for each individual config file, e.g.:

...
    volumes:
      - type: bind
        source: ./config/prometheus/prometheus.yml
        target: /etc/prometheus/prometheus.yml
      - prometheus_data:/prometheus  # Persistent local storage for Prometheus data
...

but it just thew more errors:

Failed to deploy a stack: Network netprobe_netprobe-net Creating Network netprobe_netprobe-net Created Volume "netprobe_grafana_data" Creating Volume "netprobe_grafana_data" Created Volume "netprobe_prometheus_data" Creating Volume "netprobe_prometheus_data" Created Container netprobe-grafana Creating Container netprobe-prometheus Creating Container netprobe-redis Creating Container netprobe-probe Creating Container netprobe-presentation Creating Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /data/compose/85/config/grafana/datasources/automatic.yml

Without every config file and their leading parent directories already existing at config time (e.g. docker-compose validation pre-run time), I don't think there is a solution for us.

Instead of specifying actual config files directly within the compose file, I often see image creators instead redirect the location of config and data files for their app(s) explicitly into a common directory they can then map those directories as a volumes, e.g.:

    - ./config:/myapp/config
    - ./data:/myapp/data

…then populate all their config and data files and directories into those locations at first startup.

One place to start troubleshooting might be to run docker compose config -fq within a fresh and unpopulated project folder… my hope is you will see a bunch of the same errors we are seeing 🤷

[jfstockton]

I'm having the same issue as j64bit was reporting. I tried changing the file '/netprobe_lite/config/grafana/dashboards/netprobe.json' to the same path but with 'automatic.yml' instead of 'netprobe.json' and it would then load and start. The contents of Grafana would be incorrect, but at least this was a hint as to what might be going wrong. Any help would be appreciated. Thanks for the work that you all put into this.

[imoldovavan]

Same issue here even with absolute path location of volumes.
Attaching to netprobe-grafana, netprobe-presentation, netprobe-probe, netprobe-prometheus, netprobe-redis, netprobe-speedtest
Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "/home/ionut/netprobe_lite/config/grafana/dashboards/netprobe.json" to rootfs at "/var/lib/grafana/dashboards/netprobe.json": mount /home/ionut/netprobe_lite/config/grafana/dashboards/netprobe.json:/var/lib/grafana/dashboards/netprobe.json (via /proc/self/fd/6), flags: 0x5000: not a directory: unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type

[imoldovavan]

For those running this on Linux I found a solution from this post: https://stackoverflow.com/questions/72695311/failure-starting-docker-container-failed-to-create-shim-task-oci-runtime-crea

I used chmod +x on the home/ionut/netprobe_lite/config/grafana/dashboards/netprobe.json file.
chmod +x home/ionut/netprobe_lite/config/grafana/dashboards/netprobe.json

This allowed the installation to complete.

I also did this for automatic.yml, and main.yml although I don't think these were necessary.

[edmondhma]

For those running this on Linux I found a solution from this post: https://stackoverflow.com/questions/72695311/failure-starting-docker-container-failed-to-create-shim-task-oci-runtime-crea

I used chmod +x on the home/ionut/netprobe_lite/config/grafana/dashboards/netprobe.json file. chmod +x home/ionut/netprobe_lite/config/grafana/dashboards/netprobe.json

This allowed the installation to complete.

I also did this for automatic.yml, and main.yml although I don't think these were necessary.

This doesn't work on my Ubuntu setup. Don't know why. Not even without Portainer.