Skip to content

Fix tailnet access to LAN-gated apps, and Caddy access logging - #59

Merged
mathewcsims merged 1 commit into
mainfrom
fix-tailnet-subnet-access-and-caddy-logging
Aug 7, 2026
Merged

Fix tailnet access to LAN-gated apps, and Caddy access logging#59
mathewcsims merged 1 commit into
mainfrom
fix-tailnet-subnet-access-and-caddy-logging

Conversation

@mathewcsims

Copy link
Copy Markdown
Owner

Found while debugging docs.mathewcsims.uk being unreachable from an Android client on mobile data. Two separate latent faults, neither in this repo's configuration.

Tailnet access

A subnet route is a separate ACL destination from the tailnet IPs of the devices behind it. Babel advertised 10.0.1.0/24, the route was approved in the console, and the client had "use subnet routes" enabled — all three necessary, none sufficient, because no grant ever named the CIDR. Traffic was dropped by policy at the receiving node.

That is why every server-side check came back clean. Caddy's @lan matcher, the DigitalOcean and NextDNS records, the tailnet's own resolver and HedgeDoc itself were all verified correct and none of them was the problem.

The symptoms actively misdirect:

Path Symptom Why
With exit node timeout Tailscale keeps RFC1918 destinations off the exit path, so the packet leaves via the local interface
Without exit node ERR_CONNECTION_REFUSED browser falls back to the public A record, reaches Caddy from the internet, hits handle { abort }

Neither leaves a trace on the Pi — a full packet capture of the client's traffic showed no connection attempt toward the LAN address at all.

The grant itself is applied to the live tailnet already (verified present in babel's enforced packet filter, where the 10.x list was previously empty). This PR adds scripts/tailscale-acl.sh so the policy is managed the same way as everything else here: key used only inside a Python process via urllib so it never reaches argv, /acl/validate before applying, and If-Match so a concurrent console edit is a 412 rather than a silent clobber.

Caddy access logging

Caddy emits access logs only for site blocks carrying the log directive. The global log block configures the default logger's sink, which is not the same thing — so no request has ever been logged.

The consequence is that the caddy-abuse fail2ban jail has been watching a file it was never fed. Its lifetime counter stood at exactly 3: the one synthetic and two hand-fed lines used to verify it nine days ago, and nothing since.

Fixed with an (access_log) snippet imported by all 25 site blocks. Verified after reload: a probe with a unique path lands in logs/access.log, and fail2ban-regex matches the emitted format (1 matched, 0 missed).

Docs

SETUP.md's "Accessing LAN-only apps over Tailscale" said two things were required and claimed the arrangement was "confirmed working end-to-end". It now documents three, and records plainly that no LAN-gated app had ever been reachable from a genuinely off-LAN tailnet device. Testing from a device sitting on the physical LAN passes regardless, which is how it went unnoticed.

Verification

  • ACL grant present in babel's tailscale debug netmap packet filter (was NONE)
  • User confirmed docs.mathewcsims.uk loads over the tailnet
  • Caddyfile passes caddy validate; applied with caddy reload, no downtime
  • Access logging confirmed by probe; fail2ban-regex confirmed against the real emitted format
  • tcpdump, installed on the Pi for the diagnosis, has been removed again

🤖 Generated with Claude Code

@mathewcsims mathewcsims closed this Aug 7, 2026
@mathewcsims mathewcsims reopened this Aug 7, 2026
@mathewcsims mathewcsims closed this Aug 7, 2026
@mathewcsims mathewcsims reopened this Aug 7, 2026
@mathewcsims
mathewcsims force-pushed the fix-tailnet-subnet-access-and-caddy-logging branch from 5e35396 to 2d0bbb4 Compare August 7, 2026 09:41
Two faults found while debugging docs.mathewcsims.uk being unreachable
from an Android client on mobile data. Neither was in this repo's config,
and both had been latent for a while.

The tailnet fault: a subnet route is a separate ACL destination from the
tailnet IPs of the devices behind it. Babel advertised 10.0.1.0/24, the
route was approved, and the client had "use subnet routes" on — all three
necessary, none sufficient, because no grant named the CIDR. Traffic was
dropped by policy at the receiving node, which is why every server-side
check came back clean: Caddy, DNS records, the NextDNS rewrite, the
tailnet's own resolver and HedgeDoc were all correct throughout.

The symptoms actively misdirect. With an exit node the connection times
out, because Tailscale keeps RFC1918 destinations off the exit path, so
the packet leaves via the local interface and dies. Without one it is
refused, because the browser falls back to the public A record and hits
handle { abort } from the internet. Neither leaves a trace on the Pi — a
full packet capture of the client showed no connection attempt toward the
LAN address at all.

Adds scripts/tailscale-acl.sh to manage the policy file, in the same shape
as dns-nextdns.sh: the API key is used only inside a Python process via
urllib so it never appears in argv, /acl/validate runs before anything is
applied, and If-Match makes a concurrent console edit a 412 rather than a
silent clobber.

The logging fault: Caddy emits access logs only for site blocks carrying
the log directive. The global log block configures the default logger's
sink, not access logging, so no request has ever been logged here. That
also means the caddy-abuse fail2ban jail has been watching a file it was
never fed — its lifetime counter stood at exactly 3, matching the one
synthetic and two hand-fed lines used to verify it nine days ago, and
nothing since. Fixed with an (access_log) snippet imported by all 25 site
blocks; re-verified that a probe with a unique path now lands in the log
and that fail2ban-regex matches the emitted format.

SETUP.md's "Accessing LAN-only apps over Tailscale" section said two
things were required and claimed the arrangement was confirmed working
end to end. It now documents three, and records that no LAN-gated app had
ever actually been reachable from a genuinely off-LAN tailnet device —
testing from a device on the physical LAN passes regardless, which is how
it went unnoticed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@mathewcsims
mathewcsims force-pushed the fix-tailnet-subnet-access-and-caddy-logging branch from 2d0bbb4 to dd6eebe Compare August 7, 2026 11:30
@mathewcsims
mathewcsims merged commit 6803e38 into main Aug 7, 2026
7 checks passed
@mathewcsims
mathewcsims deleted the fix-tailnet-subnet-access-and-caddy-logging branch August 7, 2026 11:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant