Skip to content

firewall: flush stale UDP conntrack entries on port_forward setup/teardown #1362

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
shivkr6 wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

firewall: flush stale UDP conntrack entries on port_forward setup/teardown #1362

shivkr6 wants to merge 1 commit into from

Conversation

@shivkr6
Copy link
Contributor

@shivkr6 shivkr6 commented Nov 18, 2025 •
edited
Loading

Add a new netlink_netfilter module to interact with the kernel's conntrack table using the netlink_packet_netfilter crate. This module allows dumping and deleting conntrack entries. All firewall drivers now call the new flush_udp_conntrack() function during port forwarding setup and teardown.

When a container with a UDP port mapping is started, stale conntrack entries can prevent traffic from reaching the new container instance. This change proactively deletes these stale entries for the mapped UDP ports, ensuring that new connections are not dropped by the kernel.

Fixes: #1045

NOTE: This PR cannot be merged right now because:

  1. I'm waiting for my PRs to be merged in the netlink-packet-netfilter crate.
    2) I have to write integration tests to test this functionality. [DONE]

CC: @Luap99 @mheon

@shivkr6
Copy link
Contributor Author

shivkr6 commented Nov 19, 2025

The unit tests seem to be failing because conntrack-tools is not installed in CI/CD

@shivkr6 shivkr6 force-pushed the netlink-packet-netfilter branch from f61d2f3 to 0731a48 Compare November 20, 2025 12:20
@shivkr6
Copy link
Contributor Author

shivkr6 commented Nov 20, 2025

Finished writing the integration tests for flushing conntrack entries.
image

Question: Is it possible to install conntrack-tools in the CI/CD test environment? conntrack-tools is needed for some of the unit tests.
CC: @Luap99 @mheon

@shivkr6 shivkr6 force-pushed the netlink-packet-netfilter branch from 0731a48 to 82cb21c Compare November 20, 2025 12:25
@packit-as-a-service
Copy link

packit-as-a-service bot commented Nov 20, 2025

Ephemeral COPR build failed. @containers/packit-build please check.

2 similar comments
@packit-as-a-service
Copy link

packit-as-a-service bot commented Nov 20, 2025

Ephemeral COPR build failed. @containers/packit-build please check.

@packit-as-a-service
Copy link

packit-as-a-service bot commented Nov 20, 2025

Ephemeral COPR build failed. @containers/packit-build please check.

@shivkr6
Copy link
Contributor Author

shivkr6 commented Nov 20, 2025

I moved the conntrack entries flushing code to the end of all setup_port_forward and teardown_port_forward functions, instead of keeping it at the start, to avoid a potential race condition where a new conntrack entry is created after the flush but before the port forwarding rules are completely set up.

@shivkr6 shivkr6 force-pushed the netlink-packet-netfilter branch from 82cb21c to 86d1881 Compare November 20, 2025 12:49
@Luap99
Copy link
Member

Luap99 commented Nov 20, 2025

Question: Is it possible to install conntrack-tools in the CI/CD test environment? conntrack-tools is needed for some of the unit tests.

Yes we can do that. I make that happen.

Luap99 added a commit to Luap99/automation_images that referenced this pull request Nov 20, 2025
@Luap99
netavark: add conntrack-tools
cdf6a38 
Needed for containers/netavark#1362

Signed-off-by: Paul Holzinger <[email protected]>
@shivkr6 shivkr6 force-pushed the netlink-packet-netfilter branch 4 times, most recently from 425517c to 107bf6b Compare November 20, 2025 13:42
@Luap99
Copy link
Member

Luap99 commented Nov 20, 2025 •
edited
Loading

#1363 this should include conntrack now

(once that merges you can rebase)

Luap99
Luap99 reviewed Nov 20, 2025
View reviewed changes
@Luap99
Copy link
Member

Luap99 commented Nov 20, 2025

#1363 was merged so CI should have access to conntrack I hope.

@shivkr6 shivkr6 force-pushed the netlink-packet-netfilter branch from 107bf6b to e6104d8 Compare November 20, 2025 18:06
@shivkr6
Copy link
Contributor Author

shivkr6 commented Nov 20, 2025 •
edited
Loading

I'll do the remaining changes tomorrow Done.

@shivkr6 shivkr6 force-pushed the netlink-packet-netfilter branch 3 times, most recently from b07535a to bf8e6dc Compare November 21, 2025 06:31
@shivkr6 shivkr6 requested a review from Luap99 November 21, 2025 07:44
mheon
mheon reviewed Nov 21, 2025
View reviewed changes
@mheon
Copy link
Member

mheon commented Nov 21, 2025

I have an error message nit, but on the whole LGTM

Luap99
Luap99 requested changes Nov 21, 2025
View reviewed changes
Cargo.toml
rand = "0.9.2"
sha2 = "0.10.9"
netlink-packet-route = "0.25.1"
netlink-packet-netfilter = { git = "https://github.com/shivkr6/netlink-packet-netfilter.git", branch = "conntrack-new" }
Copy link
Member

@Luap99 Luap99 Nov 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just so it is not overlooked, we cannot merge this while the upstream lib isn't merged so I mark as changes requested

@Luap99
Copy link
Member

Luap99 commented Nov 21, 2025

Forgot to say overall great work!

@shivkr6 shivkr6 force-pushed the netlink-packet-netfilter branch from bf8e6dc to bcb10cb Compare November 23, 2025 17:52
@shivkr6
Copy link
Contributor Author

shivkr6 commented Nov 23, 2025

I have no idea why the firewalld: receive udp traffic with pre-existing stale conntrack entry (range) test is failing even after successfully the flushing the conntrack entries. IDK why socat inside the container still cannot receive the udp traffic :(

Please help @Luap99 @mheon

@shivkr6
Copy link
Contributor Author

shivkr6 commented Nov 23, 2025

Forgot to say overall great work!

Thanks! The appreciation from you and Matt really keeps my motivation and confidence high

@shivkr6
Copy link
Contributor Author

shivkr6 commented Nov 24, 2025

It's extremely weird that the firewalld: receive udp traffic with pre-existing stale conntrack entry (range) integration test passes when I change the container_port from 8081 to 8080.
This passes successfully:

@test "firewalld: receive udp traffic with pre-existing stale conntrack entry (range)" {
    setup_firewalld
    export NETAVARK_FW="firewalld"
    # Explicitly add a rule to trigger connection tracking.
    run_in_host_netns firewall-cmd --direct --add-rule ipv4 filter OUTPUT 0 -p udp -m state --state NEW -j ACCEPT
    run_ct_udp_flush_test "${TESTSDIR}/testfiles/bridge-udp-stale-conntrack-range.json" "127.0.0.1" "10081" "8080"
}

@shivkr6 shivkr6 force-pushed the netlink-packet-netfilter branch from bcb10cb to 343b730 Compare November 24, 2025 08:05
@shivkr6
firewall: flush stale UDP conntrack entries on port_forward setup/tea…
49b6f1d 
…rdown

Add a new netlink_netfilter module to interact with the kernel's conntrack table using the netlink_packet_netfilter crate. This module allows dumping and deleting conntrack entries. All firewall drivers now call the new flush_udp_conntrack() function during port forwarding setup and teardown.

When a container with a UDP port mapping is started, stale conntrack entries can prevent traffic from reaching the new container instance. This change proactively deletes these stale entries for the mapped UDP ports, ensuring that new connections are not dropped by the kernel.

Added an integration test for the same and unit tests for dump_conntrack and del_conntrack.

Fixes: containers#1045

Signed-off-by: Shivang K Raghuvanshi <[email protected]>
@shivkr6 shivkr6 force-pushed the netlink-packet-netfilter branch from 343b730 to 49b6f1d Compare November 24, 2025 17:45
@shivkr6
Copy link
Contributor Author

shivkr6 commented Nov 24, 2025

I skipped the firewalld: receive udp traffic with pre-existing stale conntrack entry (range) test because port forwarding range seems to be broken with firewalld. All the traffic gets redirected to the first container port no matter which host range port you pick. It's not an issue related to conntrack.

@mheon
Copy link
Member

mheon commented Nov 24, 2025

Tests green, just waiting on that upstream library now. Nice work @shivkr6

@shivkr6
Copy link
Contributor Author

shivkr6 commented Nov 25, 2025

Thanks @mheon. The bigger PR, rust-netlink/netlink-packet-netfilter#12, got merged. The remaining smaller PRs yet to be merged are rust-netlink/netlink-packet-netfilter#14 and rust-netlink/netlink-packet-netfilter#15

CC @Luap99

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mheon mheon mheon left review comments

@Luap99 Luap99 Luap99 requested changes

Requested changes must be addressed to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Container with podman network not receiving UDP traffic

3 participants

@shivkr6 @Luap99 @mheon