Diagnosing Rootless Podman Port Publishing Failures

Recognizable Condition

You run podman run -d -p 8080:80 nginx (or any image). podman ps shows 0.0.0.0:8080->80/tcp. The container reports healthy. Yet curl http://localhost:8080 from the host hangs or returns Connection refused. A quick podman exec <ctr> curl http://localhost:80 works, proving the application inside the container is listening.

This pattern—mapping appears correct, container process is up, but host-to-container traffic fails—is the hallmark of a rootless networking or privilege boundary issue, not an application bug.

Cause and Diagnostic Quick-Reference

Symptom	Likely Cause	Quick Verification
Host port < 1024 (e.g., -p 80:80)	Kernel blocks unprivileged port bind	sysctl net.ipv4.ip_unprivileged_port_start (default 1024)
curl localhost:PORT works, LAN client fails	App binds 127.0.0.1 inside container	podman exec <ctr> ss -ltnp shows 127.0.0.1:80
Host 127.0.0.1:PORT works, <host-ip>:PORT fails	Firewall zone drops ingress on non-loopback interface	firewall-cmd --list-all or nft list ruleset
All host addresses fail, container IP works	Rootless network backend (slirp4netns/pasta) not exposing to host namespace	podman info --format '{{.Host.NetworkBackend}}'
Mapping shows in podman port but no listeners on host	Port collision or backend failed to create tap	ss -ltnp | grep :PORT on host

Ordered Diagnostic Checks

Run these in sequence. Stop when you find the mismatch.

1. Confirm the container process and its listening address

podman exec <ctr> ss -ltnp
# or
podman exec <ctr> netstat -ltnp

Look for the application port (e.g., :80). The Local Address column must show 0.0.0.0:80 or :::80. If it shows 127.0.0.1:80, the application is binding only to loopback inside the container—rootless port forwarding cannot reach it from the host or LAN.

2. Verify the published mapping Podman believes it created

podman port <ctr>
# Example output:
# 80/tcp -> 0.0.0.0:8080

If the host port is below 1024 and you are rootless, the mapping may be listed but the kernel silently refused the bind. Proceed to step 3.

3. Check the unprivileged port limit

sysctl net.ipv4.ip_unprivileged_port_start
# Default output: net.ipv4.ip_unprivileged_port_start = 1024

If your host port is less than this value, the rootless user cannot bind it. The container starts, but the port forwarder (slirp4netns or pasta) fails to attach to the host port. No error is surfaced to podman run.

4. Test host-loopback vs. host-IP vs. LAN

# From the same host
curl -v http://127.0.0.1:8080
curl -v http://<host-lan-ip>:8080

# From a different machine on the same LAN
curl -v http://<host-lan-ip>:8080

Three outcomes:

• Only 127.0.0.1 works → firewall zone or interface binding issue.
• Host IP works, LAN fails → host firewall or cloud security group.
• All fail → go back to step 1 or 3.

5. Inspect the active rootless network backend

podman info --format '{{.Host.NetworkBackend}}'
# Typical: netavark (with pasta or slirp4netns)

Rootless Podman 4.x+ defaults to pasta (via netavark) on newer distros; older installs use slirp4netns. Both create a user-mode network stack. pasta generally forwards host-loopback and host-IP by default; slirp4netns may require --network slirp4netns:port_handler=slirp4netns for certain UDP or broadcast cases. The backend choice affects which interfaces see the forwarded port.

6. Check host listeners and firewall

# Is the port actually listening on the host?
ss -ltnp | grep :8080

# Firewalld (RHEL/Fedora/CentOS)
firewall-cmd --list-all
firewall-cmd --get-active-zones

# nftables (Debian/Ubuntu/Arch)
nft list ruleset | grep -A2 -B2 8080

If ss shows no listener on 0.0.0.0:8080 but podman port claims the mapping, the backend failed to bind—usually due to the privileged-port limit (step 3) or a port collision.

Fixes Tied to Findings

Privileged host port (< 1024)

Preferred: Remap to an unprivileged host port.

podman run -d -p 8080:80 nginx   # host 8080 → container 80

Alternative (documented exception): Raise the kernel limit for the user session or system-wide.

# Per-session (requires root once, persists until reboot)
sudo sysctl -w net.ipv4.ip_unprivileged_port_start=80

# Permanent
sudo tee /etc/sysctl.d/99-podman-unpriv.conf <<'EOF'
net.ipv4.ip_unprivileged_port_start=80
EOF
sudo sysctl --system

Caution: Lowering this limit on a shared multi-user host allows any unprivileged user to bind well-known ports, increasing spoofing risk. Document the change and restrict to dedicated CI runners or single-tenant VMs.

Application binds 127.0.0.1 inside container

Change the application configuration to listen on 0.0.0.0 (all interfaces). For nginx:

# nginx.conf
server {
    listen 80;          # defaults to 0.0.0.0:80
    # listen 127.0.0.1:80;  ← avoid this in containers
}

For Python/Flask, Gunicorn, Node, Go, etc., ensure the bind address is 0.0.0.0 or ::, not localhost or 127.0.0.1.

Firewall blocks ingress on LAN interface

Add the port to the appropriate firewalld zone (usually public or external).

# Runtime
sudo firewall-cmd --add-port=8080/tcp
# Permanent
sudo firewall-cmd --permanent --add-port=8080/tcp
sudo firewall-cmd --reload

For nftables, add a rule to the input chain of the inet filter table:

sudo nft add rule inet filter input tcp dport 8080 accept

Rootless backend not exposing to host IP

If pasta is active and host-loopback works but host-IP does not, try forcing the port handler:

podman run -d --network pasta:port_handler=slirp4netns -p 8080:80 nginx

Or, as a diagnostic shortcut (not a production fix), use host networking to bypass port mapping entirely:

podman run -d --network host nginx
# Now container port 80 is directly host port 80 (requires root for <1024)

Warning: --network host removes network isolation. The container shares the host's network namespace, sees all interfaces, and can bind any port the user can. Use only for debugging or trusted workloads.

Escalation Criteria

Escalate to packet-level capture when:

• podman port shows the expected mapping.
• podman exec <ctr> ss -ltnp confirms 0.0.0.0:PORT inside the container.
• Host ss -ltnp shows a listener on 0.0.0.0:HOST_PORT (or :::HOST_PORT).
• Firewall is open for the port and zone.
• curl 127.0.0.1:HOST_PORT works, but curl <host-ip>:HOST_PORT and LAN clients still fail.

Capture path:

# 1. Host interface tcpdump (replace eth0 with your LAN interface)
sudo tcpdump -ni eth0 tcp port 8080

# 2. Inside container namespace
podman exec -it <ctr> tcpdump -ni any tcp port 80

# 3. Compare nftables counters
sudo nft list ruleset -a | grep -A5 -B5 8080

Look for SYN packets arriving at the host interface but not reaching the container namespace, or RST/ICMP responses indicating a reject rule. Compare against a minimal bridge-network container (podman run --network bridge -p 8080:80 nginx run as root) to isolate whether the issue is rootless-specific.

Verification Checklist After Each Change

1. Record podman info, sysctl net.ipv4.ip_unprivileged_port_start, active firewall zone.
2. Run the three curl tests (loopback, host-IP, LAN).
3. Confirm ss -ltnp on host shows the listener on the expected address family (IPv4/IPv6).
4. Check podman port <ctr> still matches intent.

If the problem persists after all fixes, file a bug with podman info, kernel version (uname -r), distro, and the tcpdump captures. Rootless networking behavior varies across kernel versions, pasta/slirp4netns releases, and firewalld/nftables integration—exact versions matter for upstream triage.