Self cross-posting from: https://lemmy.zip/post/70909658

Intention to have slightly better visibility from the self-hosted crowd and I’m interested in more general feedback on this too.

Hey everyone! I’m trying to find a solution to a really confusing problem…

I have the following simple nginx docker compose configuration on my Fedora home server that I can run without issue on my uid 1000 user, lets call this user “userA”.

services:
  nginx:
    container_name: nginx-alt
    image: docker.io/library/nginx
    restart: unless-stopped
    ports:
      - 8181:80

This exposes internal port 80 as 8181 and can be accessed in a lan in the expected matter.

However, for security reasons, I want to actually host this service eventually on a completely different user with less permissions. Let’s call this user “userB” who has a very limited scope of the file system. This is to prevent potential escaping of the rootless container causing major file system havoc (i.e. reduce the scope of the user to a very limited network of containers.)

The problem is really simple: For some reason, when userB runs this service (uid 1001), the nginx service suddenly complains about privileges. As a result, I get a “Forbidden 403” error when hosting. Turning off selinux has no affect (so setenforce 0 does nothing, meaning I can rule out secure linux interruption.)

The errors look like the following:

nginx-alt  | 2026/09/04 20:03:34 [error] 25#25: *1 "/usr/share/nginx/html/index.html" is forbidden (13: Permission denied), client: xx.xx.x.x, server: localhost, request: "GET / HTTP/1.1", host: "xxx.xxx.xxx.xxx:8181"
nginx-alt  | 10.89.0.2 - - [04/Sep/2026:20:03:34 +0000] "GET / HTTP/1.1" 403 153 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:155.0) Gecko/20100101 Firefox/155.0" "-"

For what it’s worth, both users should be relatively vanilla and all ports are appropriately exported. There shouldn’t be anything, for example, that is making userA run as “privileged” over the other users and podman should be running rootless in both containers.

I did see a note on the nginx image about running in rootless that I might try, but it doesn’t solve my bigger issue here which is the lack of consistency between the two users. Additionally, userns_mode: keep-ids only caused the container to fail to boot for other reason entirely.

There must be something fundamentally wrong with my configuration of my system. Has anyone had any experience running two podman containers on two different users simultaneously that can provide feedback?

Obviously, I’m not trying to run just an nginx server, but I found this to be the easiest configuration to reproduce.

  • Overspark@piefed.socialEnglish
    1·
    23 hours ago

    The solution to this can be multiple reverse proxies. Each user runs it’s own reverse proxy (if you actually need one, otherwise you let the container bind directly to a port on the host system). Then you run one main reverse proxy on port 80/443 that proxies to those other ports based on hostname.

    The upside of this construction is that containers running as different users can’t directly access each others internal networks, which is much better for security, as those networks often contain barely protected services, which is why you’re using a reverse proxy in the first place.

    Edit: there are different options with custom networks and such, but they’re even more complicated so I wouldn’t advise them.

    • rhymepurple@lemmy.mlEnglish
      1·
      3 hours ago

      I think this is what OP was originally trying, but this approach breaks when each service’s Podman service runs on its own localhost user.

      • Overspark@piefed.socialEnglish
        1·
        3 hours ago

        You can use the per-user reverse proxies as a bridge between the host network and the user-specific internal network. So for a user coming from outside the path looks like this:

        User -> main reverse proxy on main IP and port 80/443 -> user-specific proxy listening on main IP with port 8080 and proxying to user-specific internal network -> destination container listening on user-specific internal network.

        And for a container running as a different user the path will be the same, but the user-specific reverse proxy will be listening on port 8081 and higher for example.

        • rhymepurple@lemmy.mlEnglish
          1·
          2 hours ago

          This would require the main proxy running as root or with some other sort of elevated privileges to allow cross-user network access though, right? If so, wouldn’t that essentially make the user-specific reverse proxy unnecessary in most cases?