Convincing Docker and Iptables to Play Nicely

If you’ve ever tried to set up Docker on a server that also needs to have local firewall rules, you know it can be a problem. Docker wants to take over, and, rightly so, given how complicated the internal Docker networking is, but you still need some control.

There are multiple tutorials out there that try to solve this problem, and I’ve been through them all. Many of them are outdated, some of them work until you hit an edge case, but none that I found truly had it figured out.

My solution for this was to put my Docker servers on a private network behind another server that acts as a firewall and router, so Docker could have full control of iptables. That is still a perfectly valid approach, but there is a real cost and maintenance overhead to have those extra servers.

I now have the perfect solution; Docker gets to manage its rules, I get to manage mine, and we don’t get in each other’s way.

I like to use ferm to manage iptables. It provides some really great features like the ability to use functions, variables, and lists to create complex rules with an easy-to-read syntax. It also plays nicely with configuration management tools. In version 2.4, ferm introduced the @preserve feature, which was the missing link to get it to play nicely with Docker.

Docker already leaves the INPUT and OUTPUT chains alone, so my default ferm config still works:

# /etc/ferm/ferm.conf

# Default rules
domain (ip ip6) {
    table filter {
        # Default Policies
        chain INPUT policy DROP;
        chain OUTPUT policy ACCEPT;

        # loopback traffic
        chain INPUT interface lo ACCEPT;
        chain OUTPUT outerface lo ACCEPT;

        chain (INPUT OUTPUT) {
            # ICMP is very handy and necessary
            proto icmp ACCEPT;

            # connection tracking
            mod conntrack ctstate (RELATED ESTABLISHED) ACCEPT;
        }
    }
}

# Local rules
@include ferm.d/;

I keep all of my rules organized in /etc/ferm/ferm.d, which get included in alphabetical order by the @include at the end of the main config files. To make sure they are included in the order I expect, I add 2-digit numbered prefixes to the files. Then, I use my configuration management to manage all of those rules.

The first set of rules that I want to be sure ferm loads uses @preserve to tell it to leave all of the chains that Docker wants to control alone.

# /etc/ferm/ferm.d/00-docker.ferm

domain (ip ip6) {
    table filter {
        chain (DOCKER DOCKER-INGRESS DOCKER-ISOLATION-STAGE-1 DOCKER-ISOLATION-STAGE-2 FORWARD) @preserve;
    }

    table nat {
        chain (DOCKER DOCKER-INGRESS PREROUTING OUTPUT POSTROUTING) @preserve;
    }
}

Then, I include any local rules, like allowing SSH access to the server:

# /etc/ferm/ferm.d/20-in.ssh.ferm

domain (ip ip6) {
    table filter chain INPUT proto tcp dport 22 ACCEPT;
}

I leverage the DOCKER-USER table to manage remote access to Docker services, like allowing access to an HTTP service:

# /etc/ferm/ferm.d/20-in.docker.http.ferm

domain (ip ip6) {
    table filter chain DOCKER-USER

    # Incoming traffic bound for a docker service will come in
    # to the FORWARD chain on eth0 and exit on docker_gwbridge
    interface eth0 outerface docker_gwbridge

    # The destination port here is the port listening IN THE DOCKER CONTAINER
    # Often times that is the same as the host port, but not always
    proto tcp dport (80 443)

    ACCEPT;
}

Finally, I wrap up the config by setting the default actions for external access to Docker services:

# /etc/ferm/ferm.d/99-docker.ferm

domain (ip ip6) table filter chain DOCKER-USER {
    interface eth0 outerface docker_gwbridge {
        mod conntrack ctstate (RELATED ESTABLISHED) ACCEPT;
        DROP;
    }
    RETURN;
}

With this setup, I get the best of both worlds: fully managed iptables rules that work well with my configuration management, all while letting Docker handle all of its own rules.