Docker uses two fundamental Linux networking technologies, networking namespaces and iptables, to allow containers to communicate with each other and the outside world. This post uses the Linux commandline to explore how Docker implements networking with these technologies. Once you understand how Docker networking works, you will have the power to extend it to your needs.

Installation

When Docker is first installed, it does a few actions behind the scenes to make networking possible:

  1. it creates a bridge,
  2. it adds its own rules to iptables.

Bridge Creation

Docker creates a bridge called docker0 which is an assigned an IP address (172.17.0.1/16 in this case). If no network is specified when a container is created, it is connected to this bridge by default.

Note that the bridge state is DOWN. The bridge is activated when a container is connected to it.

$ brctl show
bridge name bridge id     STP enabled interfaces
docker0     8000.0242a690e758 no

$ ip a
...
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:4d:11:10:f1 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
...

New iptables Rules

Docker adds its own rules in IP tables to isolate containers in separate networks from each other. Since there is only one network here (docker0), these rules have no effect. We will investigate these rules later when we create other networks.

The default rules are as below. Docker adds four chains of its own, DOCKER, DOCKER-ISOLATION-1, DOCKER-ISOLATION-2 and DOCKER-USER. This last chain is a placeholder for users to implement their own rules (see documentation).

$ sudo iptables -S
-P INPUT ACCEPT
-P FORWARD DROP
-P OUTPUT ACCEPT
-N DOCKER
-N DOCKER-ISOLATION-STAGE-1
-N DOCKER-ISOLATION-STAGE-2
-N DOCKER-USER
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -j RETURN
-A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -j RETURN
-A DOCKER-USER -j RETURN

iptables has an option to group the rules by chain which can make them more understandable. From this view, the sequence of chains is:

  1. FORWARD,
  2. DOCKER-USER,
  3. DOCKER-ISOLATION-STAGE-1,
  4. DOCKER-ISOLATION-STAGE-2,
  5. DOCKER
$ sudo iptables -vL

Chain INPUT (policy ACCEPT 8832 packets, 625K bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
 2022  708K DOCKER-USER  all  --  any    any     anywhere             anywhere            
 2022  708K DOCKER-ISOLATION-STAGE-1  all  --  any    any     anywhere             anywhere            
    0     0 ACCEPT     all  --  any    docker0  anywhere             anywhere             ctstate RELATED,ESTABLISHED
    0     0 DOCKER     all  --  any    docker0  anywhere             anywhere            
    0     0 ACCEPT     all  --  docker0 !docker0  anywhere             anywhere            
    0     0 ACCEPT     all  --  docker0 docker0  anywhere             anywhere            

Chain OUTPUT (policy ACCEPT 11525 packets, 1532K bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain DOCKER (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DOCKER-ISOLATION-STAGE-2  all  --  docker0 !docker0  anywhere             anywhere            
 1938  701K RETURN     all  --  any    any     anywhere             anywhere            

Chain DOCKER-ISOLATION-STAGE-2 (1 references)
 pkts bytes target     prot opt in     out     source               destination         
   84  7056 DROP       all  --  any    docker0  anywhere             anywhere            
  812 56496 RETURN     all  --  any    any     anywhere             anywhere            

Chain DOCKER-USER (1 references)
 pkts bytes target     prot opt in     out     source               destination         
 2022  708K RETURN     all  --  any    any     anywhere             anywhere

Running a Container

When a container is created with default networking, Docker:

  1. creates a new network namespace for the container,
  2. creates a virtual Ethernet pair and places one end of the pair in the container’s network namespace,
  3. hooks the other to the bridge, docker0,
  4. assigns an IP address to the container’s ethernet device,
  5. enables the bridge.

Let’s run some commands to see evidence of the changes.

First, let’s create a container with a simple interactive shell and check its IP address. We can see that it has been assigned an address from the same subnet as the docker0 bridge.

$ docker run -it alpine /bin/ash
# ip a
...
30: eth0@if31: ...
   inet 172.17.0.2/16 brd 172.17.255.255 scope global eth0
   ...

In another shell logged into the host, let’s look at what’s changed on the host side. For one, we now see that docker0 is UP.

We also see that a new interface has appeared. It’s a veth, a virtual Ethernet device, which is one half of a pair of devices connected together by a virtual connection. The other half of the veth pair has been placed into the container’s network namespace.

$ ip a
...
3: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default 
    link/ether ...
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
    ...
31: vethe704058@if30: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP group default 
    link/ether ...
    inet6 ...
       valid_lft forever preferred_lft forever

Creating a New Docker Network

Let’s create a new docker network named ‘test1’ and verify its successful creation1:

 $ docker network create test1
 $ docker network ls

 NETWORK ID     NAME          DRIVER          SCOPE
 aec###         test1         bridge          local

This results in the creation of a new bridge named ‘br-###’

 $ ip a
 ...
 5: br-###: <NO-CARRIER,BROADCAST,MULTICAST,UP     mtu qdisc noqueue state...
 ...

Another way to see it is using the brctl command.

 $ brctl show
 bridge name    bridge id         STP enabled     interfaces
 br-###             8000.###    no

This bridge has an IP subnet assigned to it. Containers in this network are assigned addresses from this subnet.

 $ ip r
 ...
 172.19.0.0/16 dev br-### proto kernel scope link src 172.18.0.1 linkdown
 ...

These rules have been added to iptables:

 $ sudo iptables -S
 ...
 -A FORWARD -o br-### -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
 -A FORWARD -o br-### -j DOCKER
 -A FORWARD -i br-### ! -o br-### -j ACCEPT
 -A FORWARD -i br-### -o br-### -j ACCEPT
 -A DOCKER-ISOLATION-STAGE-1 -i br-### ! -o br-### -j DOCKER-ISOLATION-STAGE-2
 -A DOCKER-ISOLATION-STAGE-2 -o br-### -j DROP
 ...

Using the Network

The link is down until a container is invoked in that network. As a simple example, let’s invoke a shell in an Ubuntu container running bash.

 $ docker run -it --network=test1 ubuntu /bin/bash
 $ ip r
 ...
 172.19.0.0/16 dev br-### proto kernel scope link src 172.19.0.1
 ...

Cleaning Up

To remove networks not currently used by containers, first exit the Alpine shell, then remove the unused network

 $ docker network rm test1

You can verify using the commands above that the bridges and iptables rules have been deleted

  1. All commands run with Docker version 18.09.7, build 2d0083d ↩