What Happens When a Docker Network Is Created?
Docker uses two fundamental Linux networking technologies, networking namespaces and iptables, to allow containers to communicate with each other and the outside world. This post uses the Linux commandline to explore how Docker implements networking with these technologies. Once you understand how Docker networking works, you will have the power to extend it to your needs.
Installation
When Docker is first installed, it does a few actions behind the scenes to make networking possible:
- it creates a bridge,
- it adds its own rules to
iptables
.
Bridge Creation
Docker creates a bridge called docker0
which is an assigned an IP
address (172.17.0.1/16
in this case). If no network is specified
when a container is created, it is connected to this bridge by
default.
Note that the bridge state is DOWN
. The bridge is activated when
a container is connected to it.
$ brctl show
bridge name bridge id STP enabled interfaces
docker0 8000.0242a690e758 no
$ ip a
...
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 02:42:4d:11:10:f1 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
valid_lft forever preferred_lft forever
...
New iptables Rules
Docker adds its own rules in IP tables to isolate containers in
separate networks from each other. Since there is only one network
here (docker0
), these rules have no effect. We will investigate
these rules later when we create other networks.
The default rules are as below. Docker adds four chains of its own,
DOCKER
, DOCKER-ISOLATION-1
, DOCKER-ISOLATION-2
and
DOCKER-USER
. This last chain is a placeholder for users to
implement their own rules (see
documentation).
$ sudo iptables -S
-P INPUT ACCEPT
-P FORWARD DROP
-P OUTPUT ACCEPT
-N DOCKER
-N DOCKER-ISOLATION-STAGE-1
-N DOCKER-ISOLATION-STAGE-2
-N DOCKER-USER
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -j RETURN
-A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -j RETURN
-A DOCKER-USER -j RETURN
iptables
has an option to group the rules by chain which can make
them more understandable. From this view, the sequence of chains is:
FORWARD
,DOCKER-USER
,DOCKER-ISOLATION-STAGE-1
,DOCKER-ISOLATION-STAGE-2
,DOCKER
$ sudo iptables -vL
Chain INPUT (policy ACCEPT 8832 packets, 625K bytes)
pkts bytes target prot opt in out source destination
Chain FORWARD (policy DROP 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
2022 708K DOCKER-USER all -- any any anywhere anywhere
2022 708K DOCKER-ISOLATION-STAGE-1 all -- any any anywhere anywhere
0 0 ACCEPT all -- any docker0 anywhere anywhere ctstate RELATED,ESTABLISHED
0 0 DOCKER all -- any docker0 anywhere anywhere
0 0 ACCEPT all -- docker0 !docker0 anywhere anywhere
0 0 ACCEPT all -- docker0 docker0 anywhere anywhere
Chain OUTPUT (policy ACCEPT 11525 packets, 1532K bytes)
pkts bytes target prot opt in out source destination
Chain DOCKER (1 references)
pkts bytes target prot opt in out source destination
Chain DOCKER-ISOLATION-STAGE-1 (1 references)
pkts bytes target prot opt in out source destination
0 0 DOCKER-ISOLATION-STAGE-2 all -- docker0 !docker0 anywhere anywhere
1938 701K RETURN all -- any any anywhere anywhere
Chain DOCKER-ISOLATION-STAGE-2 (1 references)
pkts bytes target prot opt in out source destination
84 7056 DROP all -- any docker0 anywhere anywhere
812 56496 RETURN all -- any any anywhere anywhere
Chain DOCKER-USER (1 references)
pkts bytes target prot opt in out source destination
2022 708K RETURN all -- any any anywhere anywhere
Running a Container
When a container is created with default networking, Docker:
- creates a new network namespace for the container,
- creates a virtual Ethernet pair and places one end of the pair in the container’s network namespace,
- hooks the other to the bridge,
docker0
, - assigns an IP address to the container’s ethernet device,
- enables the bridge.
Let’s run some commands to see evidence of the changes.
First, let’s create a container with a simple interactive shell and
check its IP address. We can see that it has been assigned an address
from the same subnet as the docker0
bridge.
$ docker run -it alpine /bin/ash
# ip a
...
30: eth0@if31: ...
inet 172.17.0.2/16 brd 172.17.255.255 scope global eth0
...
In another shell logged into the host, let’s look at what’s changed on the
host side. For one, we now see that docker0
is UP
.
We also see that a new interface has appeared. It’s a veth, a virtual Ethernet device, which is one half of a pair of devices connected together by a virtual connection. The other half of the veth pair has been placed into the container’s network namespace.
$ ip a
...
3: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether ...
inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
valid_lft forever preferred_lft forever
...
31: vethe704058@if30: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP group default
link/ether ...
inet6 ...
valid_lft forever preferred_lft forever
Creating a New Docker Network
Let’s create a new docker network named ‘test1’ and verify its successful creation1:
$ docker network create test1
$ docker network ls
NETWORK ID NAME DRIVER SCOPE
aec### test1 bridge local
This results in the creation of a new bridge named ‘br-###’
$ ip a
...
5: br-###: <NO-CARRIER,BROADCAST,MULTICAST,UP mtu qdisc noqueue state...
...
Another way to see it is using the brctl
command.
$ brctl show
bridge name bridge id STP enabled interfaces
br-### 8000.### no
This bridge has an IP subnet assigned to it. Containers in this network are assigned addresses from this subnet.
$ ip r
...
172.19.0.0/16 dev br-### proto kernel scope link src 172.18.0.1 linkdown
...
These rules have been added to iptables
:
$ sudo iptables -S
...
-A FORWARD -o br-### -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o br-### -j DOCKER
-A FORWARD -i br-### ! -o br-### -j ACCEPT
-A FORWARD -i br-### -o br-### -j ACCEPT
-A DOCKER-ISOLATION-STAGE-1 -i br-### ! -o br-### -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-2 -o br-### -j DROP
...
Using the Network
The link is down until a container is invoked in that network. As a simple example, let’s invoke a shell in an Ubuntu container running bash.
$ docker run -it --network=test1 ubuntu /bin/bash
$ ip r
...
172.19.0.0/16 dev br-### proto kernel scope link src 172.19.0.1
...
Cleaning Up
To remove networks not currently used by containers, first exit the Alpine shell, then remove the unused network
$ docker network rm test1
You can verify using the commands above that the bridges and iptables rules have been deleted
-
All commands run with Docker version 18.09.7, build 2d0083d ↩