Networking in Docker

Networking computers is hard. Networking computers using containers, as when running Docker, can be harder still.

There are a lot of moving pieces involved when we want different computers or different services on the same machine to communicate with each other. Details like addresses and ports and protocols, not to mention authentication and permissions. And just because a service is using a specific port, say 3000, doesn't mean that it's available on that port. The host configuration, as in a Docker system, may make it available at another port, say 80.

The above scenario is not uncommon for a web service, where the app runs on port 3000, and users access it on the web via port 80.

Another example is when an application connects to a database using specific host and port settings. When both are running on the same machine, that may look like

mysql://db_user:password@tcp(127.0.0.1:3306)/my_database

If the database has its own machine, then it will have its own hostname like "my-database://" and possibly a custom port mapping.

That makes sense. What happens when Docker enters the picture?

First, it's important to understand that containers have their own rules around networking. Often, a container service will not expose its networking connections unless we tell it to. This is intentional safeguarding, so that nothing is left unprotected by mistake. Second, containers can communicate using an internal network system that can be isolated from the host system. Again, this is designed to help isolate and insulate our systems.

Scenario I

I have one service running natively on my machine that needs to connect to another service running inside a container. One example is a Golang application that uses a Postgres database inside a container.

If we download the Postgres container and execute "docker run ...", the database is not reachable. Why is that?

As we said before, Docker operates its own network for communication between containers. This is not the same as the network for the host machine. But we can tell Docker to make the services inside the container available to the host machine. There are two ways to do that.

The common way -- especially with a database container -- is to expose the port that the database uses. For Postgres that is 5432; for MySQL it is 3306. When calling "docker run ..." we include the publish flag to make that port available to the host machine.

docker run -p 5432:5432 postgres

Another option is to tell Docker to use the host machine's networking namespace, instead of its own internal one. That means all the ports would be available, without having to specify them as we do above.

On linux that's:

docker run --network host postgres

Now the Go app can reach the DB using this connection.

On Docker Desktop for Mac, however, this is currently not supported, according to this open issue.

Scenario 2

Let's move our application into its own container, so now both services are running inside containers.

If we start our database container, and then start our app container, they will not work. Why not? Docker does not assume that each container should be open to the others. The security firewall again. Instead, we need to create a named network and run both services with it.

docker network create app-network

Now we run our database container and specify the network to use

docker run --network app-network postgres

Same thing for our application

docker run --network app-network my-app

If you miss the network step when you run the container, don't worry. It's possible to modify a running container, and add it to your custom network:

docker network connect {network_name} {container_name}

More info on Docker networks

Scenario 3

Now we have our database and application in two separate containers. Let's say we want to start and stop our whole application without using all those commands above! We can do that with docker-compose.

Compose is a great way to handle multiple containers and sophisticated environment or network settings. Instead of running individual commands in the terminal, we define a manifest -- something like a recipe -- that Docker will use to build all the required resources. It makes our process much easier, as it automatically creates a network for the resources, and provides a transparent way of naming the containers for us.

To run a compose file, simply type

docker-compose up

The official Postgres Docker image has a good example of how this works in a docker-compose.yml file. (The example creates a database in one container and a web-based database-admin tool in the other. Imagine replacing adminer here with the web app we use above.)

version: '3.1'
 
services:
 
  db:
    image: postgres
    restart: always
    environment:
      POSTGRES_PASSWORD: example
 
  adminer:
    image: adminer
    restart: always
    ports:
      - 8080:8080

Notice a couple things:

• we don't need to specify the network to connect these services. Compose does that for us.
• environment variables are added to each service. That means we won't struggle to remember them later, and we don't have to re-enter them.
• only expose the ports needed for user access, in this case for the web service on port 8080.

This is a lot less to remember than all of the terminal commands above.