Docker

Docker is a type of virtualization for Gnu\Linux that allows processes to share similar resources, without having a full OS for each image.

General Tips

It helps to read a book, and then keep it as a Reference. Here Using Docker By Adrian Mouat is a decent book. In fact, you should be reading books on any subject that interests you. Physical paper books, too.

Docker is x86_64. ARM has a separate build. There is no i386.

You will want 'some' RAM. I had 1GB on a P4 machine, and that was not enough. 4GB was enough.

You should always use docker compose. Docker is possible to run on the command line, but with a compose file, you can write everything down in a much simpler fashion. Use compose. It's a separate install, currently. Install it. Seriously, just ignore the docker command lines. I consider them useless. More of a red herring for rookies.

ARM is SOL

I tried to use Docker on ARM, and while you might be able to install docker, you will find that images for your applications may not be there. e.g. you will find Nginx, but you won't find Gitlab. This means that docker is pretty much x86-64 only.

Docker Commands

Here are commands you need to know. Just the necessary ones.

docker-compose up -d

Starts the containers in the docker compose file, if they aren't already started. the -d detaches from the stdout logging. You don't need to use stdout logging, you can use docker logs, but its there if you want it.

docker ps

Lists containers running. If one fails to start, you'll see it missing from here

docker logs <containername>

Gives you some logging output from the container. Often enough to troubleshoot. (also see tips below).

docker exec -it <containername> /bin/bash

This will get you in a shell in the docker container. From here you can do what you need to. Most are debian, and need apt-get install less nano or whatever program you are missing. Ping is missing from possibly all containers, so if you want to test via ping, you'll have to apt-get it.

docker-compose restart

This will restart all containers. However, I don't recommend it. Initting containers can get corrupted this way, and also its much easier to restart a single faulty container via...

docker restart <containername>

This will restart one single container.

docker cp <containername>:/dir/to/file dest

You can copy files from local machine to docker, or vice versa with this. Extremely useful.

Deleting Containers

Less often, you might want to know

#!/bin/bash docker rm $(docker ps -a -q) docker rmi $(docker images -q) --force

This starts over from scratch. This is how easy it is to reboot a docker from square one. note: below command (rmi), only needed if you want to remove base images. Needed when updating a stable. Not needed when changing other parameters not related to base image. Also, this deletes all containers, which maybe you don't want to do, if you either A) only want to delete certain images you are testing B) have some containers with custom changes saved, and not backed up elsewhere.

Updating

docker images
(shows out of date image)
docker pull mysql
(downloads new image)
docker images
(shows two versions of mysql. old and new)
docker stop some_container_name
docker rmi -f cjklfs23404
(where cjklfs23404 is the old container alias under docker images)
docker-compose up -d mysql

Containers don't all Restart?!

Make sure to have in your docker compose for each container a restart:always Otherwise, a container won't necessarily start when docker is restarted.

Search for images

docker search <imagename>

Search for Versions of an image

What if you want to see what versions are available for a given image? Say you want php, but don't know which to get. hub.docker.com is broken. Requires js, slow, and bad. You can't get a list of images on it easily.

Solution:

$REPOSITORY=php
wget -q https://registry.hub.docker.com/v1/repositories/$REPOSITORY/tags -O - | \
   sed -e 's/[][]//g' -e 's/"//g' -e 's/ //g' | \
   tr '}' '\n'  | \
   awk -F: '{print $3}' | less

And you will get the full list. Put the above into an alias or .sh file. Ref: https://stackoverflow.com/questions/54418542/dockerhub-listing-all-available-versions-of-a-given-image

Save Changes to Docker Image

docker commit

See official documentation.

Volumes

Volumes can be handled a few ways. The simplest is to store files in a local directory which is then mapped to the remote drive.

e.g. docker-compose:

  nginx:
    image: nginx:####
    container_name: mynginxserver
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf
      - /etc/letsencrypt/:/etc/letsencrypt/
      - ./local_dir/:/var/www/html/
      - ./local_logs/:/var/log/nginx/
    ports:
      - 80:80
      - 443:443
    restart: always
    logging:
      driver: "json-file"
      options:
        max-size: "200k"
        max-file: "10"

However, if you just have a volume, without a local directory, then it will save in /var/ somewhere (unintuitive... bad). e.g. from Dolibarr:

The Dolibarr installation and all data beyond what lives in the database
 (file uploads, etc) are stored in the unnamed docker volume volume
/var/www/html and /var/www/documents. The docker daemon will store
 that data within the docker directory /var/lib/docker/volumes/....
That means your data is saved even if the container crashes, is stopped
or deleted.

Specific Tips

YAML is space sensitive

When you edit the .yml file for docker-compose, you have to hit spaces in a certain pattern (tabs not allowed). This is absurd, but just be aware. The errors are cryptic, and its often just because the spacing doesn't stick to what it expects.

If you restart a containers namesake process, it will probably restart / reset the container

So if you are troubleshooting an apache container, you edit some files, then /etc/init.d/apache2 restart, uh oh... You just undid all the edits you made, if they aren't in a permanent volume. You can shell in, make edits, and then exit the shell, but a service restart often resets the container.

Consider a single reverse proxy, to handle multiple websites

There are many ways to do this. I use an nginx proxy from scratch. You can also use some containers that are built for this purpose (I personally think it's bloated but a lot of people use Jason Wilder's proxy) https://web.archive.org/web/https://github.com/jwilder/nginx-proxy - A lot of people swear by this, but I think it's straying too far to the high level.

If you use a single reverse proxy, Lets Encrypt can be done easy

In this case scenario you would have certbot on the host and a local volume that the proxy has access to which is the webroot of the Lets Encrypt scripts. The nginx proxy entry look something like this:

location ^~ /.well-known {
     alias /var/www/html/.well-known/;
      autoindex on;
    }

And this is put in every server declaration of nginx.conf. Real simple, real easy. The docker compose of the nginx proxy is something like:

Contents of docker-compose.yml

nginx: image: nginx:latest container_name: custom_name_for_my_proxy volumes: - ./nginx.conf:/etc/nginx/nginx.conf - /etc/letsencrypt/:/etc/letsencrypt/ - ./webroot/:/var/www/html/

The volumes section is extremely simple, don't be scared. There are two entries. Local and remote. You specify what folder will be the local directory which will be cloned to the host at the remote path you specify. So, the host runs certbot at /etc/letsencrypt, and this folder is cloned to the nginx proxy container, at the same location. Finally, webroot must be set in certbot, but it prompts you for this. And if you forget or get it wrong, it can be configured somewhere in /etc/letsencrypt. it's a one liner text entry).

Give every Container a Containername

This makes it easier to refer to them later. All you need to do in the compose file is include container\_name: something. Much better than the gibberish they give these names if you don't include it.

Beware of Interrupting Initting Containers

When you first build a container, it might take 30-60 or more seconds to do whatever it needs to do. If, before then, you restart it... It may get corrupted. This has happened to me more than once. When you are testing a new container, and it doesn't seem to work for some inexplicable reason, create a container with a new name (it will create a new one), or delete the first one, and start it again.

Put Apache or Program logs from the Container in a volume that is locally accessible

This means you want a volume something like ./containerA\_files/logs:/var/www/log/apache2/ so that you can monitor the logs from your host machine easily. docker logs doesn't have everything.

Only Restart Containers you need to Restart

You can restart everything with docker-compose restart, but it's faster, and less prone to break initting containers, if you docker restart containername. Do the latter.

Volumes Mounting Over Existing Directories

As discussed here: https://web.archive.org/web/https://github.com/moby/moby/issues/4361, if you add a volume to an existing container, it will seem to delete the folder's contents. I've seen mixed behaviour with this. Sometimes it deletes it even if you start a new container with the folder... Other times it has not. In any case, just docker cp the files to the folder, then add the volume mount. This may not be the most graceful solution for upgrades, but it will work. Best practices would be here (See: dbxt commented Dec 14, 2014): https://web.archive.org/web/https://github.com/docker-library/wordpress/issues/10

If you edit a docker-compose file you must restart the container with Docker-compose

If you make a change in docker compose, you must docker stop service, then docker-compose up -d service. Otherwise the changes will not take effect. The mistake here, would be thinking that you could just docker stop service, then docker start service... That doesn't work.

Keep verbose logging off your HDD with RAM only logging

If you have e.g. apache/nginx write to your HDD with every website visit, it will quickly wear out your HDD (esp. SSD). Put verbose logging in the RAM only with a tmpfs mount. https://docs.docker.com/storage/tmpfs/

e.g. (ref: https://stackoverflow.com/questions/41902930/docker-compose-tmpfs-not-working)

services:
  ubuntu:
    image: ubuntu
    command: "bash -c 'mount'"
    volumes:
      - cache_vol:/var/cache
      - run_vol:/run

volumes:
  run_vol:
    driver_opts:
      type: tmpfs
      device: tmpfs
  cache_vol:
    driver_opts:
      type: tmpfs
      device: tmpfs

This also allows you to share the tmpfs mounts if needed.

Access it from host via:

docker exec -it nginx_server tail -F /run/shm/access.log

Note that you might think you can do

docker container run my_nginx_server tail -F /run/shm/access.log

But that will only work on container base images, not running containers. To 'execute' a command on a running or existing container, the command you want is exec. (Also used to interactively start bash shell above)

EDIT: Use with discretion. Active data in RAM may use power.

Mysql Backup / Restore

# Backup
docker exec CONTAINER /usr/bin/mysqldump -u root --password=root DATABASE > backup.sql

# Restore
cat backup.sql | docker exec -i CONTAINER /usr/bin/mysql -u root --password=root DATABASE

ref: https://gist.github.com/spalladino/6d981f7b33f6e0afe6bb May not need to do this, if you keep the DB (ibdata files) in a local directory/volume via docker-compose.yml e.g.

mydb:
   image: mysql:9999
   volumes:
      - ./db_files:/var/lib/mysql
   restart: always

Limit docker logs

Docker will keep lots of logs if you don't manage it. https://github.com/docker/compose/issues/1083 https://docs.docker.com/compose/compose-file/#logging https://stackoverflow.com/questions/31829587/docker-container-logs-taking-all-my-disk-space view (all) logs with

docker-compose logs

To limit a service's logs in docker-compose do the following:

• find a containers/services entry
• append to the end (watch the white space & no tabs)

    logging:
      driver: "json-file"
      options:
        max-size: "200k"
        max-file: "10"

The rule with logs, is nothing - unless you need it. Then everything. Any unnecessary logging is taxing the CPU / HDD. Which leads me to...

view details about container

To view paths, configuration settings, etc... See

docker inspect <name or id>

It's easier to grep these. the search in less wasn't working for me to find .LogPath for example.

docker inspect mycontainer | grep log

Will show where the log files are located.

Installing Docker-compose from pip or github binary not Debian

Subject. See official docker documentation which states the URL of github.com/docker/compose/releases or pip.

Pip is probably better.

apt-get install python3-pip 
pip3 install docker-compose

Troubleshooting

Cannot start container: [0] Id already in use #20570

• https://github.com/moby/moby/issues/20570

After update, unable to start container. Solution was found to be:

docker-compose down -v then docker-compose up -d

Container was viewable with

docker ps -a (not just docker ps)

Fails

Creating a docker-compose with a stable or latest tag

If you make a docker-compose file, you must link it to a marked version. You can later edit it, to use stable or latest, but one of the strengths of Docker is version control. If you create a docker-compose, always test on a specific version, should you need to roll back in the future. Otherwise, your scripts will fail in 2-20-200 years when the conf files are not supported anymore.

e.g. https://github.com/nucreativa/frontaccounting-docker (6th commit) Here, the nginx config is no longer supported in vhosts, it must be in server_blocks. This would've been avoided by denoting a fixed nginx release instead of tag:latest.

See Also

• samba_docker_by_dlandon

External Links

• https://github.com/LeCoupa/awesome-cheatsheets/blob/master/tools/docker.sh Information dense cheatsheet (unfortunately github)
• https://gist.github.com/spalladino/6d981f7b33f6e0afe6bb A million idiots type "thanks". Also the source for the mysql backup command.