Project Atomic – Installing VM with Vagrant, Libvirt and get more space for the /var/lib/docker directory

I’m playing with Project Atomic. I use Vagrant in my Fedora 23 desktop as helps me to increase my productivity when working with VM :D. As I prefer to use libvirt over VirtualBox as my vagrant provider I install the following packages:

sudo dnf install vagrant-libvirt virt-manager

By default the Atomic Host virtual machine has little space for new containers and images (about 2 GB) so if you don’t remove your containers often you’re not going to have much fun. In this post I’m installing the atomic-host and assign more space to the /var/lib/docker directory which is the place our images, containers and other docker files will be stored.

I create the Vagrantfile for the official atomic-host box:

vagrant init centos/atomic-host

Then I edit the Vagrantfile. I’m adding a QCOW2 file that will act as a virtual disk (I’m using 30G). I use as a reference the vagrant-libvirt documentation. I add the following lines after config.vm.box = “centos/atomic-host”

config.vm.provider :libvirt do |libvirt|
   libvirt.storage :file, :size => ’30G’
end

I start the virtual machine:

vagrant up –provider libvirt

In the vagrant up logs I can see that a new 30 GB disk has been added to the virtual machine.

==> default: — Disks: vdb(qcow2,30G)

==>default:– Disk(vdb): /var/lib/libvirt/images/atomichost_default-vdb.qcow2

Now I open a SSH session:

vagrant ssh

I create a partition for the /dev/vdb disk and change type to LVM so I can add more storage in the future easily. Here are shown only the important parts:

sudo fdisk /dev/vdb

…
Command (m for help): n
Partition type:
p primary (0 primary, 0 extended, 4 free)
e extended
Select (default p): p
Partition number (1-4, default 1): Press Enter
First sector (2048-62914559, default 2048): Press Enter
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-62914559, default 62914559): Press Enter
Using default value 62914559
Partition 1 of type Linux and of size 30 GiB is set
Command (m for help): t
Selected partition 1
Hex code (type L to list all codes): 8e
Changed type of partition ‘Linux’ to ‘Linux LVM’
Command (m for help): w

The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.

Now I’m going to use Logical Volume Management. That way if I need more space in the future I could add a new virtual disk to the logical volume. First I create the physical volume for LVM:

sudo pvcreate /dev/vdb1
Physical volume “/dev/vdb1” successfully created

I create a volume group and add the /dev/vdb1 partition to that volume group:

sudo vgcreate atomic_vg /dev/vdb1

I create a logical volume group and add all the space available in the volume group

sudo lvcreate -l 100%FREE -n atomic_lv atomic_vg

I add a filesystem to the logical volume partition. The logical volume is where I will store all the /var/lib/docker files. I’m using XFS as my filesystem type.

sudo mkfs.xfs /dev/mapper/atomic_vg-atomic_lv

I add an entry to /etc/fstab

sudo sh -c “echo ‘/dev/mapper/atomic_vg-atomic_lv /var/lib/docker xfs defaults 0 0’ >> /etc/fstab”

I stop the docker service so no newer files are copied to the existing /var/lib/docker directory

sudo systemctl stop docker

I mount temporarily the logical volume under /media

sudo mount /dev/mapper/atomic_vg-atomic_lv /media

I copy all the existing files from /var/lib/docker to the logical volume

sudo sh -c “cp -r /var/lib/docker/* /media/”

I umount the logical volume

sudo umount /media

I try to mount the new partition:

sudo mount -a

I check that the new /var/lib/docker is ready

sudo df -kh

Filesystem Size Used Avail Use% Mounted on
…
/dev/mapper/atomic_vg-atomic_lv 30G 33M 30G 1% /var/lib/docker
…

There it is 30 GB for my new images and containers!. Finally I start againt the docker engine service:

sudo systemctl start docker

Well that was long, wasn’t it, but at least I’ve more space to play now!

Note: In case you want to add more space using a new qcow2 after you’ve already run vagrant up, according to this issue,  if you’ve already instantiated the VM with vagrant up, if you change the Vagrantfile to add a new disk (e.g libvirt.storage :file, :size => ’30G’) it won’t work after a vagrant reload, no new virtual disk will be added so alternatively you can halt the virtual machine and use virt-manager to add a new disk and follow the fdisk, pvcreate, mount steps…

Docker Containers: Build and Deploy with Kubernetes, Flannel, Cockpit, and Atomic by Christopher Negus

The book Docker Containers: Build and Deploy with Kubernetes, Flannel, Cockpit, and Atomic by Christopher Negus is an awesome book for those who want to have an introduction to Docker and advanced concepts like Kubernetes. It’s easy to read and concise, one of the best books I’ve read so far and I’ve read many of them about Docker:). Chapters are clear and “short”, with short I mean they are not full of boring information, you’ll start working with containers right away.

It was published at the end of 2015 and still valid of course!, but as usual and inevitable due to the nature of Docker container evolution… some commands (really few) may fail so I’m sharing with you in this post some updates. If you’re reading the book and something doesn’t work as expected this could help. I’m using the O’Reilly Safari version in case it differs from yours.

Chapter 7 – Start a Container to inspect

The following command:

docker run -d -p 8080:8080 --name="fed_web"   \
--restart="on-failure:5"  -w /var/www/html    \
-v /var/www/html:/var/www/html                \
fedora python -m SimpleHTTPServer 8080

Should be changed to:

docker run -d -p 8080:8080 --name="fed_web"   \
--restart="on-failure:5"  -w /var/www/html    \
-v /var/www/html:/var/www/html                \
fedora python3 -u -m http.server 8080

Seems that the latest fedora container is not providing python 2.7 by default and python3 is the only available python. Thanks to the -u with python3 the outputs are not buffered and sent to STDOUT so you can use docker attach, docker logs… More info for that -u option here.

Chapter 7 – Using docker exec to Start a New Process in a Running Container

In recent Fedora versions (23 onwards) it seems that we have to start saying goodbye to yum and say hello to dnf. So to avoid warnings, it’s best to change:

docker exec -it fed_web yum install net-tools -y

With:

docker exec -it fed_web dnf install net-tools -y

Chapter 9 – Mounting sockets

Once again the dnf issue 🙂 It’s best to change:

RUN yum -y update; yum -y install systemd-libs docker; yum clean all

With:

RUN dnf -y update; dnf -y install systemd-libs docker; dnf clean all

To avoid ugly warnings.

Chapter 11 – Summary

As a rule, keep an eye on the amount of disk space available in the /var/log/dockerdirectory,

Should be:

As a rule, keep an eye on the amount of disk space available in the /var/lib/dockerdirectory,

Chapter 16 – Step 2: Set Up Kubernetes Master

The

KUBE_API_ADDRESS="--address=0.0.0.0" 

configuration line for our master server should be

KUBE_API_ADDRESS="--insecure-bind-address=0.0.0.0" as --address

has been deprecated.

The

KUBE_SERVICE_ADDRESSES="--portal_net=10.254.0.0/16" 

configuration line should be 

KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16"

Cheers!

Service definition to run Cockpit on system startup for CentOS Atomic SIG

I’m working these days with Project Atomic. You should have a look to the awesome Quickstart guide.

I’ve chosen to use Vagrant with the CentOS Atomic SIG so playing with Project Atomic is really easy (change to virtualbox if using that provider :D)

vagrant init centos/atomic-host; vagrant up –provider libvirt

One of the first things I’ve tested is Cockpit’s web server manager. It’s pretty cool and easy to install following the guide.

Once inside the Project Atomic host, Cockpit’s container is intalled with the following command:

vagrant ssh
sudo atomic run cockpit/ws

Remember, I use this blog so I don’t forget my notes. I’m just sharing with you the service definition needed to run Cockpit on system startup when working with CentOS Atomic SIG and not Fedora’s version which is explained in the source for this post. This file must be placed at /etc/systemd/system/cockpitws.service

[Unit]
Description=Cockpit Web Interface
Requires=docker.service
After=docker.service

[Service]
Restart=on-failure
RestartSec=10
ExecStart=/usr/bin/docker run --rm --privileged --pid host -v /:/host --name %p cockpit/ws /container/atomic-run --local-ssh
ExecStop=-/usr/bin/docker stop -t 2 %p

[Install]
WantedBy=multi-user.target

Then just enable and start the service and the Cockpit container will run and be ready to serve at 9090 port (user vagrant/vagrant or root/vagrant).

sudo systemctl daemon-reload
sudo systemctl enable cockpitws.service
sudo systemctl start cockpitws.service

Cool stuff Project Atomic and Cockpit.

 

 

INSTALLING THE LATEST STABLE RELEASE FOR DOCKER COMPOSE ON LINUX X86-64

Following the last post on how to install the latest stable binary for Docker Machine, If you want to install the Docker Machine client on your 64 bits Linux machine you only have to follow the official instructions and download it from GitHub.

However, if you want to download the file tagged as the latest stable release right away without checking on GitHub what is the new release, this is the one line command I’m using thanks to the tag added by the Docker team:

curl -L https://github.com/docker/compose/releases/download/`curl -s -L https://github.com/docker/compose/releases/latest| grep Linux-x86_64 | grep href | cut -f 6 -d '/'`/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose && chmod +x /usr/local/bin/docker-compose

Then you can check that docker-machine is ready if you have /usr/local/bin in your PATH, e.g I’m running the 1.7.1 version:

docker-compose version
docker-compose version 1.7.1, build 0a9ab35

Cheers!

Installing the latest stable release for Docker Machine on Linux x86-64

I’ve been playing with Docker for a while and I just love it so I’m starting to share some notes that may be useful to you. If you want to install the Docker Machine client on your 64 bits Linux machine you only have to follow the official instructions and download it from GitHub.

However, I’m lazy so if you want to download the file tagged as the latest stable release right away without checking on GitHub what is the new release, this is the one line command I’m using thanks to the tag added by the Docker team:

curl -L https://github.com/docker/machine/releases/download/`curl -s -L https://github.com/docker/machine/releases/latest| grep Linux-x86_64 | grep href | cut -f 6 -d '/'`/docker-machine-`uname -s`-`uname -m` > /usr/local/bin/docker-machine && chmod +x /usr/local/bin/docker-machine

Then you can check that docker-machine is ready if you have /usr/local/bin in your PATH, e.g I’m running the 0.7.0 version:

docker-machine version
docker-machine version 0.7.0, build a650a40

Cheers!