etcd

Installing CoreOS etcd server on CentOS 7

While I’m preparing a shell script or test some Ansible roles available at Ansible Galaxy so the installation is automatic, here I show you the steps I followed to install by hand the Etcd server on CentOS 7 as quick and fast as possible.

First of all we have to create some directories (/var/lib/etcd and /etc/etcd) and add the etcd user and group

mkdir /var/lib/etcd;mkdir /etc/etcd; groupadd -r etcd; useradd -r -g etcd -d /var/lib/etcd -s /sbin/nologin -c "etcd user" etcd;chown -R etcd:etcd /var/lib/etcd

Now we have to add a systemd service definition for our etcd service

cat << EOT > /usr/lib/systemd/system/etcd.service
[Unit]
Description=etcd service
After=network.target

[Service]
Type=notify
WorkingDirectory=/var/lib/etcd/
EnvironmentFile=-/etc/etcd/etcd.conf
User=etcd
ExecStart=/usr/bin/etcd
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
EOT

Warning: The etcd service needs a configuration file, we install a really simple one that should be modified according to your needs, e.g add urls with your server’s IP address or DNS names so your server is not only useful for localhost and secure client requests. Read https://github.com/coreos/etcd for more info.

cat &lt;&lt; EOT &gt; /etc/etcd/etcd.conf
 # [member]
 ETCD_NAME=default
 ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
 #ETCD_SNAPSHOT_COUNTER="10000"
 #ETCD_HEARTBEAT_INTERVAL="100"
 #ETCD_ELECTION_TIMEOUT="1000"
 #ETCD_LISTEN_PEER_URLS="http://localhost:2380,http://localhost:7001"
 ETCD_LISTEN_CLIENT_URLS="http://localhost:2379"
 ETCD_ADVERTISE_CLIENT_URLS="http://localhost:2379"
 #ETCD_MAX_SNAPSHOTS="5"
 #ETCD_MAX_WALS="5"
 #ETCD_CORS=""
 #
 #[cluster]
 #ETCD_INITIAL_ADVERTISE_PEER_URLS="http://localhost:2380,http://localhost:7001"
 # if you use different ETCD_NAME (e.g. test), set ETCD_INITIAL_CLUSTER value for this name, i.e. "test=http://..."
 #ETCD_INITIAL_CLUSTER="default=http://localhost:2380,default=http://localhost:7001"
 #ETCD_INITIAL_CLUSTER_STATE="new"
 #ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
 #ETCD_ADVERTISE_CLIENT_URLS="http://localhost:2379,http://localhost:4001"
 #ETCD_DISCOVERY=""
 #ETCD_DISCOVERY_SRV=""
 #ETCD_DISCOVERY_FALLBACK="proxy"
 #ETCD_DISCOVERY_PROXY=""
 #
 #[proxy]
 #ETCD_PROXY="off"
 #
 #[security]
 #ETCD_CA_FILE=""
 #ETCD_CERT_FILE=""
 #ETCD_KEY_FILE=""
 #ETCD_PEER_CA_FILE=""
 #ETCD_PEER_CERT_FILE=""
 #ETCD_PEER_KEY_FILE=""
 EOT

Time to download and install etcd binaries for Linux x86_64, the following commands should be good for any Linux distro. It downloads the latest stable version available, creates a directory for any downloaded version and changes the symbolinc link accordingly. It runs etcd with the version argument to check that the binary works fine.

ETCD_VERSION=`curl -s -L https://github.com/coreos/etcd/releases/latest | grep linux-amd64\.tar\.gz | grep href | cut -f 6 -d '/' | sort -u`; ETCD_DIR=/opt/etcd-$ETCD_VERSION; mkdir $ETCD_DIR;curl -L https://github.com/coreos/etcd/releases/download/$ETCD_VERSION/etcd-$ETCD_VERSION-linux-amd64.tar.gz | tar xz --strip-components=1 -C $ETCD_DIR; ln -sf $ETCD_DIR/etcd /usr/bin/etcd && ln -sf $ETCD_DIR/etcdctl /usr/bin/etcdctl; etcd --version

We can enable and start the etcd server with:

systemctl enable etcd; systemctl start etcd

Check etcd service status

systemctl status etcd

● etcd.service – etcd service
Loaded: loaded (/usr/lib/systemd/system/etcd.service; enabled; vendor preset: disabled)
Active: active (running) since lun 2016-08-01 10:05:51 UTC; 2s ago
Main PID: 31051 (etcd)
CGroup: /system.slice/etcd.service
└─31051 /usr/bin/etcd

ago 01 10:05:51 localhost.localdomain etcd[31051]: ready to serve client requests
ago 01 10:05:51 localhost.localdomain etcd[31051]: serving insecure client requests on localhost:2379, this is strongly discouraged!
ago 01 10:05:51 localhost.localdomain systemd[1]: Started etcd service.

As you may notice there’s a warning about “serving insecure client requests on localhost:2379, this is strongly discouraged!” once again please change the configuration for your needs and set it safely.

I’ll try update this post so you may follow this blog.

Cheers!

atomic

Project Atomic – Installing VM with Vagrant, Libvirt and get more space for the /var/lib/docker directory

I’m playing with Project Atomic. I use Vagrant in my Fedora 23 desktop as helps me to increase my productivity when working with VM😀. As I prefer to use libvirt over VirtualBox as my vagrant provider I install the following packages:

sudo dnf install vagrant-libvirt virt-manager

By default the Atomic Host virtual machine has little space for new containers and images (about 2 GB) so if you don’t remove your containers often you’re not going to have much fun. In this post I’m installing the atomic-host and assign more space to the /var/lib/docker directory which is the place our images, containers and other docker files will be stored.

I create the Vagrantfile for the official atomic-host box:

vagrant init centos/atomic-host

Then I edit the Vagrantfile. I’m adding a QCOW2 file that will act as a virtual disk (I’m using 30G). I use as a reference the vagrant-libvirt documentation. I add the following lines after config.vm.box = “centos/atomic-host”

config.vm.provider :libvirt do |libvirt|
   libvirt.storage :file, :size => ’30G’
end

I start the virtual machine:

vagrant up –provider libvirt

In the vagrant up logs I can see that a new 30 GB disk has been added to the virtual machine.

==> default: — Disks: vdb(qcow2,30G)

==>default:– Disk(vdb): /var/lib/libvirt/images/atomichost_default-vdb.qcow2

Now I open a SSH session:

vagrant ssh

I create a partition for the /dev/vdb disk and change type to LVM so I can add more storage in the future easily. Here are shown only the important parts:

sudo fdisk /dev/vdb


Command (m for help): n
Partition type:
p primary (0 primary, 0 extended, 4 free)
e extended
Select (default p): p
Partition number (1-4, default 1): Press Enter
First sector (2048-62914559, default 2048): Press Enter
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-62914559, default 62914559): Press Enter
Using default value 62914559
Partition 1 of type Linux and of size 30 GiB is set
Command (m for help): t
Selected partition 1
Hex code (type L to list all codes): 8e
Changed type of partition ‘Linux’ to ‘Linux LVM’
Command (m for help): w

The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.

Now I’m going to use Logical Volume Management. That way if I need more space in the future I could add a new virtual disk to the logical volume. First I create the physical volume for LVM:

sudo pvcreate /dev/vdb1
Physical volume “/dev/vdb1” successfully created

I create a volume group and add the /dev/vdb1 partition to that volume group:

sudo vgcreate atomic_vg /dev/vdb1

I create a logical volume group and add all the space available in the volume group

sudo lvcreate -l 100%FREE -n atomic_lv atomic_vg

I add a filesystem to the logical volume partition. The logical volume is where I will store all the /var/lib/docker files. I’m using XFS as my filesystem type.

sudo mkfs.xfs /dev/mapper/atomic_vg-atomic_lv

I add an entry to /etc/fstab

sudo sh -c “echo ‘/dev/mapper/atomic_vg-atomic_lv /var/lib/docker xfs defaults 0 0’ >> /etc/fstab”

I stop the docker service so no newer files are copied to the existing /var/lib/docker directory

sudo systemctl stop docker

I mount temporarily the logical volume under /media

sudo mount /dev/mapper/atomic_vg-atomic_lv /media

I copy all the existing files from /var/lib/docker to the logical volume

sudo sh -c “cp -r /var/lib/docker/* /media/”

I umount the logical volume

sudo umount /media

I try to mount the new partition:

sudo mount -a

I check that the new /var/lib/docker is ready

sudo df -kh

Filesystem Size Used Avail Use% Mounted on

/dev/mapper/atomic_vg-atomic_lv 30G 33M 30G 1% /var/lib/docker

There it is 30 GB for my new images and containers!. Finally I start againt the docker engine service:

sudo systemctl start docker

Well that was long, wasn’t it, but at least I’ve more space to play now!

Note: In case you want to add more space using a new qcow2 after you’ve already run vagrant up, according to this issue,  if you’ve already instantiated the VM with vagrant up, if you change the Vagrantfile to add a new disk (e.g libvirt.storage :file, :size => ’30G’) it won’t work after a vagrant reload, no new virtual disk will be added so alternatively you can halt the virtual machine and use virt-manager to add a new disk and follow the fdisk, pvcreate, mount steps…

docker_containers

Docker Containers: Build and Deploy with Kubernetes, Flannel, Cockpit, and Atomic by Christopher Negus

The book Docker Containers: Build and Deploy with Kubernetes, Flannel, Cockpit, and Atomic by Christopher Negus is an awesome book for those who want to have an introduction to Docker and advanced concepts like Kubernetes. It’s easy to read and concise, one of the best books I’ve read so far and I’ve read many of them about Docker:). Chapters are clear and “short”, with short I mean they are not full of boring information, you’ll start working with containers right away.

It was published at the end of 2015 and still valid of course!, but as usual and inevitable due to the nature of Docker container evolution… some commands (really few) may fail so I’m sharing with you in this post some updates. If you’re reading the book and something doesn’t work as expected this could help. I’m using the O’Reilly Safari version in case it differs from yours.

Chapter 7 – Start a Container to inspect

The following command:

docker run -d -p 8080:8080 --name="fed_web"   \
--restart="on-failure:5"  -w /var/www/html    \
-v /var/www/html:/var/www/html                \
fedora python -m SimpleHTTPServer 8080

Should be changed to:

docker run -d -p 8080:8080 --name="fed_web"   \
--restart="on-failure:5"  -w /var/www/html    \
-v /var/www/html:/var/www/html                \
fedora python3 -u -m http.server 8080

Seems that the latest fedora container is not providing python 2.7 by default and python3 is the only available python. Thanks to the -u with python3 the outputs are not buffered and sent to STDOUT so you can use docker attach, docker logs… More info for that -u option here.

Chapter 7 – Using docker exec to Start a New Process in a Running Container

In recent Fedora versions (23 onwards) it seems that we have to start saying goodbye to yum and say hello to dnf. So to avoid warnings, it’s best to change:

docker exec -it fed_web yum install net-tools -y

With:

docker exec -it fed_web dnf install net-tools -y

Chapter 9 – Mounting sockets

Once again the dnf issue🙂 It’s best to change:

RUN yum -y update; yum -y install systemd-libs docker; yum clean all

With:

RUN dnf -y update; dnf -y install systemd-libs docker; dnf clean all

To avoid ugly warnings.

Chapter 11 – Summary

As a rule, keep an eye on the amount of disk space available in the /var/log/dockerdirectory,

Should be:

As a rule, keep an eye on the amount of disk space available in the /var/lib/dockerdirectory,

Chapter 16 – Step 2: Set Up Kubernetes Master

The

KUBE_API_ADDRESS="--address=0.0.0.0" 

configuration line for our master server should be

KUBE_API_ADDRESS="--insecure-bind-address=0.0.0.0" as --address

has been deprecated.

The

KUBE_SERVICE_ADDRESSES="--portal_net=10.254.0.0/16" 

configuration line should be 

KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16"

Cheers!

atomic

Service definition to run Cockpit on system startup for CentOS Atomic SIG

I’m working these days with Project Atomic. You should have a look to the awesome Quickstart guide.

I’ve chosen to use Vagrant with the CentOS Atomic SIG so playing with Project Atomic is really easy (change to virtualbox if using that provider :D)

vagrant init centos/atomic-host; vagrant up –provider libvirt

One of the first things I’ve tested is Cockpit’s web server manager. It’s pretty cool and easy to install following the guide.

Once inside the Project Atomic host, Cockpit’s container is intalled with the following command:

vagrant ssh
sudo atomic run cockpit/ws

Remember, I use this blog so I don’t forget my notes. I’m just sharing with you the service definition needed to run Cockpit on system startup when working with CentOS Atomic SIG and not Fedora’s version which is explained in the source for this post. This file must be placed at /etc/systemd/system/cockpitws.service

[Unit]
Description=Cockpit Web Interface
Requires=docker.service
After=docker.service

[Service]
Restart=on-failure
RestartSec=10
ExecStart=/usr/bin/docker run --rm --privileged --pid host -v /:/host --name %p cockpit/ws /container/atomic-run --local-ssh
ExecStop=-/usr/bin/docker stop -t 2 %p

[Install]
WantedBy=multi-user.target

Then just enable and start the service and the Cockpit container will run and be ready to serve at 9090 port (user vagrant/vagrant or root/vagrant).

sudo systemctl daemon-reload
sudo systemctl enable cockpitws.service
sudo systemctl start cockpitws.service

cockpit_inicio

Cool stuff Project Atomic and Cockpit.

 

 

docker

INSTALLING THE LATEST STABLE RELEASE FOR DOCKER COMPOSE ON LINUX X86-64

Following the last post on how to install the latest stable binary for Docker Machine, If you want to install the Docker Machine client on your 64 bits Linux machine you only have to follow the official instructions and download it from GitHub.

However, if you want to download the file tagged as the latest stable release right away without checking on GitHub what is the new release, this is the one line command I’m using thanks to the tag added by the Docker team:


curl -L https://github.com/docker/compose/releases/download/`curl -s -L https://github.com/docker/compose/releases/latest| grep Linux-x86_64 | grep href | cut -f 6 -d '/'`/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose && chmod +x /usr/local/bin/docker-compose

Then you can check that docker-machine is ready if you have /usr/local/bin in your PATH, e.g I’m running the 1.7.1 version:

docker-compose version
docker-compose version 1.7.1, build 0a9ab35

Cheers!

docker

Installing the latest stable release for Docker Machine on Linux x86-64

I’ve been playing with Docker for a while and I just love it so I’m starting to share some notes that may be useful to you. If you want to install the Docker Machine client on your 64 bits Linux machine you only have to follow the official instructions and download it from GitHub.

However, I’m lazy so if you want to download the file tagged as the latest stable release right away without checking on GitHub what is the new release, this is the one line command I’m using thanks to the tag added by the Docker team:


curl -L https://github.com/docker/machine/releases/download/`curl -s -L https://github.com/docker/machine/releases/latest| grep Linux-x86_64 | grep href | cut -f 6 -d '/'`/docker-machine-`uname -s`-`uname -m` > /usr/local/bin/docker-machine && chmod +x /usr/local/bin/docker-machine

Then you can check that docker-machine is ready if you have /usr/local/bin in your PATH, e.g I’m running the 0.7.0 version:

docker-machine version
docker-machine version 0.7.0, build a650a40

Cheers!