未加星标

How Docker uses cgroups to set resource limits?

字体大小 | |
[系统(linux) 所属分类 系统(linux) | 发布者 店小二05 | 时间 2019 | 作者 红领巾 ] 0人收藏点击收藏

Today, I was interested to know how does Docker uses cgroups to set resource limits. In this short post, I will share with you what I learnt.

I will assume that you have a machine on which Docker is installed.

Docker allows you to pass resource limits using the command-line options. Let’s assume that you want to limit the IO read rate to 1mb per second for a container. You can start a new container with the device-read-bps option as shown below

$ docker run -it --device-read-bps /dev/sda:1mb centos

In the above command, we are instantiating a new centos container. We specified device-read-bps option to limit the read rate to 1mb per second for /dev/sda device.

The above command will start the container and you will inside the container shell.

We will create a new file inside the container and then try to read the file. To create a file with random content, we will use dd utility as shown below.

[root@container-id ~]# dd if=/dev/zero of=afile bs=1M count=100

The above will create a file with 100MB size.

Now, let’s try to read afile file.

But before that, we will start the iotop utility on the docker host

$ iotop -o

To do that, we will again use dd utility as shown below.

[root@container-id ~]# dd if=/root/afile of=/dev/null

As you can see below in the iotop screenshot, the disk read speed was close to 1mb per second.


How Docker uses cgroups to set resource limits?

If you do the above in an unconstrained container, you will find that read speed is much higher.

Let’s start a new container without the limits

$ docker run -it centos

Now, again create a file as we did above. This time we will create a file of 5Gb size.

[root@container-id ~]# dd if=/dev/zero of=afile bs=1M count=5000

Next, we will read the file using dd command as we did previously. This time if you look at iotop , you will find that disk read speed is 591.89 Mb per second.


How Docker uses cgroups to set resource limits?
How does Docker uses cgroup?

Cgroup is a linux feature to limit, police, and account the resource usage for a set of processes. It provides mechanism to limit and monitor system resources like CPU time, system memory, disk bandwidth, network bandwidth, etc.

The cgroups works by dividing resources into groups and then assigning tasks to those groups.

Docker uses cgroups to limit the system resources.

When you install Docker binary on a linux box like ubuntu it will install cgroup related packages and create subsystem directories. You can list all the subsystems that you can manage using cgroups via the lscgroup command.

$ lscgroup cpuset:/ cpu:/ cpuacct:/ memory:/ devices:/ freezer:/ blkio:/ perf_event:/ hugetlb:/

If lscgroup is not installed, then you can install it using sudo apt-get install cgroup-bin command.

On Ubuntu, these corresponds to directories inside the /sys/fs/cgroup directory.

$ cd /sys/fs/cgroup/

Once inside the cgroup directory you can list its contents.

$ ls -l total 0 drwxr-xr-x 2 root root 0 Jan 3 14:50 blkio drwxr-xr-x 2 root root 0 Jan 3 14:50 cpu drwxr-xr-x 2 root root 0 Jan 3 14:50 cpuacct drwxr-xr-x 2 root root 0 Jan 3 14:50 cpuset drwxr-xr-x 2 root root 0 Jan 3 14:50 devices drwxr-xr-x 2 root root 0 Jan 3 14:50 freezer drwxr-xr-x 2 root root 0 Jan 3 14:50 hugetlb drwxr-xr-x 2 root root 0 Jan 3 14:50 memory drwxr-xr-x 2 root root 0 Jan 3 14:50 perf_event drwxr-xr-x 3 root root 0 Jan 3 14:45 systemd

The blkio directory is used to manage block devices. Similarly other directories are used to manage other system resources.

Let’s look inside the contents of blkio directory.

/sys/fs/cgroup/blkio$ ls -l total 0 -r--r--r-- 1 root root 0 Jan 3 14:50 blkio.io_merged -r--r--r-- 1 root root 0 Jan 3 14:50 blkio.io_merged_recursive -r--r--r-- 1 root root 0 Jan 3 14:50 blkio.io_queued -r--r--r-- 1 root root 0 Jan 3 14:50 blkio.io_queued_recursive -r--r--r-- 1 root root 0 Jan 3 14:50 blkio.io_service_bytes -r--r--r-- 1 root root 0 Jan 3 14:50 blkio.io_service_bytes_recursive -r--r--r-- 1 root root 0 Jan 3 14:50 blkio.io_service_time -r--r--r-- 1 root root 0 Jan 3 14:50 blkio.io_service_time_recursive -r--r--r-- 1 root root 0 Jan 3 14:50 blkio.io_serviced -r--r--r-- 1 root root 0 Jan 3 14:50 blkio.io_serviced_recursive -r--r--r-- 1 root root 0 Jan 3 14:50 blkio.io_wait_time -r--r--r-- 1 root root 0 Jan 3 14:50 blkio.io_wait_time_recursive -rw-r--r-- 1 root root 0 Jan 3 14:50 blkio.leaf_weight -rw-r--r-- 1 root root 0 Jan 3 14:50 blkio.leaf_weight_device --w------- 1 root root 0 Jan 3 14:50 blkio.reset_stats -r--r--r-- 1 root root 0 Jan 3 14:50 blkio.sectors -r--r--r-- 1 root root 0 Jan 3 14:50 blkio.sectors_recursive -r--r--r-- 1 root root 0 Jan 3 14:50 blkio.throttle.io_service_bytes -r--r--r-- 1 root root 0 Jan 3 14:50 blkio.throttle.io_serviced -rw-r--r-- 1 root root 0 Jan 3 14:50 blkio.throttle.read_bps_device -rw-r--r-- 1 root root 0 Jan 3 14:50 blkio.throttle.read_iops_device -rw-r--r-- 1 root root 0 Jan 3 14:50 blkio.throttle.write_bps_device -rw-r--r-- 1 root root 0 Jan 3 14:50 blkio.throttle.write_iops_device -r--r--r-- 1 root root 0 Jan 3 14:50 blkio.time -r--r--r-- 1 root root 0 Jan 3 14:50 blkio.time_recursive -rw-r--r-- 1 root root 0 Jan 3 14:50 blkio.weight -rw-r--r-- 1 root root 0 Jan 3 14:50 blkio.weight_device -rw-r--r-- 1 root root 0 Jan 3 14:50 cgroup.clone_children --w--w--w- 1 root root 0 Jan 3 14:50 cgroup.event_control -rw-r--r-- 1 root root 0 Jan 3 14:50 cgroup.procs -r--r--r-- 1 root root 0 Jan 3 14:50 cgroup.sane_behavior -rw-r--r-- 1 root root 0 Jan 3 14:50 notify_on_release -rw-r--r-- 1 root root 0 Jan 3 14:50 release_agent -rw-r--r-- 1 root root 0 Jan 3 14:50 tasks

The three important file from the above are:

tasks: This contains pids for the tasks attached to this control group cgroup.procs: This file contain thread group ids which is useful if you have multi threaded application. cgroup.event_control: This file is used to hook in to notification API.

When you run a new docker container using docker run command then docker will create a new child group under each of the sub systems. The name of the child group will be docker/container_id .

So, when you run a new container using the command shown below

$ docker run -it --device-read-bps /dev/sda:1mb centos

Then, directories will be created for the container. If you list contents of the directory blkio you will notice following

$ ls -l blkio/docker/26dc49635757074a2119039dc74634f72e9eddff41bee9dd8f761d73d3780a5c/ total 0 -r--r--r-- 1 root root 0 Jan 3 15:10 blkio.io_merged -r--r--r-- 1 root root 0 Jan 3 15:10 blkio.io_merged_recursive -r--r--r-- 1 root root 0 Jan 3 15:10 blkio.io_queued -r--r--r-- 1 root root 0 Jan 3 15:10 blkio.io_queued_recursive -r--r--r-- 1 root root 0 Jan 3 15:10 blkio.io_service_bytes -r--r--r-- 1 root root 0 Jan 3 15:10 blkio.io_service_bytes_recursive -r--r--r-- 1 root root 0 Jan 3 15:10 blkio.io_service_time -r--r--r-- 1 root root 0 Jan 3 15:10 blkio.io_service_time_recursive -r--r--r-- 1 root root 0 Jan 3 15:10 blkio.io_serviced -r--r--r-- 1 root root 0 Jan 3 15:10 blkio.io_serviced_recursive -r--r--r-- 1 root root 0 Jan 3 15:10 blkio.io_wait_time -r--r--r-- 1 root root 0 Jan 3 15:10 blkio.io_wait_time_recursive -rw-r--r-- 1 root root 0 Jan 3 15:10 blkio.leaf_weight -rw-r--r-- 1 root root 0 Jan 3 15:10 blkio.leaf_weight_device --w------- 1 root root 0 Jan 3 15:10 blkio.reset_stats -r--r--r-- 1 root root 0 Jan 3 15:10 blkio.sectors -r--r--r-- 1 root root 0 Jan 3 15:10 blkio.sectors_recursive -r--r--r-- 1 root root 0 Jan 3 15:10 blkio.throttle.io_service_bytes -r--r--r-- 1 root root 0 Jan 3 15:10 blkio.throttle.io_serviced -rw-r--r-- 1 root root 0 Jan 3 15:10 blkio.throttle.read_bps_device -rw-r--r-- 1 root root 0 Jan 3 15:10 blkio.throttle.read_iops_device -rw-r--r-- 1 root root 0 Jan 3 15:10 blkio.throttle.write_bps_device -rw-r--r-- 1 root root 0 Jan 3 15:10 blkio.throttle.write_iops_device -r--r--r-- 1 root root 0 Jan 3 15:10 blkio.time -r--r--r-- 1 root root 0 Jan 3 15:10 blkio.time_recursive -rw-r--r-- 1 root root 0 Jan 3 15:10 blkio.weight -rw-r--r-- 1 root root 0 Jan 3 15:10 blkio.weight_device -rw-r--r-- 1 root root 0 Jan 3 15:10 cgroup.clone_children --w--w--w- 1 root root 0 Jan 3 15:10 cgroup.event_control -rw-r--r-- 1 root root 0 Jan 3 15:10 cgroup.procs -rw-r--r-- 1 root root 0 Jan 3 15:10 notify_on_release -rw-r--r-- 1 root root 0 Jan 3 15:10 tasks

This has the same file structure as the blkio directory.

The two important things to note are:

If you cat the contents of the tasks

file then you will notice that it has the process id of the container.

:/sys/fs/cgroup/blkio/docker/26dc49635757074a2119039dc74634f72e9eddff41bee9dd8f761d73d3780a5c$ cat tasks 6347

This is the process id of the bash process running inside the container.

vagrant@vagrant-ubuntu-trusty-64:/sys/fs/cgroup/blkio/docker/26dc49635757074a2119039dc74634f72e9eddff41bee9dd8f761d73d3780a5c$ ps -ef|grep bash root 6347 6328 0 15:10 pts/0 00:00:00 /bin/bash There is an entry made to the blkio.throttle.read_bps_device with the read limit on the device. $ cat blkio.throttle.read_bps_device 8:0 1048576

The above shows how Docker uses Cgroup to define limits on different resources. The similar happen for other resources like CPU, memory, etc.

Conclusion

In this post, we learn how Docker uses Cgroups to set resource constraints. Docker provides the plumbing and tooling that make it easy for developer to consume advance linux features.

本文系统(linux)相关术语:linux系统 鸟哥的linux私房菜 linux命令大全 linux操作系统

代码区博客精选文章
分页:12
转载请注明
本文标题:How Docker uses cgroups to set resource limits?
本站链接:https://www.codesec.net/view/628494.html


1.凡CodeSecTeam转载的文章,均出自其它媒体或其他官网介绍,目的在于传递更多的信息,并不代表本站赞同其观点和其真实性负责;
2.转载的文章仅代表原创作者观点,与本站无关。其原创性以及文中陈述文字和内容未经本站证实,本站对该文以及其中全部或者部分内容、文字的真实性、完整性、及时性,不作出任何保证或承若;
3.如本站转载稿涉及版权等问题,请作者及时联系本站,我们会及时处理。
登录后可拥有收藏文章、关注作者等权限...
技术大类 技术大类 | 系统(linux) | 评论(0) | 阅读(150)