Document that tmpfs storage is charged to container memory usage

[antonionovelli]

Description

I am writing because I am wondering if I am wrongly interpretating the docker online documentation. Thus I’d like to understand
Here is a docker run call

Reproduce

docker run -it --rm --memory=5000m --memory-swap=5000m --mount type=tmpfs,destination=/app,tmpfs-size=25000m ubuntu:22.04 /bin/bash

or alternatively

sudo mount -o size=25G -t tmpfs none /mnt/tmpfs 

followed by

docker run -it --rm --memory=5000m --memory-swap=5000m  -v /mnt/tmpfs:/app ubuntu:22.04 /bin/bash

Inside the container, I can see the 25000m allocated to the /app mount point - this is the partial output of the df -h.

tmpfs                           25G     0   25G   0% /app

This is consistent with the documentation tmpfs mounts | Docker Docs

but In both cases (i.e. "-mount type=tmpfs,destination=/app,tmpfs-size=25000m" or "-v /mnt/tmpfs:/app") I am not able to write more than --memory limit

I tested this with sequential dd calls e.g

dd if=/dev/zero of=/app/zero_1.img bs=1G

dd if=/dev/zero of=/app/zero_2.img bs=1G

dd if=/dev/zero of=/apps/zero_3.img bs=1G

...

Expected behavior

Thus, I am not able to write more than 5000m on /app (I tested this with dd by writing more files). Apparently, this seems to be not consistent with documentation. Indeed neither the previous link nor Runtime options with Memory, CPUs, and GPUs | Docker Docs report any interaction among --memory --memory-swap and --mount type=tmpfs

Apparently, it seems that docker is not able to distinguish between the ram allocated (5000m ) and the tmpfs (independently on how the tempfs has been created.

And when the tempfs reaches the --memory --memory-swap limit, the container is killed (or I dd is killed)

But is this the expected behaviour?

If the answer is yes:

• why am I able to execute this kind of command " docker run -it --rm --memory=5000m --memory-swap=5000m --mount type=tmpfs,destination=/app,tmpfs-size=25000m ubuntu:22.04 /bin/bash" .In this case the expectation is that I cannot execute the command or at least I get a warning
• whereas I do not know what to expect in this second case "docker run -it --rm --memory=5000m --memory-swap=5000m -v /mnt/tmpfs:/app ubuntu:22.04 /bin/bash" since the tmpfs is independently generated outside the docker run command
• the documentation should warn the user about this

if the answer is not (i.e. the behaviour described is not expected), then the expectation is that the usage of the tmpfs should be independent of the --memory=5000m --memory-swap=5000m

docker version

Client:
 Version:           20.10.24+dfsg1
 API version:       1.41
 Go version:        go1.19.8
 Git commit:        297e128
 Built:             Thu May 18 08:38:34 2023
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server:
 Engine:
  Version:          20.10.24+dfsg1
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.19.8
  Git commit:       5d6db84
  Built:            Thu May 18 08:38:34 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.20~ds1
  GitCommit:        1.6.20~ds1-1+b1
 runc:
  Version:          1.1.5+ds1
  GitCommit:        1.1.5+ds1-1+deb12u1
 docker-init:
  Version:          0.19.0
  GitCommit:

docker info

Client:
 Context:    default
 Debug Mode: false

Server:
 Containers: 1
  Running: 0
  Paused: 0
  Stopped: 1
 Images: 90
 Server Version: 20.10.24+dfsg1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 1.6.20~ds1-1+b1
 runc version: 1.1.5+ds1-1+deb12u1
 init version: 
 Security Options:
  apparmor
  seccomp
   Profile: default
  cgroupns
 Kernel Version: 6.1.0-10-amd64
 Operating System: Debian GNU/Linux 12 (bookworm)
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 11.64GiB
 Name: debian
 ID: HAUY:ZNXD:HVDH:T2RY:ILGB:V3JT:ELJY:HOXR:4ATV:HZB3:6LIL:ATIK
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Additional Info

I have checked other opened issues but, apparently, I was not able to find a similar topic
I reported the same question here https://forums.docker.com/t/docker-mount-type-tmpfs-memory-usage/141924
Please let me know if you need further info
Kind regards
Antonio

[antonionovelli]

the docker info will show a different amount of memory since I tested this in different machines (one virtualized and one physical with less memory ... the docker info reports the memory of the physical one)

[corhere]

Yes, this is expected behaviour. tmpfs stores its contents in memory. That memory usage is charged towards the memory usage of the container, same as memory used by processes running in the container. This is all implemented in the Linux kernel: the tmpfs filesystem and cgroups. Docker is just the messenger; it does not enforce the memory limits itself.

[antonionovelli]

Dear @corhere thank you very much for the clarifications! It would be indeed very appreciated if more explicit documentation is made available since if this is expected, the fact that a user can execute that command might be a source of confusion.

Apart from the documentation, would be possible to include a warning message (at least in my first case where the tmpfs is instantiated within the docker run command)?

In the other case, the only source to avoid any error would be the documentation as already done for the conjunct usage of --memory and --memory-swap to finetune the access to the swap memory.

Kind regards,
Antonio