Re: [meta-virtualization] Kubernetes & Docker on Xilinx Versal Yocto

["Robert Berger" <robert.berger.yocto.user@gmail.com>]

Hi,

My comments are inline

On 10/08/2021 18:06, Ramon Fried wrote:
> Hi all,
> I'm working on the bring-up of Kubernetes worker node on Xilinx
> Versal. (ARM A72)
> The storage media (NOR flash) is too small, and I'm running all the
> filesystems as tmpfs, which I load at boot time.
> 
> I'm experiencing a lot of issues bringing up Docker and Kubernetes
> together this way, it seems that running these from tmpfs is not that
> trivial.

I am not sure you can run an overlayfs over a tmpfs - never tried that.

> The overlayfs causes real weird issues where suddenly the proc fs is
> unmounted, symbolic links that stop working, etc.
> 
> Did anyone succeed running Kubernetes with Docker using the overlayfs2 plugin ?
> I saw on the Docker manual, that overlayfs2 only supports ext4, I'm
> wondering if I'm on the right track.

I typically use an SD card (big enough) and dedicate a btrfs partition 
for containers and/or kubernetes.

e.g. /dev/mmcblk0p3 on /var/lib/rancher type btrfs 
(rw,noatime,ssd,space_cache,subvolid=5,subvol=/)

This seems to work fine and as a nice side effect I can use btrfs 
subvolumes for snapshots.

kubernetes writes to a subvolume /var/lib/rancher/k3s and I can do 
snapshots e.g. to /var/lib/rancher/snapshot/k3s/xxx

root@k3s-node-1:/var/lib/rancher# btrfs subvolume list /var/lib/rancher
ID 257 gen 16 top level 5 path k3s
ID 258 gen 22 top level 5 path snapshot
ID 260 gen 21 top level 258 path snapshot/k3s/2021-08-10-11:25:17
ID 261 gen 21 top level 258 path snapshot/k3s/2021-08-10-11:28:00

> 
> Would love to share notes.
> Thanks,
> Ramon.

Regards,

Robert

> 
> 
> 
> 
>