Re: No files in snapshot

[Thomas Schneider <74cmonty@gmail.com>]

Hi,

I would agree with Oliver's conclusion:
the relevant directories are all tmpfs mounts.

root@ld5505:~# mount | grep /var
/dev/sdbq3 on /var/cache type btrfs 
(rw,noatime,compress=lzo,ssd,space_cache=v2,subvolid=260,subvol=/@cache)
/dev/sdbq3 on /var/lib/vz/images type btrfs 
(rw,noatime,compress=lzo,ssd,space_cache=v2,subvolid=262,subvol=/@images)
lxcfs on /var/lib/lxcfs type fuse.lxcfs 
(rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
tmpfs on /var/lib/ceph/osd/ceph-122 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-123 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-105 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-92 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-112 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-77 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-76 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-87 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-119 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-79 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-115 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-98 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-100 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-89 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-108 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-120 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-82 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-118 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-104 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-95 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-1 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-96 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-117 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-116 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-99 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-106 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-110 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-97 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-81 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-121 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-88 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-0 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-94 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-113 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-107 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-101 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-78 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-93 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-85 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-103 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-102 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-109 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-114 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-80 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-111 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-83 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-86 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-91 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-84 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-90 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-8 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-9 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-10 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-11 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-12 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-13 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-14 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-15 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-16 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-17 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-18 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-19 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-20 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-21 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-22 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-23 type tmpfs (rw,relatime)
tmpfs on /var/lib/ceph/osd/ceph-24 type tmpfs (rw,relatime)


Am 04.09.2019 um 22:52 schrieb Oliver Freyermuth:
> Am 04.09.19 um 21:07 schrieb Chris Murphy:
>> On Wed, Sep 4, 2019 at 12:24 PM Remi Gauvin <remi@georgianit.com> wrote:
>>> On 2019-09-04 1:36 p.m., Chris Murphy wrote:
>>>
>>>> I don't really know how snapper works.
>>>>
>>>> The way 'btrfs subvolume snapshot' works,  you must point it to a
>>>> subvolume. It won't snapshot a regular directory and from what you
>>>> posted above, there are no subvolumes in /var or /var/lib which means
>>>> trying to snapshot /var/lib/ceph/osd/ceph-....  would fail. So maybe
>>>> it's failing but snapper doesn't show the error. I'm not really sure.
>>>>
>>> In this case, his snapshots are all of the root.
>>>
>>> I don't know how Ceph works, but since we already confirmed that there
>>> are no subvolumes under /var, the only other explanation is that
>>> /var/lib/ceph/osd/ceph-<n> is a submount
>>>
>>> What is the the result of running:
>>> mount | grep /var
>>>
>> Yep.
>>
>>
> Looking at Thomas' mail on the Ceph-users list:
> http://lists.ceph.com/pipermail/ceph-users-ceph.com/2019-August/036679.html
> I deduce he is using Ceph with the Bluestore backend, which indeed means that /var/lib/ceph/osd/ceph.<n>
> are tmpfs mounts which are completeley ephemeral and are created from LVM metadata of the actual Ceph OSD disks.
>
> That would of course also explain why they are not part of any btrfs snapshot of / (and that also means there is no need to backup anything).