Re: Object size

[Zenon Panoussis <oracle@provocation.net>]

On 04/28/2011 10:02 PM, Gregory Farnum wrote:

[various explanations]

Thanks Greg, that's very helpful towards graspings ceph's workings. I'll
put it in the wiki.

> The relation between these reports and your data can be a bit fuzzy, 
> though. When looking at the disk space used the OSD is just relying on 
> a df for the mount it's on -- if it's sharing that mount with anything 
> else (eg, the node OS) then it's not distinguishing between OSD data, 
> and data on the disk. Something like that must be going on if you've 
> got a 4.4x ratio. (An example is below. [1]) Based on what you're giving 
> us here:

> 1) You have 9791 MB of data in the filesystem.
> 2) You have (12222MB - 9791 MB=) 2431MB of metadata maintaining the Ceph tree.
> 3) RADOS is using 24444MB of disk space amongst all your OSDs to store this.
> 4) Your nodes have other stuff installed to the tune of (29135MB/2=)14567MB or (29135/3=)9711MB per OSD.

1 and 3 are correct. 2 is presumably correct; it makes perfect sense and
there's no reason to question it. 4 is not correct:

# df -m
[...]
/dev/mapper/sda6        232003     26913    191832  13% /mnt/osd

# grep /mnt/osd /etc/ceph/ceph.conf
	osd data = /mnt/osd

# ls -a /mnt/osd/
.  ..  ceph_fsid  current  fsid  lost+found  magic  whoami

So the OSD lives in its own exclusive partition and nothing else uses that
partition. The other node is done the same way. The "53579 MB used" reported
by ceph matches the aggregated "Used" output of df -m on both nodes. And
I checked, lost+found is empty on both. Something here is trying to be elusive
(and is succeeding).

Z