Re: Questions about Exofs

[Boaz Harrosh <bharrosh@panasas.com>]

On 05/15/2012 07:21 PM, Idan Kedar wrote:

>>> By the way, several weeks I have tried setting up a RAID 5 environment
>>> >> with 8 OSDs, 1 mirror and RAID nesting. I then tried cloning and
>>> >> compiling the kernel tree over this pNFS-OSD-RAID. The result was that
>>> >> otgtd died, and I don't know why. It didn't dump core anywhere I could
>>> >> find and the only "log" it has - stdout - didn't give any useful info.
>>> >> I was going to inquire about this in a couple of weeks when I need to
>>> >> get this environment working, but since this issue came up, maybe we
>>> >> can somehow resolve it sooner.
>> >
>> >
>> > 8 OSDs with a mirror ? what was the mkfs.exofs command line you used?
> Something along the lines of
> # LD_LIBRARY_PATH=lib ./usr/mkfs.exofs --pid=0x10000 --format
> --mirrors=1 --group_width=2 --group_depth=2 --dev=/dev/osd0
> --osdname=$(uuid) --dev=/dev/osd1 --osdname=$(uuid) --dev=/dev/osd2
> --osdname=$(uuid) ...
> 
> I don't remember exactly at the moment, but I will bump this thread
> when I'll start using RAID again.
>> >


BTW I don't see a --raid=5 above but a raid5 of --group_width=2 is just
a mirror. The minimum for --raid=5 is 3. (I'm not sure if
the system checks and optimizes for this)

So the above should probably be:

# LD_LIBRARY_PATH=lib ./usr/mkfs.exofs --pid=0x10000 	\
  --mirrors=2 --group_width=2 --group_depth=1000 	\
  --dev=/dev/osd0 --format --osdname=$(uuid) 		\
  --dev=/dev/osd1 --format --osdname=$(uuid)		\
  --dev=/dev/osd2 --format --osdname=$(uuid)		\
  ...

group_depth is how many times to repeat this group until
advancing to the next group. 

The group_count is:
	group_count = int( (total_devices / mirrors+1) / group_width )

an interesting raid5 configuration with 8 devices mirrors and groups
can be:

# LD_LIBRARY_PATH=lib ./usr/mkfs.exofs --pid=0x10000 	\
  --mirrors=1 --raid=5 --group_width=3 --group_depth=1000 	\
  --dev=/dev/osd0 --format --osdname=$(uuid) 		\
  --dev=/dev/osd1 --format --osdname=$(uuid)		\
  --dev=/dev/osd2 --format --osdname=$(uuid)		\

Which means each file will have only a single group  - 3-out-of-4.
but the next file will use another set of 3 devices out of the 4.

And I think you need a --stripe_pages= right?

Cheers
Boaz