RE: Scalability of MD raid 1 mirror devices

[Neil Brown <neilb@suse.de>]

[-- Attachment #1: Type: text/plain, Size: 4307 bytes --]

Ankur Bose <ankur.bose@oracle.com> writes:

> Hi, it seems like this is very active group , I don't see any reason why anyone is not replying to the below mail.
>  
>
> Kindly reply we are in middle of something.

We are all in the middle of something.


>
> Thanks,
> ankur.
> -----Original Message-----
> From: Suresh Babu Kandukuru 
> Sent: 12 October 2015 17:34
> To: linux-raid@vger.kernel.org; neilb@suse.de
> Subject: Scalability of MD raid 1 mirror devices
>
>
> Dear Group,
>
> We are doing scalability of MD raid 1  mirror devices on the Linux host running the 3.17.2 .   we see the number of RAID 1 devices limited to 128 in one case and 511 in another case .  we would like to know why this limitation ?. Appreciate any kind of inputs and pointers 
>
> We can create the RAID 1 device in 3 ways.
>
> 1.  /dev/mdX   -> here X is the number, we can specify from 0 to 511.
> 2. /dev/md/X  -> here also X is a number from 0 to 511. It creates it as a link to the actual device /dev/mdX. ( similar to above but creates a link also).
> 3. /dev/md/”name”  -> This creates a link to actual device, whichever is free starting from 127 to 0. Below is the function which is responsible for it.
>
> char *find_free_devnm(int use_partitions) {
>         static char devnm[32];
>         int devnum;
>         for (devnum = 127; devnum != 128;  devnum = devnum ? devnum-1 : (1<<20)-1) {
>
>                 if (use_partitions)
>                         sprintf(devnm, "md_d%d", devnum);
>                 else
>                         sprintf(devnm, "md%d", devnum);
>                 if (mddev_busy(devnm))
>                         continue;
>                 if (!conf_name_is_free(devnm))
>                         continue;
>                 if (!use_udev()) {
>                         /* make sure it is new to /dev too, at least as a
>                          * non-standard */
>                         int devid = devnm2devid(devnm);
>                         if (devid) {
>                                 char *dn = map_dev(major(devid),
>                                                    minor(devid), 0);
>                                 if (dn && ! is_standard(dn, NULL))
>                                         continue;
>                         }
>                 }
>                 break;
>         }
>         if (devnum == 128)
>                 return NULL;
>         return devnm;
> }
>
> So ideally we should not create a device which is more than 128 { The
> program may crash }.

Please explain why you think the program would crash?


>
> Then we  tried to find how we are able to create up to 511 and why it is failing after that.
>
> int dev_open(char *dev, int flags)
>
> inside this function 
>
> fd = open(dev, flags);  / this line is assigning fd to -1 , which is causing the program to fail. So I wrote a simple program to crosscheck it.
>
> int main(){
>
>         char devname[32] = "/dev/hello1";
>
> // The flags I have set according to the code.
>
>         int flags = O_RDWR;
>         flags |= O_DIRECT;
>         if (mknod(devname, S_IFBLK|0600, makedev(9,511)) == 0) {
>
>                 int  fd = open(devname, flags);
>                 cout<<fd<<endl;
>                 unlink(devname);
>
>         }
> }
>
> So if the minor number is more than 511, the “fd” is assigned to -1,
> if it is in the range of 0 to 511 It is working fine.

Hmmm... you are right.
Probably due to this:

	blk_register_region(MKDEV(MD_MAJOR, 0), 512, THIS_MODULE,
			    md_probe, NULL, NULL);

Try changing the "512" to "1<<MINORBITS".


NeilBrown

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]