* Scalability of MD raid 1 mirror devices [not found] <98ad0e44-1ac1-4451-a3d3-0a48f3a1d9e7@default> @ 2015-10-12 12:03 ` Suresh Babu Kandukuru 2015-10-15 9:04 ` Ankur Bose 0 siblings, 1 reply; 4+ messages in thread From: Suresh Babu Kandukuru @ 2015-10-12 12:03 UTC (permalink / raw) To: linux-raid, neilb Dear Group, We are doing scalability of MD raid 1 mirror devices on the Linux host running the 3.17.2 . we see the number of RAID 1 devices limited to 128 in one case and 511 in another case . we would like to know why this limitation ?. Appreciate any kind of inputs and pointers We can create the RAID 1 device in 3 ways. 1. /dev/mdX -> here X is the number, we can specify from 0 to 511. 2. /dev/md/X -> here also X is a number from 0 to 511. It creates it as a link to the actual device /dev/mdX. ( similar to above but creates a link also). 3. /dev/md/”name” -> This creates a link to actual device, whichever is free starting from 127 to 0. Below is the function which is responsible for it. char *find_free_devnm(int use_partitions) { static char devnm[32]; int devnum; for (devnum = 127; devnum != 128; devnum = devnum ? devnum-1 : (1<<20)-1) { if (use_partitions) sprintf(devnm, "md_d%d", devnum); else sprintf(devnm, "md%d", devnum); if (mddev_busy(devnm)) continue; if (!conf_name_is_free(devnm)) continue; if (!use_udev()) { /* make sure it is new to /dev too, at least as a * non-standard */ int devid = devnm2devid(devnm); if (devid) { char *dn = map_dev(major(devid), minor(devid), 0); if (dn && ! is_standard(dn, NULL)) continue; } } break; } if (devnum == 128) return NULL; return devnm; } So ideally we should not create a device which is more than 128 { The program may crash }. Then we tried to find how we are able to create up to 511 and why it is failing after that. int dev_open(char *dev, int flags) inside this function fd = open(dev, flags); / this line is assigning fd to -1 , which is causing the program to fail. So I wrote a simple program to crosscheck it. int main(){ char devname[32] = "/dev/hello1"; // The flags I have set according to the code. int flags = O_RDWR; flags |= O_DIRECT; if (mknod(devname, S_IFBLK|0600, makedev(9,511)) == 0) { int fd = open(devname, flags); cout<<fd<<endl; unlink(devname); } } So if the minor number is more than 511, the “fd” is assigned to -1, if it is in the range of 0 to 511 It is working fine. So should we go with the range of 511 or stick to 128 ? Thanks /Suresh -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 4+ messages in thread
* RE: Scalability of MD raid 1 mirror devices 2015-10-12 12:03 ` Scalability of MD raid 1 mirror devices Suresh Babu Kandukuru @ 2015-10-15 9:04 ` Ankur Bose 2015-10-16 1:11 ` Neil Brown 0 siblings, 1 reply; 4+ messages in thread From: Ankur Bose @ 2015-10-15 9:04 UTC (permalink / raw) To: linux-raid, neilb; +Cc: Suresh Babu Kandukuru Hi, it seems like this is very active group , I don't see any reason why anyone is not replying to the below mail. Kindly reply we are in middle of something. Thanks, ankur. -----Original Message----- From: Suresh Babu Kandukuru Sent: 12 October 2015 17:34 To: linux-raid@vger.kernel.org; neilb@suse.de Subject: Scalability of MD raid 1 mirror devices Dear Group, We are doing scalability of MD raid 1 mirror devices on the Linux host running the 3.17.2 . we see the number of RAID 1 devices limited to 128 in one case and 511 in another case . we would like to know why this limitation ?. Appreciate any kind of inputs and pointers We can create the RAID 1 device in 3 ways. 1. /dev/mdX -> here X is the number, we can specify from 0 to 511. 2. /dev/md/X -> here also X is a number from 0 to 511. It creates it as a link to the actual device /dev/mdX. ( similar to above but creates a link also). 3. /dev/md/”name” -> This creates a link to actual device, whichever is free starting from 127 to 0. Below is the function which is responsible for it. char *find_free_devnm(int use_partitions) { static char devnm[32]; int devnum; for (devnum = 127; devnum != 128; devnum = devnum ? devnum-1 : (1<<20)-1) { if (use_partitions) sprintf(devnm, "md_d%d", devnum); else sprintf(devnm, "md%d", devnum); if (mddev_busy(devnm)) continue; if (!conf_name_is_free(devnm)) continue; if (!use_udev()) { /* make sure it is new to /dev too, at least as a * non-standard */ int devid = devnm2devid(devnm); if (devid) { char *dn = map_dev(major(devid), minor(devid), 0); if (dn && ! is_standard(dn, NULL)) continue; } } break; } if (devnum == 128) return NULL; return devnm; } So ideally we should not create a device which is more than 128 { The program may crash }. Then we tried to find how we are able to create up to 511 and why it is failing after that. int dev_open(char *dev, int flags) inside this function fd = open(dev, flags); / this line is assigning fd to -1 , which is causing the program to fail. So I wrote a simple program to crosscheck it. int main(){ char devname[32] = "/dev/hello1"; // The flags I have set according to the code. int flags = O_RDWR; flags |= O_DIRECT; if (mknod(devname, S_IFBLK|0600, makedev(9,511)) == 0) { int fd = open(devname, flags); cout<<fd<<endl; unlink(devname); } } So if the minor number is more than 511, the “fd” is assigned to -1, if it is in the range of 0 to 511 It is working fine. So should we go with the range of 511 or stick to 128 ? Thanks /Suresh -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 4+ messages in thread
* RE: Scalability of MD raid 1 mirror devices 2015-10-15 9:04 ` Ankur Bose @ 2015-10-16 1:11 ` Neil Brown 2015-10-16 2:29 ` Neil Brown 0 siblings, 1 reply; 4+ messages in thread From: Neil Brown @ 2015-10-16 1:11 UTC (permalink / raw) To: Ankur Bose, linux-raid; +Cc: Suresh Babu Kandukuru [-- Attachment #1: Type: text/plain, Size: 4307 bytes --] Ankur Bose <ankur.bose@oracle.com> writes: > Hi, it seems like this is very active group , I don't see any reason why anyone is not replying to the below mail. > > > Kindly reply we are in middle of something. We are all in the middle of something. > > Thanks, > ankur. > -----Original Message----- > From: Suresh Babu Kandukuru > Sent: 12 October 2015 17:34 > To: linux-raid@vger.kernel.org; neilb@suse.de > Subject: Scalability of MD raid 1 mirror devices > > > Dear Group, > > We are doing scalability of MD raid 1 mirror devices on the Linux host running the 3.17.2 . we see the number of RAID 1 devices limited to 128 in one case and 511 in another case . we would like to know why this limitation ?. Appreciate any kind of inputs and pointers > > We can create the RAID 1 device in 3 ways. > > 1. /dev/mdX -> here X is the number, we can specify from 0 to 511. > 2. /dev/md/X -> here also X is a number from 0 to 511. It creates it as a link to the actual device /dev/mdX. ( similar to above but creates a link also). > 3. /dev/md/”name” -> This creates a link to actual device, whichever is free starting from 127 to 0. Below is the function which is responsible for it. > > char *find_free_devnm(int use_partitions) { > static char devnm[32]; > int devnum; > for (devnum = 127; devnum != 128; devnum = devnum ? devnum-1 : (1<<20)-1) { > > if (use_partitions) > sprintf(devnm, "md_d%d", devnum); > else > sprintf(devnm, "md%d", devnum); > if (mddev_busy(devnm)) > continue; > if (!conf_name_is_free(devnm)) > continue; > if (!use_udev()) { > /* make sure it is new to /dev too, at least as a > * non-standard */ > int devid = devnm2devid(devnm); > if (devid) { > char *dn = map_dev(major(devid), > minor(devid), 0); > if (dn && ! is_standard(dn, NULL)) > continue; > } > } > break; > } > if (devnum == 128) > return NULL; > return devnm; > } > > So ideally we should not create a device which is more than 128 { The > program may crash }. Please explain why you think the program would crash? > > Then we tried to find how we are able to create up to 511 and why it is failing after that. > > int dev_open(char *dev, int flags) > > inside this function > > fd = open(dev, flags); / this line is assigning fd to -1 , which is causing the program to fail. So I wrote a simple program to crosscheck it. > > int main(){ > > char devname[32] = "/dev/hello1"; > > // The flags I have set according to the code. > > int flags = O_RDWR; > flags |= O_DIRECT; > if (mknod(devname, S_IFBLK|0600, makedev(9,511)) == 0) { > > int fd = open(devname, flags); > cout<<fd<<endl; > unlink(devname); > > } > } > > So if the minor number is more than 511, the “fd” is assigned to -1, > if it is in the range of 0 to 511 It is working fine. Hmmm... you are right. Probably due to this: blk_register_region(MKDEV(MD_MAJOR, 0), 512, THIS_MODULE, md_probe, NULL, NULL); Try changing the "512" to "1<<MINORBITS". NeilBrown [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 818 bytes --] ^ permalink raw reply [flat|nested] 4+ messages in thread
* RE: Scalability of MD raid 1 mirror devices 2015-10-16 1:11 ` Neil Brown @ 2015-10-16 2:29 ` Neil Brown 0 siblings, 0 replies; 4+ messages in thread From: Neil Brown @ 2015-10-16 2:29 UTC (permalink / raw) To: Ankur Bose, linux-raid; +Cc: Suresh Babu Kandukuru Neil Brown <neilb@suse.de> writes: >> So if the minor number is more than 511, the “fd” is assigned to -1, >> if it is in the range of 0 to 511 It is working fine. > > Hmmm... you are right. > Probably due to this: > > blk_register_region(MKDEV(MD_MAJOR, 0), 512, THIS_MODULE, > md_probe, NULL, NULL); > > Try changing the "512" to "1<<MINORBITS". Actually don't change that. That is deliberate. add "CREATE named=yes" to /etc/mdadm.conf, then create arrays with names like /dev/md/whatever-you-like and you can create as many as you like. The arrays are create by effectively doing echo md_whatever-you-like > /sys/module/md_mod/parameters/new_array NeilBrown -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2015-10-16 2:29 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <98ad0e44-1ac1-4451-a3d3-0a48f3a1d9e7@default>
2015-10-12 12:03 ` Scalability of MD raid 1 mirror devices Suresh Babu Kandukuru
2015-10-15 9:04 ` Ankur Bose
2015-10-16 1:11 ` Neil Brown
2015-10-16 2:29 ` Neil Brown
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).