* [PATCH v2 0/9] fix max discard sectors limit @ 2013-04-19 16:40 Namjae Jeon 2013-04-20 19:50 ` James Bottomley 2013-04-22 21:23 ` Kent Overstreet 0 siblings, 2 replies; 6+ messages in thread From: Namjae Jeon @ 2013-04-19 16:40 UTC (permalink / raw) To: dwmw2, axboe, shli, Paul.Clements, npiggin, neilb, cjb, adrian.hunter, James.Bottomley, JBottomley Cc: nbd-general, Namjae Jeon, linux-scsi, Namjae Jeon, linux-mmc, linux-kernel, linux-raid, jcmvbkbc, linux-mtd From: Namjae Jeon <namjae.jeon@samsung.com> linux-v3.8-rc1 and later support for plug for blkdev_issue_discard with commit 0cfbcafcae8b7364b5fa96c2b26ccde7a3a296a9 (block: add plug for blkdev_issue_discard ) For example, 1) DISCARD rq-1 with size size 4GB 2) DISCARD rq-2 with size size 1GB If these 2 discard requests get merged, final request size will be 5GB. In this case, request's __data_len field may overflow as it can store max 4GB(unsigned int). This issue was observed while doing mkfs.f2fs on 5GB SD card: https://lkml.org/lkml/2013/4/1/292 # mkfs.f2fs /dev/mmcblk0p3 Info: sector size = 512 Info: total sectors = 11370496 (in 512bytes) Info: zone aligned segment0 blkaddr: 512 [ 257.789764] blk_update_request: bio idx 0 >= vcnt 0 mkfs process gets stuck in D state and I see the following in the dmesg: [ 257.789733] __end_that: dev mmcblk0: type=1, flags=122c8081 [ 257.789764] sector 4194304, nr/cnr 2981888/4294959104 [ 257.789764] bio df3840c0, biotail df3848c0, buffer (null), len 1526726656 [ 257.789764] blk_update_request: bio idx 0 >= vcnt 0 [ 257.794921] request botched: dev mmcblk0: type=1, flags=122c8081 [ 257.794921] sector 4194304, nr/cnr 2981888/4294959104 [ 257.794921] bio df3840c0, biotail df3848c0, buffer (null), len 1526726656 Few drivers(e.g. mmc, mtd..) set q->limits.max_discard_sectors more than UINT_MAX >> 9 sectors which is incorrect and it may lead to overflow of request's __data_len field if merged discard request's size exceeds 4GB. This patchset fixes this issue by updating helper function blk_queue_max_discard_sectors which is used to set max_discard_sectors limit. This patchset also replaces "q->limits.max_discard_sector = max_discard_sectors" with blk_queue_max_discard_sectors call in other drivers like mmc, mtd etc. Namjae Jeon (9): block: fix max discard sectors limit mmc: fix max_discard_sectors sd: use generic helper to set max_discard_sectors mtd: use generic helper to set max_discard_sectors loop: use generic helper to set max_discard_sectors nbd: use generic helper to set max_discard_sectors brd: use generic helper to set max_discard_sectors dm thin: use generic helper to set max_discard_sectors bcache: use generic helper to set max_discard_sectors -- 1.7.9.5 ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v2 0/9] fix max discard sectors limit 2013-04-19 16:40 [PATCH v2 0/9] fix max discard sectors limit Namjae Jeon @ 2013-04-20 19:50 ` James Bottomley 2013-04-21 1:37 ` Namjae Jeon 2013-04-26 7:33 ` Hannes Reinecke 2013-04-22 21:23 ` Kent Overstreet 1 sibling, 2 replies; 6+ messages in thread From: James Bottomley @ 2013-04-20 19:50 UTC (permalink / raw) To: Namjae Jeon Cc: axboe, nbd-general, npiggin, linux-scsi, neilb, dwmw2, Namjae Jeon, Paul.Clements, adrian.hunter, linux-kernel, linux-raid, jcmvbkbc, linux-mtd, linux-mmc, cjb, shli On Sat, 2013-04-20 at 01:40 +0900, Namjae Jeon wrote: > From: Namjae Jeon <namjae.jeon@samsung.com> > > linux-v3.8-rc1 and later support for plug for blkdev_issue_discard with > commit 0cfbcafcae8b7364b5fa96c2b26ccde7a3a296a9 > (block: add plug for blkdev_issue_discard ) > > For example, > 1) DISCARD rq-1 with size size 4GB > 2) DISCARD rq-2 with size size 1GB > > If these 2 discard requests get merged, final request size will be 5GB. > > In this case, request's __data_len field may overflow as it can store > max 4GB(unsigned int). > > This issue was observed while doing mkfs.f2fs on 5GB SD card: > https://lkml.org/lkml/2013/4/1/292 > > # mkfs.f2fs /dev/mmcblk0p3 > Info: sector size = 512 > Info: total sectors = 11370496 (in 512bytes) > Info: zone aligned segment0 blkaddr: 512 > [ 257.789764] blk_update_request: bio idx 0 >= vcnt 0 > > mkfs process gets stuck in D state and I see the following in the dmesg: > > [ 257.789733] __end_that: dev mmcblk0: type=1, flags=122c8081 > [ 257.789764] sector 4194304, nr/cnr 2981888/4294959104 > [ 257.789764] bio df3840c0, biotail df3848c0, buffer (null), len 1526726656 > [ 257.789764] blk_update_request: bio idx 0 >= vcnt 0 > [ 257.794921] request botched: dev mmcblk0: type=1, flags=122c8081 > [ 257.794921] sector 4194304, nr/cnr 2981888/4294959104 > [ 257.794921] bio df3840c0, biotail df3848c0, buffer (null), len 1526726656 > > Few drivers(e.g. mmc, mtd..) set q->limits.max_discard_sectors > more than UINT_MAX >> 9 sectors which is incorrect and it may lead to overflow > of request's __data_len field if merged discard request's size exceeds 4GB. > > This patchset fixes this issue by updating helper function > blk_queue_max_discard_sectors which is used to set max_discard_sectors limit. > > This patchset also replaces "q->limits.max_discard_sector = max_discard_sectors" > with blk_queue_max_discard_sectors call in other drivers like mmc, mtd etc. I really don't understand this explanation. How can you be affected by the incorrect setting of q->limits.max_discard sectors when n the blkdev_issue_discard() code you see: max_discard_sectors = min(q->limits.max_discard_sectors, UINT_MAX >> 9); ? The problem is not that we issue discards bigger than __data_len can allow, the problem is that we merge them larger than __data_len will allow. That means the merge code needs fixing to pay attention to max_discard_sectors, so isn't this the correct fix: James --- diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 78feda9..33f358f 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -838,7 +838,7 @@ static inline unsigned int blk_queue_get_max_sectors(struct request_queue *q, unsigned int cmd_flags) { if (unlikely(cmd_flags & REQ_DISCARD)) - return q->limits.max_discard_sectors; + return min(q->limits.max_discard_sectors, UINT_MAX >> 9); if (unlikely(cmd_flags & REQ_WRITE_SAME)) return q->limits.max_write_same_sectors; ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH v2 0/9] fix max discard sectors limit 2013-04-20 19:50 ` James Bottomley @ 2013-04-21 1:37 ` Namjae Jeon 2013-04-26 7:33 ` Hannes Reinecke 1 sibling, 0 replies; 6+ messages in thread From: Namjae Jeon @ 2013-04-21 1:37 UTC (permalink / raw) To: James Bottomley Cc: axboe, nbd-general, npiggin, linux-scsi, neilb, dwmw2, Namjae Jeon, Paul.Clements, adrian.hunter, linux-kernel, linux-raid, jcmvbkbc, linux-mtd, linux-mmc, cjb, shli 2013/4/21 James Bottomley <James.Bottomley@hansenpartnership.com>: > On Sat, 2013-04-20 at 01:40 +0900, Namjae Jeon wrote: >> From: Namjae Jeon <namjae.jeon@samsung.com> >> >> linux-v3.8-rc1 and later support for plug for blkdev_issue_discard with >> commit 0cfbcafcae8b7364b5fa96c2b26ccde7a3a296a9 >> (block: add plug for blkdev_issue_discard ) >> >> For example, >> 1) DISCARD rq-1 with size size 4GB >> 2) DISCARD rq-2 with size size 1GB >> >> If these 2 discard requests get merged, final request size will be 5GB. >> >> In this case, request's __data_len field may overflow as it can store >> max 4GB(unsigned int). >> >> This issue was observed while doing mkfs.f2fs on 5GB SD card: >> https://lkml.org/lkml/2013/4/1/292 >> >> # mkfs.f2fs /dev/mmcblk0p3 >> Info: sector size = 512 >> Info: total sectors = 11370496 (in 512bytes) >> Info: zone aligned segment0 blkaddr: 512 >> [ 257.789764] blk_update_request: bio idx 0 >= vcnt 0 >> >> mkfs process gets stuck in D state and I see the following in the dmesg: >> >> [ 257.789733] __end_that: dev mmcblk0: type=1, flags=122c8081 >> [ 257.789764] sector 4194304, nr/cnr 2981888/4294959104 >> [ 257.789764] bio df3840c0, biotail df3848c0, buffer (null), len 1526726656 >> [ 257.789764] blk_update_request: bio idx 0 >= vcnt 0 >> [ 257.794921] request botched: dev mmcblk0: type=1, flags=122c8081 >> [ 257.794921] sector 4194304, nr/cnr 2981888/4294959104 >> [ 257.794921] bio df3840c0, biotail df3848c0, buffer (null), len 1526726656 >> >> Few drivers(e.g. mmc, mtd..) set q->limits.max_discard_sectors >> more than UINT_MAX >> 9 sectors which is incorrect and it may lead to overflow >> of request's __data_len field if merged discard request's size exceeds 4GB. >> >> This patchset fixes this issue by updating helper function >> blk_queue_max_discard_sectors which is used to set max_discard_sectors limit. >> >> This patchset also replaces "q->limits.max_discard_sector = max_discard_sectors" >> with blk_queue_max_discard_sectors call in other drivers like mmc, mtd etc. > Hi. James. > I really don't understand this explanation. How can you be affected by > the incorrect setting of q->limits.max_discard sectors when n the > blkdev_issue_discard() code you see: > > max_discard_sectors = min(q->limits.max_discard_sectors, UINT_MAX >> > 9); > > ? > > The problem is not that we issue discards bigger than __data_len can > allow, the problem is that we merge them larger than __data_len will > allow. That means the merge code needs fixing to pay attention to > max_discard_sectors, so isn't this the correct fix: Yes, I agree. And the below patch looks good to fix this issue. Thanks for your comment. > > James > > --- > > diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h > index 78feda9..33f358f 100644 > --- a/include/linux/blkdev.h > +++ b/include/linux/blkdev.h > @@ -838,7 +838,7 @@ static inline unsigned int blk_queue_get_max_sectors(struct request_queue *q, > unsigned int cmd_flags) > { > if (unlikely(cmd_flags & REQ_DISCARD)) > - return q->limits.max_discard_sectors; > + return min(q->limits.max_discard_sectors, UINT_MAX >> 9); > > if (unlikely(cmd_flags & REQ_WRITE_SAME)) > return q->limits.max_write_same_sectors; > > ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v2 0/9] fix max discard sectors limit 2013-04-20 19:50 ` James Bottomley 2013-04-21 1:37 ` Namjae Jeon @ 2013-04-26 7:33 ` Hannes Reinecke 1 sibling, 0 replies; 6+ messages in thread From: Hannes Reinecke @ 2013-04-26 7:33 UTC (permalink / raw) To: James Bottomley Cc: axboe, nbd-general, Namjae Jeon, npiggin, linux-scsi, neilb, dwmw2, Namjae Jeon, Paul.Clements, adrian.hunter, linux-kernel, linux-raid, jcmvbkbc, linux-mtd, linux-mmc, cjb, shli On 04/20/2013 09:50 PM, James Bottomley wrote: > On Sat, 2013-04-20 at 01:40 +0900, Namjae Jeon wrote: >> From: Namjae Jeon <namjae.jeon@samsung.com> >> >> linux-v3.8-rc1 and later support for plug for blkdev_issue_discard with >> commit 0cfbcafcae8b7364b5fa96c2b26ccde7a3a296a9 >> (block: add plug for blkdev_issue_discard ) >> >> For example, >> 1) DISCARD rq-1 with size size 4GB >> 2) DISCARD rq-2 with size size 1GB >> >> If these 2 discard requests get merged, final request size will be 5GB. >> >> In this case, request's __data_len field may overflow as it can store >> max 4GB(unsigned int). >> >> This issue was observed while doing mkfs.f2fs on 5GB SD card: >> https://lkml.org/lkml/2013/4/1/292 >> >> # mkfs.f2fs /dev/mmcblk0p3 >> Info: sector size = 512 >> Info: total sectors = 11370496 (in 512bytes) >> Info: zone aligned segment0 blkaddr: 512 >> [ 257.789764] blk_update_request: bio idx 0 >= vcnt 0 >> >> mkfs process gets stuck in D state and I see the following in the dmesg: >> >> [ 257.789733] __end_that: dev mmcblk0: type=1, flags=122c8081 >> [ 257.789764] sector 4194304, nr/cnr 2981888/4294959104 >> [ 257.789764] bio df3840c0, biotail df3848c0, buffer (null), len 1526726656 >> [ 257.789764] blk_update_request: bio idx 0 >= vcnt 0 >> [ 257.794921] request botched: dev mmcblk0: type=1, flags=122c8081 >> [ 257.794921] sector 4194304, nr/cnr 2981888/4294959104 >> [ 257.794921] bio df3840c0, biotail df3848c0, buffer (null), len 1526726656 >> >> Few drivers(e.g. mmc, mtd..) set q->limits.max_discard_sectors >> more than UINT_MAX >> 9 sectors which is incorrect and it may lead to overflow >> of request's __data_len field if merged discard request's size exceeds 4GB. >> >> This patchset fixes this issue by updating helper function >> blk_queue_max_discard_sectors which is used to set max_discard_sectors limit. >> >> This patchset also replaces "q->limits.max_discard_sector = max_discard_sectors" >> with blk_queue_max_discard_sectors call in other drivers like mmc, mtd etc. > > I really don't understand this explanation. How can you be affected by > the incorrect setting of q->limits.max_discard sectors when n the > blkdev_issue_discard() code you see: > > max_discard_sectors = min(q->limits.max_discard_sectors, UINT_MAX >> > 9); > > ? > > The problem is not that we issue discards bigger than __data_len can > allow, the problem is that we merge them larger than __data_len will > allow. That means the merge code needs fixing to pay attention to > max_discard_sectors, so isn't this the correct fix: > > James > > --- > > diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h > index 78feda9..33f358f 100644 > --- a/include/linux/blkdev.h > +++ b/include/linux/blkdev.h > @@ -838,7 +838,7 @@ static inline unsigned int blk_queue_get_max_sectors(struct request_queue *q, > unsigned int cmd_flags) > { > if (unlikely(cmd_flags & REQ_DISCARD)) > - return q->limits.max_discard_sectors; > + return min(q->limits.max_discard_sectors, UINT_MAX >> 9); > > if (unlikely(cmd_flags & REQ_WRITE_SAME)) > return q->limits.max_write_same_sectors; > > Patch works, and fixes the discard failing issue I've been pestering mkp with at LSF. Please apply. Cheers, Hannes -- Dr. Hannes Reinecke zSeries & Storage hare@suse.de +49 911 74053 688 SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: J. Hawn, J. Guild, F. Imendörffer, HRB 16746 (AG Nürnberg) ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v2 0/9] fix max discard sectors limit 2013-04-19 16:40 [PATCH v2 0/9] fix max discard sectors limit Namjae Jeon 2013-04-20 19:50 ` James Bottomley @ 2013-04-22 21:23 ` Kent Overstreet 2013-04-23 5:16 ` Namjae Jeon 1 sibling, 1 reply; 6+ messages in thread From: Kent Overstreet @ 2013-04-22 21:23 UTC (permalink / raw) To: Namjae Jeon Cc: axboe, nbd-general, linux-raid, npiggin, linux-scsi, neilb, dwmw2, Namjae Jeon, Paul.Clements, adrian.hunter, JBottomley, linux-kernel, James.Bottomley, jcmvbkbc, linux-mtd, linux-mmc, cjb, shli On Sat, Apr 20, 2013 at 01:40:02AM +0900, Namjae Jeon wrote: > From: Namjae Jeon <namjae.jeon@samsung.com> > > linux-v3.8-rc1 and later support for plug for blkdev_issue_discard with > commit 0cfbcafcae8b7364b5fa96c2b26ccde7a3a296a9 > (block: add plug for blkdev_issue_discard ) > > For example, > 1) DISCARD rq-1 with size size 4GB > 2) DISCARD rq-2 with size size 1GB > > If these 2 discard requests get merged, final request size will be 5GB. > > In this case, request's __data_len field may overflow as it can store > max 4GB(unsigned int). Complete NACK - like James said, this fix is nonsensical. This should be fixed by just checking for overflow before merging requests, not with weird hacks in the drivers. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v2 0/9] fix max discard sectors limit 2013-04-22 21:23 ` Kent Overstreet @ 2013-04-23 5:16 ` Namjae Jeon 0 siblings, 0 replies; 6+ messages in thread From: Namjae Jeon @ 2013-04-23 5:16 UTC (permalink / raw) To: Kent Overstreet Cc: axboe, nbd-general, linux-raid, npiggin, linux-scsi, neilb, dwmw2, Namjae Jeon, Paul.Clements, adrian.hunter, JBottomley, linux-kernel, James.Bottomley, jcmvbkbc, linux-mtd, linux-mmc, cjb, shli 2013/4/23, Kent Overstreet <koverstreet@google.com>: > On Sat, Apr 20, 2013 at 01:40:02AM +0900, Namjae Jeon wrote: >> From: Namjae Jeon <namjae.jeon@samsung.com> >> >> linux-v3.8-rc1 and later support for plug for blkdev_issue_discard with >> commit 0cfbcafcae8b7364b5fa96c2b26ccde7a3a296a9 >> (block: add plug for blkdev_issue_discard ) >> >> For example, >> 1) DISCARD rq-1 with size size 4GB >> 2) DISCARD rq-2 with size size 1GB >> >> If these 2 discard requests get merged, final request size will be 5GB. >> >> In this case, request's __data_len field may overflow as it can store >> max 4GB(unsigned int). > > Complete NACK - like James said, this fix is nonsensical. This should be > fixed by just checking for overflow before merging requests, not with > weird hacks in the drivers. that agree to your opinion, and will change at only appropriate place(James'change) and will send the updated patch soon. Thanks. > ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2013-04-26 7:33 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2013-04-19 16:40 [PATCH v2 0/9] fix max discard sectors limit Namjae Jeon 2013-04-20 19:50 ` James Bottomley 2013-04-21 1:37 ` Namjae Jeon 2013-04-26 7:33 ` Hannes Reinecke 2013-04-22 21:23 ` Kent Overstreet 2013-04-23 5:16 ` Namjae Jeon
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox