mkfs.f2fs gets stuck with "blk_update_request: bio idx 0 >= vcnt 0" on 3.8

mkfs.f2fs gets stuck with "blk_update_request: bio idx 0 >= vcnt 0" on 3.8

[Max Filippov]

Hi,

I'm trying to create f2fs filesystem on SD card on pandaboard using
f2fs-tools v1.0.0.
It works fine on Linus' v3.6, but fails on both v3.8 and stable v3.8.5:

# mkfs.f2fs /dev/mmcblk0p3
Info: sector size = 512
Info: total sectors = 11370496 (in 512bytes)
Info: zone aligned segment0 blkaddr: 512
[  257.789764] blk_update_request: bio idx 0 >= vcnt 0

mkfs process gets stuck in D state and I see the following in the dmesg:

[  257.789733] __end_that: dev mmcblk0: type=1, flags=122c8081
[  257.789764]   sector 4194304, nr/cnr 2981888/4294959104
[  257.789764]   bio df3840c0, biotail df3848c0, buffer   (null), len 1526726656
[  257.789764] blk_update_request: bio idx 0 >= vcnt 0
[  257.794921] request botched: dev mmcblk0: type=1, flags=122c8081
[  257.794921]   sector 4194304, nr/cnr 2981888/4294959104
[  257.794921]   bio df3840c0, biotail df3848c0, buffer   (null), len 1526726656

I'd appreciate any suggestion on what to try before I try to bisect it.

-- 
Thanks.
-- Max

Re: mkfs.f2fs gets stuck with "blk_update_request: bio idx 0 >= vcnt 0" on 3.8

[Vyacheslav Dubeyko]

On Apr 1, 2013, at 9:19 PM, Max Filippov wrote:

> Hi,
> 
> I'm trying to create f2fs filesystem on SD card on pandaboard using
> f2fs-tools v1.0.0.
> It works fine on Linus' v3.6, but fails on both v3.8 and stable v3.8.5:
> 
> # mkfs.f2fs /dev/mmcblk0p3
> Info: sector size = 512
> Info: total sectors = 11370496 (in 512bytes)
> Info: zone aligned segment0 blkaddr: 512
> [  257.789764] blk_update_request: bio idx 0 >= vcnt 0
> 
> mkfs process gets stuck in D state and I see the following in the dmesg:
> 
> [  257.789733] __end_that: dev mmcblk0: type=1, flags=122c8081
> [  257.789764]   sector 4194304, nr/cnr 2981888/4294959104
> [  257.789764]   bio df3840c0, biotail df3848c0, buffer   (null), len 1526726656
> [  257.789764] blk_update_request: bio idx 0 >= vcnt 0
> [  257.794921] request botched: dev mmcblk0: type=1, flags=122c8081
> [  257.794921]   sector 4194304, nr/cnr 2981888/4294959104
> [  257.794921]   bio df3840c0, biotail df3848c0, buffer   (null), len 1526726656
> 
> I'd appreciate any suggestion on what to try before I try to bisect it.
> 

Could you share "cat /proc/partitions" and "strace mkfs.f2fs /dev/mmcblk0p3" outputs? I think that these outputs can be very useful for issue analysis.

By the way, can you reproduce the issue on another SD-card? Do you reproduce the issue only under pandaboard?

Thanks,
Vyacheslav Dubeyko.

> -- 
> Thanks.
> -- Max
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: mkfs.f2fs gets stuck with "blk_update_request: bio idx 0 >= vcnt 0" on 3.8

[Max Filippov]

On 04/01/2013 11:27 PM, Vyacheslav Dubeyko wrote:
> 
> On Apr 1, 2013, at 9:19 PM, Max Filippov wrote:
> 
>> Hi,
>>
>> I'm trying to create f2fs filesystem on SD card on pandaboard using
>> f2fs-tools v1.0.0.
>> It works fine on Linus' v3.6, but fails on both v3.8 and stable v3.8.5:
>>
>> # mkfs.f2fs /dev/mmcblk0p3
>> Info: sector size = 512
>> Info: total sectors = 11370496 (in 512bytes)
>> Info: zone aligned segment0 blkaddr: 512
>> [  257.789764] blk_update_request: bio idx 0 >= vcnt 0
>>
>> mkfs process gets stuck in D state and I see the following in the dmesg:
>>
>> [  257.789733] __end_that: dev mmcblk0: type=1, flags=122c8081
>> [  257.789764]   sector 4194304, nr/cnr 2981888/4294959104
>> [  257.789764]   bio df3840c0, biotail df3848c0, buffer   (null), len 1526726656
>> [  257.789764] blk_update_request: bio idx 0 >= vcnt 0
>> [  257.794921] request botched: dev mmcblk0: type=1, flags=122c8081
>> [  257.794921]   sector 4194304, nr/cnr 2981888/4294959104
>> [  257.794921]   bio df3840c0, biotail df3848c0, buffer   (null), len 1526726656
>>
>> I'd appreciate any suggestion on what to try before I try to bisect it.
>>
> 
> Could you share "cat /proc/partitions" and "strace mkfs.f2fs /dev/mmcblk0p3" outputs? I think that these outputs can be very useful for issue analysis.
> 
> By the way, can you reproduce the issue on another SD-card? Do you reproduce the issue only under pandaboard?

Yes, I can reproduce it on three SD cards of different vendors and different sizes.
Unfortunately ATM I don't have any other board to try it on.

# cat /proc/partitions
major minor  #blocks  name

 179        0    7782400 mmcblk0
 179        1      40131 mmcblk0p1
 179        2      32130 mmcblk0p2
 179        3    5685248 mmcblk0p3

strace output is the following:

execve("/home/jcmvbkbc/opt/bin/mkfs.f2fs", ["mkfs.f2fs", "/dev/mmcblk0p3"], [/* 17 vars */]) = 0
brk(0)                                  = 0x15000
uname({sys="Linux", node="zoo.metropolis", ...}) = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
mmap2(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb6f39000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=15792, ...}) = 0
mmap2(NULL, 15792, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb6f35000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/arm-linux-gnueabihf/libgcc_s.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0(\0\1\0\0\0\340%\0\0004\0\0\0"..., 512) = 512
lseek(3, 33324, SEEK_SET)               = 33324
read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1120) = 1120
lseek(3, 32988, SEEK_SET)               = 32988
read(3, "A6\0\0\0aeabi\0\1,\0\0\0\0057-A\0\6\n\7A\10\1\t\2\n\4\22"..., 55) = 55
fstat64(3, {st_mode=S_IFREG|0644, st_size=34444, ...}) = 0
mmap2(NULL, 65812, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb6f0c000
mprotect(0xb6f14000, 28672, PROT_NONE)  = 0
mmap2(0xb6f1b000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x7) = 0xb6f1b000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/arm-linux-gnueabihf/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0(\0\1\0\0\0\355p\1\0004\0\0\0"..., 512) = 512
lseek(3, 888764, SEEK_SET)              = 888764
read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1360) = 1360
lseek(3, 888324, SEEK_SET)              = 888324
read(3, "A4\0\0\0aeabi\0\1*\0\0\0\0057-A\0\6\n\7A\10\1\t\2\n\4\22"..., 53) = 53
fstat64(3, {st_mode=S_IFREG|0755, st_size=890124, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb6f34000
mmap2(NULL, 931200, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb6e28000
mprotect(0xb6efe000, 32768, PROT_NONE)  = 0
mmap2(0xb6f06000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xd6) = 0xb6f06000
mmap2(0xb6f09000, 9600, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb6f09000
close(3)                                = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb6e27000
set_tls(0xb6e274c0, 0xb6e27b98, 0xb6f3c048, 0xb6e274c0, 0xb6f34570) = 0
mprotect(0xb6f06000, 8192, PROT_READ)   = 0
mprotect(0xb6f1b000, 4096, PROT_READ)   = 0
mprotect(0x13000, 4096, PROT_READ)      = 0
mprotect(0xb6f3b000, 4096, PROT_READ)   = 0
munmap(0xb6f35000, 15792)               = 0
brk(0)                                  = 0x15000
brk(0x36000)                            = 0x36000
open("/etc/mtab", O_RDONLY|O_CLOEXEC)   = 3
fcntl64(3, F_GETFD)                     = 0x1 (flags FD_CLOEXEC)
fstat64(3, {st_mode=S_IFREG|0644, st_size=506, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb6f38000
read(3, "192.168.0.1:/tftpboot/zoo/panda_"..., 4096) = 506
read(3, "", 4096)                       = 0
close(3)                                = 0
munmap(0xb6f38000, 4096)                = 0
open("/dev/mmcblk0p3", O_RDWR)          = 3
fstat64(3, {st_mode=S_IFBLK|0660, st_rdev=makedev(179, 3), ...}) = 0
ioctl(3, BLKSSZGET, 0xbeffc27c)         = 0
ioctl(3, BLKGETSIZE, 0x140d8)           = 0
ioctl(3, 0x301, 0xbeffc274)             = 0
fstat64(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 2), ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb6f38000
write(1, "Info: sector size = 512\n", 24) = 24
write(1, "Info: total sectors = 11370496 ("..., 45) = 45
write(1, "Info: zone aligned segment0 blka"..., 41) = 41
fstat64(3, {st_mode=S_IFBLK|0660, st_rdev=makedev(179, 3), ...}) = 0
ioctl(3, BLKDISCARD


-- 
Thanks.
-- Max

Re: mkfs.f2fs gets stuck with "blk_update_request: bio idx 0 >= vcnt 0" on 3.8

[Vyacheslav Dubeyko]

On Tue, 2013-04-02 at 00:21 +0400, Max Filippov wrote:

[snip]
> > 
> > Could you share "cat /proc/partitions" and "strace mkfs.f2fs /dev/mmcblk0p3" outputs? I think that these outputs can be very useful for issue analysis.
> > 
> > By the way, can you reproduce the issue on another SD-card? Do you reproduce the issue only under pandaboard?
> 
> Yes, I can reproduce it on three SD cards of different vendors and different sizes.
> Unfortunately ATM I don't have any other board to try it on.
> 
> # cat /proc/partitions
> major minor  #blocks  name
> 
>  179        0    7782400 mmcblk0
>  179        1      40131 mmcblk0p1
>  179        2      32130 mmcblk0p2
>  179        3    5685248 mmcblk0p3
> 


As I see, you have several partition on your SD-card. How did it
prepared?

> strace output is the following:
> 
[snip]
> write(1, "Info: sector size = 512\n", 24) = 24
> write(1, "Info: total sectors = 11370496 ("..., 45) = 45
> write(1, "Info: zone aligned segment0 blka"..., 41) = 41
> fstat64(3, {st_mode=S_IFBLK|0660, st_rdev=makedev(179, 3), ...}) = 0
> ioctl(3, BLKDISCARD

As I understand, the BLKDISCARD ioctl (to pre-discard all blocks on an
ssd, or a thinly-provisioned storage device) is a visible reason of the
issue. Unfortunately, as I see, mkfs.f2fs doesn't support option to
format partition without blocks discard step.

So, I think that it needs to investigate issue in the direction of
BLKDISCARD code on the kernel side. It makes sense to debug
f2fs_trim_device() method of mkfs.f2fs utility too. But I can't see
anything strange in this function at a glance.

With the best regards,
Vyacheslav Dubeyko.

> 
> 

Re: mkfs.f2fs gets stuck with "blk_update_request: bio idx 0 >= vcnt 0" on 3.8

[Max Filippov]

On Tue, Apr 2, 2013 at 10:27 AM, Vyacheslav Dubeyko <slava@dubeyko.com> wrote:
> On Tue, 2013-04-02 at 00:21 +0400, Max Filippov wrote:

[...]

>> # cat /proc/partitions
>> major minor  #blocks  name
>>
>>  179        0    7782400 mmcblk0
>>  179        1      40131 mmcblk0p1
>>  179        2      32130 mmcblk0p2
>>  179        3    5685248 mmcblk0p3
>>
>
> As I see, you have several partition on your SD-card. How did it
> prepared?

I made it with fdisk, just as with any other block device.

>> strace output is the following:
>>
> [snip]
>> write(1, "Info: sector size = 512\n", 24) = 24
>> write(1, "Info: total sectors = 11370496 ("..., 45) = 45
>> write(1, "Info: zone aligned segment0 blka"..., 41) = 41
>> fstat64(3, {st_mode=S_IFBLK|0660, st_rdev=makedev(179, 3), ...}) = 0
>> ioctl(3, BLKDISCARD
>
> As I understand, the BLKDISCARD ioctl (to pre-discard all blocks on an
> ssd, or a thinly-provisioned storage device) is a visible reason of the
> issue. Unfortunately, as I see, mkfs.f2fs doesn't support option to
> format partition without blocks discard step.
>
> So, I think that it needs to investigate issue in the direction of
> BLKDISCARD code on the kernel side. It makes sense to debug
> f2fs_trim_device() method of mkfs.f2fs utility too. But I can't see
> anything strange in this function at a glance.

Ok, I'll try to find what has changed in that ioctl handler since 3.6.

-- 
Thanks.
-- Max

Re: mkfs.f2fs gets stuck with "blk_update_request: bio idx 0 >= vcnt 0" on 3.8

[Max Filippov]

Hi,

On Tue, Apr 2, 2013 at 7:41 PM, Max Filippov <jcmvbkbc@gmail.com> wrote:
> I'm trying to create f2fs filesystem on SD card on pandaboard using
> f2fs-tools v1.0.0.
> It works fine on Linus' v3.6, but fails on both v3.8 and stable v3.8.5:
>
> # mkfs.f2fs /dev/mmcblk0p3
> Info: sector size = 512
> Info: total sectors = 11370496 (in 512bytes)
> Info: zone aligned segment0 blkaddr: 512
> [  257.789764] blk_update_request: bio idx 0 >= vcnt 0
>
> mkfs process gets stuck in D state and I see the following in the dmesg:
>
> [  257.789733] __end_that: dev mmcblk0: type=1, flags=122c8081
> [  257.789764]   sector 4194304, nr/cnr 2981888/4294959104
> [  257.789764]   bio df3840c0, biotail df3848c0, buffer   (null), len 1526726656
> [  257.789764] blk_update_request: bio idx 0 >= vcnt 0
> [  257.794921] request botched: dev mmcblk0: type=1, flags=122c8081
> [  257.794921]   sector 4194304, nr/cnr 2981888/4294959104
> [  257.794921]   bio df3840c0, biotail df3848c0, buffer   (null), len 1526726656

[...]

>> So, I think that it needs to investigate issue in the direction of
>> BLKDISCARD code on the kernel side. It makes sense to debug
>> f2fs_trim_device() method of mkfs.f2fs utility too. But I can't see
>> anything strange in this function at a glance.
>
> Ok, I'll try to find what has changed in that ioctl handler since 3.6.

the commit 0cfbcafcae8b7364b5fa96c2b26ccde7a3a296a9 'block: add plug
for blkdev_issue_discard'
have added merge opportunity for DISCARD requests. When I do
mkfs.f2fs on a 5G partition (0xad8000 sectors) it submits two bios,
one for 0x7fe000 sectors (0xffc00000 bytes) and another for
0x2da000 sectors (0x5b400000 bytes). Prior to that commit these
bios weren't merged into one request. Now the second bio gets
merged with the first, but the request's __data_len field is unsigned int
and it gets wrapped to 0x5b000000 bytes instead of 0x15b000000
in the bio_attempt_back_merge. Later this reduced size is passed to
the blk_update_request causing KERN_ERR and not completed
request. Reverting this commit fixes mkfs.f2fs for me.

-- 
Thanks.
-- Max

Re: mkfs.f2fs gets stuck with "blk_update_request: bio idx 0 >= vcnt 0" on 3.8

[Shaohua Li]

On Thu, Apr 04, 2013 at 06:00:18AM +0400, Max Filippov wrote:
> Hi,
> 
> On Tue, Apr 2, 2013 at 7:41 PM, Max Filippov <jcmvbkbc@gmail.com> wrote:
> > I'm trying to create f2fs filesystem on SD card on pandaboard using
> > f2fs-tools v1.0.0.
> > It works fine on Linus' v3.6, but fails on both v3.8 and stable v3.8.5:
> >
> > # mkfs.f2fs /dev/mmcblk0p3
> > Info: sector size = 512
> > Info: total sectors = 11370496 (in 512bytes)
> > Info: zone aligned segment0 blkaddr: 512
> > [  257.789764] blk_update_request: bio idx 0 >= vcnt 0
> >
> > mkfs process gets stuck in D state and I see the following in the dmesg:
> >
> > [  257.789733] __end_that: dev mmcblk0: type=1, flags=122c8081
> > [  257.789764]   sector 4194304, nr/cnr 2981888/4294959104
> > [  257.789764]   bio df3840c0, biotail df3848c0, buffer   (null), len 1526726656
> > [  257.789764] blk_update_request: bio idx 0 >= vcnt 0
> > [  257.794921] request botched: dev mmcblk0: type=1, flags=122c8081
> > [  257.794921]   sector 4194304, nr/cnr 2981888/4294959104
> > [  257.794921]   bio df3840c0, biotail df3848c0, buffer   (null), len 1526726656
> 
> [...]
> 
> >> So, I think that it needs to investigate issue in the direction of
> >> BLKDISCARD code on the kernel side. It makes sense to debug
> >> f2fs_trim_device() method of mkfs.f2fs utility too. But I can't see
> >> anything strange in this function at a glance.
> >
> > Ok, I'll try to find what has changed in that ioctl handler since 3.6.
> 
> the commit 0cfbcafcae8b7364b5fa96c2b26ccde7a3a296a9 'block: add plug
> for blkdev_issue_discard'
> have added merge opportunity for DISCARD requests. When I do
> mkfs.f2fs on a 5G partition (0xad8000 sectors) it submits two bios,
> one for 0x7fe000 sectors (0xffc00000 bytes) and another for
> 0x2da000 sectors (0x5b400000 bytes). Prior to that commit these
> bios weren't merged into one request. Now the second bio gets
> merged with the first, but the request's __data_len field is unsigned int
> and it gets wrapped to 0x5b000000 bytes instead of 0x15b000000
> in the bio_attempt_back_merge. Later this reduced size is passed to
> the blk_update_request causing KERN_ERR and not completed
> request. Reverting this commit fixes mkfs.f2fs for me.

A workaround is setting limits.max_discard_sectors to a smaller value.

So the question is why __data_len isn't sector based? Since disk is sector
based, is there any disk finishing IO in byte granularity? Maybe Jens can
answer.

Thanks,
Shaohua

Re: mkfs.f2fs gets stuck with "blk_update_request: bio idx 0 >= vcnt 0" on 3.8

[Max Filippov]

On Fri, Apr 5, 2013 at 5:53 AM, Shaohua Li <shli@kernel.org> wrote:
> On Thu, Apr 04, 2013 at 06:00:18AM +0400, Max Filippov wrote:

[...]

>> the commit 0cfbcafcae8b7364b5fa96c2b26ccde7a3a296a9 'block: add plug
>> for blkdev_issue_discard'
>> have added merge opportunity for DISCARD requests. When I do
>> mkfs.f2fs on a 5G partition (0xad8000 sectors) it submits two bios,
>> one for 0x7fe000 sectors (0xffc00000 bytes) and another for
>> 0x2da000 sectors (0x5b400000 bytes). Prior to that commit these
>> bios weren't merged into one request. Now the second bio gets
>> merged with the first, but the request's __data_len field is unsigned int
>> and it gets wrapped to 0x5b000000 bytes instead of 0x15b000000
>> in the bio_attempt_back_merge. Later this reduced size is passed to
>> the blk_update_request causing KERN_ERR and not completed
>> request. Reverting this commit fixes mkfs.f2fs for me.
>
> A workaround is setting limits.max_discard_sectors to a smaller value.

I'm not sure:
1) in my case max_discard_sectors is 0x7fe000 (0xffc00000 bytes,
    which still fits into 32 bits) and
2) this parameter will only change size of individual discard requests for
    the discarded range, but as long as these requests are done inside
    the plug they will be merged anyway with an overflow if we try
    to discard more than 4G at once.

> So the question is why __data_len isn't sector based? Since disk is sector
> based, is there any disk finishing IO in byte granularity? Maybe Jens can
> answer.

-- 
Thanks.
-- Max

Re: mkfs.f2fs gets stuck with "blk_update_request: bio idx 0 >= vcnt 0" on 3.8

[Namjae Jeon]

Hi. Max.

I have a question.
Your mmc host driver set to host->max_discard_to by some value instead
of not zero ?

Thanks.

2013/4/5, Max Filippov <jcmvbkbc@gmail.com>:
> On Fri, Apr 5, 2013 at 5:53 AM, Shaohua Li <shli@kernel.org> wrote:
>> On Thu, Apr 04, 2013 at 06:00:18AM +0400, Max Filippov wrote:
>
> [...]
>
>>> the commit 0cfbcafcae8b7364b5fa96c2b26ccde7a3a296a9 'block: add plug
>>> for blkdev_issue_discard'
>>> have added merge opportunity for DISCARD requests. When I do
>>> mkfs.f2fs on a 5G partition (0xad8000 sectors) it submits two bios,
>>> one for 0x7fe000 sectors (0xffc00000 bytes) and another for
>>> 0x2da000 sectors (0x5b400000 bytes). Prior to that commit these
>>> bios weren't merged into one request. Now the second bio gets
>>> merged with the first, but the request's __data_len field is unsigned
>>> int
>>> and it gets wrapped to 0x5b000000 bytes instead of 0x15b000000
>>> in the bio_attempt_back_merge. Later this reduced size is passed to
>>> the blk_update_request causing KERN_ERR and not completed
>>> request. Reverting this commit fixes mkfs.f2fs for me.
>>
>> A workaround is setting limits.max_discard_sectors to a smaller value.
>
> I'm not sure:
> 1) in my case max_discard_sectors is 0x7fe000 (0xffc00000 bytes,
>     which still fits into 32 bits) and
> 2) this parameter will only change size of individual discard requests for
>     the discarded range, but as long as these requests are done inside
>     the plug they will be merged anyway with an overflow if we try
>     to discard more than 4G at once.
>
>> So the question is why __data_len isn't sector based? Since disk is
>> sector
>> based, is there any disk finishing IO in byte granularity? Maybe Jens can
>> answer.
>
> --
> Thanks.
> -- Max
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: mkfs.f2fs gets stuck with "blk_update_request: bio idx 0 >= vcnt 0" on 3.8

[Max Filippov]

Hi Namjae,

On Fri, Apr 5, 2013 at 11:57 AM, Namjae Jeon <linkinjeon@gmail.com> wrote:
> Hi. Max.
>
> I have a question.
> Your mmc host driver set to host->max_discard_to by some value instead
> of not zero ?

I believe it's zero, because the only place where I can see it initialized
(sdhci_add_host in the drivers/mmc/host/sdhci.c) is not built in my
configuration.

> 2013/4/5, Max Filippov <jcmvbkbc@gmail.com>:
>> On Fri, Apr 5, 2013 at 5:53 AM, Shaohua Li <shli@kernel.org> wrote:
>>> On Thu, Apr 04, 2013 at 06:00:18AM +0400, Max Filippov wrote:
>>
>> [...]
>>
>>>> the commit 0cfbcafcae8b7364b5fa96c2b26ccde7a3a296a9 'block: add plug
>>>> for blkdev_issue_discard'
>>>> have added merge opportunity for DISCARD requests. When I do
>>>> mkfs.f2fs on a 5G partition (0xad8000 sectors) it submits two bios,
>>>> one for 0x7fe000 sectors (0xffc00000 bytes) and another for
>>>> 0x2da000 sectors (0x5b400000 bytes). Prior to that commit these
>>>> bios weren't merged into one request. Now the second bio gets
>>>> merged with the first, but the request's __data_len field is unsigned
>>>> int
>>>> and it gets wrapped to 0x5b000000 bytes instead of 0x15b000000
>>>> in the bio_attempt_back_merge. Later this reduced size is passed to
>>>> the blk_update_request causing KERN_ERR and not completed
>>>> request. Reverting this commit fixes mkfs.f2fs for me.
>>>
>>> A workaround is setting limits.max_discard_sectors to a smaller value.
>>
>> I'm not sure:
>> 1) in my case max_discard_sectors is 0x7fe000 (0xffc00000 bytes,
>>     which still fits into 32 bits) and
>> 2) this parameter will only change size of individual discard requests for
>>     the discarded range, but as long as these requests are done inside
>>     the plug they will be merged anyway with an overflow if we try
>>     to discard more than 4G at once.
>>
>>> So the question is why __data_len isn't sector based? Since disk is
>>> sector
>>> based, is there any disk finishing IO in byte granularity? Maybe Jens can
>>> answer.

-- 
Thanks.
-- Max

Re: mkfs.f2fs gets stuck with "blk_update_request: bio idx 0 >= vcnt 0" on 3.8

[Shaohua Li]

On Fri, Apr 05, 2013 at 06:18:10AM +0400, Max Filippov wrote:
> On Fri, Apr 5, 2013 at 5:53 AM, Shaohua Li <shli@kernel.org> wrote:
> > On Thu, Apr 04, 2013 at 06:00:18AM +0400, Max Filippov wrote:
> 
> [...]
> 
> >> the commit 0cfbcafcae8b7364b5fa96c2b26ccde7a3a296a9 'block: add plug
> >> for blkdev_issue_discard'
> >> have added merge opportunity for DISCARD requests. When I do
> >> mkfs.f2fs on a 5G partition (0xad8000 sectors) it submits two bios,
> >> one for 0x7fe000 sectors (0xffc00000 bytes) and another for
> >> 0x2da000 sectors (0x5b400000 bytes). Prior to that commit these
> >> bios weren't merged into one request. Now the second bio gets
> >> merged with the first, but the request's __data_len field is unsigned int
> >> and it gets wrapped to 0x5b000000 bytes instead of 0x15b000000
> >> in the bio_attempt_back_merge. Later this reduced size is passed to
> >> the blk_update_request causing KERN_ERR and not completed
> >> request. Reverting this commit fixes mkfs.f2fs for me.
> >
> > A workaround is setting limits.max_discard_sectors to a smaller value.
> 
> I'm not sure:
> 1) in my case max_discard_sectors is 0x7fe000 (0xffc00000 bytes,
>     which still fits into 32 bits) and
> 2) this parameter will only change size of individual discard requests for
>     the discarded range, but as long as these requests are done inside
>     the plug they will be merged anyway with an overflow if we try
>     to discard more than 4G at once.

No, merge in plug list checks max_discard_sectors. Please see
ll_back_merge_fn() in bio_attempt_back_merge(). Merge in unplug checks
max_discard_sectors too. I didn't see any chance a merge doesn't check
max_discard_sectors. Can you post a blktrace when the discard ioctl runs and
double check if max_discard_sectors is really 0x7fe000?

Re: mkfs.f2fs gets stuck with "blk_update_request: bio idx 0 >= vcnt 0" on 3.8

[Max Filippov]

On Sun, Apr 7, 2013 at 5:58 AM, Shaohua Li <shli@kernel.org> wrote:
> On Fri, Apr 05, 2013 at 06:18:10AM +0400, Max Filippov wrote:
>> On Fri, Apr 5, 2013 at 5:53 AM, Shaohua Li <shli@kernel.org> wrote:
>> > On Thu, Apr 04, 2013 at 06:00:18AM +0400, Max Filippov wrote:
>>
>> [...]
>>
>> >> the commit 0cfbcafcae8b7364b5fa96c2b26ccde7a3a296a9 'block: add plug
>> >> for blkdev_issue_discard'
>> >> have added merge opportunity for DISCARD requests. When I do
>> >> mkfs.f2fs on a 5G partition (0xad8000 sectors) it submits two bios,
>> >> one for 0x7fe000 sectors (0xffc00000 bytes) and another for
>> >> 0x2da000 sectors (0x5b400000 bytes). Prior to that commit these
>> >> bios weren't merged into one request. Now the second bio gets
>> >> merged with the first, but the request's __data_len field is unsigned int
>> >> and it gets wrapped to 0x5b000000 bytes instead of 0x15b000000
>> >> in the bio_attempt_back_merge. Later this reduced size is passed to
>> >> the blk_update_request causing KERN_ERR and not completed
>> >> request. Reverting this commit fixes mkfs.f2fs for me.
>> >
>> > A workaround is setting limits.max_discard_sectors to a smaller value.
>>
>> I'm not sure:
>> 1) in my case max_discard_sectors is 0x7fe000 (0xffc00000 bytes,
>>     which still fits into 32 bits) and
>> 2) this parameter will only change size of individual discard requests for
>>     the discarded range, but as long as these requests are done inside
>>     the plug they will be merged anyway with an overflow if we try
>>     to discard more than 4G at once.
>
> No, merge in plug list checks max_discard_sectors. Please see
> ll_back_merge_fn() in bio_attempt_back_merge(). Merge in unplug checks
> max_discard_sectors too. I didn't see any chance a merge doesn't check
> max_discard_sectors. Can you post a blktrace when the discard ioctl runs and
> double check if max_discard_sectors is really 0x7fe000?

Ok, max_discard_sectors is a bit ambiguous here.
There's a variable inside blkdev_issue_discard called max_discard_sectors,
I looked at it and it was 0x7fe000. But the q->limits.max_discard_sectors
is initialized to 0xffffffff in the mmc_queue_setup_discard. The following
change fixes the original issue too:

diff --git a/drivers/mmc/core/core.c b/drivers/mmc/core/core.c
index aaed768..0753986 100644
--- a/drivers/mmc/core/core.c
+++ b/drivers/mmc/core/core.c
@@ -1920,7 +1920,7 @@ unsigned int mmc_calc_max_discard(struct mmc_card *card)
        unsigned int max_discard, max_trim;

        if (!host->max_discard_to)
-               return UINT_MAX;
+               return UINT_MAX >> 9;

        /*
         * Without erase_group_def set, MMC erase timeout depends on clock


-- 
Thanks.
-- Max

Re: mkfs.f2fs gets stuck with "blk_update_request: bio idx 0 >= vcnt 0" on 3.8

[Namjae Jeon]

2013/4/7, Max Filippov <jcmvbkbc@gmail.com>:
> On Sun, Apr 7, 2013 at 4:36 AM, Namjae Jeon <linkinjeon@gmail.com> wrote:
>>
>>
>> 2013/4/6 Max Filippov <jcmvbkbc@gmail.com>
>>>
>>> Hi Namjae,
>>>
>>> On Fri, Apr 5, 2013 at 11:57 AM, Namjae Jeon <linkinjeon@gmail.com>
>>> wrote:
>>> > Hi. Max.
>>> >
>>> > I have a question.
>>> > Your mmc host driver set to host->max_discard_to by some value instead
>>> > of not zero ?
>>>
>>> I believe it's zero, because the only place where I can see it
>>> initialized
>>> (sdhci_add_host in the drivers/mmc/host/sdhci.c) is not built in my
>>> configuration.
>>>
>>  -> sorry for private mail. There is confusing to me about your previous
>> mail.
>> I think that max_discard_sectors is set to 0xFFFFFFFF(UINT_MAX).
>> because mmc core set to it by UINT_MAX when host-<max_discard_to is zero.
>> So I guess it can try to be merged to over 4GB size.
>>
>> unsigned int mmc_calc_max_discard(struct mmc_card *card)
>> {
>> struct mmc_host *host = card->host;
>> unsigned int max_discard, max_trim;
>>
>> if (!host->max_discard_to)
>> return UINT_MAX; ==> It should be UINT_MAX >> 9;
>>
>> unfortunely, I don't have mmc device and card had over 4GB size.
>> So If you agree, Could you try to test after changing  above code
>> UINT_MAX
>> => UINT_MAX >> 9 ?
>
> This fixes the issue too. Requests do not merge, although
> attempt_plug_merge
> returns ELEVATOR_BACK_MERGE.
Okay, Thanks for your test.
As I guess, this issue is  caused from max discards setting of mmc core.
I will try to commit this patch to mmc driver.


Thanks Max.

>
> --
> Thanks.
> -- Max
>