* fio causing kernel panic due to zero size requests
@ 2012-04-20 17:27 Vikram Seth
2012-04-20 17:49 ` Jens Axboe
0 siblings, 1 reply; 4+ messages in thread
From: Vikram Seth @ 2012-04-20 17:27 UTC (permalink / raw)
To: fio
Hi Jens,
Recently we saw an issue where fio created kernel panic because of 0
size requests.
Pasting the stack trace below.
<d>Pid: 13846, comm: fio Tainted: P M 2.6.38.4-
<d>RIP: 0010:[<ffffffff8111d90d>] [<ffffffff8111d90d>]
__blockdev_direct_IO+0x6a7/0x9ce
<d>RSP: 0018:ffff8823fb7e9c18 EFLAGS: 00010246
<d>RAX: 0000000000000000 RBX: ffffea007d55d460 RCX: 000000000000000a
<d>RDX: 0000000000000c00 RSI: 0000000000000000 RDI: ffff8812342b80e0
<d>RBP: ffff8823fb7e9cd8 R08: 0000000000000001 R09: 0000000000000000
<d>R10: 0000000000000000 R11: ffffea0000000000 R12: ffff881233d64400
<d>R13: 0000000000000003 R14: ffff881233d64498 R15: ffff8812342b80e0
<d>FS: 00007f048cddb6e0(0000) GS:ffff88007e200000(0000) knlGS:0000000000000000
<d>CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<d>CR2: 000000000040d040 CR3: 000000238ea69000 CR4: 00000000000406f0
<d>DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<d>DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process fio (pid: 13846, threadinfo ffff8823fb7e8000, task ffff882432ead120)
<0>Stack:
<c> ffff8823fb7e9c68<c> ffffffff814478ce<c> ffff882300000001<c>
ffff8823cf614000<c>
<c> ffff882300000004<c> ffffffff814478ce<c> fffffffffffffc00<c>
00000000814478ce<c>
<c> ffff8823fb7e8000<c> 0000000000000400<c> 0000000a00000001<c>
ffff8823fb7e8000<c>
Checking that function it's
0xffffffff8111d90d is in __blockdev_direct_IO (fs/direct-io.c:955).
and that points to following line in direct-io.c:
955 BUG_ON(this_chunk_bytes == 0);
We also tested with following patch from kernel.org
f9b5570 fs: simplify handling of zero sized reads in __blockdev_direct_IO
That resolved the panic problem.
Can you please add check to fio that it should error out OR skip 0
size accesses ?
Thanks,
Vikram.
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: fio causing kernel panic due to zero size requests
2012-04-20 17:27 fio causing kernel panic due to zero size requests Vikram Seth
@ 2012-04-20 17:49 ` Jens Axboe
2012-04-20 18:13 ` Vikram Seth
0 siblings, 1 reply; 4+ messages in thread
From: Jens Axboe @ 2012-04-20 17:49 UTC (permalink / raw)
To: Vikram Seth; +Cc: fio
On 2012-04-20 19:27, Vikram Seth wrote:
> Hi Jens,
>
> Recently we saw an issue where fio created kernel panic because of 0
> size requests.
> Pasting the stack trace below.
>
> <d>Pid: 13846, comm: fio Tainted: P M 2.6.38.4-
> <d>RIP: 0010:[<ffffffff8111d90d>] [<ffffffff8111d90d>]
> __blockdev_direct_IO+0x6a7/0x9ce
> <d>RSP: 0018:ffff8823fb7e9c18 EFLAGS: 00010246
> <d>RAX: 0000000000000000 RBX: ffffea007d55d460 RCX: 000000000000000a
> <d>RDX: 0000000000000c00 RSI: 0000000000000000 RDI: ffff8812342b80e0
> <d>RBP: ffff8823fb7e9cd8 R08: 0000000000000001 R09: 0000000000000000
> <d>R10: 0000000000000000 R11: ffffea0000000000 R12: ffff881233d64400
> <d>R13: 0000000000000003 R14: ffff881233d64498 R15: ffff8812342b80e0
> <d>FS: 00007f048cddb6e0(0000) GS:ffff88007e200000(0000) knlGS:0000000000000000
> <d>CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> <d>CR2: 000000000040d040 CR3: 000000238ea69000 CR4: 00000000000406f0
> <d>DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> <d>DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> Process fio (pid: 13846, threadinfo ffff8823fb7e8000, task ffff882432ead120)
> <0>Stack:
> <c> ffff8823fb7e9c68<c> ffffffff814478ce<c> ffff882300000001<c>
> ffff8823cf614000<c>
> <c> ffff882300000004<c> ffffffff814478ce<c> fffffffffffffc00<c>
> 00000000814478ce<c>
> <c> ffff8823fb7e8000<c> 0000000000000400<c> 0000000a00000001<c>
> ffff8823fb7e8000<c>
>
> Checking that function it's
>
> 0xffffffff8111d90d is in __blockdev_direct_IO (fs/direct-io.c:955).
>
> and that points to following line in direct-io.c:
>
> 955 BUG_ON(this_chunk_bytes == 0);
>
> We also tested with following patch from kernel.org
>
> f9b5570 fs: simplify handling of zero sized reads in __blockdev_direct_IO
>
> That resolved the panic problem.
>
> Can you please add check to fio that it should error out OR skip 0
> size accesses ?
It seems you forgot to include the job file that triggered this :-)
Fio should not be making zero sized reads, but pretty embarassing that
the kernel oopsed on getting one. It could be a file system issue, a bug
there triggering a zero sized read.
So how did you trigger this? And on what fs?
--
Jens Axboe
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: fio causing kernel panic due to zero size requests
2012-04-20 17:49 ` Jens Axboe
@ 2012-04-20 18:13 ` Vikram Seth
2012-04-20 18:20 ` Jens Axboe
0 siblings, 1 reply; 4+ messages in thread
From: Vikram Seth @ 2012-04-20 18:13 UTC (permalink / raw)
To: Jens Axboe; +Cc: fio
[-- Attachment #1: Type: text/plain, Size: 2510 bytes --]
On Fri, Apr 20, 2012 at 10:49 AM, Jens Axboe <axboe@kernel.dk> wrote:
> On 2012-04-20 19:27, Vikram Seth wrote:
>> Hi Jens,
>>
>> Recently we saw an issue where fio created kernel panic because of 0
>> size requests.
>> Pasting the stack trace below.
>>
>> <d>Pid: 13846, comm: fio Tainted: P M 2.6.38.4-
>> <d>RIP: 0010:[<ffffffff8111d90d>] [<ffffffff8111d90d>]
>> __blockdev_direct_IO+0x6a7/0x9ce
>> <d>RSP: 0018:ffff8823fb7e9c18 EFLAGS: 00010246
>> <d>RAX: 0000000000000000 RBX: ffffea007d55d460 RCX: 000000000000000a
>> <d>RDX: 0000000000000c00 RSI: 0000000000000000 RDI: ffff8812342b80e0
>> <d>RBP: ffff8823fb7e9cd8 R08: 0000000000000001 R09: 0000000000000000
>> <d>R10: 0000000000000000 R11: ffffea0000000000 R12: ffff881233d64400
>> <d>R13: 0000000000000003 R14: ffff881233d64498 R15: ffff8812342b80e0
>> <d>FS: 00007f048cddb6e0(0000) GS:ffff88007e200000(0000) knlGS:0000000000000000
>> <d>CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>> <d>CR2: 000000000040d040 CR3: 000000238ea69000 CR4: 00000000000406f0
>> <d>DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>> <d>DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
>> Process fio (pid: 13846, threadinfo ffff8823fb7e8000, task ffff882432ead120)
>> <0>Stack:
>> <c> ffff8823fb7e9c68<c> ffffffff814478ce<c> ffff882300000001<c>
>> ffff8823cf614000<c>
>> <c> ffff882300000004<c> ffffffff814478ce<c> fffffffffffffc00<c>
>> 00000000814478ce<c>
>> <c> ffff8823fb7e8000<c> 0000000000000400<c> 0000000a00000001<c>
>> ffff8823fb7e8000<c>
>>
>> Checking that function it's
>>
>> 0xffffffff8111d90d is in __blockdev_direct_IO (fs/direct-io.c:955).
>>
>> and that points to following line in direct-io.c:
>>
>> 955 BUG_ON(this_chunk_bytes == 0);
>>
>> We also tested with following patch from kernel.org
>>
>> f9b5570 fs: simplify handling of zero sized reads in __blockdev_direct_IO
>>
>> That resolved the panic problem.
>>
>> Can you please add check to fio that it should error out OR skip 0
>> size accesses ?
>
> It seems you forgot to include the job file that triggered this :-)
> Fio should not be making zero sized reads, but pretty embarassing that
> the kernel oopsed on getting one. It could be a file system issue, a bug
> there triggering a zero sized read.
>
> So how did you trigger this? And on what fs?
>
> --
> Jens Axboe
>
The job file is attached. It is a simple job file doing direct IO (so
no fs) on a flash device.
Vikram.
[-- Attachment #2: fio-test-file --]
[-- Type: application/octet-stream, Size: 1344 bytes --]
# This job file tries to run standard perf tests.
[global]
description=Standard Performance test
runtime=1280
direct=1
filename=/dev/sda
numjobs=20
group_reporting
[seqread_4]
rw=read
bs=4k
[seqread_8]
stonewall
rw=read
bs=8k
[seqread_16]
stonewall
rw=read
bs=16k
[seqread_32]
stonewall
rw=read
bs=32k
[seqread_64]
stonewall
rw=read
bs=64k
[seqread_128]
stonewall
rw=read
bs=128k
[seqread_256]
stonewall
rw=read
bs=256k
[seqwrite_4]
stonewall
rw=write
bs=4k
[seqwrite_8]
stonewall
rw=write
bs=8k
[seqwrite_16]
stonewall
rw=write
bs=16k
[seqwrite_32]
stonewall
rw=write
bs=32k
[seqwrite_64]
stonewall
rw=write
bs=64k
[seqwrite_128]
stonewall
rw=write
bs=128k
[seqwrite_256]
stonewall
rw=write
bs=256k
[randread_4]
stonewall
rw=randread
bs=4k
[randread_8]
stonewall
rw=randread
bs=8k
[randread_16]
stonewall
rw=randread
bs=16k
[randread_32]
stonewall
rw=randread
bs=32k
[randread_64]
stonewall
rw=randread
bs=64k
[randread_128]
stonewall
rw=randread
bs=128k
[randread_256]
stonewall
rw=randread
bs=256k
[randwrite_4]
stonewall
rw=randwrite
bs=4k
[randwrite_8]
stonewall
rw=randwrite
bs=8k
[randwrite_16]
stonewall
rw=randwrite
bs=16k
[randwrite_32]
stonewall
rw=randwrite
bs=32k
[randwrite_64]
stonewall
rw=randwrite
bs=64k
[randwrite_128]
stonewall
rw=randwrite
bs=128k
[randwrite_256]
stonewall
rw=randwrite
bs=256k
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: fio causing kernel panic due to zero size requests
2012-04-20 18:13 ` Vikram Seth
@ 2012-04-20 18:20 ` Jens Axboe
0 siblings, 0 replies; 4+ messages in thread
From: Jens Axboe @ 2012-04-20 18:20 UTC (permalink / raw)
To: Vikram Seth; +Cc: fio
On 2012-04-20 20:13, Vikram Seth wrote:
> On Fri, Apr 20, 2012 at 10:49 AM, Jens Axboe <axboe@kernel.dk> wrote:
>> On 2012-04-20 19:27, Vikram Seth wrote:
>>> Hi Jens,
>>>
>>> Recently we saw an issue where fio created kernel panic because of 0
>>> size requests.
>>> Pasting the stack trace below.
>>>
>>> <d>Pid: 13846, comm: fio Tainted: P M 2.6.38.4-
>>> <d>RIP: 0010:[<ffffffff8111d90d>] [<ffffffff8111d90d>]
>>> __blockdev_direct_IO+0x6a7/0x9ce
>>> <d>RSP: 0018:ffff8823fb7e9c18 EFLAGS: 00010246
>>> <d>RAX: 0000000000000000 RBX: ffffea007d55d460 RCX: 000000000000000a
>>> <d>RDX: 0000000000000c00 RSI: 0000000000000000 RDI: ffff8812342b80e0
>>> <d>RBP: ffff8823fb7e9cd8 R08: 0000000000000001 R09: 0000000000000000
>>> <d>R10: 0000000000000000 R11: ffffea0000000000 R12: ffff881233d64400
>>> <d>R13: 0000000000000003 R14: ffff881233d64498 R15: ffff8812342b80e0
>>> <d>FS: 00007f048cddb6e0(0000) GS:ffff88007e200000(0000) knlGS:0000000000000000
>>> <d>CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>>> <d>CR2: 000000000040d040 CR3: 000000238ea69000 CR4: 00000000000406f0
>>> <d>DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>>> <d>DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
>>> Process fio (pid: 13846, threadinfo ffff8823fb7e8000, task ffff882432ead120)
>>> <0>Stack:
>>> <c> ffff8823fb7e9c68<c> ffffffff814478ce<c> ffff882300000001<c>
>>> ffff8823cf614000<c>
>>> <c> ffff882300000004<c> ffffffff814478ce<c> fffffffffffffc00<c>
>>> 00000000814478ce<c>
>>> <c> ffff8823fb7e8000<c> 0000000000000400<c> 0000000a00000001<c>
>>> ffff8823fb7e8000<c>
>>>
>>> Checking that function it's
>>>
>>> 0xffffffff8111d90d is in __blockdev_direct_IO (fs/direct-io.c:955).
>>>
>>> and that points to following line in direct-io.c:
>>>
>>> 955 BUG_ON(this_chunk_bytes == 0);
>>>
>>> We also tested with following patch from kernel.org
>>>
>>> f9b5570 fs: simplify handling of zero sized reads in __blockdev_direct_IO
>>>
>>> That resolved the panic problem.
>>>
>>> Can you please add check to fio that it should error out OR skip 0
>>> size accesses ?
>>
>> It seems you forgot to include the job file that triggered this :-)
>> Fio should not be making zero sized reads, but pretty embarassing that
>> the kernel oopsed on getting one. It could be a file system issue, a bug
>> there triggering a zero sized read.
>>
>> So how did you trigger this? And on what fs?
>>
>> --
>> Jens Axboe
>>
>
> The job file is attached. It is a simple job file doing direct IO (so
> no fs) on a flash device.
What fio version did you use?
--
Jens Axboe
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2012-04-20 18:20 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2012-04-20 17:27 fio causing kernel panic due to zero size requests Vikram Seth
2012-04-20 17:49 ` Jens Axboe
2012-04-20 18:13 ` Vikram Seth
2012-04-20 18:20 ` Jens Axboe
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox