From: Jens Axboe <axboe@kernel.dk>
To: Guenter Roeck <linux@roeck-us.net>
Cc: "linux-block@vger.kernel.org" <linux-block@vger.kernel.org>,
Ming Lei <ming.lei@redhat.com>
Subject: Re: [PATCH] blk-mq: fix corruption with direct issue
Date: Tue, 4 Dec 2018 19:25:05 -0700 [thread overview]
Message-ID: <7aa746ff-58ab-e0e9-7058-3086a7f19c47@kernel.dk> (raw)
In-Reply-To: <20181205013821.GA19605@roeck-us.net>
On 12/4/18 6:38 PM, Guenter Roeck wrote:
> On Tue, Dec 04, 2018 at 03:47:46PM -0700, Jens Axboe wrote:
>> If we attempt a direct issue to a SCSI device, and it returns BUSY, then
>> we queue the request up normally. However, the SCSI layer may have
>> already setup SG tables etc for this particular command. If we later
>> merge with this request, then the old tables are no longer valid. Once
>> we issue the IO, we only read/write the original part of the request,
>> not the new state of it.
>>
>> This causes data corruption, and is most often noticed with the file
>> system complaining about the just read data being invalid:
>>
>> [ 235.934465] EXT4-fs error (device sda1): ext4_iget:4831: inode #7142: comm dpkg-query: bad extra_isize 24937 (inode size 256)
>>
>> because most of it is garbage...
>>
>> This doesn't happen from the normal issue path, as we will simply defer
>> the request to the hardware queue dispatch list if we fail. Once it's on
>> the dispatch list, we never merge with it.
>>
>> Fix this from the direct issue path by flagging the request as
>> REQ_NOMERGE so we don't change the size of it before issue.
>>
>> See also:
>> https://bugzilla.kernel.org/show_bug.cgi?id=201685
>>
>> Fixes: 6ce3dd6eec1 ("blk-mq: issue directly if hw queue isn't busy in case of 'none'")
>> Signed-off-by: Jens Axboe <axboe@kernel.dk>
>
> Tested-by: Guenter Roeck <linux@roeck-us.net>
>
> ... on two systems affected by the problem.
Thanks for testing! And for being persistent in reproducing and
providing clues for getting this nailed.
--
Jens Axboe
next prev parent reply other threads:[~2018-12-05 2:25 UTC|newest]
Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-12-04 22:47 [PATCH] blk-mq: fix corruption with direct issue Jens Axboe
2018-12-05 1:37 ` Ming Lei
2018-12-05 2:16 ` Jens Axboe
2018-12-05 2:23 ` Jens Axboe
2018-12-05 2:27 ` Ming Lei
2018-12-05 2:30 ` Jens Axboe
2018-12-05 2:58 ` Ming Lei
2018-12-05 3:03 ` Ming Lei
2018-12-05 3:05 ` Jens Axboe
2018-12-07 2:46 ` Theodore Y. Ts'o
2018-12-07 3:04 ` Jens Axboe
2018-12-07 3:44 ` Ming Lei
2018-12-07 9:30 ` Ming Lei
2018-12-05 3:04 ` Jens Axboe
2018-12-05 1:38 ` Guenter Roeck
2018-12-05 2:25 ` Jens Axboe [this message]
2018-12-05 17:55 ` Guenter Roeck
2018-12-05 17:59 ` Jens Axboe
2018-12-05 19:09 ` Guenter Roeck
2018-12-05 20:11 ` Jens Axboe
2018-12-05 14:41 ` Christoph Hellwig
2018-12-05 15:15 ` Jens Axboe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=7aa746ff-58ab-e0e9-7058-3086a7f19c47@kernel.dk \
--to=axboe@kernel.dk \
--cc=linux-block@vger.kernel.org \
--cc=linux@roeck-us.net \
--cc=ming.lei@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox