From: "J. Hart" <jfhart085@gmail.com>
To: Christoph Hellwig <hch@lst.de>
Cc: linux-nvme@lists.infradead.org, kbusch@kernel.org, axboe@fb.com,
sagi@grimberg.me, jfhart085@gmail.com
Subject: Re: nvme nvme0: I/O 0 (I/O Cmd) QID 1 timeout, aborting, source drive corruption observed
Date: Thu, 15 Dec 2022 18:07:32 +0900 [thread overview]
Message-ID: <d4c4eea9-e573-df0c-cca5-536bcff6c0a9@gmail.com> (raw)
In-Reply-To: <20221215082344.GB3816@lst.de>
On 12/15/22 5:23 PM, Christoph Hellwig wrote:
>>
>> dmesg reports the following:
>
> nvme0 is the destination driver I guess?
That is correct. The device /dev/nvme0n1p3 is the destination partition
on the NVME drive, and was mounted at /mnt/root_new.
>> [Dec14 19:24] nvme nvme0: I/O 0 (I/O Cmd) QID 1 timeout, aborting
>
> Can you enable CONFIG_NVME_VERBOSE_ERRORS so that we can see what
> commands are hanging?
I will do this and run the requisite testing right away. I'll reply
with the results as soon as I have them.
>> I have also observed file system corruption on the source drive of the
>> transfer. I would not normally think this to be related, except that after
>> the first time I observed it, I made certain that I corrected the file
>> content before any additional attempts, but have seen this again after
>> every attempt. The modification dates and file sizes did not change, but
>> the file content on the source drive did. I confirmed this using the
>> "diff" utility, and again using a rsync dry run with the check sum test
>> enabled.
I should also note that I did a third test using md5sum and confirmed
that the sums obtained thereby were different.
> Ok, that's really odd. The only way I could think of that happening
> is if the driver does stay DMAs, which would be really grave.
My apologies....I am not sure what is meant by "stay DMAs". Is there
something I can look for here ?
> Do you have CONFIG_INTEL_IOMMU and CONFIG_INTEL_IOMMU_DEFAULT_ON enabled?
> If not, it would be good to enable those to see if the iommu catches
> any stray DMAs.
I will enable these as well and reply with the test results. Thanks
very much for your generous assistance.
J. Hart
next prev parent reply other threads:[~2022-12-15 9:07 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-12-15 1:38 nvme nvme0: I/O 0 (I/O Cmd) QID 1 timeout, aborting, source drive corruption observed J. Hart
2022-12-15 8:23 ` Christoph Hellwig
2022-12-15 9:07 ` J. Hart [this message]
2022-12-15 9:09 ` Christoph Hellwig
2022-12-15 9:15 ` J. Hart
2022-12-15 13:33 ` J. Hart
2022-12-15 17:34 ` Keith Busch
2022-12-15 22:30 ` J. Hart
2022-12-16 6:39 ` Christoph Hellwig
2022-12-16 19:08 ` Keith Busch
2023-01-18 10:27 ` Mark Ruijter
2022-12-16 23:16 ` Keith Busch
2022-12-17 1:28 ` J. Hart
2022-12-19 14:41 ` Keith Busch
2022-12-20 1:10 ` J. Hart
2022-12-20 16:56 ` Keith Busch
2022-12-21 7:50 ` Christoph Hellwig
-- strict thread matches above, loose matches on Subject: below --
2022-12-17 12:07 J. Hart
2022-12-17 15:07 J. Hart
2022-12-17 16:14 J. Hart
2022-12-17 21:57 J. Hart
2022-12-18 6:20 J. Hart
2022-12-18 12:08 J. Hart
2022-12-19 14:45 ` Keith Busch
2022-12-19 23:40 ` J. Hart
2022-12-20 18:10 ` Keith Busch
2022-12-20 14:04 ` J. Hart
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=d4c4eea9-e573-df0c-cca5-536bcff6c0a9@gmail.com \
--to=jfhart085@gmail.com \
--cc=axboe@fb.com \
--cc=hch@lst.de \
--cc=kbusch@kernel.org \
--cc=linux-nvme@lists.infradead.org \
--cc=sagi@grimberg.me \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox