Re: [PATCH] nvme-pci: slimmer CQ head update

From: John Garry <john.garry@huawei.com>
To: Keith Busch <kbusch@kernel.org>
Cc: sagi@grimberg.me, Robin Murphy <robin.murphy@arm.com>,
	linux-nvme@lists.infradead.org,
	Alexey Dobriyan <adobriyan@gmail.com>,
	axboe@fb.com, Will Deacon <will@kernel.org>,
	Christoph Hellwig <hch@lst.de>
Subject: Re: [PATCH] nvme-pci: slimmer CQ head update
Date: Thu, 7 May 2020 16:41:01 +0100	[thread overview]
Message-ID: <6b573185-c6a5-501c-7db2-befacefa3701@huawei.com> (raw)
In-Reply-To: <20200507153506.GA2621480@dhcp-10-100-145-180.wdl.wdc.com>

>>
>> Or maybe exacerbating some other fault?
>>
>>   Do you see this with different nvme controllers?
>>
>> I only have 3x, and they are all ES3000 V3 NVMe PCIe SSD
>>
>>> Does your
>>> controller write the phase bit before writing the command id in the cqe?
>>
>> I don't know. Is that sort of info available from nvme-cli?
> 
> No, the only way to 100% confirm is with bus protocol analyzers. It's
> a protocol violation if a controller was behaving that way. We've seen
> devices broken like that before, though it's been a while since I've
> seen such behvaior.

I have an FAE contact, who I can ask. That may take a few days.

> 
>> [  885.344575] WARNING: CPU: 41 PID: 4565 at block/blk-mq.c:665  blk_mq_start_request+0xc4/0xcc
> 
> This warning appears to support my suspicion: the completion side is
> observing a new phase with a stale command id, and that command id was
> reallocated as a new request that we're still constructing at the time
> the double-completion occured.
> 
> Host software is supposed to be guaranteed the entire CQE is written
> once we see an updated phase, per spec: "If a Completion Queue
> Entry is constructed via multiple writes, the Phase Tag bit shall be
> updated in the last write of that Completion Queue Entry."

Thanks,
John

_______________________________________________
linux-nvme mailing list
linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme