Re: reservation errors during fstests on pNFS block

[Chuck Lever III <chuck.lever@oracle.com>]

> On Jun 14, 2024, at 2:33 PM, Chuck Lever III <chuck.lever@oracle.com> wrote:
> 
>> On Jun 14, 2024, at 2:26 PM, Christoph Hellwig <hch@infradead.org> wrote:
>> 
>> On Fri, Jun 14, 2024 at 05:46:21PM +0000, Chuck Lever III wrote:
>>> 
>>> I can go back and try reproducing with just generic/069 and
>>> tcpdump as a first step. Is there a way I can tell that the
>>> PR errors are not reporting a possible data corruption?
>> 
>> xfstests in general does data verifycation to check for data integrity,
>> so we should not rely on kernel messages.
>> 
>> I'm a bit busy right now, but I'll try to reproduce this locally next
>> week.
> 
> Thanks, I'll also try to investigate further.

This is 100% reproducible in my set up.

bl_alloc_lseg() calls this:

 561 static struct nfs4_deviceid_node *
 562 bl_find_get_deviceid(struct nfs_server *server,
 563                 const struct nfs4_deviceid *id, const struct cred *cred,
 564                 gfp_t gfp_mask)
 565 { 
 566         struct nfs4_deviceid_node *node;
 567         unsigned long start, end;
 568
 569 retry:
 570         node = nfs4_find_get_deviceid(server, id, cred, gfp_mask);
 571         if (!node)
 572                 return ERR_PTR(-ENODEV);

nfs4_find_get_deviceid() tries to be clever and do a lookup without
the spin lock first.

If it can't find a matching deviceid, it creates a new device_info
(which calls bl_alloc_deviceid_node, and that registers the device's
PR key).

Then it takes the nfs4_deviceid_lock and looks up the deviceid again.
If it finds it this time, bl_find_get_deviceid() frees the spare
(new) device_info, which unregisters the PR key for the same device.

Any subsequent I/O from this client on that device gets EBADE.

The umount later unregisters the device's PR key again.

Seems like PR key registration should be done from a more
idempotent context...?


--
Chuck Lever