Re: PROBLEM: nfs I/O errors with sqlite applications

[Lutz Vieweg <lvml@5t9.de>]

[-- Attachment #1: Type: text/plain, Size: 2011 bytes --]

On 06/07/2017 05:08 AM, NeilBrown wrote:
>>>   fcntl(3, F_SETLK, {type=F_RDLCK, whence=SEEK_SET, start=1073741824, len=1}) = -1 EIO (Input/output error)
>>>   write(2, "Error: disk I/O error\n", 22Error: disk I/O error
>>
>> But unlike the original reporter, we use the NFS v3 protocol:
>>> myserver:/data on /data type nfs (rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,soft,proto=tcp,timeo=600,retrans=2,sec=sys,mountvers=3,mountport=20048,mountproto=udp,local_lock=none)
>
> Using "soft" is not a good idea.  It could be the cause, but it isn't very
> likely if NFS is otherwise working OK.

NFS v3 has been working very well for us for many years.
When we upgraded those two servers ~3 years ago, we did try NFS v4 first, but
that had caused frequent occurences of "un-killable processes in D state",
so we had to revert to v3 to allow for stable operation.

> It might help to run
>    rpcdebug -m nfs -s all; rpcdebug -m nlm -s all ;rpcdebug -m rpc -s all
>    #repeat your test
>    rpcdebug -m nfs -c all; rpcdebug -m nlm -c all ;rpcdebug -m rpc -c all
>
> then collect the kernel logs (possibly just run "dmesg") and post all
> the messages which happened at that time.

Ok, attaching a log generated like this while running:

sqlite3 x.sqlite "PRAGMA case_sensitive_like=1;PRAGMA synchronous=OFF;PRAGMA 
recursive_triggers=ON;PRAGMA foreign_keys=OFF;PRAGMA locking_mode = NORMAL;PRAGMA journal_mode = 
TRUNCATE;"

> It might also help to find the port number that lockd is running on
>     rpcinfo -p $SERVER | grep 'tcp.*nlockmgr'

None of the ports reported this way contains the string "nlockmgr":
> rpcinfo -p myserver
>    program vers proto   port  service
>     100000    4   tcp    111  portmapper
>     100000    3   tcp    111  portmapper
>     100000    2   tcp    111  portmapper
>     100000    4   udp    111  portmapper
>     100000    3   udp    111  portmapper
>     100000    2   udp    111  portmapper

Regards,

Lutz Vieweg

[-- Attachment #2: log.txt.xz --]
[-- Type: application/x-xz, Size: 7544 bytes --]