Linux CIFS filesystem development
 help / color / mirror / Atom feed
* blocking write() after disconnecting cifs server
@ 2013-12-03 13:16 mail654-Mmb7MZpHnFY
  2013-12-23 10:46 ` Jeff Layton
  0 siblings, 1 reply; 7+ messages in thread
From: mail654-Mmb7MZpHnFY @ 2013-12-03 13:16 UTC (permalink / raw)
  To: linux-cifs-u79uwXL29TY76Z2rM5mHXA

Hello,

write() from cifs kernel driver blocks when disconnecting the cifs server. The blocking call didn't return after 30 minutes. Client and server are connected via a switch and server's LAN cable is unplugged during the write call. I use kernel 3.11.8 and mounted without "hard" option.

Is there a possibility for an non-blocking write() without using O_SYNC or "directio" mount option?

Way to reproduce the scenario: Below is a sample program which calls write() in a loop. The error messages appear when unplugging the cable during this loop.

Kind regards,
Hagen

CIFS VFS: sends on sock ffff88003710c280 stuck for 15 seconds
CIFS VFS: Error -11 sending data on socket to server

#include <fstream>
#include <iostream>
int main () {
  const int size = 100000;
  char buffer[size];
  std::ofstream outfile("/mnt/new.bin",std::ofstream::binary);
  if (!outfile.is_open())
  {
    return 1;
  }
  for (int idx=0; idx<10000 && outfile.good(); idx++)
  {
    outfile.write(buffer,size);
    std::cout << "written, size=" << size << std::endl;
  }
  std::cout << "finished " << outfile.good() << std::endl;
  outfile.close();
  return 0;
}

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: blocking write() after disconnecting cifs server
  2013-12-03 13:16 blocking write() after disconnecting cifs server mail654-Mmb7MZpHnFY
@ 2013-12-23 10:46 ` Jeff Layton
       [not found]   ` <trinity-2ffa0afb-c02b-46c6-9e2e-31b209aebd8b-1388678667254@3capp-gmx-bs34>
  0 siblings, 1 reply; 7+ messages in thread
From: Jeff Layton @ 2013-12-23 10:46 UTC (permalink / raw)
  To: mail654-Mmb7MZpHnFY; +Cc: linux-cifs-u79uwXL29TY76Z2rM5mHXA

On Tue, 3 Dec 2013 14:16:08 +0100 (CET)
mail654-Mmb7MZpHnFY@public.gmane.org wrote:

> Hello,
> 
> write() from cifs kernel driver blocks when disconnecting the cifs server. The blocking call didn't return after 30 minutes. Client and server are connected via a switch and server's LAN cable is unplugged during the write call. I use kernel 3.11.8 and mounted without "hard" option.
> 
> Is there a possibility for an non-blocking write() without using O_SYNC or "directio" mount option?
> 
> Way to reproduce the scenario: Below is a sample program which calls write() in a loop. The error messages appear when unplugging the cable during this loop.
> 
> Kind regards,
> Hagen
> 
> CIFS VFS: sends on sock ffff88003710c280 stuck for 15 seconds
> CIFS VFS: Error -11 sending data on socket to server
> 
> #include <fstream>
> #include <iostream>
> int main () {
>   const int size = 100000;
>   char buffer[size];
>   std::ofstream outfile("/mnt/new.bin",std::ofstream::binary);
>   if (!outfile.is_open())
>   {
>     return 1;
>   }
>   for (int idx=0; idx<10000 && outfile.good(); idx++)
>   {
>     outfile.write(buffer,size);
>     std::cout << "written, size=" << size << std::endl;
>   }
>   std::cout << "finished " << outfile.good() << std::endl;
>   outfile.close();
>   return 0;
> }

A hang of that length is unexpected. If you're able to reproduce this,
can you get the stack from the task issuing the write at the time?

    $ cat /proc/<pid>/stack

That might give us a clue as to what it's doing.
-- 
Jeff Layton <jlayton-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org>

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: blocking write() after disconnecting cifs server
       [not found]   ` <trinity-2ffa0afb-c02b-46c6-9e2e-31b209aebd8b-1388678667254@3capp-gmx-bs34>
@ 2014-01-02 19:31     ` Jeff Layton
       [not found]       ` <20140102143157.3232068c-4QP7MXygkU+dMjc06nkz3ljfA9RmPOcC@public.gmane.org>
  0 siblings, 1 reply; 7+ messages in thread
From: Jeff Layton @ 2014-01-02 19:31 UTC (permalink / raw)
  To: mail654-Mmb7MZpHnFY; +Cc: linux-cifs-u79uwXL29TY76Z2rM5mHXA

On Thu, 2 Jan 2014 17:04:27 +0100 (CET)
mail654-Mmb7MZpHnFY@public.gmane.org wrote:

> > > write() from cifs kernel driver blocks when disconnecting the cifs server. The blocking call didn't return after 30 minutes. Client and server are connected via a switch and server's LAN cable is unplugged during the write call. I use kernel 3.11.8 and mounted without "hard" option.
> > > 
> > > Is there a possibility for an non-blocking write() without using O_SYNC or "directio" mount option?
> > > 
> > > Way to reproduce the scenario: Below is a sample program which calls write() in a loop. The error messages appear when unplugging the cable during this loop.
> > > 
> > > Kind regards,
> > > Hagen
> > > 
> > > CIFS VFS: sends on sock ffff88003710c280 stuck for 15 seconds
> > > CIFS VFS: Error -11 sending data on socket to server
> > > 
> > > #include <fstream>
> > > #include <iostream>
> > > int main () {
> > >   const int size = 100000;
> > >   char buffer[size];
> > >   std::ofstream outfile("/mnt/new.bin",std::ofstream::binary);
> > >   if (!outfile.is_open())
> > >   {
> > >     return 1;
> > >   }
> > >   for (int idx=0; idx<10000 && outfile.good(); idx++)
> > >   {
> > >     outfile.write(buffer,size);
> > >     std::cout << "written, size=" << size << std::endl;
> > >   }
> > >   std::cout << "finished " << outfile.good() << std::endl;
> > >   outfile.close();
> > >   return 0;
> > > }
> > 
> > A hang of that length is unexpected. If you're able to reproduce this,
> > can you get the stack from the task issuing the write at the time?
> > 
> >     $ cat /proc/<pid>/stack
> > 
> > That might give us a clue as to what it's doing.
> 
> [<ffffffff8170ab8c>] balance_dirty_pages.isra.19+0x4ac/0x55c
> [<ffffffff8115455b>] balance_dirty_pages_ratelimited+0xeb/0x110
> [<ffffffff81148f3a>] generic_perform_write+0x16a/0x210
> [<ffffffff8114903d>] generic_file_buffered_write+0x5d/0x90
> [<ffffffff8114aa66>] __generic_file_aio_write+0x1b6/0x3b0
> [<ffffffff8114acc9>] generic_file_aio_write+0x69/0xd0
> [<ffffffffa03ef225>] cifs_strict_writev+0xa5/0xd0 [cifs]
> [<ffffffff811b2b95>] do_sync_readv_writev+0x65/0x90
> [<ffffffff811b4312>] do_readv_writev+0xd2/0x2b0
> [<ffffffff811b452c>] vfs_writev+0x3c/0x50
> [<ffffffff811b46a2>] SyS_writev+0x52/0xc0
> [<ffffffff8172976f>] tracesys+0xe1/0xe6
> [<ffffffffffffffff>] 0xffffffffffffffff
> 

Looks like it's stuck in dirty page throttling.

What's likely happening is that you have a bunch of dirty pages when
you go to pull the cable. At that point the system is trying to flush
the pages so that this task can try to dirty more of them.

What *should* happen (at least if this is a soft mount) is that the
writeback of those pages eventually times out, the pages get their
error bit set and eventually the write() syscalls go through.

Have you tried stracing this and are able to tell that the write
syscall never returns in this situation? Is it possible that the
write() syscalls are returning, albeit slowly?

-- 
Jeff Layton <jlayton-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org>

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Aw: Re: blocking write() after disconnecting cifs server
       [not found]       ` <20140102143157.3232068c-4QP7MXygkU+dMjc06nkz3ljfA9RmPOcC@public.gmane.org>
@ 2014-01-03  9:10         ` mail654-Mmb7MZpHnFY
  2014-01-11  2:45         ` Shirish Pargaonkar
  2014-01-19  4:40         ` Shirish Pargaonkar
  2 siblings, 0 replies; 7+ messages in thread
From: mail654-Mmb7MZpHnFY @ 2014-01-03  9:10 UTC (permalink / raw)
  To: Jeff Layton; +Cc: linux-cifs-u79uwXL29TY76Z2rM5mHXA

jlayton-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org wrote:
> On Thu, 2 Jan 2014 17:04:27 +0100 (CET)
> mail654-Mmb7MZpHnFY@public.gmane.org wrote:
> 
> > > > write() from cifs kernel driver blocks when disconnecting the cifs server. The blocking call didn't return after 30 minutes. Client and server are connected via a switch and server's LAN cable is unplugged during the write call. I use kernel 3.11.8 and mounted without "hard" option.
> > > > 
> > > > Is there a possibility for an non-blocking write() without using O_SYNC or "directio" mount option?
> > > > 
> > > > Way to reproduce the scenario: Below is a sample program which calls write() in a loop. The error messages appear when unplugging the cable during this loop.
> > > > 
> > > > Kind regards,
> > > > Hagen
> > > > 
> > > > CIFS VFS: sends on sock ffff88003710c280 stuck for 15 seconds
> > > > CIFS VFS: Error -11 sending data on socket to server
> > > > 
> > > > #include <fstream>
> > > > #include <iostream>
> > > > int main () {
> > > >   const int size = 100000;
> > > >   char buffer[size];
> > > >   std::ofstream outfile("/mnt/new.bin",std::ofstream::binary);
> > > >   if (!outfile.is_open())
> > > >   {
> > > >     return 1;
> > > >   }
> > > >   for (int idx=0; idx<10000 && outfile.good(); idx++)
> > > >   {
> > > >     outfile.write(buffer,size);
> > > >     std::cout << "written, size=" << size << std::endl;
> > > >   }
> > > >   std::cout << "finished " << outfile.good() << std::endl;
> > > >   outfile.close();
> > > >   return 0;
> > > > }
> > > 
> > > A hang of that length is unexpected. If you're able to reproduce this,
> > > can you get the stack from the task issuing the write at the time?
> > > 
> > >     $ cat /proc/<pid>/stack
> > > 
> > > That might give us a clue as to what it's doing.
> > 
> > [<ffffffff8170ab8c>] balance_dirty_pages.isra.19+0x4ac/0x55c
> > [<ffffffff8115455b>] balance_dirty_pages_ratelimited+0xeb/0x110
> > [<ffffffff81148f3a>] generic_perform_write+0x16a/0x210
> > [<ffffffff8114903d>] generic_file_buffered_write+0x5d/0x90
> > [<ffffffff8114aa66>] __generic_file_aio_write+0x1b6/0x3b0
> > [<ffffffff8114acc9>] generic_file_aio_write+0x69/0xd0
> > [<ffffffffa03ef225>] cifs_strict_writev+0xa5/0xd0 [cifs]
> > [<ffffffff811b2b95>] do_sync_readv_writev+0x65/0x90
> > [<ffffffff811b4312>] do_readv_writev+0xd2/0x2b0
> > [<ffffffff811b452c>] vfs_writev+0x3c/0x50
> > [<ffffffff811b46a2>] SyS_writev+0x52/0xc0
> > [<ffffffff8172976f>] tracesys+0xe1/0xe6
> > [<ffffffffffffffff>] 0xffffffffffffffff
> > 
> 
> Looks like it's stuck in dirty page throttling.
> 
> What's likely happening is that you have a bunch of dirty pages when
> you go to pull the cable. At that point the system is trying to flush
> the pages so that this task can try to dirty more of them.
> 
> What *should* happen (at least if this is a soft mount) is that the
> writeback of those pages eventually times out, the pages get their
> error bit set and eventually the write() syscalls go through.
> 
> Have you tried stracing this and are able to tell that the write
> syscall never returns in this situation? Is it possible that the
> write() syscalls are returning, albeit slowly?

No, during several straces I've never seen a write() syscall returning after
pulling the cable.

Hagen

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: blocking write() after disconnecting cifs server
       [not found]       ` <20140102143157.3232068c-4QP7MXygkU+dMjc06nkz3ljfA9RmPOcC@public.gmane.org>
  2014-01-03  9:10         ` Aw: " mail654-Mmb7MZpHnFY
@ 2014-01-11  2:45         ` Shirish Pargaonkar
  2014-01-19  4:40         ` Shirish Pargaonkar
  2 siblings, 0 replies; 7+ messages in thread
From: Shirish Pargaonkar @ 2014-01-11  2:45 UTC (permalink / raw)
  To: Jeff Layton; +Cc: mail654-Mmb7MZpHnFY, linux-cifs

I think there are two things going on.

The flusher thread for the backing device is blocked trying to
reconnect.  So no dirty pages are getting flushed out.

Meanwhile application thread just keeps writing to the page cache.

So both threads are oblivious to each other.

Should not cifs check, somehow, that an app is dirtying too many
pages and just error out the writes and mark error bits on all dirty
pages of this app?




On Thu, Jan 2, 2014 at 1:31 PM, Jeff Layton <jlayton-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org> wrote:
> On Thu, 2 Jan 2014 17:04:27 +0100 (CET)
> mail654-Mmb7MZpHnFY@public.gmane.org wrote:
>
>> > > write() from cifs kernel driver blocks when disconnecting the cifs server. The blocking call didn't return after 30 minutes. Client and server are connected via a switch and server's LAN cable is unplugged during the write call. I use kernel 3.11.8 and mounted without "hard" option.
>> > >
>> > > Is there a possibility for an non-blocking write() without using O_SYNC or "directio" mount option?
>> > >
>> > > Way to reproduce the scenario: Below is a sample program which calls write() in a loop. The error messages appear when unplugging the cable during this loop.
>> > >
>> > > Kind regards,
>> > > Hagen
>> > >
>> > > CIFS VFS: sends on sock ffff88003710c280 stuck for 15 seconds
>> > > CIFS VFS: Error -11 sending data on socket to server
>> > >
>> > > #include <fstream>
>> > > #include <iostream>
>> > > int main () {
>> > >   const int size = 100000;
>> > >   char buffer[size];
>> > >   std::ofstream outfile("/mnt/new.bin",std::ofstream::binary);
>> > >   if (!outfile.is_open())
>> > >   {
>> > >     return 1;
>> > >   }
>> > >   for (int idx=0; idx<10000 && outfile.good(); idx++)
>> > >   {
>> > >     outfile.write(buffer,size);
>> > >     std::cout << "written, size=" << size << std::endl;
>> > >   }
>> > >   std::cout << "finished " << outfile.good() << std::endl;
>> > >   outfile.close();
>> > >   return 0;
>> > > }
>> >
>> > A hang of that length is unexpected. If you're able to reproduce this,
>> > can you get the stack from the task issuing the write at the time?
>> >
>> >     $ cat /proc/<pid>/stack
>> >
>> > That might give us a clue as to what it's doing.
>>
>> [<ffffffff8170ab8c>] balance_dirty_pages.isra.19+0x4ac/0x55c
>> [<ffffffff8115455b>] balance_dirty_pages_ratelimited+0xeb/0x110
>> [<ffffffff81148f3a>] generic_perform_write+0x16a/0x210
>> [<ffffffff8114903d>] generic_file_buffered_write+0x5d/0x90
>> [<ffffffff8114aa66>] __generic_file_aio_write+0x1b6/0x3b0
>> [<ffffffff8114acc9>] generic_file_aio_write+0x69/0xd0
>> [<ffffffffa03ef225>] cifs_strict_writev+0xa5/0xd0 [cifs]
>> [<ffffffff811b2b95>] do_sync_readv_writev+0x65/0x90
>> [<ffffffff811b4312>] do_readv_writev+0xd2/0x2b0
>> [<ffffffff811b452c>] vfs_writev+0x3c/0x50
>> [<ffffffff811b46a2>] SyS_writev+0x52/0xc0
>> [<ffffffff8172976f>] tracesys+0xe1/0xe6
>> [<ffffffffffffffff>] 0xffffffffffffffff
>>
>
> Looks like it's stuck in dirty page throttling.
>
> What's likely happening is that you have a bunch of dirty pages when
> you go to pull the cable. At that point the system is trying to flush
> the pages so that this task can try to dirty more of them.
>
> What *should* happen (at least if this is a soft mount) is that the
> writeback of those pages eventually times out, the pages get their
> error bit set and eventually the write() syscalls go through.
>
> Have you tried stracing this and are able to tell that the write
> syscall never returns in this situation? Is it possible that the
> write() syscalls are returning, albeit slowly?
>
> --
> Jeff Layton <jlayton-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-cifs" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: blocking write() after disconnecting cifs server
       [not found]       ` <20140102143157.3232068c-4QP7MXygkU+dMjc06nkz3ljfA9RmPOcC@public.gmane.org>
  2014-01-03  9:10         ` Aw: " mail654-Mmb7MZpHnFY
  2014-01-11  2:45         ` Shirish Pargaonkar
@ 2014-01-19  4:40         ` Shirish Pargaonkar
       [not found]           ` <CADT32eLOnvVO3phJTqC7TATdJuhgUXgxvAVvDZS+kmDDr-MmiA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
  2 siblings, 1 reply; 7+ messages in thread
From: Shirish Pargaonkar @ 2014-01-19  4:40 UTC (permalink / raw)
  To: Jeff Layton; +Cc: mail654-Mmb7MZpHnFY, linux-cifs

If an address_space flag has AS_EIO bit set, should the subsequent
writes should_fail/start_failing instead of them writing to the page cache?
Also not sure what happens to pages with PG_error bit set, probably
get discarded.

On Thu, Jan 2, 2014 at 1:31 PM, Jeff Layton <jlayton-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org> wrote:
> On Thu, 2 Jan 2014 17:04:27 +0100 (CET)
> mail654-Mmb7MZpHnFY@public.gmane.org wrote:
>
>> > > write() from cifs kernel driver blocks when disconnecting the cifs server. The blocking call didn't return after 30 minutes. Client and server are connected via a switch and server's LAN cable is unplugged during the write call. I use kernel 3.11.8 and mounted without "hard" option.
>> > >
>> > > Is there a possibility for an non-blocking write() without using O_SYNC or "directio" mount option?
>> > >
>> > > Way to reproduce the scenario: Below is a sample program which calls write() in a loop. The error messages appear when unplugging the cable during this loop.
>> > >
>> > > Kind regards,
>> > > Hagen
>> > >
>> > > CIFS VFS: sends on sock ffff88003710c280 stuck for 15 seconds
>> > > CIFS VFS: Error -11 sending data on socket to server
>> > >
>> > > #include <fstream>
>> > > #include <iostream>
>> > > int main () {
>> > >   const int size = 100000;
>> > >   char buffer[size];
>> > >   std::ofstream outfile("/mnt/new.bin",std::ofstream::binary);
>> > >   if (!outfile.is_open())
>> > >   {
>> > >     return 1;
>> > >   }
>> > >   for (int idx=0; idx<10000 && outfile.good(); idx++)
>> > >   {
>> > >     outfile.write(buffer,size);
>> > >     std::cout << "written, size=" << size << std::endl;
>> > >   }
>> > >   std::cout << "finished " << outfile.good() << std::endl;
>> > >   outfile.close();
>> > >   return 0;
>> > > }
>> >
>> > A hang of that length is unexpected. If you're able to reproduce this,
>> > can you get the stack from the task issuing the write at the time?
>> >
>> >     $ cat /proc/<pid>/stack
>> >
>> > That might give us a clue as to what it's doing.
>>
>> [<ffffffff8170ab8c>] balance_dirty_pages.isra.19+0x4ac/0x55c
>> [<ffffffff8115455b>] balance_dirty_pages_ratelimited+0xeb/0x110
>> [<ffffffff81148f3a>] generic_perform_write+0x16a/0x210
>> [<ffffffff8114903d>] generic_file_buffered_write+0x5d/0x90
>> [<ffffffff8114aa66>] __generic_file_aio_write+0x1b6/0x3b0
>> [<ffffffff8114acc9>] generic_file_aio_write+0x69/0xd0
>> [<ffffffffa03ef225>] cifs_strict_writev+0xa5/0xd0 [cifs]
>> [<ffffffff811b2b95>] do_sync_readv_writev+0x65/0x90
>> [<ffffffff811b4312>] do_readv_writev+0xd2/0x2b0
>> [<ffffffff811b452c>] vfs_writev+0x3c/0x50
>> [<ffffffff811b46a2>] SyS_writev+0x52/0xc0
>> [<ffffffff8172976f>] tracesys+0xe1/0xe6
>> [<ffffffffffffffff>] 0xffffffffffffffff
>>
>
> Looks like it's stuck in dirty page throttling.
>
> What's likely happening is that you have a bunch of dirty pages when
> you go to pull the cable. At that point the system is trying to flush
> the pages so that this task can try to dirty more of them.
>
> What *should* happen (at least if this is a soft mount) is that the
> writeback of those pages eventually times out, the pages get their
> error bit set and eventually the write() syscalls go through.
>
> Have you tried stracing this and are able to tell that the write
> syscall never returns in this situation? Is it possible that the
> write() syscalls are returning, albeit slowly?
>
> --
> Jeff Layton <jlayton-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-cifs" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: blocking write() after disconnecting cifs server
       [not found]           ` <CADT32eLOnvVO3phJTqC7TATdJuhgUXgxvAVvDZS+kmDDr-MmiA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
@ 2014-01-20 12:40             ` Jeff Layton
  0 siblings, 0 replies; 7+ messages in thread
From: Jeff Layton @ 2014-01-20 12:40 UTC (permalink / raw)
  To: Shirish Pargaonkar; +Cc: mail654-Mmb7MZpHnFY, linux-cifs

On Sat, 18 Jan 2014 22:40:15 -0600
Shirish Pargaonkar <shirishpargaonkar-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:

> If an address_space flag has AS_EIO bit set, should the subsequent
> writes should_fail/start_failing instead of them writing to the page cache?
> Also not sure what happens to pages with PG_error bit set, probably
> get discarded.
> 

Yeah, possibly.

What the NFS client does is switch to synchronous writes whenever there
is an issue. See nfs_need_sync_write(). Perhaps cifs should do
something similar?


> On Thu, Jan 2, 2014 at 1:31 PM, Jeff Layton <jlayton-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org> wrote:
> > On Thu, 2 Jan 2014 17:04:27 +0100 (CET)
> > mail654-Mmb7MZpHnFY@public.gmane.org wrote:
> >
> >> > > write() from cifs kernel driver blocks when disconnecting the cifs server. The blocking call didn't return after 30 minutes. Client and server are connected via a switch and server's LAN cable is unplugged during the write call. I use kernel 3.11.8 and mounted without "hard" option.
> >> > >
> >> > > Is there a possibility for an non-blocking write() without using O_SYNC or "directio" mount option?
> >> > >
> >> > > Way to reproduce the scenario: Below is a sample program which calls write() in a loop. The error messages appear when unplugging the cable during this loop.
> >> > >
> >> > > Kind regards,
> >> > > Hagen
> >> > >
> >> > > CIFS VFS: sends on sock ffff88003710c280 stuck for 15 seconds
> >> > > CIFS VFS: Error -11 sending data on socket to server
> >> > >
> >> > > #include <fstream>
> >> > > #include <iostream>
> >> > > int main () {
> >> > >   const int size = 100000;
> >> > >   char buffer[size];
> >> > >   std::ofstream outfile("/mnt/new.bin",std::ofstream::binary);
> >> > >   if (!outfile.is_open())
> >> > >   {
> >> > >     return 1;
> >> > >   }
> >> > >   for (int idx=0; idx<10000 && outfile.good(); idx++)
> >> > >   {
> >> > >     outfile.write(buffer,size);
> >> > >     std::cout << "written, size=" << size << std::endl;
> >> > >   }
> >> > >   std::cout << "finished " << outfile.good() << std::endl;
> >> > >   outfile.close();
> >> > >   return 0;
> >> > > }
> >> >
> >> > A hang of that length is unexpected. If you're able to reproduce this,
> >> > can you get the stack from the task issuing the write at the time?
> >> >
> >> >     $ cat /proc/<pid>/stack
> >> >
> >> > That might give us a clue as to what it's doing.
> >>
> >> [<ffffffff8170ab8c>] balance_dirty_pages.isra.19+0x4ac/0x55c
> >> [<ffffffff8115455b>] balance_dirty_pages_ratelimited+0xeb/0x110
> >> [<ffffffff81148f3a>] generic_perform_write+0x16a/0x210
> >> [<ffffffff8114903d>] generic_file_buffered_write+0x5d/0x90
> >> [<ffffffff8114aa66>] __generic_file_aio_write+0x1b6/0x3b0
> >> [<ffffffff8114acc9>] generic_file_aio_write+0x69/0xd0
> >> [<ffffffffa03ef225>] cifs_strict_writev+0xa5/0xd0 [cifs]
> >> [<ffffffff811b2b95>] do_sync_readv_writev+0x65/0x90
> >> [<ffffffff811b4312>] do_readv_writev+0xd2/0x2b0
> >> [<ffffffff811b452c>] vfs_writev+0x3c/0x50
> >> [<ffffffff811b46a2>] SyS_writev+0x52/0xc0
> >> [<ffffffff8172976f>] tracesys+0xe1/0xe6
> >> [<ffffffffffffffff>] 0xffffffffffffffff
> >>
> >
> > Looks like it's stuck in dirty page throttling.
> >
> > What's likely happening is that you have a bunch of dirty pages when
> > you go to pull the cable. At that point the system is trying to flush
> > the pages so that this task can try to dirty more of them.
> >
> > What *should* happen (at least if this is a soft mount) is that the
> > writeback of those pages eventually times out, the pages get their
> > error bit set and eventually the write() syscalls go through.
> >
> > Have you tried stracing this and are able to tell that the write
> > syscall never returns in this situation? Is it possible that the
> > write() syscalls are returning, albeit slowly?
> >
> > --
> > Jeff Layton <jlayton-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org>
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-cifs" in
> > the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe linux-cifs" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


-- 
Jeff Layton <jlayton-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org>

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2014-01-20 12:40 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-12-03 13:16 blocking write() after disconnecting cifs server mail654-Mmb7MZpHnFY
2013-12-23 10:46 ` Jeff Layton
     [not found]   ` <trinity-2ffa0afb-c02b-46c6-9e2e-31b209aebd8b-1388678667254@3capp-gmx-bs34>
2014-01-02 19:31     ` Jeff Layton
     [not found]       ` <20140102143157.3232068c-4QP7MXygkU+dMjc06nkz3ljfA9RmPOcC@public.gmane.org>
2014-01-03  9:10         ` Aw: " mail654-Mmb7MZpHnFY
2014-01-11  2:45         ` Shirish Pargaonkar
2014-01-19  4:40         ` Shirish Pargaonkar
     [not found]           ` <CADT32eLOnvVO3phJTqC7TATdJuhgUXgxvAVvDZS+kmDDr-MmiA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2014-01-20 12:40             ` Jeff Layton

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox