Re: [PATCH 20/20] iscsi: support for swapping over iSCSI.

[Mike Christie <michaelc@cs.wisc.edu>]

Peter Zijlstra wrote:
> On Thu, 2006-09-14 at 14:22 -0500, Mike Christie wrote:
>> Peter Zijlstra wrote:
>>> On Wed, 2006-09-13 at 15:50 -0500, Mike Christie wrote:
>>>> Peter Zijlstra wrote:
>>>>> Implement sht->swapdev() for iSCSI. This method takes care of reserving
>>>>> the extra memory needed and marking all relevant sockets with SOCK_VMIO.
>>>>>
>>>>> When used for swapping, TCP socket creation is done under GFP_MEMALLOC and
>>>>> the TCP connect is done with SOCK_VMIO to ensure their success. Also the
>>>>> netlink userspace interface is marked SOCK_VMIO, this will ensure that even
>>>>> under pressure we can still communicate with the daemon (which runs as
>>>>> mlockall() and needs no additional memory to operate).
>>>>>
>>>>> Netlink requests are handled under the new PF_MEM_NOWAIT when a swapper is
>>>>> present. This ensures that the netlink socket will not block. User-space will
>>>>> need to retry failed requests.
>>>>>
>>>>> The TCP receive path is handled under PF_MEMALLOC for SOCK_VMIO sockets.
>>>>> This makes sure we do not block the critical socket, and that we do not
>>>>> fail to process incomming data.
>>>>>
>>>>> Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
>>>>> CC: Mike Christie <michaelc@cs.wisc.edu>
>>>>> ---
>>>>>  drivers/scsi/iscsi_tcp.c            |  103 +++++++++++++++++++++++++++++++-----
>>>>>  drivers/scsi/scsi_transport_iscsi.c |   23 +++++++-
>>>>>  include/scsi/libiscsi.h             |    1 
>>>>>  include/scsi/scsi_transport_iscsi.h |    2 
>>>>>  4 files changed, 113 insertions(+), 16 deletions(-)
>>>>>
>>>>> Index: linux-2.6/drivers/scsi/iscsi_tcp.c
>>>>> ===================================================================
>>>>> --- linux-2.6.orig/drivers/scsi/iscsi_tcp.c
>>>>> +++ linux-2.6/drivers/scsi/iscsi_tcp.c
>>>>> @@ -42,6 +42,7 @@
>>>>>  #include <scsi/scsi_host.h>
>>>>>  #include <scsi/scsi.h>
>>>>>  #include <scsi/scsi_transport_iscsi.h>
>>>>> +#include <scsi/scsi_device.h>
>>>>>  
>>>>>  #include "iscsi_tcp.h"
>>>>>  
>>>>> @@ -845,9 +846,13 @@ iscsi_tcp_data_recv(read_descriptor_t *r
>>>>>  	int rc;
>>>>>  	struct iscsi_conn *conn = rd_desc->arg.data;
>>>>>  	struct iscsi_tcp_conn *tcp_conn = conn->dd_data;
>>>>> -	int processed;
>>>>> +	int processed = 0;
>>>>>  	char pad[ISCSI_PAD_LEN];
>>>>>  	struct scatterlist sg;
>>>>> +	unsigned long pflags = current->flags;
>>>>> +
>>>>> +	if (sk_has_vmio(tcp_conn->sock->sk))
>>>>> +		current->flags |= PF_MEMALLOC;
>>>>>  
>>>> Is this too late or not needed or what is it for? This function gets run
>>>> from the network layer's softirq and at this point we have a skbuff with
>>>> data that we want to process. The iscsi layer also does not allocate
>>>> memory for read or write IO in this path.
>>> I thought I found allocations in that path, lemme search...
>>> found this:
>>>
>>> iscsi_tcp_data_recv()
>>>   iscsi_data_rescv()
>>>     iscsi_complete_pdu()
>>>       __iscsi_complete_pdu()
>>>         iscsi_recv_pdu()
>>>           alloc_skb( GFP_ATOMIC);
>>>
>> You are right that is for the netlink interface. Could we move the
>> PF_MEMALLOC setting and clearing to iscsi_recv_pdu and and add it to
>> iscsi_conn_error in scsi_transport_iscsi.c so that iscsi_iser and
>> qla4xxx will have it set when they need it. I will send a patch for this
>> along with a way to have the netlink sock vmio set for all iscsi drivers
>> that need it.
> 
> I already have such a patch, look at:
> http://programming.kicks-ass.net/kernel-patches/vm_deadlock/current/iscsi_vmio.patch
> 

You are drowning me in patches :) I did not see that one. I was still
commenting on this patch :)

The new patch looks ok.


> but what conditional do you want to use for PF_MEMALLOC, an
> unconditional setting will be highly unpopular.
> 
> Hmm, perhaps you could key it of sk_has_vmio(nls)...

Yes.