Re: tcp_sendpage and page allocation lifetime vs. iscsi

[Avi Kivity <avi@argo.co.il>]

Olivier Galibert wrote:

>I have a problem with the iscsi driver (both 4.x and 5.x) and scsi
>tape I'm not sure how to solve.  It may linked to some specific
>characteristics of the tg3 network driver.
>
>What happens is, from what I can trace:
>1- st alloc_pages a bunch of pages for buffering
>
>2- st sends a bunch of them to iscsi for writing (32K is common when
>     labelling a tape for instance)
>
>3- iscsi sends whatever header is needed followed by the data using
>     tcp_sendpage
>
>4- tcp_sendpage copies from of the pages but get_page() others,
>     probably depending on the state of the socket buffer.  It returns
>     immediatly anyway, leaving some pages with an elevated count (which, I
>     guess, it will eventually decrement again)
>
>5- iscsi returns to st
>
>6- st reuses the buffer immediatly, and/or frees it if the device is
>     closed.  Silent corruption in one case, bad_page in __free_page_ok
>     called from normalize_buffer in the other.
>
>I'm going to complete my traces to be sure that's really what's going
>on (I don't have a log immediatly after sendpage yet).  But in any
>case, what would the solution be?
>
>  
>
you need a completion to tell you when your buffer has been sent. you 
can use the kiocb parameter to tcp_sendmsg, as it has a completion. 
however, tcp_sendmsg does not appear to use it.

in effect, you need tcp aio, but the mainline kernel does not support it 
yet.

-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.