Re: [BUG KERNEL 2.6.20-rc1] ftp: get or put stops during file-transfer

Re: [BUG KERNEL 2.6.20-rc1] ftp: get or put stops during file-transfer

[Adrian Bunk]

On Sun, Dec 17, 2006 at 11:23:11PM +0900, Komuro wrote:
> On Sun, 17 Dec 2006 04:02:22 +0000
> Al Viro <viro@ftp.linux.org.uk> wrote:
> 
> > On Sun, Dec 17, 2006 at 09:27:52PM +0900, Komuro wrote:
> > > 
> > > Hello,
> > > 
> > > On kernel 2.6.20-rc1, ftp (get or put) stops
> > > during file-transfer.
> > > 
> > > Client: ftp-0.17-33.fc6  (192.168.1.1)
> > > Server: vsftpd-2.0.5-8   (192.168.1.3)
> > > 
> > > This problem does _not_ happen on kernel-2.6.19.
> > > is it caused by network-subsystem change on 2.6.20-rc1??
> > 
> > Do you have NAT between you and server?
> 
> No. I don't have NAT between the client and the server.
> Actually, the client and the sever is located in same room.
> 
> client -- 100MbpsHub -- server.

What network cards are in the client and the server?

Are there any error messages your client gives or in the log files?

> Thanks!
> 
> Best Regards
> Komuro

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: [BUG KERNEL 2.6.20-rc1] ftp: get or put stops during file-transfer

[Komuro]

> 
> What network cards are in the client and the server?

DL10022-based pcmcia network card(both client and server)
The driver name is pcnet_cs.
 
> Are there any error messages your client gives or in the log files?

no error messages.

I capture the packet of ftp transfer by ethereal.
I found the malformed packet when it stops.

I will investigate it further.

Thanks!

Best Regards
Komuro

Re: [BUG KERNEL 2.6.20-rc1] ftp: get or put stops during file-transfer

[Komuro]

Hi,

I investigated the ftp-file-transfer-stop problem by git-bisect method,
and found this problem was introduced by
"[TCP]: MD5 Signature Option (RFC2385) support" patch.

Mr.YOSHIFUJI san, please fix this problem.

>commit cfb6eeb4c860592edd123fdea908d23c6ad1c7dc
>Author: YOSHIFUJI Hideaki <yoshfuji@linux-ipv6.org>
>Date:   Tue Nov 14 19:07:45 2006 -0800
>
>    [TCP]: MD5 Signature Option (RFC2385) support.
>    
>    Based on implementation by Rick Payne.
>    
>    Signed-off-by: YOSHIFUJI Hideaki <yoshfuji@linux-ipv6.org>
>    Signed-off-by: David S. Miller <davem@davemloft.net>

Best Regards
Komuro


> On Sun, Dec 17, 2006 at 11:23:11PM +0900, Komuro wrote:
> > On Sun, 17 Dec 2006 04:02:22 +0000
> > Al Viro <viro@ftp.linux.org.uk> wrote:
> > 
> > > On Sun, Dec 17, 2006 at 09:27:52PM +0900, Komuro wrote:
> > > > 
> > > > Hello,
> > > > 
> > > > On kernel 2.6.20-rc1, ftp (get or put) stops
> > > > during file-transfer.
> > > > 
> > > > Client: ftp-0.17-33.fc6  (192.168.1.1)
> > > > Server: vsftpd-2.0.5-8   (192.168.1.3)
> > > > 
> > > > This problem does _not_ happen on kernel-2.6.19.
> > > > is it caused by network-subsystem change on 2.6.20-rc1??
> > > 
> > > Do you have NAT between you and server?
> > 
> > No. I don't have NAT between the client and the server.
> > Actually, the client and the sever is located in same room.
> > 
> > client -- 100MbpsHub -- server.

Re: [BUG KERNEL 2.6.20-rc1] ftp: get or put stops during file-transfer

[YOSHIFUJI Hideaki / 吉藤英明]

In article <20061230185043.d31d2104.komurojun-mbn@nifty.com> (at Sat, 30 Dec 2006 18:50:43 +0900), Komuro <komurojun-mbn@nifty.com> says:

> I investigated the ftp-file-transfer-stop problem by git-bisect method,
> and found this problem was introduced by
> "[TCP]: MD5 Signature Option (RFC2385) support" patch.
> 
> Mr.YOSHIFUJI san, please fix this problem.

Hmm, have you try disabling CONFIG_TCP_MD5SIG?
(Is it already disabled?)

Are there any specific size of transfer to reproduce this?
Do you see similar issue with other simple application?

--yoshfuji

Re: [BUG KERNEL 2.6.20-rc1] ftp: get or put stops during file-transfer

[Komuro]

> 
> > I investigated the ftp-file-transfer-stop problem by git-bisect method,
> > and found this problem was introduced by
> > "[TCP]: MD5 Signature Option (RFC2385) support" patch.
> > 
> > Mr.YOSHIFUJI san, please fix this problem.
> 
> Hmm, have you try disabling CONFIG_TCP_MD5SIG?
> (Is it already disabled?)

This problem happens both CONFIG_TCP_MD5SIG is disabled and enabled.

> Are there any specific size of transfer to reproduce this?

When I do ftp 40Mbytes file for 5-times or more,
 this problem happens.


> Do you see similar issue with other simple application?

sorry, I don't reproduce this problem on other application.

Thanks,

Best Regards
Komuro.

Re: [BUG KERNEL 2.6.20-rc1] ftp: get or put stops during file-transfer

[YOSHIFUJI Hideaki / 吉藤英明]

In article <20061230205931.9e430173.komurojun-mbn@nifty.com> (at Sat, 30 Dec 2006 20:59:31 +0900), Komuro <komurojun-mbn@nifty.com> says:

> > Do you see similar issue with other simple application?
> 
> sorry, I don't reproduce this problem on other application.

Can you reproduce it with other ftp client and/or server?


Anyway...

Please provide the output of "netstat -na" command during the
transfer, and the output of "lsmod | grep conntrack" (just for
sure).


More questions:

What kind of mode do you use? e.g. PORT/EPRT/LPRT/PASV/EPSV/LPSV

When the transfer get stuck, are other communication still working?

Are there any workaround?
e.g. stop-start vsftpd cycle, ifdown-ifup cycle, rmmod/insmod cycle etc.


Regards,

--yoshfuji

Re: [BUG KERNEL 2.6.20-rc1] ftp: get or put stops during file-transfer

[Komuro]

> Can you reproduce it with other ftp client and/or server?
 
O.K. I wiil try to test other ftp client and server.
 

> Please provide the output of "netstat -na" command during the
> transfer, and the output of "lsmod | grep conntrack" (just for
> sure).

Please see the output of "netstat -na" when stuck. (below)

CONFIG_NETFILER is diabled in my test configuration
,conntrack modules is not loaded.
(CONFIG_IP_DCCP, CONFIG_IP_SCTP, CONFIG_TIPC, CONFIG_IPV6 is
also disabled) 

 
> What kind of mode do you use? e.g. PORT/EPRT/LPRT/PASV/EPSV/LPSV

PASV mode.
 
> When the transfer get stuck, are other communication still working?

Other communication works properly.
Actually, I can start other ftp session on other console of the same PC.

> Are there any workaround?
> e.g. stop-start vsftpd cycle, ifdown-ifup cycle, rmmod/insmod cycle etc.

I only need to do the killall command.


>> output of netstat -na

Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address               Foreign Address             State      
tcp        0      0 0.0.0.0:873                 0.0.0.0:*                   LISTEN      
tcp        0      0 0.0.0.0:111                 0.0.0.0:*                   LISTEN      
tcp        0      0 192.168.0.6:35737           192.168.0.2:26827           TIME_WAIT   
tcp        0      0 192.168.0.6:51036           192.168.0.2:21              ESTABLISHED 
udp        0      0 0.0.0.0:68                  0.0.0.0:*                               
udp        0      0 0.0.0.0:867                 0.0.0.0:*                               
udp        0      0 0.0.0.0:870                 0.0.0.0:*                               
udp        0      0 0.0.0.0:111                 0.0.0.0:*                               
Active UNIX domain sockets (servers and established)
Proto RefCnt Flags       Type       State         I-Node Path
unix  2      [ ACC ]     STREAM     LISTENING     5056   /tmp/.font-unix/fs7100
unix  2      [ ]         DGRAM                    1234   @/org/kernel/udev/udevd
unix  5      [ ]         DGRAM                    4748   /dev/log
unix  2      [ ACC ]     STREAM     LISTENING     4917   /var/run/dbus/system_bus_socket
unix  2      [ ACC ]     STREAM     LISTENING     4989   /var/run/acpid.socket
unix  2      [ ]         DGRAM                    5390   
unix  3      [ ]         STREAM     CONNECTED     4920   
unix  3      [ ]         STREAM     CONNECTED     4919   
unix  2      [ ]         DGRAM                    4866   
unix  2      [ ]         DGRAM                    4756   



Thanks,

Best Regards
Komuro

Re: [BUG KERNEL 2.6.20-rc1] ftp: get or put stops during file-transfer

[Komuro]

> 
> Can you reproduce it with other ftp client and/or server?

I tried the proftpd-1.3.0a-1.fc6(kernel version is 2.6.19).
The ftp stop problem does not happen.

Therefore, this problem is reproduced when
client's kernel-version is 2.6.20-rc1 or later
and server is vsftpd.
Server's kernel-version is not related with this problem.

The ftp-stop-problem happens on client's PC.

Please advise.

Best Regards
Komuro



 

Re: [BUG KERNEL 2.6.20-rc1] ftp: get or put stops during file-transfer

[Komuro]

Hi,

I made a patch below.
With this patch, the ftp-transfer-stop problem does not happen.
Therefore, I think this is not a problem of vsftpd.

Mr.YOSHIFUJI san, why did you set TCPOLEN_TSTAMP_ALIGNED
to iov_len?



--- linux-2.6.20-rc3/net/ipv4/tcp_ipv4.c.orig	2007-01-03 11:50:04.000000000 +0900
+++ linux-2.6.20-rc3/net/ipv4/tcp_ipv4.c	2007-01-03 15:30:44.000000000 +0900
@@ -648,7 +648,7 @@ static void tcp_v4_send_ack(struct tcp_t
 				   TCPOLEN_TIMESTAMP);
 		rep.opt[1] = htonl(tcp_time_stamp);
 		rep.opt[2] = htonl(ts);
-		arg.iov[0].iov_len = TCPOLEN_TSTAMP_ALIGNED;
+		arg.iov[0].iov_len = sizeof(rep);
 	}
 
 	/* Swap the send and the receive. */


Best Regards
Komuro

Re: [BUG KERNEL 2.6.20-rc1] ftp: get or put stops during file-transfer

[Craig Schlenter]

On Fri, Jan 05, 2007 at 05:45:46AM +0900, Komuro wrote:
> Hi,
> 
> I made a patch below.
> With this patch, the ftp-transfer-stop problem does not happen.
> Therefore, I think this is not a problem of vsftpd.
> 
> Mr.YOSHIFUJI san, why did you set TCPOLEN_TSTAMP_ALIGNED
> to iov_len?
> 
> 
> 
> --- linux-2.6.20-rc3/net/ipv4/tcp_ipv4.c.orig	2007-01-03 11:50:04.000000000 +0900
> +++ linux-2.6.20-rc3/net/ipv4/tcp_ipv4.c	2007-01-03 15:30:44.000000000 +0900
> @@ -648,7 +648,7 @@ static void tcp_v4_send_ack(struct tcp_t
>  				   TCPOLEN_TIMESTAMP);
>  		rep.opt[1] = htonl(tcp_time_stamp);
>  		rep.opt[2] = htonl(ts);
> -		arg.iov[0].iov_len = TCPOLEN_TSTAMP_ALIGNED;
> +		arg.iov[0].iov_len = sizeof(rep);

Perhaps this was supposed to be
                arg.iov[0].iov_len += TCPOLEN_TSTAMP_ALIGNED;

That's what the ipv6 stuff does in places.

bye,

--Craig

Re: [BUG KERNEL 2.6.20-rc1] ftp: get or put stops during file-transfer

[YOSHIFUJI Hideaki / 吉藤英明]

In article <20070104122330.GA2233@craigdell.detnet.com> (at Thu, 4 Jan 2007 14:23:30 +0200), Craig Schlenter <craig@codefountain.com> says:

> On Fri, Jan 05, 2007 at 05:45:46AM +0900, Komuro wrote:
> > Hi,
> > 
> > I made a patch below.
> > With this patch, the ftp-transfer-stop problem does not happen.
> > Therefore, I think this is not a problem of vsftpd.
> > 
> > Mr.YOSHIFUJI san, why did you set TCPOLEN_TSTAMP_ALIGNED
> > to iov_len?
> > 
> > 
> > 
> > --- linux-2.6.20-rc3/net/ipv4/tcp_ipv4.c.orig	2007-01-03 11:50:04.000000000 +0900
> > +++ linux-2.6.20-rc3/net/ipv4/tcp_ipv4.c	2007-01-03 15:30:44.000000000 +0900
> > @@ -648,7 +648,7 @@ static void tcp_v4_send_ack(struct tcp_t
> >  				   TCPOLEN_TIMESTAMP);
> >  		rep.opt[1] = htonl(tcp_time_stamp);
> >  		rep.opt[2] = htonl(ts);
> > -		arg.iov[0].iov_len = TCPOLEN_TSTAMP_ALIGNED;
> > +		arg.iov[0].iov_len = sizeof(rep);
> 
> Perhaps this was supposed to be
>                 arg.iov[0].iov_len += TCPOLEN_TSTAMP_ALIGNED;
> 
> That's what the ipv6 stuff does in places.

Good catch! I agree.
Craig, please provide a patch for us, please.

Thank you again.

--yoshfuji

[PATCH] Re: [BUG KERNEL 2.6.20-rc1] ftp: get or put stops during file-transfer

[Craig Schlenter]

Hi Dave

YOSHIFUJI Hideaki / 吉藤英明 has suggested that I send the patch
below to fix the ftp stalls present in the current kernels.

All credit goes to Komuro <komurojun-mbn@nifty.com> for tracking
this down. The patch is untested but it looks *cough* obviously
correct.

Signed-off-by: Craig Schlenter <craig@codefountain.com>

Thank you!

--Craig

diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c
index bf7a224..12de90a 100644
--- a/net/ipv4/tcp_ipv4.c
+++ b/net/ipv4/tcp_ipv4.c
@@ -648,7 +648,7 @@ static void tcp_v4_send_ack(struct tcp_timewait_sock *twsk,
 				   TCPOLEN_TIMESTAMP);
 		rep.opt[1] = htonl(tcp_time_stamp);
 		rep.opt[2] = htonl(ts);
-		arg.iov[0].iov_len = TCPOLEN_TSTAMP_ALIGNED;
+		arg.iov[0].iov_len += TCPOLEN_TSTAMP_ALIGNED;
 	}
 
 	/* Swap the send and the receive. */

Re: [PATCH] Re: [BUG KERNEL 2.6.20-rc1] ftp: get or put stops during file-transfer

[YOSHIFUJI Hideaki / 吉藤英明]

[-- Attachment #1: Type: Text/Plain, Size: 438 bytes --]

Dave, please apply.  Thank you.

In article <20070109051139.GA2229@craigdell.detnet.com> (at Tue, 9 Jan 2007 07:11:39 +0200), Craig Schlenter <craig@codefountain.com> says:

> All credit goes to Komuro <komurojun-mbn@nifty.com> for tracking
> this down. The patch is untested but it looks *cough* obviously
> correct.
> 
> Signed-off-by: Craig Schlenter <craig@codefountain.com>
Signed-off-by: YOSHIFUJI Hideaki <yoshfuji@linux-ipv6.org>

[-- Attachment #2: Type: Message/Rfc822, Size: 3419 bytes --]

From: Craig Schlenter <craig@codefountain.com>
To: "YOSHIFUJI Hideaki / 吉藤英明" <yoshfuji@linux-ipv6.org>, davem@davemloft.net
Cc: komurojun-mbn@nifty.com, bunk@stusta.de, jgarzik@pobox.com, viro@ftp.linux.org.uk, linux-kernel@vger.kernel.org, netdev@vger.kernel.org
Subject: [PATCH] Re: [BUG KERNEL 2.6.20-rc1] ftp: get or put stops during file-transfer
Date: Tue, 9 Jan 2007 07:11:39 +0200
Message-ID: <20070109051139.GA2229@craigdell.detnet.com>

Hi Dave

YOSHIFUJI Hideaki / 吉藤英明 has suggested that I send the patch
below to fix the ftp stalls present in the current kernels.

All credit goes to Komuro <komurojun-mbn@nifty.com> for tracking
this down. The patch is untested but it looks *cough* obviously
correct.

Signed-off-by: Craig Schlenter <craig@codefountain.com>

Thank you!

--Craig

diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c
index bf7a224..12de90a 100644
--- a/net/ipv4/tcp_ipv4.c
+++ b/net/ipv4/tcp_ipv4.c
@@ -648,7 +648,7 @@ static void tcp_v4_send_ack(struct tcp_timewait_sock *twsk,
 				   TCPOLEN_TIMESTAMP);
 		rep.opt[1] = htonl(tcp_time_stamp);
 		rep.opt[2] = htonl(ts);
-		arg.iov[0].iov_len = TCPOLEN_TSTAMP_ALIGNED;
+		arg.iov[0].iov_len += TCPOLEN_TSTAMP_ALIGNED;
 	}
 
 	/* Swap the send and the receive. */

Re: [PATCH] Re: [BUG KERNEL 2.6.20-rc1] ftp: get or put stops during file-transfer

[David Miller]

From: YOSHIFUJI Hideaki <yoshfuji@linux-ipv6.org>
Date: Tue, 09 Jan 2007 14:22:44 +0900 (JST)

> Dave, please apply.  Thank you.
> 
> In article <20070109051139.GA2229@craigdell.detnet.com> (at Tue, 9 Jan 2007 07:11:39 +0200), Craig Schlenter <craig@codefountain.com> says:
> 
> > All credit goes to Komuro <komurojun-mbn@nifty.com> for tracking
> > this down. The patch is untested but it looks *cough* obviously
> > correct.
> > 
> > Signed-off-by: Craig Schlenter <craig@codefountain.com>
> Signed-off-by: YOSHIFUJI Hideaki <yoshfuji@linux-ipv6.org>

Applied, thanks everyone.

Re: [BUG KERNEL 2.6.20-rc1] ftp: get or put stops during file-transfer

[Komuro]

On Thu, 4 Jan 2007 14:23:30 +0200
Craig Schlenter <craig@codefountain.com> wrote:


> > --- linux-2.6.20-rc3/net/ipv4/tcp_ipv4.c.orig	2007-01-03 11:50:04.000000000 +0900
> > +++ linux-2.6.20-rc3/net/ipv4/tcp_ipv4.c	2007-01-03 15:30:44.000000000 +0900
> > @@ -648,7 +648,7 @@ static void tcp_v4_send_ack(struct tcp_t
> >  				   TCPOLEN_TIMESTAMP);
> >  		rep.opt[1] = htonl(tcp_time_stamp);
> >  		rep.opt[2] = htonl(ts);
> > -		arg.iov[0].iov_len = TCPOLEN_TSTAMP_ALIGNED;
> > +		arg.iov[0].iov_len = sizeof(rep);
> 
> Perhaps this was supposed to be
>                 arg.iov[0].iov_len += TCPOLEN_TSTAMP_ALIGNED;
> 
> That's what the ipv6 stuff does in places.

It works properly.
Thanks!


Best Regards
Komuro