copying large files over NFS locks up machine on -testing from Thursday

copying large files over NFS locks up machine on -testing from Thursday

[Kip Macy]

I've locked up my dom0 a couple of times this morning copying a 3GB
file from local disk to an NFS mount(neither xend nor guests running).
I don't encounter this problem on the stock CentOS 4 kernel. The
machine is a PowerEdge 2850 with 2 e1000 cards - the one in use is
connected to a PowerConnect 2216 10/100 switch and has negotiated
100Mbit. I'll check if the stock Cambridge isn't negotiating full
duplex but that shouldn't cause lockups.

My mount options are:
defaults,intr,rsize=32768,wsize=32768,nfsvers=3,tcp,timeo=600

This isn't at NetApp (no 100Mbit there), but these are the options
used by all their linux NFS clients.

Re: copying large files over NFS locks up machine on -testing from Thursday

[Robert Read]

Does it happen only when using the bridge?

robert

On May 21, 2005, at 11:06, Kip Macy wrote:

> I've locked up my dom0 a couple of times this morning copying a 3GB
> file from local disk to an NFS mount(neither xend nor guests running).
> I don't encounter this problem on the stock CentOS 4 kernel. The
> machine is a PowerEdge 2850 with 2 e1000 cards - the one in use is
> connected to a PowerConnect 2216 10/100 switch and has negotiated
> 100Mbit. I'll check if the stock Cambridge isn't negotiating full
> duplex but that shouldn't cause lockups.
>
> My mount options are:
> defaults,intr,rsize=32768,wsize=32768,nfsvers=3,tcp,timeo=600
>
> This isn't at NetApp (no 100Mbit there), but these are the options
> used by all their linux NFS clients.
>
>

--
Robert Read
robert@xensource.com

Re: copying large files over NFS locks up machine on -testing from Thursday

[Chris Wright]

* Kip Macy (kip.macy@gmail.com) wrote:
> I've locked up my dom0 a couple of times this morning copying a 3GB
> file from local disk to an NFS mount(neither xend nor guests running).
> I don't encounter this problem on the stock CentOS 4 kernel. The
> machine is a PowerEdge 2850 with 2 e1000 cards - the one in use is
> connected to a PowerConnect 2216 10/100 switch and has negotiated
> 100Mbit. I'll check if the stock Cambridge isn't negotiating full
> duplex but that shouldn't cause lockups.
> 
> My mount options are:
> defaults,intr,rsize=32768,wsize=32768,nfsvers=3,tcp,timeo=600

Hmm, I've seen this on stock 2.6.11 kernels as well (no xen).  Any chance
you can get useful debugging out it?

thanks,
-chris

Re: copying large files over NFS locks up machine on -testing from Thursday

[Kip Macy]

If it happens on -unstable in domUs (it probably does) I can grab a
coredump or attach to it while it is running and see where it is
wedged. I don't have a port master where I'm at so I can't use pdb or
cdb.

      -Kip


On 5/21/05, Chris Wright <chrisw@osdl.org> wrote:
> * Kip Macy (kip.macy@gmail.com) wrote:
> > I've locked up my dom0 a couple of times this morning copying a 3GB
> > file from local disk to an NFS mount(neither xend nor guests running).
> > I don't encounter this problem on the stock CentOS 4 kernel. The
> > machine is a PowerEdge 2850 with 2 e1000 cards - the one in use is
> > connected to a PowerConnect 2216 10/100 switch and has negotiated
> > 100Mbit. I'll check if the stock Cambridge isn't negotiating full
> > duplex but that shouldn't cause lockups.
> >
> > My mount options are:
> > defaults,intr,rsize=32768,wsize=32768,nfsvers=3,tcp,timeo=600
> 
> Hmm, I've seen this on stock 2.6.11 kernels as well (no xen).  Any chance
> you can get useful debugging out it?
> 
> thanks,
> -chris
>

Re: copying large files over NFS locks up machine on -testing from Thursday

[Kip Macy]

I just tried copying the 3GB file over NFS from within a domU on
-testing in hopes of getting some debug info. dom0 became unresponsive
for a few seconds after close to a minute. It had successfully copied
2.3GB when I hit ^C and then started a copy from NFS to the domU's /
which itself is a loopback device mounted over NFS in dom0 - shortly
thereafter the machine locked up, the only output being complaints
from megaraid about aborted SCSI commands. It seems possible that this
is a dom0 issue.

I'll try building a stock linux 2.6.11.10 kernel with the same non-xen
options as the Cambridge kernel to see if it has the same issues.
After that I'll just stick to having everything on local disk until I
get moved to a GigE network at which point I can start testing
migration.



        -Kip


On 5/21/05, Chris Wright <chrisw@osdl.org> wrote:
> * Kip Macy (kip.macy@gmail.com) wrote:
> > I've locked up my dom0 a couple of times this morning copying a 3GB
> > file from local disk to an NFS mount(neither xend nor guests running).
> > I don't encounter this problem on the stock CentOS 4 kernel. The
> > machine is a PowerEdge 2850 with 2 e1000 cards - the one in use is
> > connected to a PowerConnect 2216 10/100 switch and has negotiated
> > 100Mbit. I'll check if the stock Cambridge isn't negotiating full
> > duplex but that shouldn't cause lockups.
> >
> > My mount options are:
> > defaults,intr,rsize=32768,wsize=32768,nfsvers=3,tcp,timeo=600
> 
> Hmm, I've seen this on stock 2.6.11 kernels as well (no xen).  Any chance
> you can get useful debugging out it?
> 
> thanks,
> -chris
>

Re: copying large files over NFS locks up machine on -testing from Thursday

[Kip Macy]

It appears to be isolated to Cambridge's 2.6.11.10 - I just built an
unmodified 2.6.11.10 and copied the 3GB file over NFS without issue.
I'm 5 for 5 locking up -testing's 2.6.11.10 so I think it isn't an
issue with mainline.

     -Kip

On 5/21/05, Kip Macy <kip.macy@gmail.com> wrote:
> I just tried copying the 3GB file over NFS from within a domU on
> -testing in hopes of getting some debug info. dom0 became unresponsive
> for a few seconds after close to a minute. It had successfully copied
> 2.3GB when I hit ^C and then started a copy from NFS to the domU's /
> which itself is a loopback device mounted over NFS in dom0 - shortly
> thereafter the machine locked up, the only output being complaints
> from megaraid about aborted SCSI commands. It seems possible that this
> is a dom0 issue.
> 
> I'll try building a stock linux 2.6.11.10 kernel with the same non-xen
> options as the Cambridge kernel to see if it has the same issues.
> After that I'll just stick to having everything on local disk until I
> get moved to a GigE network at which point I can start testing
> migration.
> 
> 
> 
>         -Kip
> 
> 
> On 5/21/05, Chris Wright <chrisw@osdl.org> wrote:
> > * Kip Macy (kip.macy@gmail.com) wrote:
> > > I've locked up my dom0 a couple of times this morning copying a 3GB
> > > file from local disk to an NFS mount(neither xend nor guests running).
> > > I don't encounter this problem on the stock CentOS 4 kernel. The
> > > machine is a PowerEdge 2850 with 2 e1000 cards - the one in use is
> > > connected to a PowerConnect 2216 10/100 switch and has negotiated
> > > 100Mbit. I'll check if the stock Cambridge isn't negotiating full
> > > duplex but that shouldn't cause lockups.
> > >
> > > My mount options are:
> > > defaults,intr,rsize=32768,wsize=32768,nfsvers=3,tcp,timeo=600
> >
> > Hmm, I've seen this on stock 2.6.11 kernels as well (no xen).  Any chance
> > you can get useful debugging out it?
> >
> > thanks,
> > -chris
> >
>