BUG: Mellanox crash with iommu=soft and swiotlb=force

BUG: Mellanox crash with iommu=soft and swiotlb=force

[Christoffer Dall]

Hi,

In doing some performance experiments I found that using a 10G Mellanox
MX354A Dual port FDR CX3 device on a server running Apache and running
ab against that server causes the system to crash with 'iommu=soft
swiotlb=force'.  The same behavior is seen without these options on Dom0
running under Xen.

I have tried this on v4.0 and v4.3-rc3.

Examples of crashes:
http://pastebin.ubuntu.com/11953498/
http://pastebin.ubuntu.com/11953443/

Kernel config (v4.3):
http://paste.ubuntu.com/12605446/

To reproduce, host a file in apache server on a system with the Mellanox
NIC (I used the 41K index file of the GCC manual) and run ab (apache
bench) from a client machine, something like this:

$ ab -n 100000 -c 100 http://10.10.1.2/gcc/index.html

I'm afraid debugging this is out of my area of expertise.  Is this
something you could have a look at?


Thanks,
-Christoffer

Re: BUG: Mellanox crash with iommu=soft and swiotlb=force

[Or Gerlitz]

On Tue, Sep 29, 2015 at 12:04 AM, Christoffer Dall
<christoffer.dall@linaro.org> wrote:
> Hi,
>
> In doing some performance experiments I found that using a 10G Mellanox
> MX354A Dual port FDR CX3 device on a server running Apache and running
> ab against that server causes the system to crash with 'iommu=soft
> swiotlb=force'.  The same behavior is seen without these options on Dom0
> running under Xen.
>
> I have tried this on v4.0 and v4.3-rc3.

Woops, needs looking indeed. Unfortunately many people in the team are
off for the Sukkot holiday with real backing to business coming to
play on Oct 6th -- not sure we can really respond on that before.

Are you running over ARM? which? if not, is that x86 64bit?

Or.

> Examples of crashes:
> http://pastebin.ubuntu.com/11953498/
> http://pastebin.ubuntu.com/11953443/
>
> Kernel config (v4.3):
> http://paste.ubuntu.com/12605446/
>
> To reproduce, host a file in apache server on a system with the Mellanox
> NIC (I used the 41K index file of the GCC manual) and run ab (apache
> bench) from a client machine, something like this:
>
> $ ab -n 100000 -c 100 http://10.10.1.2/gcc/index.html
>
> I'm afraid debugging this is out of my area of expertise.  Is this
> something you could have a look at?
>
>
> Thanks,
> -Christoffer
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: BUG: Mellanox crash with iommu=soft and swiotlb=force

[Christoffer Dall]

On Tue, Sep 29, 2015 at 12:59:35AM +0300, Or Gerlitz wrote:
> On Tue, Sep 29, 2015 at 12:04 AM, Christoffer Dall
> <christoffer.dall@linaro.org> wrote:
> > Hi,
> >
> > In doing some performance experiments I found that using a 10G Mellanox
> > MX354A Dual port FDR CX3 device on a server running Apache and running
> > ab against that server causes the system to crash with 'iommu=soft
> > swiotlb=force'.  The same behavior is seen without these options on Dom0
> > running under Xen.
> >
> > I have tried this on v4.0 and v4.3-rc3.
> 
> Woops, needs looking indeed. Unfortunately many people in the team are
> off for the Sukkot holiday with real backing to business coming to
> play on Oct 6th -- not sure we can really respond on that before.
> 
> Are you running over ARM? which? if not, is that x86 64bit?
> 
I'm running on x86_64.

Thanks,
-Christoffer

Re: BUG: Mellanox crash with iommu=soft and swiotlb=force

[Christoffer Dall]

Hi,

On Tue, Sep 29, 2015 at 12:59:35AM +0300, Or Gerlitz wrote:
> On Tue, Sep 29, 2015 at 12:04 AM, Christoffer Dall
> <christoffer.dall@linaro.org> wrote:
> > Hi,
> >
> > In doing some performance experiments I found that using a 10G Mellanox
> > MX354A Dual port FDR CX3 device on a server running Apache and running
> > ab against that server causes the system to crash with 'iommu=soft
> > swiotlb=force'.  The same behavior is seen without these options on Dom0
> > running under Xen.
> >
> > I have tried this on v4.0 and v4.3-rc3.
> 
> Woops, needs looking indeed. Unfortunately many people in the team are
> off for the Sukkot holiday with real backing to business coming to
> play on Oct 6th -- not sure we can really respond on that before.
> 
Friendly reminder on this one.  Did anyone have a look?

Thanks,
-Christoffer