Fwd: SWIOTLB on 32-bit PAE.

Fwd: SWIOTLB on 32-bit PAE.

[Christian Melki]

I discovered a strange error on my machine. 32-bit PAE 4.2.0 without
IOMMU code (yeah, I know).
When writing to an ext4 filesystem on a USB disk my kernel would hang
and not return control to userspace. It would spew kernel messages to
syslog as fast as it could without giving userspace any time, ie "hang"
as far as a user sees it. The device itself is ok and the filesystem is
clean.
I get messages like this, millions of them.

Oct  2 14:33:06 voodoochild kernel: [  223.287447] nommu_map_sg:
overflow 25dcac000+1024 of device mask ffffffff
Oct  2 14:33:06 voodoochild kernel: [  223.287448] nommu_map_sg:
overflow 25dcac000+1024 of device mask ffffffff
Oct  2 14:33:06 voodoochild kernel: [  223.287449] nommu_map_sg:
overflow 25dcac000+1024 of device mask ffffffff

I disovered that SWIOTLB was not on. It is only provided for 64-bit
machines where it is default. On 32-bit SWIOTLB seems disabled unless 
you are running with some IOMMU/AGPGART code.

But what about 32-bit PAE? Would that not qualify for SWIOTLB without IOMMU?

I am a novice in this area. Maybe this is a Kconfig "bug" for us that 
like to hang ourselves with CONFIG_* rope? :) I'd like to think that 
SWIOTLB should be default for 32-bit PAE too?

Best regards,
Christian

Re: Fwd: SWIOTLB on 32-bit PAE.

[Bjorn Helgaas]

[+cc Konrad, Joerg, iommu list]

On Fri, Oct 02, 2015 at 04:20:16PM +0200, Christian Melki wrote:
> I discovered a strange error on my machine. 32-bit PAE 4.2.0 without
> IOMMU code (yeah, I know).
> When writing to an ext4 filesystem on a USB disk my kernel would hang
> and not return control to userspace. It would spew kernel messages to
> syslog as fast as it could without giving userspace any time, ie "hang"
> as far as a user sees it. The device itself is ok and the filesystem is
> clean.
> I get messages like this, millions of them.
> 
> Oct  2 14:33:06 voodoochild kernel: [  223.287447] nommu_map_sg:
> overflow 25dcac000+1024 of device mask ffffffff
> Oct  2 14:33:06 voodoochild kernel: [  223.287448] nommu_map_sg:
> overflow 25dcac000+1024 of device mask ffffffff
> Oct  2 14:33:06 voodoochild kernel: [  223.287449] nommu_map_sg:
> overflow 25dcac000+1024 of device mask ffffffff
> 
> I disovered that SWIOTLB was not on. It is only provided for 64-bit
> machines where it is default. On 32-bit SWIOTLB seems disabled
> unless you are running with some IOMMU/AGPGART code.
> 
> But what about 32-bit PAE? Would that not qualify for SWIOTLB without IOMMU?
> 
> I am a novice in this area. Maybe this is a Kconfig "bug" for us
> that like to hang ourselves with CONFIG_* rope? :) I'd like to think
> that SWIOTLB should be default for 32-bit PAE too?
> 
> Best regards,
> Christian
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-pci" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Fwd: SWIOTLB on 32-bit PAE.

[Konrad Rzeszutek Wilk]

On Sat, Oct 03, 2015 at 09:36:30AM -0500, Bjorn Helgaas wrote:
> [+cc Konrad, Joerg, iommu list]
> 
> On Fri, Oct 02, 2015 at 04:20:16PM +0200, Christian Melki wrote:
> > I discovered a strange error on my machine. 32-bit PAE 4.2.0 without
> > IOMMU code (yeah, I know).
> > When writing to an ext4 filesystem on a USB disk my kernel would hang
> > and not return control to userspace. It would spew kernel messages to
> > syslog as fast as it could without giving userspace any time, ie "hang"
> > as far as a user sees it. The device itself is ok and the filesystem is
> > clean.
> > I get messages like this, millions of them.
> > 
> > Oct  2 14:33:06 voodoochild kernel: [  223.287447] nommu_map_sg:
> > overflow 25dcac000+1024 of device mask ffffffff
> > Oct  2 14:33:06 voodoochild kernel: [  223.287448] nommu_map_sg:
> > overflow 25dcac000+1024 of device mask ffffffff
> > Oct  2 14:33:06 voodoochild kernel: [  223.287449] nommu_map_sg:
> > overflow 25dcac000+1024 of device mask ffffffff
> > 
> > I disovered that SWIOTLB was not on. It is only provided for 64-bit
> > machines where it is default. On 32-bit SWIOTLB seems disabled
> > unless you are running with some IOMMU/AGPGART code.
> > 
> > But what about 32-bit PAE? Would that not qualify for SWIOTLB without IOMMU?

It most certainly would!
> > 
> > I am a novice in this area. Maybe this is a Kconfig "bug" for us
> > that like to hang ourselves with CONFIG_* rope? :) I'd like to think
> > that SWIOTLB should be default for 32-bit PAE too?

Aye. I presume that you had done a small change already for this?
Would you be willing to post it on lkml and CC me ?

Thank you.
> > 
> > Best regards,
> > Christian
> > 
> > 
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-pci" in
> > the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Fwd: SWIOTLB on 32-bit PAE.

[Konrad Rzeszutek Wilk]

On Sat, Oct 03, 2015 at 09:36:30AM -0500, Bjorn Helgaas wrote:
> [+cc Konrad, Joerg, iommu list]
> 
> On Fri, Oct 02, 2015 at 04:20:16PM +0200, Christian Melki wrote:
> > I discovered a strange error on my machine. 32-bit PAE 4.2.0 without
> > IOMMU code (yeah, I know).
> > When writing to an ext4 filesystem on a USB disk my kernel would hang
> > and not return control to userspace. It would spew kernel messages to
> > syslog as fast as it could without giving userspace any time, ie "hang"
> > as far as a user sees it. The device itself is ok and the filesystem is
> > clean.
> > I get messages like this, millions of them.
> > 
> > Oct  2 14:33:06 voodoochild kernel: [  223.287447] nommu_map_sg:
> > overflow 25dcac000+1024 of device mask ffffffff
> > Oct  2 14:33:06 voodoochild kernel: [  223.287448] nommu_map_sg:
> > overflow 25dcac000+1024 of device mask ffffffff
> > Oct  2 14:33:06 voodoochild kernel: [  223.287449] nommu_map_sg:
> > overflow 25dcac000+1024 of device mask ffffffff
> > 
> > I disovered that SWIOTLB was not on. It is only provided for 64-bit
> > machines where it is default. On 32-bit SWIOTLB seems disabled
> > unless you are running with some IOMMU/AGPGART code.
> > 
> > But what about 32-bit PAE? Would that not qualify for SWIOTLB without IOMMU?

It most certainly would!
> > 
> > I am a novice in this area. Maybe this is a Kconfig "bug" for us
> > that like to hang ourselves with CONFIG_* rope? :) I'd like to think
> > that SWIOTLB should be default for 32-bit PAE too?

Aye. I presume that you had done a small change already for this?
Would you be willing to post it on lkml and CC me ?

Thank you.
> > 
> > Best regards,
> > Christian
> > 
> > 
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-pci" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html

SWIOTLB on 32-bit PAE

[Christian Melki]

[-- Attachment #1: Type: text/plain, Size: 1232 bytes --]

Hi,

I discovered that my 32-bit PAE 4.2.0 kernel (no IOMMU code) would hang 
when writing to my USB disk. The kernel spews million(-ish messages per 
sec) to syslog, effectively "hanging" userspace with my kernel.

Oct  2 14:33:06 voodoochild kernel: [  223.287447] nommu_map_sg: 
overflow 25dcac000+1024 of device mask ffffffff
Oct  2 14:33:06 voodoochild kernel: [  223.287448] nommu_map_sg: 
overflow 25dcac000+1024 of device mask ffffffff
Oct  2 14:33:06 voodoochild kernel: [  223.287449] nommu_map_sg: 
overflow 25dcac000+1024 of device mask ffffffff
... etc ...

In my kernel config I noticed that SWIOTLB was not on. It seems SWIOTLB 
is provided for 64-bit and 32-bit with IOMMU/AGPGART code. But if I 
compiled the kernel with PAE and no IOMMU and no other GART code, I 
would not get SWIOTLB. I'd like to think that SWIOTLB should be selected 
for 32-bit PAE as default.

I have attached a oneliner patch which does that.
The patch works for me. The issue where the kernel more or less runs 
endless bashing of (nommu_?)map_sg when failing is another problem I 
guess. I expected the kernel/drivers to have some more graceful 
handling, but I know to little about this area to have a proper opinion.

Regards,
Christian

[-- Attachment #2: swiotlb.patch --]
[-- Type: text/x-patch, Size: 481 bytes --]

diff -urN linux-4.2.orig/arch/x86/Kconfig linux-4.2/arch/x86/Kconfig
--- linux-4.2.orig/arch/x86/Kconfig	2015-10-05 08:56:58.933313678 +0200
+++ linux-4.2/arch/x86/Kconfig	2015-10-05 09:00:00.916306025 +0200
@@ -1282,6 +1282,7 @@
 config X86_PAE
 	bool "PAE (Physical Address Extension) Support"
 	depends on X86_32 && !HIGHMEM4G
+	select SWIOTLB
 	---help---
 	  PAE is required for NX support, and furthermore enables
 	  larger swapspace support for non-overcommit purposes. It

Re: Fwd: SWIOTLB on 32-bit PAE.

[Joerg Roedel]

On Sat, Oct 03, 2015 at 02:00:19PM -0400, Konrad Rzeszutek Wilk wrote:
> Aye. I presume that you had done a small change already for this?
> Would you be willing to post it on lkml and CC me ?

Yes, I also think it is needed. SWIOTLB already correctly handles
buffers in HighMem, so enabling it for 32-bit PAE should be easy. Please
also CC me on the patch.


	Joerg

Re: Fwd: SWIOTLB on 32-bit PAE.

[Christian Melki]

Hi.

I think I rang the bell a bit early. The corruption is in grub legacy (I 
assumed that would be in Linux too, but it wasn't). The on disk format 
of ext4 apparently stores byte ordering somehow? I thought the on disk 
format of ext4 was a specific endian (little). Since only that file is 
unreadable in grub, but readable in Linux.. I am guessing endian issues 
somehow.. which still is strange.

Regards,
Christian

On 10/03/2015 08:00 PM, Konrad Rzeszutek Wilk wrote:
> On Sat, Oct 03, 2015 at 09:36:30AM -0500, Bjorn Helgaas wrote:
>> [+cc Konrad, Joerg, iommu list]
>>
>> On Fri, Oct 02, 2015 at 04:20:16PM +0200, Christian Melki wrote:
>>> I discovered a strange error on my machine. 32-bit PAE 4.2.0 without
>>> IOMMU code (yeah, I know).
>>> When writing to an ext4 filesystem on a USB disk my kernel would hang
>>> and not return control to userspace. It would spew kernel messages to
>>> syslog as fast as it could without giving userspace any time, ie "hang"
>>> as far as a user sees it. The device itself is ok and the filesystem is
>>> clean.
>>> I get messages like this, millions of them.
>>>
>>> Oct  2 14:33:06 voodoochild kernel: [  223.287447] nommu_map_sg:
>>> overflow 25dcac000+1024 of device mask ffffffff
>>> Oct  2 14:33:06 voodoochild kernel: [  223.287448] nommu_map_sg:
>>> overflow 25dcac000+1024 of device mask ffffffff
>>> Oct  2 14:33:06 voodoochild kernel: [  223.287449] nommu_map_sg:
>>> overflow 25dcac000+1024 of device mask ffffffff
>>>
>>> I disovered that SWIOTLB was not on. It is only provided for 64-bit
>>> machines where it is default. On 32-bit SWIOTLB seems disabled
>>> unless you are running with some IOMMU/AGPGART code.
>>>
>>> But what about 32-bit PAE? Would that not qualify for SWIOTLB without IOMMU?
>
> It most certainly would!
>>>
>>> I am a novice in this area. Maybe this is a Kconfig "bug" for us
>>> that like to hang ourselves with CONFIG_* rope? :) I'd like to think
>>> that SWIOTLB should be default for 32-bit PAE too?
>
> Aye. I presume that you had done a small change already for this?
> Would you be willing to post it on lkml and CC me ?
>
> Thank you.
>>>
>>> Best regards,
>>> Christian
>>>
>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-pci" in
>>> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Fwd: SWIOTLB on 32-bit PAE.

[Christian Melki]

Never mind my ranting.
The problem was in the extent disk format of ext4. I had forgot that 
ext4 uses extents for all new files by default. Old grub won't read extents.

Oh, well... nothing like wasting hard working peoples time with stupid 
things. Anyway. SWIOTLB for 32-bit PAE should be a good thing though. :)

Regards,
Christian

On 10/05/2015 01:35 PM, Christian Melki wrote:
> Hi.
>
> I think I rang the bell a bit early. The corruption is in grub legacy (I
> assumed that would be in Linux too, but it wasn't). The on disk format
> of ext4 apparently stores byte ordering somehow? I thought the on disk
> format of ext4 was a specific endian (little). Since only that file is
> unreadable in grub, but readable in Linux.. I am guessing endian issues
> somehow.. which still is strange.
>
> Regards,
> Christian
>
> On 10/03/2015 08:00 PM, Konrad Rzeszutek Wilk wrote:
>> On Sat, Oct 03, 2015 at 09:36:30AM -0500, Bjorn Helgaas wrote:
>>> [+cc Konrad, Joerg, iommu list]
>>>
>>> On Fri, Oct 02, 2015 at 04:20:16PM +0200, Christian Melki wrote:
>>>> I discovered a strange error on my machine. 32-bit PAE 4.2.0 without
>>>> IOMMU code (yeah, I know).
>>>> When writing to an ext4 filesystem on a USB disk my kernel would hang
>>>> and not return control to userspace. It would spew kernel messages to
>>>> syslog as fast as it could without giving userspace any time, ie "hang"
>>>> as far as a user sees it. The device itself is ok and the filesystem is
>>>> clean.
>>>> I get messages like this, millions of them.
>>>>
>>>> Oct  2 14:33:06 voodoochild kernel: [  223.287447] nommu_map_sg:
>>>> overflow 25dcac000+1024 of device mask ffffffff
>>>> Oct  2 14:33:06 voodoochild kernel: [  223.287448] nommu_map_sg:
>>>> overflow 25dcac000+1024 of device mask ffffffff
>>>> Oct  2 14:33:06 voodoochild kernel: [  223.287449] nommu_map_sg:
>>>> overflow 25dcac000+1024 of device mask ffffffff
>>>>
>>>> I disovered that SWIOTLB was not on. It is only provided for 64-bit
>>>> machines where it is default. On 32-bit SWIOTLB seems disabled
>>>> unless you are running with some IOMMU/AGPGART code.
>>>>
>>>> But what about 32-bit PAE? Would that not qualify for SWIOTLB
>>>> without IOMMU?
>>
>> It most certainly would!
>>>>
>>>> I am a novice in this area. Maybe this is a Kconfig "bug" for us
>>>> that like to hang ourselves with CONFIG_* rope? :) I'd like to think
>>>> that SWIOTLB should be default for 32-bit PAE too?
>>
>> Aye. I presume that you had done a small change already for this?
>> Would you be willing to post it on lkml and CC me ?
>>
>> Thank you.
>>>>
>>>> Best regards,
>>>> Christian
>>>>
>>>>
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe linux-pci" in
>>>> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Fwd: SWIOTLB on 32-bit PAE

[Konrad Rzeszutek Wilk]

On Mon, Oct 05, 2015 at 12:44:55PM +0200, Christian Melki wrote:
> Joerg,
> 
> I already sent the patch for Kconfig change to the list.
> But I discovered something even more disturbing which I mailed Konrad about.
> With the change my machine does not hang any more, but instead corrupts data
> which is written. The data however is readable on the machine which
> corrupted it, unreadable on the other machine without the patch. I don't
> understand what is going on. I have never encountered "selective" file
> corruption of fs on different machines. The fs itself checks out cleanly
> with badblock control (ext4). I fail to understand what is happening.

And I believe this ended up being an operator error?

With the patch you sent, you didn't include your SoB 
(Signed-off-By). I would like to include your patch but I cannot
with your SoB. It is matter of reading to Documentation/SubmittingPatches
and just consenting to the Certificate in there.

If you are uncomfortable with that - that is OK, I can make this
patch myself (a one liner :-)) but I figured it owuld be nice
for you to have all the credit.
> 
> Regards,
> Christian
> 
> -------- Forwarded Message --------
> Subject: SWIOTLB on 32-bit PAE
> Date: Mon, 5 Oct 2015 10:31:39 +0200
> From: Christian Melki <christian.melki@t2data.com>
> To: linux-kernel@vger.kernel.org
> CC: konrad.wilk@oracle.com
> 
> Hi,
> 
> I discovered that my 32-bit PAE 4.2.0 kernel (no IOMMU code) would hang
> when writing to my USB disk. The kernel spews million(-ish messages per
> sec) to syslog, effectively "hanging" userspace with my kernel.
> 
> Oct  2 14:33:06 voodoochild kernel: [  223.287447] nommu_map_sg:
> overflow 25dcac000+1024 of device mask ffffffff
> Oct  2 14:33:06 voodoochild kernel: [  223.287448] nommu_map_sg:
> overflow 25dcac000+1024 of device mask ffffffff
> Oct  2 14:33:06 voodoochild kernel: [  223.287449] nommu_map_sg:
> overflow 25dcac000+1024 of device mask ffffffff
> ... etc ...
> 
> In my kernel config I noticed that SWIOTLB was not on. It seems SWIOTLB
> is provided for 64-bit and 32-bit with IOMMU/AGPGART code. But if I
> compiled the kernel with PAE and no IOMMU and no other GART code, I
> would not get SWIOTLB. I'd like to think that SWIOTLB should be selected
> for 32-bit PAE as default.
> 
> I have attached a oneliner patch which does that.
> The patch works for me. The issue where the kernel more or less runs
> endless bashing of (nommu_?)map_sg when failing is another problem I
> guess. I expected the kernel/drivers to have some more graceful
> handling, but I know to little about this area to have a proper opinion.
> 
> Regards,
> Christian
> 
> 
> 

> diff -urN linux-4.2.orig/arch/x86/Kconfig linux-4.2/arch/x86/Kconfig
> --- linux-4.2.orig/arch/x86/Kconfig	2015-10-05 08:56:58.933313678 +0200
> +++ linux-4.2/arch/x86/Kconfig	2015-10-05 09:00:00.916306025 +0200
> @@ -1282,6 +1282,7 @@
>  config X86_PAE
>  	bool "PAE (Physical Address Extension) Support"
>  	depends on X86_32 && !HIGHMEM4G
> +	select SWIOTLB
>  	---help---
>  	  PAE is required for NX support, and furthermore enables
>  	  larger swapspace support for non-overcommit purposes. It
> 

Re: Fwd: SWIOTLB on 32-bit PAE

[Christian Melki]

[-- Attachment #1: Type: text/plain, Size: 3391 bytes --]

Yes. Operator error indeed.

Thanks for asking. Included is the same patch with the SoB.

On 10/05/2015 04:21 PM, Konrad Rzeszutek Wilk wrote:
> On Mon, Oct 05, 2015 at 12:44:55PM +0200, Christian Melki wrote:
>> Joerg,
>>
>> I already sent the patch for Kconfig change to the list.
>> But I discovered something even more disturbing which I mailed Konrad about.
>> With the change my machine does not hang any more, but instead corrupts data
>> which is written. The data however is readable on the machine which
>> corrupted it, unreadable on the other machine without the patch. I don't
>> understand what is going on. I have never encountered "selective" file
>> corruption of fs on different machines. The fs itself checks out cleanly
>> with badblock control (ext4). I fail to understand what is happening.
>
> And I believe this ended up being an operator error?
>
> With the patch you sent, you didn't include your SoB
> (Signed-off-By). I would like to include your patch but I cannot
> with your SoB. It is matter of reading to Documentation/SubmittingPatches
> and just consenting to the Certificate in there.
>
> If you are uncomfortable with that - that is OK, I can make this
> patch myself (a one liner :-)) but I figured it owuld be nice
> for you to have all the credit.
>>
>> Regards,
>> Christian
>>
>> -------- Forwarded Message --------
>> Subject: SWIOTLB on 32-bit PAE
>> Date: Mon, 5 Oct 2015 10:31:39 +0200
>> From: Christian Melki <christian.melki@t2data.com>
>> To: linux-kernel@vger.kernel.org
>> CC: konrad.wilk@oracle.com
>>
>> Hi,
>>
>> I discovered that my 32-bit PAE 4.2.0 kernel (no IOMMU code) would hang
>> when writing to my USB disk. The kernel spews million(-ish messages per
>> sec) to syslog, effectively "hanging" userspace with my kernel.
>>
>> Oct  2 14:33:06 voodoochild kernel: [  223.287447] nommu_map_sg:
>> overflow 25dcac000+1024 of device mask ffffffff
>> Oct  2 14:33:06 voodoochild kernel: [  223.287448] nommu_map_sg:
>> overflow 25dcac000+1024 of device mask ffffffff
>> Oct  2 14:33:06 voodoochild kernel: [  223.287449] nommu_map_sg:
>> overflow 25dcac000+1024 of device mask ffffffff
>> ... etc ...
>>
>> In my kernel config I noticed that SWIOTLB was not on. It seems SWIOTLB
>> is provided for 64-bit and 32-bit with IOMMU/AGPGART code. But if I
>> compiled the kernel with PAE and no IOMMU and no other GART code, I
>> would not get SWIOTLB. I'd like to think that SWIOTLB should be selected
>> for 32-bit PAE as default.
>>
>> I have attached a oneliner patch which does that.
>> The patch works for me. The issue where the kernel more or less runs
>> endless bashing of (nommu_?)map_sg when failing is another problem I
>> guess. I expected the kernel/drivers to have some more graceful
>> handling, but I know to little about this area to have a proper opinion.
>>
>> Regards,
>> Christian
>>
>>
>>
>
>> diff -urN linux-4.2.orig/arch/x86/Kconfig linux-4.2/arch/x86/Kconfig
>> --- linux-4.2.orig/arch/x86/Kconfig	2015-10-05 08:56:58.933313678 +0200
>> +++ linux-4.2/arch/x86/Kconfig	2015-10-05 09:00:00.916306025 +0200
>> @@ -1282,6 +1282,7 @@
>>   config X86_PAE
>>   	bool "PAE (Physical Address Extension) Support"
>>   	depends on X86_32 && !HIGHMEM4G
>> +	select SWIOTLB
>>   	---help---
>>   	  PAE is required for NX support, and furthermore enables
>>   	  larger swapspace support for non-overcommit purposes. It
>>
>


[-- Attachment #2: swiotlb.patch --]
[-- Type: text/plain, Size: 542 bytes --]

diff -urN linux-4.2.orig/arch/x86/Kconfig linux-4.2/arch/x86/Kconfig
--- linux-4.2.orig/arch/x86/Kconfig	2015-10-05 08:56:58.933313678 +0200
+++ linux-4.2/arch/x86/Kconfig	2015-10-05 09:00:00.916306025 +0200
@@ -1282,6 +1282,7 @@
 config X86_PAE
 	bool "PAE (Physical Address Extension) Support"
 	depends on X86_32 && !HIGHMEM4G
+	select SWIOTLB
 	---help---
 	  PAE is required for NX support, and furthermore enables
 	  larger swapspace support for non-overcommit purposes. It

Signed-off-by: Christian Melki <christian.melki@t2data.com>

Re: Fwd: SWIOTLB on 32-bit PAE

[Konrad Rzeszutek Wilk]

On Mon, Oct 05, 2015 at 05:31:33PM +0200, Christian Melki wrote:
> Yes. Operator error indeed.
> 
> Thanks for asking. Included is the same patch with the SoB.

I've stuck it at 
https://git.kernel.org/cgit/linux/kernel/git/konrad/swiotlb.git/ #stable/for-linus-4.3

Will send it out tomorrow if there are no #linux-next issues.
> 
> On 10/05/2015 04:21 PM, Konrad Rzeszutek Wilk wrote:
> >On Mon, Oct 05, 2015 at 12:44:55PM +0200, Christian Melki wrote:
> >>Joerg,
> >>
> >>I already sent the patch for Kconfig change to the list.
> >>But I discovered something even more disturbing which I mailed Konrad about.
> >>With the change my machine does not hang any more, but instead corrupts data
> >>which is written. The data however is readable on the machine which
> >>corrupted it, unreadable on the other machine without the patch. I don't
> >>understand what is going on. I have never encountered "selective" file
> >>corruption of fs on different machines. The fs itself checks out cleanly
> >>with badblock control (ext4). I fail to understand what is happening.
> >
> >And I believe this ended up being an operator error?
> >
> >With the patch you sent, you didn't include your SoB
> >(Signed-off-By). I would like to include your patch but I cannot
> >with your SoB. It is matter of reading to Documentation/SubmittingPatches
> >and just consenting to the Certificate in there.
> >
> >If you are uncomfortable with that - that is OK, I can make this
> >patch myself (a one liner :-)) but I figured it owuld be nice
> >for you to have all the credit.
> >>
> >>Regards,
> >>Christian
> >>
> >>-------- Forwarded Message --------
> >>Subject: SWIOTLB on 32-bit PAE
> >>Date: Mon, 5 Oct 2015 10:31:39 +0200
> >>From: Christian Melki <christian.melki@t2data.com>
> >>To: linux-kernel@vger.kernel.org
> >>CC: konrad.wilk@oracle.com
> >>
> >>Hi,
> >>
> >>I discovered that my 32-bit PAE 4.2.0 kernel (no IOMMU code) would hang
> >>when writing to my USB disk. The kernel spews million(-ish messages per
> >>sec) to syslog, effectively "hanging" userspace with my kernel.
> >>
> >>Oct  2 14:33:06 voodoochild kernel: [  223.287447] nommu_map_sg:
> >>overflow 25dcac000+1024 of device mask ffffffff
> >>Oct  2 14:33:06 voodoochild kernel: [  223.287448] nommu_map_sg:
> >>overflow 25dcac000+1024 of device mask ffffffff
> >>Oct  2 14:33:06 voodoochild kernel: [  223.287449] nommu_map_sg:
> >>overflow 25dcac000+1024 of device mask ffffffff
> >>... etc ...
> >>
> >>In my kernel config I noticed that SWIOTLB was not on. It seems SWIOTLB
> >>is provided for 64-bit and 32-bit with IOMMU/AGPGART code. But if I
> >>compiled the kernel with PAE and no IOMMU and no other GART code, I
> >>would not get SWIOTLB. I'd like to think that SWIOTLB should be selected
> >>for 32-bit PAE as default.
> >>
> >>I have attached a oneliner patch which does that.
> >>The patch works for me. The issue where the kernel more or less runs
> >>endless bashing of (nommu_?)map_sg when failing is another problem I
> >>guess. I expected the kernel/drivers to have some more graceful
> >>handling, but I know to little about this area to have a proper opinion.
> >>
> >>Regards,
> >>Christian
> >>
> >>
> >>
> >
> >>diff -urN linux-4.2.orig/arch/x86/Kconfig linux-4.2/arch/x86/Kconfig
> >>--- linux-4.2.orig/arch/x86/Kconfig	2015-10-05 08:56:58.933313678 +0200
> >>+++ linux-4.2/arch/x86/Kconfig	2015-10-05 09:00:00.916306025 +0200
> >>@@ -1282,6 +1282,7 @@
> >>  config X86_PAE
> >>  	bool "PAE (Physical Address Extension) Support"
> >>  	depends on X86_32 && !HIGHMEM4G
> >>+	select SWIOTLB
> >>  	---help---
> >>  	  PAE is required for NX support, and furthermore enables
> >>  	  larger swapspace support for non-overcommit purposes. It
> >>
> >
> 

> diff -urN linux-4.2.orig/arch/x86/Kconfig linux-4.2/arch/x86/Kconfig
> --- linux-4.2.orig/arch/x86/Kconfig	2015-10-05 08:56:58.933313678 +0200
> +++ linux-4.2/arch/x86/Kconfig	2015-10-05 09:00:00.916306025 +0200
> @@ -1282,6 +1282,7 @@
>  config X86_PAE
>  	bool "PAE (Physical Address Extension) Support"
>  	depends on X86_32 && !HIGHMEM4G
> +	select SWIOTLB
>  	---help---
>  	  PAE is required for NX support, and furthermore enables
>  	  larger swapspace support for non-overcommit purposes. It
> 
> Signed-off-by: Christian Melki <christian.melki@t2data.com>