Re: linux-next: Tree for May 3

Re: linux-next: Tree for May 3

[Conor.Dooley]

On 03/05/2022 08:29, Stephen Rothwell wrote:
> Hi all,
> 
> Changes since 20220502:
> 
> The input tree still had its build failure so I used the version from
> next-20220426.
> 
> The usb tree still had its build failure for which I reverted a commit.
> 
> The bitmap tree lost its build failure.
> 
> The mm tree still had its build failures for which I applied a supplied
> patch and gained a conflict against the folio tree.
> 
> Non-merge commits (relative to Linus' tree): 7492
>   8226 files changed, 410422 insertions(+), 168696 deletions(-)

Hey all,

Not done this before so apologies if I am not reporting this correctly.
The tree for May 3rd does not boot on my PolarFire SoC Icicle kit,
which I have attempted to bisect. The culprit seems to be commit
6424e31b1c05 ("swiotlb: remove swiotlb_init_with_tbl and
swiotlb_init_late_with_tbl")

Please lmk if you need any further information,
Conor.

config: arch/riscv/configs/defconfig
log:
git bisect start
# bad: [44a2f39e611ac0bc1f17c288a583d7f2e5684aa7] Add linux-next specific files for 20220503
git bisect bad 44a2f39e611ac0bc1f17c288a583d7f2e5684aa7
# good: [672c0c5173427e6b3e2a9bbb7be51ceeec78093a] Linux 5.18-rc5
git bisect good 672c0c5173427e6b3e2a9bbb7be51ceeec78093a
# bad: [80a7d9b5288b4c5ad900c7716d851e5d285f6d80] Merge branch 'drm-next' of git://git.freedesktop.org/git/drm/drm.git
git bisect bad 80a7d9b5288b4c5ad900c7716d851e5d285f6d80
# bad: [7c344e7d5be6b8bc9d753294878468cbba96af97] Merge branch 'master' of git://linuxtv.org/mchehab/media-next.git
git bisect bad 7c344e7d5be6b8bc9d753294878468cbba96af97
# bad: [7534d13c1fc508697ea76dc1c403151465ba50ff] Merge branch 'reset/next' of https://git.pengutronix.de/git/pza/linux
git bisect bad 7534d13c1fc508697ea76dc1c403151465ba50ff
# bad: [bfad554357b0b99ca95fbc79016c3fd9559c7205] Merge branch 'for-next' of https://git.kernel.org/pub/scm/linux/kernel/git/krzk/linux-mem-ctrl.git
git bisect bad bfad554357b0b99ca95fbc79016c3fd9559c7205
# bad: [5075bc6921b420b16162bfeef073f75f7c43b038] Merge branch 'for-next' of git://git.armlinux.org.uk/~rmk/linux-arm.git
git bisect bad 5075bc6921b420b16162bfeef073f75f7c43b038
# good: [cf46297bbd341132c42e2a47e3e05b2c261c99d1] Merge branch 'fixes' of git://linuxtv.org/mchehab/media-next.git
git bisect good cf46297bbd341132c42e2a47e3e05b2c261c99d1
# good: [c23552d1a15a89db37d4fa95aa8d0a30ad3c4482] Merge branch 'for-next' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git
git bisect good c23552d1a15a89db37d4fa95aa8d0a30ad3c4482
# good: [d7e3c397087fffde68389e7530093dbc2b70c48a] perf stat: Support hybrid --topdown option
git bisect good d7e3c397087fffde68389e7530093dbc2b70c48a
# bad: [3cb4503a330159dc5cf2f8382181ccbabbbaa5b2] x86: remove cruft from <asm/dma-mapping.h>
git bisect bad 3cb4503a330159dc5cf2f8382181ccbabbbaa5b2
# good: [78013eaadf696d2105982abb4018fbae394ca08f] x86: remove the IOMMU table infrastructure
git bisect good 78013eaadf696d2105982abb4018fbae394ca08f
# good: [742519538e6b07250c8085bbff4bd358bc03bf16] swiotlb: pass a gfp_mask argument to swiotlb_init_late
git bisect good 742519538e6b07250c8085bbff4bd358bc03bf16
# good: [3f70356edf5611c28a68d8d5a9c2b442c9eb81e6] swiotlb: merge swiotlb-xen initialization into swiotlb
git bisect good 3f70356edf5611c28a68d8d5a9c2b442c9eb81e6
# bad: [6424e31b1c050a25aea033206d5f626f3523448c] swiotlb: remove swiotlb_init_with_tbl and swiotlb_init_late_with_tbl
git bisect bad 6424e31b1c050a25aea033206d5f626f3523448c
# first bad commit: [6424e31b1c050a25aea033206d5f626f3523448c] swiotlb: remove swiotlb_init_with_tbl and swiotlb_init_late_with_tbl
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: linux-next: Tree for May 3

[Conor.Dooley]

On 04/05/2022 09:31, Conor Dooley wrote:
> On 03/05/2022 08:29, Stephen Rothwell wrote:
>> Hi all,
>>
>> Changes since 20220502:
>>
>> The input tree still had its build failure so I used the version from
>> next-20220426.
>>
>> The usb tree still had its build failure for which I reverted a commit.
>>
>> The bitmap tree lost its build failure.
>>
>> The mm tree still had its build failures for which I applied a supplied
>> patch and gained a conflict against the folio tree.
>>
>> Non-merge commits (relative to Linus' tree): 7492
>>   8226 files changed, 410422 insertions(+), 168696 deletions(-)
> 
> Hey all,
> 
> Not done this before so apologies if I am not reporting this correctly.
> The tree for May 3rd does not boot on my PolarFire SoC Icicle kit,
> which I have attempted to bisect. The culprit seems to be commit
> 6424e31b1c05 ("swiotlb: remove swiotlb_init_with_tbl and
> swiotlb_init_late_with_tbl")
> 
> Please lmk if you need any further information,
> Conor.

Hey all,
Problem still exists in today's tree for the 9th of May.
After sending this email last week, I reverted the patch and was able
to boot as normal.

@Christoph, I know /nothing/ about swiotlb, so if you have any
suggestions for debugging that you would like me to try, let me
know please.

Again, if this is not a valid way to report a problem, please also
let me know.

Thanks,
Conor.

> 
> config: arch/riscv/configs/defconfig
> log:
> git bisect start
> # bad: [44a2f39e611ac0bc1f17c288a583d7f2e5684aa7] Add linux-next specific files for 20220503
> git bisect bad 44a2f39e611ac0bc1f17c288a583d7f2e5684aa7
> # good: [672c0c5173427e6b3e2a9bbb7be51ceeec78093a] Linux 5.18-rc5
> git bisect good 672c0c5173427e6b3e2a9bbb7be51ceeec78093a
> # bad: [80a7d9b5288b4c5ad900c7716d851e5d285f6d80] Merge branch 'drm-next' of git://git.freedesktop.org/git/drm/drm.git
> git bisect bad 80a7d9b5288b4c5ad900c7716d851e5d285f6d80
> # bad: [7c344e7d5be6b8bc9d753294878468cbba96af97] Merge branch 'master' of git://linuxtv.org/mchehab/media-next.git
> git bisect bad 7c344e7d5be6b8bc9d753294878468cbba96af97
> # bad: [7534d13c1fc508697ea76dc1c403151465ba50ff] Merge branch 'reset/next' of https://git.pengutronix.de/git/pza/linux
> git bisect bad 7534d13c1fc508697ea76dc1c403151465ba50ff
> # bad: [bfad554357b0b99ca95fbc79016c3fd9559c7205] Merge branch 'for-next' of https://git.kernel.org/pub/scm/linux/kernel/git/krzk/linux-mem-ctrl.git
> git bisect bad bfad554357b0b99ca95fbc79016c3fd9559c7205
> # bad: [5075bc6921b420b16162bfeef073f75f7c43b038] Merge branch 'for-next' of git://git.armlinux.org.uk/~rmk/linux-arm.git
> git bisect bad 5075bc6921b420b16162bfeef073f75f7c43b038
> # good: [cf46297bbd341132c42e2a47e3e05b2c261c99d1] Merge branch 'fixes' of git://linuxtv.org/mchehab/media-next.git
> git bisect good cf46297bbd341132c42e2a47e3e05b2c261c99d1
> # good: [c23552d1a15a89db37d4fa95aa8d0a30ad3c4482] Merge branch 'for-next' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git
> git bisect good c23552d1a15a89db37d4fa95aa8d0a30ad3c4482
> # good: [d7e3c397087fffde68389e7530093dbc2b70c48a] perf stat: Support hybrid --topdown option
> git bisect good d7e3c397087fffde68389e7530093dbc2b70c48a
> # bad: [3cb4503a330159dc5cf2f8382181ccbabbbaa5b2] x86: remove cruft from <asm/dma-mapping.h>
> git bisect bad 3cb4503a330159dc5cf2f8382181ccbabbbaa5b2
> # good: [78013eaadf696d2105982abb4018fbae394ca08f] x86: remove the IOMMU table infrastructure
> git bisect good 78013eaadf696d2105982abb4018fbae394ca08f
> # good: [742519538e6b07250c8085bbff4bd358bc03bf16] swiotlb: pass a gfp_mask argument to swiotlb_init_late
> git bisect good 742519538e6b07250c8085bbff4bd358bc03bf16
> # good: [3f70356edf5611c28a68d8d5a9c2b442c9eb81e6] swiotlb: merge swiotlb-xen initialization into swiotlb
> git bisect good 3f70356edf5611c28a68d8d5a9c2b442c9eb81e6
> # bad: [6424e31b1c050a25aea033206d5f626f3523448c] swiotlb: remove swiotlb_init_with_tbl and swiotlb_init_late_with_tbl
> git bisect bad 6424e31b1c050a25aea033206d5f626f3523448c
> # first bad commit: [6424e31b1c050a25aea033206d5f626f3523448c] swiotlb: remove swiotlb_init_with_tbl and swiotlb_init_late_with_tbl

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: linux-next: Tree for May 3

[Christoph Hellwig]

On Mon, May 09, 2022 at 01:33:07PM +0000, Conor.Dooley@microchip.com wrote:
> @Christoph, I know /nothing/ about swiotlb, so if you have any
> suggestions for debugging that you would like me to try, let me
> know please.

Hi Conor,

sorry for dropping this on the flor.  I was at LSF/MM least week and
my plan to go through my backlog today didn't go to plan as I unepectedly
spent half the day at doctors appointments.

The commit looks like a somewhat unusual culprit for a boot failure,
so any chance you could do another manual verifiation pass where
you checkout 6424e31b1c05 and then the commit before it (i.e. as
git checkout 6424e31b1c05^) to make sure it really is this commits?
Some of the commits around it just seems like more likely culprits to
me, so I'd like to really be 100% sure here.  In the meantime I'll
look through the patch.  Also you don't happen to have earlycon support
on this plaform to see if there are any interesting messages on the
serial console?

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: linux-next: Tree for May 3

[Conor.Dooley]

On 09/05/2022 15:11, Christoph Hellwig wrote:
> EXTERNAL EMAIL: Do not click links or open attachments unless you know the content is safe
> 
> On Mon, May 09, 2022 at 01:33:07PM +0000, Conor.Dooley@microchip.com wrote:
>> @Christoph, I know /nothing/ about swiotlb, so if you have any
>> suggestions for debugging that you would like me to try, let me
>> know please.
> 
> Hi Conor,
> 
> sorry for dropping this on the flor.  I was at LSF/MM least week and
> my plan to go through my backlog today didn't go to plan as I unepectedly
> spent half the day at doctors appointments.

Ye, no worries. Such is life.

> The commit looks like a somewhat unusual culprit for a boot failure,
> so any chance you could do another manual verifiation pass where
> you checkout 6424e31b1c05 and then the commit before it (i.e. as
> git checkout 6424e31b1c05^) to make sure it really is this commits?

I reverted that commit & that was sufficient to boot again. I'll give
that a go, but it might be tomorrow morning before I get there.

> Some of the commits around it just seems like more likely culprits to
> me, so I'd like to really be 100% sure here.  In the meantime I'll
> look through the patch.

Sure, I can have a poke around the commits in that area some more.

> Also you don't happen to have earlycon support
> on this plaform to see if there are any interesting messages on the
> serial console?

Aye, I should've done that in the beginning... I'll let you know.

Thanks,
Conor.
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: linux-next: Tree for May 3

[Conor.Dooley]

On 09/05/2022 15:38, Conor Dooley wrote:
> On 09/05/2022 15:11, Christoph Hellwig wrote:
>> EXTERNAL EMAIL: Do not click links or open attachments unless you know the content is safe
>>
>> On Mon, May 09, 2022 at 01:33:07PM +0000, Conor.Dooley@microchip.com wrote:
>>> @Christoph, I know /nothing/ about swiotlb, so if you have any
>>> suggestions for debugging that you would like me to try, let me
>>> know please.
>>
>> Hi Conor,
>>
>> sorry for dropping this on the flor.  I was at LSF/MM least week and
>> my plan to go through my backlog today didn't go to plan as I unepectedly
>> spent half the day at doctors appointments.
> 
> Ye, no worries. Such is life.
> 
>> The commit looks like a somewhat unusual culprit for a boot failure,
>> so any chance you could do another manual verifiation pass where
>> you checkout 6424e31b1c05 and then the commit before it (i.e. as
>> git checkout 6424e31b1c05^) to make sure it really is this commits?
> 
> I reverted that commit & that was sufficient to boot again. I'll give
> that a go, but it might be tomorrow morning before I get there.

Hey Christoph,
Sory for the delay - I've been trying to kick the earlycon into working
but something w/ bootloader/firmware is not playing ball.

git checkout 6424e31b1c05
Previous HEAD position was c5eb0a61238d Linux 5.18-rc6
HEAD is now at 6424e31b1c05 swiotlb: remove swiotlb_init_with_tbl and swiotlb_init_late_with_tbl
git checkout 6424e31b1c05^
Previous HEAD position was 6424e31b1c05 swiotlb: remove swiotlb_init_with_tbl and swiotlb_init_late_with_tbl
HEAD is now at 3f70356edf56 swiotlb: merge swiotlb-xen initialization into swiotlb

3f70356edf56 boots fine. 6424e31b1c05 does not boot.

> 
>> Some of the commits around it just seems like more likely culprits to
>> me, so I'd like to really be 100% sure here.  In the meantime I'll
>> look through the patch.
> 
> Sure, I can have a poke around the commits in that area some more.
> 
>> Also you don't happen to have earlycon support
>> on this plaform to see if there are any interesting messages on the
>> serial console?
> 
> Aye, I should've done that in the beginning... I'll let you know.

As I said, earlycon isn't playing ball rn, but I will keep kicking it
& will let you know once it does..

Thanks,
Conor.

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: linux-next: Tree for May 3

[Christoph Hellwig]

Can you try this patch?

diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index e2ef0864eb1e5..856179f27f608 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -234,7 +234,7 @@ void __init swiotlb_init_remap(bool addressing_limit, unsigned int flags,
 {
 	struct io_tlb_mem *mem = &io_tlb_default_mem;
 	unsigned long nslabs = default_nslabs;
-	size_t alloc_size = PAGE_ALIGN(array_size(sizeof(*mem->slots), nslabs));
+	size_t alloc_size;
 	size_t bytes;
 	void *tlb;
 
@@ -267,12 +267,13 @@ void __init swiotlb_init_remap(bool addressing_limit, unsigned int flags,
 		goto retry;
 	}
 
+	alloc_size = PAGE_ALIGN(array_size(sizeof(*mem->slots), nslabs));
 	mem->slots = memblock_alloc(alloc_size, PAGE_SIZE);
 	if (!mem->slots)
 		panic("%s: Failed to allocate %zu bytes align=0x%lx\n",
 		      __func__, alloc_size, PAGE_SIZE);
 
-	swiotlb_init_io_tlb_mem(mem, __pa(tlb), default_nslabs, false);
+	swiotlb_init_io_tlb_mem(mem, __pa(tlb), nslabs, false);
 	mem->force_bounce = flags & SWIOTLB_FORCE;
 
 	if (flags & SWIOTLB_VERBOSE)

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: linux-next: Tree for May 3

[Conor.Dooley]

On 11/05/2022 07:22, Christoph Hellwig wrote:
> EXTERNAL EMAIL: Do not click links or open attachments unless you know the content is safe
> 
> Can you try this patch?

Hey Christoph, gave it a try but nfortunately, no joy!
Thanks,
Conor.

> 
> diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
> index e2ef0864eb1e5..856179f27f608 100644
> --- a/kernel/dma/swiotlb.c
> +++ b/kernel/dma/swiotlb.c
> @@ -234,7 +234,7 @@ void __init swiotlb_init_remap(bool addressing_limit, unsigned int flags,
>   {
>          struct io_tlb_mem *mem = &io_tlb_default_mem;
>          unsigned long nslabs = default_nslabs;
> -       size_t alloc_size = PAGE_ALIGN(array_size(sizeof(*mem->slots), nslabs));
> +       size_t alloc_size;
>          size_t bytes;
>          void *tlb;
> 
> @@ -267,12 +267,13 @@ void __init swiotlb_init_remap(bool addressing_limit, unsigned int flags,
>                  goto retry;
>          }
> 
> +       alloc_size = PAGE_ALIGN(array_size(sizeof(*mem->slots), nslabs));
>          mem->slots = memblock_alloc(alloc_size, PAGE_SIZE);
>          if (!mem->slots)
>                  panic("%s: Failed to allocate %zu bytes align=0x%lx\n",
>                        __func__, alloc_size, PAGE_SIZE);
> 
> -       swiotlb_init_io_tlb_mem(mem, __pa(tlb), default_nslabs, false);
> +       swiotlb_init_io_tlb_mem(mem, __pa(tlb), nslabs, false);
>          mem->force_bounce = flags & SWIOTLB_FORCE;
> 
>          if (flags & SWIOTLB_VERBOSE)

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: linux-next: Tree for May 3

[Christoph Hellwig]

On Wed, May 11, 2022 at 06:44:22AM +0000, Conor.Dooley@microchip.com wrote:
> On 11/05/2022 07:22, Christoph Hellwig wrote:
> > EXTERNAL EMAIL: Do not click links or open attachments unless you know the content is safe
> > 
> > Can you try this patch?
> 
> Hey Christoph, gave it a try but nfortunately, no joy!

Yes, while it is a real fix, the problem it fixes can only happen
with Xen, which is not relevant to riscv.  The only other thing I
can think off is that the allocations were always failing on your
board, and the patch makes that failure fatal.  For that try the
patch below.  I'd also be really curious by now about the kernel
logs from a successful boot.

diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index e2ef0864eb1e5..3e992a308c8a1 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -254,8 +254,10 @@ void __init swiotlb_init_remap(bool addressing_limit, unsigned int flags,
 		tlb = memblock_alloc(bytes, PAGE_SIZE);
 	else
 		tlb = memblock_alloc_low(bytes, PAGE_SIZE);
-	if (!tlb)
-		panic("%s: failed to allocate tlb structure\n", __func__);
+	if (!tlb) {
+		pr_warn("%s: failed to allocate tlb structure\n", __func__);
+		return;
+	}
 
 	if (remap && remap(tlb, nslabs) < 0) {
 		memblock_free(tlb, PAGE_ALIGN(bytes));

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: linux-next: Tree for May 3

[Conor.Dooley]

On 11/05/2022 07:48, Christoph Hellwig wrote:
> On Wed, May 11, 2022 at 06:44:22AM +0000, Conor.Dooley@microchip.com wrote:
>> On 11/05/2022 07:22, Christoph Hellwig wrote:
>>> EXTERNAL EMAIL: Do not click links or open attachments unless you know the content is safe
>>>
>>> Can you try this patch?
>>
>> Hey Christoph, gave it a try but nfortunately, no joy!
> 
> Yes, while it is a real fix, the problem it fixes can only happen
> with Xen, which is not relevant to riscv.  The only other thing I
> can think off is that the allocations were always failing on your
> board, and the patch makes that failure fatal.  For that try the
> patch below.  I'd also be really curious by now about the kernel
> logs from a successful boot.

Without even trying the patch, I double checked the boot log from
3f70356edf56 and I get a "software IO TLB: Cannot allocate buffer"
With the patch its a "software IO TLB: swiotlb_init_remap: failed
to allocate tlb structure". So spot on & I feel like an idiot for
not spotting that before!

Is failing being fatal valid, or should it fail gracefully like it
used to do? To me, blissfully unaware about swiotlb, the "current"
behaviour of failing gracefully makes more sense.

Thanks,
Conor.

Here's the start of a boot log from v5.18-rc6:
[    0.000000] Linux version 5.18.0-rc6-dirty (conor@wendy) (riscv64-unknown-linux-gnu-gcc (GCC) 10.2.0, GNU ld (GNU B
inutils) 2.36.1) #1 SMP Tue May 10 08:25:21 IST 2022
[    0.000000] OF: fdt: Ignoring memory block 0x80000000 - 0xae000000
[    0.000000] OF: fdt: Ignoring memory range 0x1000000000 - 0x1000200000
[    0.000000] Machine model: Microchip PolarFire-SoC Icicle Kit
[    0.000000] efi: UEFI not found.
[    0.000000] Zone ranges:
[    0.000000]   DMA32    empty
[    0.000000]   Normal   [mem 0x0000001000200000-0x000000103fffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000001000200000-0x000000103fffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000001000200000-0x000000103fffffff]
[    0.000000] SBI specification v0.3 detected
[    0.000000] SBI implementation ID=0x1 Version=0x9
[    0.000000] SBI TIME extension detected
[    0.000000] SBI IPI extension detected
[    0.000000] SBI RFENCE extension detected
[    0.000000] SBI HSM extension detected
[    0.000000] CPU with hartid=0 is not available
[    0.000000] CPU with hartid=0 is not available
[    0.000000] riscv: base ISA extensions acdfim
[    0.000000] riscv: ELF capabilities acdfim
[    0.000000] percpu: Embedded 18 pages/cpu s34040 r8192 d31496 u73728
[    0.000000] pcpu-alloc: s34040 r8192 d31496 u73728 alloc=18*4096
[    0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 258055
[    0.000000] Kernel command line: earlycon=sbi debug
[    0.000000] Dentry cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)
[    0.000000] Inode-cache hash table entries: 65536 (order: 7, 524288 bytes, linear)
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] software IO TLB: Cannot allocate buffer
[    0.000000] Virtual kernel memory layout:
[    0.000000]       fixmap : 0xffffffc6fee00000 - 0xffffffc6ff000000   (2048 kB)
[    0.000000]       pci io : 0xffffffc6ff000000 - 0xffffffc700000000   (  16 MB)
[    0.000000]      vmemmap : 0xffffffc700000000 - 0xffffffc800000000   (4096 MB)
[    0.000000]      vmalloc : 0xffffffc800000000 - 0xffffffd800000000   (65536 MB)
[    0.000000]       lowmem : 0xffffffd800000000 - 0xffffffd83fe00000   (1022 MB)
[    0.000000]       kernel : 0xffffffff80000000 - 0xffffffffffffffff   (2047 MB)
[    0.000000] Memory: 991276K/1046528K available (6534K kernel code, 4865K rwdata, 2048K rodata, 2165K init, 334K bss
, 55252K reserved, 0K cma-reserved)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
--8<--
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: linux-next: Tree for May 3

[Christoph Hellwig]

On Wed, May 11, 2022 at 10:10:40AM +0000, Conor.Dooley@microchip.com wrote:
> Without even trying the patch, I double checked the boot log from
> 3f70356edf56 and I get a "software IO TLB: Cannot allocate buffer"
> With the patch its a "software IO TLB: swiotlb_init_remap: failed
> to allocate tlb structure". So spot on & I feel like an idiot for
> not spotting that before!
> 
> Is failing being fatal valid, or should it fail gracefully like it
> used to do? To me, blissfully unaware about swiotlb, the "current"
> behaviour of failing gracefully makes more sense.

Given that we're at -rc6 I think the most important thing for now is to
avoid a regression and restore the old behavior.  I'll send out a
series with this and the nslab related fixes for Xen today.

But we should look into why allocating the memory fails for your
plaforms.  Does it have very little memory?  I can't really think
of why else the memblock allocation for swiotlb would fail.

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: linux-next: Tree for May 3

[Mike Rapoport]

On Wed, May 11, 2022 at 02:37:24PM +0200, Christoph Hellwig wrote:
> On Wed, May 11, 2022 at 10:10:40AM +0000, Conor.Dooley@microchip.com wrote:
> > Without even trying the patch, I double checked the boot log from
> > 3f70356edf56 and I get a "software IO TLB: Cannot allocate buffer"
> > With the patch its a "software IO TLB: swiotlb_init_remap: failed
> > to allocate tlb structure". So spot on & I feel like an idiot for
> > not spotting that before!
> > 
> > Is failing being fatal valid, or should it fail gracefully like it
> > used to do? To me, blissfully unaware about swiotlb, the "current"
> > behaviour of failing gracefully makes more sense.
> 
> Given that we're at -rc6 I think the most important thing for now is to
> avoid a regression and restore the old behavior.  I'll send out a
> series with this and the nslab related fixes for Xen today.
> 
> But we should look into why allocating the memory fails for your
> plaforms.  Does it have very little memory?  I can't really think
> of why else the memblock allocation for swiotlb would fail.

I guess the default to use memblock_alloc_low() backfires on system with
physical memory living at 0x1000200000:

[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000001000200000-0x000000103fffffff]

The default limit for "low" memory is 0xffffffff and there is simply no
memory there.

-- 
Sincerely yours,
Mike.

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: linux-next: Tree for May 3

[Christoph Hellwig]

On Wed, May 11, 2022 at 05:08:52PM +0300, Mike Rapoport wrote:
> I guess the default to use memblock_alloc_low() backfires on system with
> physical memory living at 0x1000200000:
> 
> [    0.000000] Early memory node ranges
> [    0.000000]   node   0: [mem 0x0000001000200000-0x000000103fffffff]
> 
> The default limit for "low" memory is 0xffffffff and there is simply no
> memory there.

Is there any way to ask memblock for a specific address limit?
swiotlb just wants <= 32-bit by default.  With the little caveat
that it should be 32-bit addressable for all devices, and we don't
know the physical to dma address mapping at time of allocation.

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: linux-next: Tree for May 3

[Mike Rapoport]

On Wed, May 11, 2022 at 04:10:34PM +0200, Christoph Hellwig wrote:
> On Wed, May 11, 2022 at 05:08:52PM +0300, Mike Rapoport wrote:
> > I guess the default to use memblock_alloc_low() backfires on system with
> > physical memory living at 0x1000200000:
> > 
> > [    0.000000] Early memory node ranges
> > [    0.000000]   node   0: [mem 0x0000001000200000-0x000000103fffffff]
> > 
> > The default limit for "low" memory is 0xffffffff and there is simply no
> > memory there.
> 
> Is there any way to ask memblock for a specific address limit?
> swiotlb just wants <= 32-bit by default.  With the little caveat
> that it should be 32-bit addressable for all devices, and we don't
> know the physical to dma address mapping at time of allocation.

There is 

void *memblock_alloc_try_nid(phys_addr_t size, phys_addr_t align,
			     phys_addr_t min_addr, phys_addr_t max_addr,
			     int nid);

that lets caller to specify min and max limits

Presuming that devices see [0x1000200000-0x103fffffff] as
[0x200000-0x3fffffff] we may try something like

	min = memblock_start_of_DRAM();
	max = min + 0xffffffff;

	if (flags & SWIOTLB_ANY)
		max = MEMBLOCK_ALLOC_ACCESSIBLE;

	tlb = memblock_alloc_try_nid(bytes, PAGE_SIZE, min, max, NUMA_NO_NODE);

-- 
Sincerely yours,
Mike.

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: linux-next: Tree for May 3

[Christoph Hellwig]

On Wed, May 11, 2022 at 05:37:50PM +0300, Mike Rapoport wrote:
> Presuming that devices see [0x1000200000-0x103fffffff] as
> [0x200000-0x3fffffff] we may try something like
> 
> 	min = memblock_start_of_DRAM();
> 	max = min + 0xffffffff;
> 
> 	if (flags & SWIOTLB_ANY)
> 		max = MEMBLOCK_ALLOC_ACCESSIBLE;
> 
> 	tlb = memblock_alloc_try_nid(bytes, PAGE_SIZE, min, max, NUMA_NO_NODE);

While there is still no guarantee the first 32-bits worth of DRAM
are actually mapped to a usable address, this looks like a much better
default than what we have right now.

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: linux-next: Tree for May 3

[Conor.Dooley]

On 11/05/2022 13:37, Christoph Hellwig wrote:
> EXTERNAL EMAIL: Do not click links or open attachments unless you know the content is safe
> 
> On Wed, May 11, 2022 at 10:10:40AM +0000, Conor.Dooley@microchip.com wrote:
>> Without even trying the patch, I double checked the boot log from
>> 3f70356edf56 and I get a "software IO TLB: Cannot allocate buffer"
>> With the patch its a "software IO TLB: swiotlb_init_remap: failed
>> to allocate tlb structure". So spot on & I feel like an idiot for
>> not spotting that before!
>>
>> Is failing being fatal valid, or should it fail gracefully like it
>> used to do? To me, blissfully unaware about swiotlb, the "current"
>> behaviour of failing gracefully makes more sense.
> 
> Given that we're at -rc6 I think the most important thing for now is to
> avoid a regression and restore the old behavior.  I'll send out a
> series with this and the nslab related fixes for Xen today.
> 
> But we should look into why allocating the memory fails for your
> plaforms.  Does it have very little memory?  I can't really think
> of why else the memblock allocation for swiotlb would fail.

My assumption would be that it is the complete lack of usable memory
at 32 bit bit addresses? Even though there is memory physically there
it is just being ignored as you can see in the boot log (because it's
below the entry point?).

My question is, does failing to even boot make sense if that buffer
can't be allocated? Can argue that SWIOTLB should'nt be enabled in the
first place if there is no DMA32, but if it is enabled in error would
it not be better to at least let the system boot & spit out an uglier
warning?
(Obviously I missed the current warning, hence the "uglier" bit.)

Thanks,
Conor.

(readding log for context)
[    0.000000] OF: fdt: Ignoring memory block 0x80000000 - 0xae000000
[    0.000000] OF: fdt: Ignoring memory range 0x1000000000 - 0x1000200000
[    0.000000] Machine model: Microchip PolarFire-SoC Icicle Kit
[    0.000000] efi: UEFI not found.
[    0.000000] Zone ranges:
[    0.000000]   DMA32    empty
[    0.000000]   Normal   [mem 0x0000001000200000-0x000000103fffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000001000200000-0x000000103fffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000001000200000-0x000000103fffffff]
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: linux-next: Tree for May 3

[Geert Uytterhoeven]

Hi Conor,

On Wed, May 11, 2022 at 12:13 PM <Conor.Dooley@microchip.com> wrote:
> [    0.000000] Kernel command line: earlycon=sbi debug

That should work without the "=sbi" part.

I've used "earlycon" and "earlycon keep_bootcon" successfully on
icicle before.

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: linux-next: Tree for May 3

[Conor.Dooley]

On 14/05/2022 13:18, Geert Uytterhoeven wrote:
> EXTERNAL EMAIL: Do not click links or open attachments unless you know the content is safe
> 
> Hi Conor,
> 
> On Wed, May 11, 2022 at 12:13 PM <Conor.Dooley@microchip.com> wrote:
>> [    0.000000] Kernel command line: earlycon=sbi debug
> 
> That should work without the "=sbi" part.
> 
> I've used "earlycon" and "earlycon keep_bootcon" successfully on
> icicle before.

Yeah, honestly just entirely forgot I could do it without =sbi, d'oh.
Thanks for reminding me :)

The HSS is hogging the sbi console & that was what I was trying to kick.
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv