Touching the initrd before paging_init

Touching the initrd before paging_init

[Mark Rutland]

Hi,

On Sun, Dec 13, 2015 at 03:47:12AM -0500, Jon Masters wrote:
> Hi Folks,
> 
> Just a few notes that might be useful to someone else.
> 
> I mentioned before that we'll want to see ACPI_INITRD_TABLE_OVERRIDE
> support on arm64 in due course for use during bringup of new systems. It
> is commonly used on x86 to provide handy updated ACPI tables via a
> prepended initrd cpio that is attached to the regular initrd image,
> which is infinitely better than respinning firmware each time you want
> to provide updated tables (yes yes, we all know the "devicetree"
> directive in GRUB does this with DT and that's great and wonderful). I
> know that someone is working on it, but I had a need for an interim
> solution over the weekend for some hardware I am bringing up here.

Which table(s) are you trying to override?

It might be possible to have an EFI application prior to the kernel which
override the relevant table, or you may be able to override the tables in the
EFI stub. From the PoV of Linux proper, things would then be the same as a
normal boot -- nothing special to be done.

If you're able to do this in a separate application prior to the stub (or even
prior to the main bootloader), that same override should work for any OS.

> The problem is that there's no mapping for the pages containing the
> ramdisk data prior to paging_init.Can we not move paging_init earlier?

Why can't we move paging_init earlier? Which information do we need prior to
paging_init?

The only thing I can think of are the SLIT and SRAT. Are you trying to override
those?

> x86 does a bit of hoop jumping (and I think they have certain assumptions
> about linear mapping too) and splits the initrd setup into a couple of
> different phases, which we may well need to do also if we want to be able to
> call something like:
>
> #if defined(CONFIG_ACPI) && defined(CONFIG_BLK_DEV_INITRD)
>         acpi_initrd_override((void *)initrd_start, initrd_end -
> initrd_start);
> #endif
> 
> prior to acpi_boot_table_init in arch/arm64/kernel/setup.c. That routine
> will correctly map memory for data it copies out of the cpio, but it
> assumes that the initrd is already mapped. For the very nasty hack I
> want over the weekend, I added some extra fixmaps and then do an
> early_memremap on the physical address of first page of the initrd (the
> cpio is tiny, and is always prepended and this is very nasty). Viz:
> 
> [    0.000000] JCM: using fixmap for first page of ramdisk...
> [    0.000000] JCM: first physical page of ramdisk: [mem REDACTED]
> [    0.000000] JCM: first virtual page of ramdisk: [mem REDACTED]
> [    0.000000] XXXX ACPI table found in initrd
> [kernel/firmware/acpi/xxxx.aml][0xxxx]
> [    0.000000] XXXX ACPI table found in initrd
> [kernel/firmware/acpi/xxxx.aml][0xxx]
> [    0.000000] XXXX ACPI table found in initrd
> [kernel/firmware/acpi/xxxx.aml][0xxxx]
> 
> So maybe I'll get what I want working, but a head's up to whoever from
> Linaro ends up looking into the broader implementation that I think
> you'll need to rework initrd mapping as part of that. Unless of course
> I'm missing something. Which is more than possible.

If we really need to map the initrd this early, one option would be to make it
possible to perform arbitrarily large memremaps early on, following the
approach I posted for the linear mapping [1]. That would require some more
invasive rework, but we'd be able to reuse the logic for other things we want
to map early on.

Thanks,
Mark.

[1] http://lists.infradead.org/pipermail/linux-arm-kernel/2015-December/392292.html

Touching the initrd before paging_init

[Jon Masters]

Hi Folks,

Just a few notes that might be useful to someone else.

I mentioned before that we'll want to see ACPI_INITRD_TABLE_OVERRIDE
support on arm64 in due course for use during bringup of new systems. It
is commonly used on x86 to provide handy updated ACPI tables via a
prepended initrd cpio that is attached to the regular initrd image,
which is infinitely better than respinning firmware each time you want
to provide updated tables (yes yes, we all know the "devicetree"
directive in GRUB does this with DT and that's great and wonderful). I
know that someone is working on it, but I had a need for an interim
solution over the weekend for some hardware I am bringing up here.

The problem is that there's no mapping for the pages containing the
ramdisk data prior to paging_init. x86 does a bit of hoop jumping (and I
think they have certain assumptions about linear mapping too) and splits
the initrd setup into a couple of different phases, which we may well
need to do also if we want to be able to call something like:

#if defined(CONFIG_ACPI) && defined(CONFIG_BLK_DEV_INITRD)
        acpi_initrd_override((void *)initrd_start, initrd_end -
initrd_start);
#endif

prior to acpi_boot_table_init in arch/arm64/kernel/setup.c. That routine
will correctly map memory for data it copies out of the cpio, but it
assumes that the initrd is already mapped. For the very nasty hack I
want over the weekend, I added some extra fixmaps and then do an
early_memremap on the physical address of first page of the initrd (the
cpio is tiny, and is always prepended and this is very nasty). Viz:

[    0.000000] JCM: using fixmap for first page of ramdisk...
[    0.000000] JCM: first physical page of ramdisk: [mem REDACTED]
[    0.000000] JCM: first virtual page of ramdisk: [mem REDACTED]
[    0.000000] XXXX ACPI table found in initrd
[kernel/firmware/acpi/xxxx.aml][0xxxx]
[    0.000000] XXXX ACPI table found in initrd
[kernel/firmware/acpi/xxxx.aml][0xxx]
[    0.000000] XXXX ACPI table found in initrd
[kernel/firmware/acpi/xxxx.aml][0xxxx]

So maybe I'll get what I want working, but a head's up to whoever from
Linaro ends up looking into the broader implementation that I think
you'll need to rework initrd mapping as part of that. Unless of course
I'm missing something. Which is more than possible.

Jon.

[Linaro-acpi] Touching the initrd before paging_init

[G Gregory]

On 7 December 2015 at 06:05, Mark Rutland <mark.rutland@arm.com> wrote:
> Hi,
>
> On Sun, Dec 13, 2015 at 03:47:12AM -0500, Jon Masters wrote:
>> Hi Folks,
>>
>> Just a few notes that might be useful to someone else.
>>
>> I mentioned before that we'll want to see ACPI_INITRD_TABLE_OVERRIDE
>> support on arm64 in due course for use during bringup of new systems. It
>> is commonly used on x86 to provide handy updated ACPI tables via a
>> prepended initrd cpio that is attached to the regular initrd image,
>> which is infinitely better than respinning firmware each time you want
>> to provide updated tables (yes yes, we all know the "devicetree"
>> directive in GRUB does this with DT and that's great and wonderful). I
>> know that someone is working on it, but I had a need for an interim
>> solution over the weekend for some hardware I am bringing up here.
>
> Which table(s) are you trying to override?
>
> It might be possible to have an EFI application prior to the kernel which
> override the relevant table, or you may be able to override the tables in the
> EFI stub. From the PoV of Linux proper, things would then be the same as a
> normal boot -- nothing special to be done.
>
> If you're able to do this in a separate application prior to the stub (or even
> prior to the main bootloader), that same override should work for any OS.
>

grub2 already has a acpi module which should allow this.

Graeme

[Linaro-acpi] Touching the initrd before paging_init

[Leif Lindholm]

On Sun, Dec 13, 2015 at 03:47:12AM -0500, Jon Masters wrote:
> I mentioned before that we'll want to see ACPI_INITRD_TABLE_OVERRIDE
> support on arm64 in due course for use during bringup of new systems. It
> is commonly used on x86 to provide handy updated ACPI tables via a
> prepended initrd cpio that is attached to the regular initrd image,
> which is infinitely better than respinning firmware each time you want
> to provide updated tables (yes yes, we all know the "devicetree"
> directive in GRUB does this with DT and that's great and wonderful). I
> know that someone is working on it, but I had a need for an interim
> solution over the weekend for some hardware I am bringing up here.

Why would we do this through an initrd?
That sounds an awful lot like the per-kernel-image dtbs that are the whole
thing we're trying to avoid.

Implementing this as a grub command (or kernel command line) for debug
purposes would be trivial - and I'm pretty sure Graeme/Al already did
this in the past.

/
    Leif

[Linaro-acpi] Touching the initrd before paging_init

[Jon Masters]

Just thinking from a parity point of view - if you can do it on x86, it should be doable on ARM. But that GRUB module approach I quite like! Let's get it documented prominently in the kernel doc as the alternative to updating an initrd then :)

(In our case we would only turn this on in -debug RHEL kernels so as to keep it very clear this is never for production systems, can't trust folks not to hack things if you give them half a chance)

Jon.

-- 
Computer Architect | Sent from my 64-bit #ARM Powered phone

> On Dec 13, 2015, at 11:59, Leif Lindholm <leif.lindholm@linaro.org> wrote:
> 
>> On Sun, Dec 13, 2015 at 03:47:12AM -0500, Jon Masters wrote:
>> I mentioned before that we'll want to see ACPI_INITRD_TABLE_OVERRIDE
>> support on arm64 in due course for use during bringup of new systems. It
>> is commonly used on x86 to provide handy updated ACPI tables via a
>> prepended initrd cpio that is attached to the regular initrd image,
>> which is infinitely better than respinning firmware each time you want
>> to provide updated tables (yes yes, we all know the "devicetree"
>> directive in GRUB does this with DT and that's great and wonderful). I
>> know that someone is working on it, but I had a need for an interim
>> solution over the weekend for some hardware I am bringing up here.
> 
> Why would we do this through an initrd?
> That sounds an awful lot like the per-kernel-image dtbs that are the whole
> thing we're trying to avoid.
> 
> Implementing this as a grub command (or kernel command line) for debug
> purposes would be trivial - and I'm pretty sure Graeme/Al already did
> this in the past.
> 
> /
>    Leif

[Linaro-acpi] Touching the initrd before paging_init

[Jon Masters]

On 12/13/2015 04:36 PM, Jon Masters wrote:

> Just thinking from a parity point of view - if you can do it on x86, it should be
> doable on ARM. But that GRUB module approach I quite like!

Sorry for top post earlier. Was on my phone. In any case, it looks like
the "acpi" command in GRUB currently does an all-or-nothing replace of
all of the tables, not just a named table. We need to be able to
override e.g. just an DSDT or SSDT with a replacement test one.

Jon.

[Linaro-acpi] Touching the initrd before paging_init

[G Gregory]

On 13 December 2015 at 22:02, Jon Masters <jcm@redhat.com> wrote:
> On 12/13/2015 04:36 PM, Jon Masters wrote:
>
>> Just thinking from a parity point of view - if you can do it on x86, it should be
>> doable on ARM. But that GRUB module approach I quite like!
>
> Sorry for top post earlier. Was on my phone. In any case, it looks like
> the "acpi" command in GRUB currently does an all-or-nothing replace of
> all of the tables, not just a named table. We need to be able to
> override e.g. just an DSDT or SSDT with a replacement test one.
>
I think thats just clumsily worded documentation. The code certainly
looks like you can replace them individually.

I think its trying to say in the case RSDP,XSDT,RSDT are in ROM it
will create new ones to point to the new tables you've just imported.

I had planned to try it this week anyway.

Graeme

[Linaro-acpi] Touching the initrd before paging_init

[Jon Masters]

On 12/13/2015 05:02 PM, Jon Masters wrote:
> On 12/13/2015 04:36 PM, Jon Masters wrote:
> 
>> Just thinking from a parity point of view - if you can do it on x86, it should be
>> doable on ARM. But that GRUB module approach I quite like!
> 
> Sorry for top post earlier. Was on my phone. In any case, it looks like
> the "acpi" command in GRUB currently does an all-or-nothing replace of
> all of the tables, not just a named table. We need to be able to
> override e.g. just an DSDT or SSDT with a replacement test one.

I rescind that. Looking at the GRUB source clarifies things. It will
copy all host tables, then update the copied version if there are
additional tables with changes and recalculate checksums/pointers in the
XSDT etc. I seem to recall having looked at this before a couple years
ago or something and it might have grown some of the 64-bit XSDT logic
in that time. Either way, it does seem to do what I wanted. I will
followup. I'll probably still use the initrd approach for now.

Jon.

[Linaro-acpi] Touching the initrd before paging_init

[Jon Masters]

On 12/13/2015 05:15 PM, G Gregory wrote:
> On 13 December 2015 at 22:02, Jon Masters <jcm@redhat.com> wrote:
>> On 12/13/2015 04:36 PM, Jon Masters wrote:
>>
>>> Just thinking from a parity point of view - if you can do it on x86, it should be
>>> doable on ARM. But that GRUB module approach I quite like!
>>
>> Sorry for top post earlier. Was on my phone. In any case, it looks like
>> the "acpi" command in GRUB currently does an all-or-nothing replace of
>> all of the tables, not just a named table. We need to be able to
>> override e.g. just an DSDT or SSDT with a replacement test one.
>>
> I think thats just clumsily worded documentation. The code certainly
> looks like you can replace them individually.

Yea, it does. It's unfortunate that the code is more "obvious" than the
docs, but hardly the first time ;)

> I think its trying to say in the case RSDP,XSDT,RSDT are in ROM it
> will create new ones to point to the new tables you've just imported.

Yea. I think it actually ends up creating new tables in any case, then
copies over the bits that change. It'll actually need to do that anyway
because it'll want to update the pointers to new tables. I'll try it.
But I also want to see the initrd approach working in the not too
distant future - everything someone can do on an x86 system should
translate 1:1 in terms of experience. It'll help with bringup of newer
platforms such as the one I have that needs some NUMA related tweaks.

> I had planned to try it this week anyway.

Great :)

Jon.

[Linaro-acpi] Touching the initrd before paging_init

[Jon Masters]

On 12/13/2015 05:15 PM, Jon Masters wrote:
> On 12/13/2015 05:02 PM, Jon Masters wrote:
>> On 12/13/2015 04:36 PM, Jon Masters wrote:
>>
>>> Just thinking from a parity point of view - if you can do it on x86, it should be
>>> doable on ARM. But that GRUB module approach I quite like!
>>
>> Sorry for top post earlier. Was on my phone. In any case, it looks like
>> the "acpi" command in GRUB currently does an all-or-nothing replace of
>> all of the tables, not just a named table. We need to be able to
>> override e.g. just an DSDT or SSDT with a replacement test one.
> 
> I rescind that. Looking at the GRUB source clarifies things. It will
> copy all host tables, then update the copied version if there are
> additional tables with changes and recalculate checksums/pointers in the
> XSDT etc. I seem to recall having looked at this before a couple years
> ago or something and it might have grown some of the 64-bit XSDT logic
> in that time. Either way, it does seem to do what I wanted. I will
> followup. I'll probably still use the initrd approach for now.

Addendum. I was able to get the initrd based ACPI table override
working...HOWEVER...

*** I had to implement a custom memcpy routine to do so ***

Per some internal suggestions, I tried moving map_mem earlier (prior to
doing boot time ACPI table parsing, in order to avoid needing to abuse
fixmaps to touch the initrd contents), which works. For reading the
ramdisk cpio content (the kernel code still uses a fixmap in the initrd
override driver code to map the newly created tables in memory).

But the reading of that cpio content into the new table locations is
done using the kernel memcpy routine to early_ioremap'd memory (Device
memory), which is architecturally sensitive to missaligned accesses. The
in-kernel memcpy routine only checks the alignment of the source pointed
when it begins copying, and doesn't handle the case that the natural
alignment differs between source and destination. Therefore, the kernel
rolls over and plays dead unless I provide a hacked up jcm_memcpy that
just does byte copies. Then everything "works".

Now sure, the official version of the initrd override code is going to
work differently from my hack I'm never posting that is for bringup of a
specific system, and it will all nicely be done by Linaro and so on, but
per my other thread, I am also not convinced that the kernel's
"optimized" memcpy library routine is behaving correctly in the case
that alignment differs. Surely it should handle the case that the
alignment is off on either side of the source and destination.

Once I figured out what in the heck was going on with memcpy nonsense:

SRAT ACPI table found in initrd [kernel/firmware/acpi/srat.aml][REDACT]
SLIT ACPI table found in initrd [kernel/firmware/acpi/slit.aml][REDACT]
SSDT ACPI table found in initrd [kernel/firmware/acpi/ssdt1.aml][REDACT]
ACPI: Early table checksum verification disabled
ACPI: RSDP REDACTED_ADDRESS REDACT (REDACTED)
ACPI: XSDT REDACTED_ADDRESS REDACT (REDACTED OEM)
ACPI: FACP REDACTED_ADDRESS REDACT (REDACTED OEM)
ACPI: DSDT REDACTED_ADDRESS REDACT (REDACTED OEM)
ACPI: APIC REDACTED_ADDRESS REDACT (REDACTED OEM)
ACPI: SSDT REDACTED_ADDRESS REDACT (REDACTED OEM)
ACPI: Override [SSDT- REDACTED], this is unsafe: tainting kernel

Needless to say rewriting the kernel memcpy library was slightly more
than I planned on doing to get a simple initrd override to work...

Jon.

[Linaro-acpi] Touching the initrd before paging_init

[Leif Lindholm]

On Mon, Dec 14, 2015 at 11:51:09PM -0500, Jon Masters wrote:
> Addendum. I was able to get the initrd based ACPI table override
> working...HOWEVER...
> 
> *** I had to implement a custom memcpy routine to do so ***
> 
> Per some internal suggestions, I tried moving map_mem earlier (prior to
> doing boot time ACPI table parsing, in order to avoid needing to abuse
> fixmaps to touch the initrd contents), which works. For reading the
> ramdisk cpio content (the kernel code still uses a fixmap in the initrd
> override driver code to map the newly created tables in memory).
> 
> But the reading of that cpio content into the new table locations is
> done using the kernel memcpy routine to early_ioremap'd memory (Device
> memory), which is architecturally sensitive to missaligned accesses.

Why is it mapped as device memory?

If that is a legal thing, shouldn't drivers/acpi/osl.c be using
memcpy_fromio(), which (on ARM*) does the alignment fixups if
required, without going full-bytewise?

/
    Leif

[Linaro-acpi] Touching the initrd before paging_init

[Mark Rutland]

On Mon, Dec 14, 2015 at 11:51:09PM -0500, Jon Masters wrote:
> Addendum. I was able to get the initrd based ACPI table override
> working...HOWEVER...
> 
> *** I had to implement a custom memcpy routine to do so ***
> 
> Per some internal suggestions, I tried moving map_mem earlier (prior to
> doing boot time ACPI table parsing, in order to avoid needing to abuse
> fixmaps to touch the initrd contents), which works. For reading the
> ramdisk cpio content (the kernel code still uses a fixmap in the initrd
> override driver code to map the newly created tables in memory).
> 
> But the reading of that cpio content into the new table locations is
> done using the kernel memcpy routine to early_ioremap'd memory (Device
> memory), which is architecturally sensitive to missaligned accesses. The
> in-kernel memcpy routine only checks the alignment of the source pointed
> when it begins copying, and doesn't handle the case that the natural
> alignment differs between source and destination. Therefore, the kernel
> rolls over and plays dead unless I provide a hacked up jcm_memcpy that
> just does byte copies. Then everything "works".

You don't need a custom memcpy. All you need to do is use
early_memremap, as we do for relocate_initrd, to get a Normal Cacheable
mapping. See:

	setup_arch()
		relocate_initrd()
			copy_from_early_mem()
				early_memremap()
				memcpy()
				early_memunmap()

Mark.

[Linaro-acpi] Touching the initrd before paging_init

[G Gregory]

On 13 December 2015 at 22:19, Jon Masters <jcm@redhat.com> wrote:
> On 12/13/2015 05:15 PM, G Gregory wrote:
>> On 13 December 2015 at 22:02, Jon Masters <jcm@redhat.com> wrote:
>>> On 12/13/2015 04:36 PM, Jon Masters wrote:
>>>
>>>> Just thinking from a parity point of view - if you can do it on x86, it should be
>>>> doable on ARM. But that GRUB module approach I quite like!
>>>
>>> Sorry for top post earlier. Was on my phone. In any case, it looks like
>>> the "acpi" command in GRUB currently does an all-or-nothing replace of
>>> all of the tables, not just a named table. We need to be able to
>>> override e.g. just an DSDT or SSDT with a replacement test one.
>>>
>> I think thats just clumsily worded documentation. The code certainly
>> looks like you can replace them individually.
>
> Yea, it does. It's unfortunate that the code is more "obvious" than the
> docs, but hardly the first time ;)
>
>> I think its trying to say in the case RSDP,XSDT,RSDT are in ROM it
>> will create new ones to point to the new tables you've just imported.
>
> Yea. I think it actually ends up creating new tables in any case, then
> copies over the bits that change. It'll actually need to do that anyway
> because it'll want to update the pointers to new tables. I'll try it.
> But I also want to see the initrd approach working in the not too
> distant future - everything someone can do on an x86 system should
> translate 1:1 in terms of experience. It'll help with bringup of newer
> platforms such as the one I have that needs some NUMA related tweaks.
>
>> I had planned to try it this week anyway.
>
> Great :)
>
> Jon.
>

Tried it, results were not great :-(

set root='hd0,gpt2'
insmod acpi
acpi /DSDT.aml
linux /Image-leg earlycon=pl011,0xe1010000 console=ttyAMA0 acpi=force root
=/dev/sda2

[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Initializing cgroup subsys cpuset
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Initializing cgroup subsys cpuacct
[    0.000000] Linux version 4.4.0-rc1-00058-g5e47260-dirty
(graeme at linaro-seattle.xora.org.uk) (gcc version 4.8.3 20140911 (Red
Hat 4.8.3-9) (GCC) ) #172 SMP PREEMPT Tue Dec 1 17:01:23 GMT 2015
[    0.000000] Boot CPU: AArch64 Processor [411fd072]
[    0.000000] earlycon: Early serial console at MMIO 0xe1010000 (options '')
[    0.000000] bootconsole [uart0] enabled
[    0.000000] Bad mode in Error handler detected, code 0xbf000000 -- SError
[    0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted
4.4.0-rc1-00058-g5e47260-dirty #172
[    0.000000] Hardware name: AMD Seattle (RevB) Development Board
(Overdrive) (DT)
[    0.000000] task: fffffe0000dc7500 ti: fffffe0000d90000 task.ti:
fffffe0000d90000
[    0.000000] PC is at setup_arch+0xfc/0x564
[    0.000000] LR is at setup_arch+0xf8/0x564
[    0.000000] pc : [<fffffe0000cb35e8>] lr : [<fffffe0000cb35e4>]
pstate: 000002c5
[    0.000000] sp : fffffe0000d93f00
[    0.000000] x29: fffffe0000d93f00 x28: 00000083f0fa22c0
[    0.000000] x27: fffffe0000081198 x26: 0000008002040000
[    0.000000] x25: 0000008002010000 x24: 0000008001000000
[    0.000000] x23: fffffe0000dc0000 x22: 00000083f0f15228
[    0.000000] x21: fffffe0000080000 x20: fffffe0000cffc00
[    0.000000] x19: fffffdfffa800000 x18: fffffe00008d0bf0
[    0.000000] x17: 000000000000000e x16: 0000000000000007
[    0.000000] x15: 0000000000000001 x14: 0ffffffffffffffe
[    0.000000] x13: 0000000000000020 x12: 0000000000000038
[    0.000000] x11: 0000000000000007 x10: 0101010101010101
[    0.000000] x9 : fffffffffffffffd x8 : 0000000000000008
[    0.000000] x7 : 0000000000000005 x6 : 0000000000000080
[    0.000000] x5 : 000000000000005f x4 : 0000000000000072
[    0.000000] x3 : 0000000000000063 x2 : 0000000000000072
[    0.000000] x1 : 0000000000000000 x0 : 0000000000000001
[    0.000000]
[    0.000000] Internal error: Oops - bad mode: 0 [#1] PREEMPT SMP
[    0.000000] Modules linked in:
[    0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted
4.4.0-rc1-00058-g5e47260-dirty #172
[    0.000000] Hardware name: AMD Seattle (RevB) Development Board
(Overdrive) (DT)
[    0.000000] task: fffffe0000dc7500 ti: fffffe0000d90000 task.ti:
fffffe0000d90000
[    0.000000] PC is at setup_arch+0xfc/0x564
[    0.000000] LR is at setup_arch+0xf8/0x564
[    0.000000] pc : [<fffffe0000cb35e8>] lr : [<fffffe0000cb35e4>]
pstate: 000002c5
[    0.000000] sp : fffffe0000d93f00
[    0.000000] x29: fffffe0000d93f00 x28: 00000083f0fa22c0
[    0.000000] x27: fffffe0000081198 x26: 0000008002040000
[    0.000000] x25: 0000008002010000 x24: 0000008001000000
[    0.000000] x23: fffffe0000dc0000 x22: 00000083f0f15228
[    0.000000] x21: fffffe0000080000 x20: fffffe0000cffc00
[    0.000000] x19: fffffdfffa800000 x18: fffffe00008d0bf0
[    0.000000] x17: 000000000000000e x16: 0000000000000007
[    0.000000] x15: 0000000000000001 x14: 0ffffffffffffffe
[    0.000000] x13: 0000000000000020 x12: 0000000000000038
[    0.000000] x11: 0000000000000007 x10: 0101010101010101
[    0.000000] x9 : fffffffffffffffd x8 : 0000000000000008
[    0.000000] x7 : 0000000000000005 x6 : 0000000000000080
[    0.000000] x5 : 000000000000005f x4 : 0000000000000072
[    0.000000] x7 : 0000000000000005 x6 : 0000000000000080

                                                         [0/1977]
[    0.000000] x5 : 000000000000005f x4 : 0000000000000072
[    0.000000] x3 : 0000000000000063 x2 : 0000000000000072
[    0.000000] x1 : 0000000000000000 x0 : 0000000000000001
[    0.000000]
[    0.000000] Process swapper (pid: 0, stack limit = 0xfffffe0000d90020)
[    0.000000] Stack: (0xfffffe0000d93f00 to 0xfffffe0000d94000)
[    0.000000] 3f00: fffffe0000d93fa0 fffffe0000cb06a0
0000000000000001 fffffe0000cffc00
[    0.000000] 3f20: 000000801fe00000 00000083f0f15228
fffffe0000dc0000 0000008001000000
[    0.000000] 3f40: 0000008002010000 0000008002040000
fffffe0000081198 00000000ffffffc8
[    0.000000] 3f60: 00000083f0f155a0 fffffe0000860080
0000000000000001 000000801fe00000
[    0.000000] 3f80: ffffffffffffffff 8080808080800000
0000808080808080 fefefefefeff736d
[    0.000000] 3fa0: 0000000000000000 000000800184d000
00000083f0f155a0 0000000000000e12
[    0.000000] 3fc0: 000000801fe00000 00000083f0f15228
00000083f0f1523d 0000008001000000
[    0.000000] 3fe0: 0000000000000000 fffffe0000d00428
0000000000000000 0000000000000000
[    0.000000] Call trace:
[    0.000000] [<fffffe0000cb35e8>] setup_arch+0xfc/0x564
[    0.000000] [<fffffe0000cb06a0>] start_kernel+0xd4/0x400
[    0.000000] [<000000800184d000>] 0x800184d000
[    0.000000] Code: 91318000 94002b79 97fff3e7 d50344ff (94000760)
[    0.000000] ---[ end trace f24b6c88ae00fa9a ]---
[    0.000000] Kernel panic - not syncing: Attempted to kill the idle task!
[    0.000000] ---[ end Kernel panic - not syncing: Attempted to kill
the idle task!

[Linaro-acpi] Touching the initrd before paging_init

[Måns Rullgård]

Jon Masters <jcm@redhat.com> writes:

> On 12/13/2015 05:15 PM, Jon Masters wrote:
>> On 12/13/2015 05:02 PM, Jon Masters wrote:
>>> On 12/13/2015 04:36 PM, Jon Masters wrote:
>>>
>>>> Just thinking from a parity point of view - if you can do it on x86, it should be
>>>> doable on ARM. But that GRUB module approach I quite like!
>>>
>>> Sorry for top post earlier. Was on my phone. In any case, it looks like
>>> the "acpi" command in GRUB currently does an all-or-nothing replace of
>>> all of the tables, not just a named table. We need to be able to
>>> override e.g. just an DSDT or SSDT with a replacement test one.
>> 
>> I rescind that. Looking at the GRUB source clarifies things. It will
>> copy all host tables, then update the copied version if there are
>> additional tables with changes and recalculate checksums/pointers in the
>> XSDT etc. I seem to recall having looked at this before a couple years
>> ago or something and it might have grown some of the 64-bit XSDT logic
>> in that time. Either way, it does seem to do what I wanted. I will
>> followup. I'll probably still use the initrd approach for now.
>
> Addendum. I was able to get the initrd based ACPI table override
> working...HOWEVER...

Why do you need to override ACPI tables?  Isn't ACPI supposed to be
perfect?

> *** I had to implement a custom memcpy routine to do so ***
>
> Per some internal suggestions, I tried moving map_mem earlier (prior to
> doing boot time ACPI table parsing, in order to avoid needing to abuse
> fixmaps to touch the initrd contents), which works. For reading the
> ramdisk cpio content (the kernel code still uses a fixmap in the initrd
> override driver code to map the newly created tables in memory).
>
> But the reading of that cpio content into the new table locations is
> done using the kernel memcpy routine to early_ioremap'd memory (Device
> memory), which is architecturally sensitive to missaligned accesses.

As others have pointed out, your error is twofold: first using
early_ioremap() when early_memremap() would do, then using regular
memcpy() and not memcpy_fromio() on that.  Seems like your precious ACPI
code is to blame here.

> The in-kernel memcpy routine only checks the alignment of the source
> pointed when it begins copying, and doesn't handle the case that the
> natural alignment differs between source and destination. Therefore,
> the kernel rolls over and plays dead unless I provide a hacked up
> jcm_memcpy that just does byte copies. Then everything "works".

That's the right thing to do for normal memory on ARMv6 and up which
support unaligned accesses.  Hardware unaligned accesses are slightly
slower than aligned, so aligning the loads saves a few cycles.  The
manual shifts required to also align the destination are, however, more
expensive than simply letting the hardware do it.

-- 
M?ns Rullg?rd

[Linaro-acpi] Touching the initrd before paging_init

[Jon Masters]

On 12/15/2015 06:43 AM, M?ns Rullg?rd wrote:
> Jon Masters <jcm@redhat.com> writes:

>> But the reading of that cpio content into the new table locations is
>> done using the kernel memcpy routine to early_ioremap'd memory (Device
>> memory), which is architecturally sensitive to missaligned accesses.
> 
> As others have pointed out, your error is twofold: first using
> early_ioremap() when early_memremap() would do, then using regular
> memcpy() and not memcpy_fromio() on that.  Seems like your precious ACPI
> code is to blame here.

So I get that this could be changed. But I still want to understand
whether memcpy is behaving correctly. Will it guaranteed /never/ occur
that a copy will involve Device memory? All other such occurrences in
the kernel will be caught and fixed so this will never be an issue?

Jon.

[Linaro-acpi] Touching the initrd before paging_init

[Jon Masters]

On 12/15/2015 06:19 AM, G Gregory wrote:

<below refers to the GRUB-based approach not the initrd override>

> Tried it, results were not great :-(
> 
> set root='hd0,gpt2'
> insmod acpi
> acpi /DSDT.aml
> linux /Image-leg earlycon=pl011,0xe1010000 console=ttyAMA0 acpi=force root
> =/dev/sda2

<snip>

> [    0.000000] PC is at setup_arch+0xfc/0x564
> [    0.000000] LR is at setup_arch+0xf8/0x564

<snip>

So we unmask SError in setup_arch, which happens now later enough that
it'll come out of the UART where you can see it. However there are still
occasions on some of these early platforms where an unhandled SError can
exist as GRUB exits. I have seen that a number of times on Seattle if
there's a pending error from one of the IO IP blocks on the SoC You
might need a firmware update, but can you also confirm that this
happened reproducibly?

Jon.

[Linaro-acpi] Touching the initrd before paging_init

[Mark Rutland]

On Tue, Dec 15, 2015 at 10:29:54AM -0500, Jon Masters wrote:
> On 12/15/2015 06:43 AM, M?ns Rullg?rd wrote:
> > Jon Masters <jcm@redhat.com> writes:
> 
> >> But the reading of that cpio content into the new table locations is
> >> done using the kernel memcpy routine to early_ioremap'd memory (Device
> >> memory), which is architecturally sensitive to missaligned accesses.
> > 
> > As others have pointed out, your error is twofold: first using
> > early_ioremap() when early_memremap() would do, then using regular
> > memcpy() and not memcpy_fromio() on that.  Seems like your precious ACPI
> > code is to blame here.
> 
> So I get that this could be changed. But I still want to understand
> whether memcpy is behaving correctly. Will it guaranteed /never/ occur
> that a copy will involve Device memory?

It is guaranteed in the same sense that other problems resulting from
misuses of kernel APIs are guaranteed never to occur -- so long as the
contract between the API and the programmer is respected.

It is physically possible that a programmer may violate that contract.
It is a programming error if that happens.

> All other such occurrences in the kernel will be caught and fixed so
> this will never be an issue?

No-one has the prescience to be able to tell you whether a particular
bug will or will not be introduced in future. If the programmer pays
attention to their tools (e.g. sparse warnings about address space
mismatches), this class of error is fairly easy to avoid.

If you have found another occurrence of this issue, please report it.

Mark.

[Linaro-acpi] Touching the initrd before paging_init

[G Gregory]

On 15 December 2015 at 15:36, Jon Masters <jcm@redhat.com> wrote:
> On 12/15/2015 06:19 AM, G Gregory wrote:
>
> <below refers to the GRUB-based approach not the initrd override>
>
>> Tried it, results were not great :-(
>>
>> set root='hd0,gpt2'
>> insmod acpi
>> acpi /DSDT.aml
>> linux /Image-leg earlycon=pl011,0xe1010000 console=ttyAMA0 acpi=force root
>> =/dev/sda2
>
> <snip>
>
>> [    0.000000] PC is at setup_arch+0xfc/0x564
>> [    0.000000] LR is at setup_arch+0xf8/0x564
>
> <snip>
>
> So we unmask SError in setup_arch, which happens now later enough that
> it'll come out of the UART where you can see it. However there are still
> occasions on some of these early platforms where an unhandled SError can
> exist as GRUB exits. I have seen that a number of times on Seattle if
> there's a pending error from one of the IO IP blocks on the SoC You
> might need a firmware update, but can you also confirm that this
> happened reproducibly?
>
It is Seattle RevB I am using and it is repeatable. I have pre-release
firmware on there!

Graeme

[Linaro-acpi] Touching the initrd before paging_init

[G Gregory]

On 15 December 2015 at 16:13, G Gregory <graeme.gregory@linaro.org> wrote:
> On 15 December 2015 at 15:36, Jon Masters <jcm@redhat.com> wrote:
>> On 12/15/2015 06:19 AM, G Gregory wrote:
>>
>> <below refers to the GRUB-based approach not the initrd override>
>>
>>> Tried it, results were not great :-(
>>>
>>> set root='hd0,gpt2'
>>> insmod acpi
>>> acpi /DSDT.aml
>>> linux /Image-leg earlycon=pl011,0xe1010000 console=ttyAMA0 acpi=force root
>>> =/dev/sda2
>>
>> <snip>
>>
>>> [    0.000000] PC is at setup_arch+0xfc/0x564
>>> [    0.000000] LR is at setup_arch+0xf8/0x564
>>
>> <snip>
>>
>> So we unmask SError in setup_arch, which happens now later enough that
>> it'll come out of the UART where you can see it. However there are still
>> occasions on some of these early platforms where an unhandled SError can
>> exist as GRUB exits. I have seen that a number of times on Seattle if
>> there's a pending error from one of the IO IP blocks on the SoC You
>> might need a firmware update, but can you also confirm that this
>> happened reproducibly?
>>
> It is Seattle RevB I am using and it is repeatable. I have pre-release
> firmware on there!
>
Repeatable on ROD0084E as well.

Graeme

[Linaro-acpi] Touching the initrd before paging_init

[Måns Rullgård]

Jon Masters <jcm@redhat.com> writes:

> On 12/15/2015 06:43 AM, M?ns Rullg?rd wrote:
>> Jon Masters <jcm@redhat.com> writes:
>
>>> But the reading of that cpio content into the new table locations is
>>> done using the kernel memcpy routine to early_ioremap'd memory (Device
>>> memory), which is architecturally sensitive to missaligned accesses.
>> 
>> As others have pointed out, your error is twofold: first using
>> early_ioremap() when early_memremap() would do, then using regular
>> memcpy() and not memcpy_fromio() on that.  Seems like your precious ACPI
>> code is to blame here.
>
> So I get that this could be changed. But I still want to understand
> whether memcpy is behaving correctly.

It is.  I already explained why it's the correct way.

-- 
M?ns Rullg?rd

[Linaro-acpi] Touching the initrd before paging_init

[Jon Masters]

On 12/15/2015 11:28 AM, G Gregory wrote:
> On 15 December 2015 at 16:13, G Gregory <graeme.gregory@linaro.org> wrote:
>> On 15 December 2015 at 15:36, Jon Masters <jcm@redhat.com> wrote:
>>> On 12/15/2015 06:19 AM, G Gregory wrote:
>>>
>>> <below refers to the GRUB-based approach not the initrd override>
>>>
>>>> Tried it, results were not great :-(
>>>>
>>>> set root='hd0,gpt2'
>>>> insmod acpi
>>>> acpi /DSDT.aml
>>>> linux /Image-leg earlycon=pl011,0xe1010000 console=ttyAMA0 acpi=force root
>>>> =/dev/sda2
>>>
>>> <snip>
>>>
>>>> [    0.000000] PC is at setup_arch+0xfc/0x564
>>>> [    0.000000] LR is at setup_arch+0xf8/0x564
>>>
>>> <snip>
>>>
>>> So we unmask SError in setup_arch, which happens now later enough that
>>> it'll come out of the UART where you can see it. However there are still
>>> occasions on some of these early platforms where an unhandled SError can
>>> exist as GRUB exits. I have seen that a number of times on Seattle if
>>> there's a pending error from one of the IO IP blocks on the SoC You
>>> might need a firmware update, but can you also confirm that this
>>> happened reproducibly?
>>>
>> It is Seattle RevB I am using and it is repeatable. I have pre-release
>> firmware on there!
>>
> Repeatable on ROD0084E as well.

I'll try it on a couple of other platforms later this week. I've
pondered before whether GRUB should unmask SError and report this prior
to entering the kernel, because today Linux always gets the blame ;)

Jon.

[Linaro-acpi] Touching the initrd before paging_init

[G Gregory]

On 15 December 2015 at 16:31, Jon Masters <jcm@redhat.com> wrote:
> On 12/15/2015 11:28 AM, G Gregory wrote:
>> On 15 December 2015 at 16:13, G Gregory <graeme.gregory@linaro.org> wrote:
>>> On 15 December 2015 at 15:36, Jon Masters <jcm@redhat.com> wrote:
>>>> On 12/15/2015 06:19 AM, G Gregory wrote:
>>>>
>>>> <below refers to the GRUB-based approach not the initrd override>
>>>>
>>>>> Tried it, results were not great :-(
>>>>>
>>>>> set root='hd0,gpt2'
>>>>> insmod acpi
>>>>> acpi /DSDT.aml
>>>>> linux /Image-leg earlycon=pl011,0xe1010000 console=ttyAMA0 acpi=force root
>>>>> =/dev/sda2
>>>>
>>>> <snip>
>>>>
>>>>> [    0.000000] PC is at setup_arch+0xfc/0x564
>>>>> [    0.000000] LR is at setup_arch+0xf8/0x564
>>>>
>>>> <snip>
>>>>
>>>> So we unmask SError in setup_arch, which happens now later enough that
>>>> it'll come out of the UART where you can see it. However there are still
>>>> occasions on some of these early platforms where an unhandled SError can
>>>> exist as GRUB exits. I have seen that a number of times on Seattle if
>>>> there's a pending error from one of the IO IP blocks on the SoC You
>>>> might need a firmware update, but can you also confirm that this
>>>> happened reproducibly?
>>>>
>>> It is Seattle RevB I am using and it is repeatable. I have pre-release
>>> firmware on there!
>>>
>> Repeatable on ROD0084E as well.
>
> I'll try it on a couple of other platforms later this week. I've
> pondered before whether GRUB should unmask SError and report this prior
> to entering the kernel, because today Linux always gets the blame ;)
>
Well IMO it almost certainly should not pass control of a "broken"
machine. But I do not know anything about SError.

Graeme

[Linaro-acpi] Touching the initrd before paging_init

[G Gregory]

On 15 December 2015 at 17:08, G Gregory <graeme.gregory@linaro.org> wrote:
> On 15 December 2015 at 16:31, Jon Masters <jcm@redhat.com> wrote:
>> On 12/15/2015 11:28 AM, G Gregory wrote:
>>> On 15 December 2015 at 16:13, G Gregory <graeme.gregory@linaro.org> wrote:
>>>> On 15 December 2015 at 15:36, Jon Masters <jcm@redhat.com> wrote:
>>>>> On 12/15/2015 06:19 AM, G Gregory wrote:
>>>>>
>>>>> <below refers to the GRUB-based approach not the initrd override>
>>>>>
>>>>>> Tried it, results were not great :-(
>>>>>>
>>>>>> set root='hd0,gpt2'
>>>>>> insmod acpi
>>>>>> acpi /DSDT.aml
>>>>>> linux /Image-leg earlycon=pl011,0xe1010000 console=ttyAMA0 acpi=force root
>>>>>> =/dev/sda2
>>>>>
>>>>> <snip>
>>>>>
>>>>>> [    0.000000] PC is at setup_arch+0xfc/0x564
>>>>>> [    0.000000] LR is at setup_arch+0xf8/0x564
>>>>>
>>>>> <snip>
>>>>>
>>>>> So we unmask SError in setup_arch, which happens now later enough that
>>>>> it'll come out of the UART where you can see it. However there are still
>>>>> occasions on some of these early platforms where an unhandled SError can
>>>>> exist as GRUB exits. I have seen that a number of times on Seattle if
>>>>> there's a pending error from one of the IO IP blocks on the SoC You
>>>>> might need a firmware update, but can you also confirm that this
>>>>> happened reproducibly?
>>>>>
>>>> It is Seattle RevB I am using and it is repeatable. I have pre-release
>>>> firmware on there!
>>>>
>>> Repeatable on ROD0084E as well.
>>
>> I'll try it on a couple of other platforms later this week. I've
>> pondered before whether GRUB should unmask SError and report this prior
>> to entering the kernel, because today Linux always gets the blame ;)
>>
> Well IMO it almost certainly should not pass control of a "broken"
> machine. But I do not know anything about SError.
>
Tested on QEMU and command works as expected

[    0.000000] ACPI: Early table checksum verification disabled
[    0.000000] ACPI: RSDP 0x00000000B69F732A 000024 (v02 BOCHS )
[    0.000000] ACPI: XSDT 0x00000000B69F72DE 00004C (v01 BOCHS  BXPCFACP 0000000
1 BXPC 00000001)
[    0.000000] ACPI: SPCR 0x00000000B69F6FBA 000050 (v02 BOCHS  BXPCSPCR 0000000
1 BXPC 00000001)
[    0.000000] ACPI: MCFG 0x00000000B69F700A 00003C (v01 BOCHS  BXPCMCFG 0000000
1 BXPC 00000001)
[    0.000000] ACPI: GTDT 0x00000000B69F7046 000060 (v02 BOCHS  BXPCGTDT 0000000
1 BXPC 00000001)
[    0.000000] ACPI: APIC 0x00000000B69F70A6 0000F4 (v03 BOCHS  BXPCAPIC 0000000
1 BXPC 00000001)
[    0.000000] ACPI: FACP 0x00000000B69F719A 00010C (v05 BOCHS  BXPCFACP 0000000
1 BXPC 00000001)
[    0.000000] ACPI: DSDT 0x00000000B69F6000 000FBA (v02 XORAS  XORAXORA 0000000
1 INTL 20140926)

Can see DSDT is not the one generated from QEMU

Graeme