[PATCH] [ARM] Feroceon: fix kexec by setting outer_cache.inv_all

[PATCH] [ARM] Feroceon: fix kexec by setting outer_cache.inv_all

[Elijah Ragozin]

Hello,

I have a simple fix for kexec on Marvell Feroceon SoC.
Originally created and tested on kernel version 2.6.39.2,
but appliable for the 3.9 kernel as well.
Could you pls review. The patch is below.

From: Illia Ragozin <illia.ragozin@grapecom.com>
Subject: [PATCH] [ARM] Feroceon: fix kexec by setting outer_cache.inv_all

Before the calling kexec l2 cache must be invalidated,
this is done by outer_cache.inv_all()
For Feroceon SoC inv_all function does not set in outer_cache structure
and new kernel unpacking failed due to l2 cache still was not invalidated
at this point.

Setting outer_cache.inv_all fixes the issue.

Signed-off-by: Illia Ragozin <illia.ragozin@grapecom.com>
---
  arch/arm/mm/cache-feroceon-l2.c | 1 +
  1 file changed, 1 insertion(+)

diff --git a/arch/arm/mm/cache-feroceon-l2.c 
b/arch/arm/mm/cache-feroceon-l2.c
index e0b0e7a..09f8851 100644
--- a/arch/arm/mm/cache-feroceon-l2.c
+++ b/arch/arm/mm/cache-feroceon-l2.c
@@ -342,6 +342,7 @@ void __init feroceon_l2_init(int __l2_wt_override)
         outer_cache.inv_range = feroceon_l2_inv_range;
         outer_cache.clean_range = feroceon_l2_clean_range;
         outer_cache.flush_range = feroceon_l2_flush_range;
+       outer_cache.inv_all = l2_inv_all;

         enable_l2();

-- 
1.8.1.4

-- 
Best Regards,
Illia Ragozin.

[PATCH] [ARM] Feroceon: fix kexec by setting outer_cache.inv_all

[Jason Gunthorpe]

On Sat, Mar 23, 2013 at 10:15:44PM +0200, Elijah Ragozin wrote:
> Hello,
> 
> I have a simple fix for kexec on Marvell Feroceon SoC.
> Originally created and tested on kernel version 2.6.39.2,
> but appliable for the 3.9 kernel as well.
> Could you pls review. The patch is below.

Hmm, I am using kexec on Kirkwood (a Feroceon SOC) and haven't had any
problems with L2 coherency.. What exactly goes wrong here?

The only reason I could think of that this would make a difference is
if something along the boot path was turning on/off the L2? However if
you turn off the L2 then it needs to be invalidated before it is
turned on again. enable_l2 already does this properly..

Is the decompressor fiddling with the L2?

Jason

[PATCH] [ARM] Feroceon: fix kexec by setting outer_cache.inv_all

[Elijah Ragozin]

Hi Jason,

thanks for your answer.

That's not about the boot path. I have zImage decompression error after 
kexec.

So on this stage boot process is not started yet, enable_l2 is called 
later - after zImage decompressed and started.

Please look at arch/arm/kernel/machine_kexec.c
Just before the run the code which load and run the new kernel we do:
         setup_mm_for_reboot(0);
         flush_cache_all();
         outer_flush_all(); // <- this has no map in cache-feroceon-l2.c 
, do nothing
         outer_disable(); // <- and this has no map in 
cache-feroceon-l2.c , do nothing
         cpu_proc_fin();
         outer_inv_all(); // <- and my patch is supposed to add this 
function, w/o the patch do nothing
         flush_cache_all();
         cpu_reset(reboot_code_buffer_phys);

So there is no actually disable and invalidation cache after the old 
kernel finished and the new one is not started yet, we're not safe to 
relocate and decompress the kernel.

If in your case everything is working could you please send for me 
.config and your kernel version, probably I also miss smth.
But I think the patch is necessary, it just adds the call of function 
used for other ARMs, but missed for Feroceon.
--
BR, Illia.

On 25.03.2013 19:21, Jason Gunthorpe wrote:
> On Sat, Mar 23, 2013 at 10:15:44PM +0200, Elijah Ragozin wrote:
>> Hello,
>>
>> I have a simple fix for kexec on Marvell Feroceon SoC.
>> Originally created and tested on kernel version 2.6.39.2,
>> but appliable for the 3.9 kernel as well.
>> Could you pls review. The patch is below.
> Hmm, I am using kexec on Kirkwood (a Feroceon SOC) and haven't had any
> problems with L2 coherency.. What exactly goes wrong here?
>
> The only reason I could think of that this would make a difference is
> if something along the boot path was turning on/off the L2? However if
> you turn off the L2 then it needs to be invalidated before it is
> turned on again. enable_l2 already does this properly..
>
> Is the decompressor fiddling with the L2?
>
> Jason
>
-- 
Best Regards,
Illia Ragozin.

[PATCH] [ARM] Feroceon: fix kexec by setting outer_cache.inv_all

[Jason Gunthorpe]

On Mon, Mar 25, 2013 at 07:49:11PM +0200, Elijah Ragozin wrote:

> So there is no actually disable and invalidation cache after the old
> kernel finished and the new one is not started yet, we're not safe
> to relocate and decompress the kernel.

Upon closer inspection I found the spec passage that the L2 becomes
non-coherent with the CPU once the L1 D cache is turned off.

So, the L2 has to be invalidated after the L1 D cache is turned off,
or before the L1 D cache is turned on..

Looks likes 'cpu_proc_fin' turns off the both L1 caches, so the
invalidate is properly after.

This looks right to me then.

You might want to add some additional info to the the commit message
along the lines of:

  On Feroceon the L2 cache becomes non-coherent with the CPU when the
  L1 caches are disabled. Thus the L2 needs to be invalidated after
  both L1 caches are disabled.

  .. text about kexec being the motivation ..


> If in your case everything is working could you please send for me
> .config and your kernel version, probably I also miss smth. 

Upon deeper reflection, in my case the L2 is re-initialized by a boot
stub we use on the kexec path...

Also, I will warn you that I could not figure out how to get Linux to
shut off the DMA's in things like the ethernet, so kexec is very flaky
on these systems.

Jason

[PATCH] [ARM] Feroceon: fix kexec by setting outer_cache.inv_all

[Elijah Ragozin]

On 25.03.2013 20:13, Jason Gunthorpe wrote:
> On Mon, Mar 25, 2013 at 07:49:11PM +0200, Elijah Ragozin wrote:
>
>> So there is no actually disable and invalidation cache after the old
>> kernel finished and the new one is not started yet, we're not safe
>> to relocate and decompress the kernel.
> Upon closer inspection I found the spec passage that the L2 becomes
> non-coherent with the CPU once the L1 D cache is turned off.
>
> So, the L2 has to be invalidated after the L1 D cache is turned off,
> or before the L1 D cache is turned on..
>
> Looks likes 'cpu_proc_fin' turns off the both L1 caches, so the
> invalidate is properly after.
>
> This looks right to me then.
>
> You might want to add some additional info to the the commit message
> along the lines of:
>
>    On Feroceon the L2 cache becomes non-coherent with the CPU when the
>    L1 caches are disabled. Thus the L2 needs to be invalidated after
>    both L1 caches are disabled.
>
>    .. text about kexec being the motivation ..
>
>
>> If in your case everything is working could you please send for me
>> .config and your kernel version, probably I also miss smth.
> Upon deeper reflection, in my case the L2 is re-initialized by a boot
> stub we use on the kexec path...
>
> Also, I will warn you that I could not figure out how to get Linux to
> shut off the DMA's in things like the ethernet, so kexec is very flaky
> on these systems.
>
> Jason
>

Thank you for this clarification,
I'll rework the commit message and resend the patch then.

-- 
Best Regards,
Illia Ragozin.