* Re: Upgrading cramfs root file system while running (DENX wrote that is not possible)
From: David Jander @ 2006-04-21 6:53 UTC (permalink / raw)
To: linuxppc-embedded
In-Reply-To: <200604202303.21680.antonio.dibacco@aruba.it>
Hi,
On Thursday 20 April 2006 23:03, Antonio Di Bacco wrote:
> Yes, I also thought this too. Anything important should stay already in RAM
> but there is a chance that something bad could happen. Probably the best
What do you mean with "something bad could happen"?
The only thing I can think of is pulling the power plug while flash is being
erased or written. What else could go wrong?
We do the following: system running from read-only jffs2 partition. Sometimes
that partition is remounted read-write and single files are replaced, but in
some occasions we need to upgrade the whole fs. In that case a CGI lodas the
image into a ramdisk, and the upgrade process is started. For upgrade,
'eraseall' and 'dd' (from busybox) are needed. First, all unnecessary
processes are killed (the webserver stays alive to be able to report the
status when finished), then "dd" is called for a dummy operation (to have it
cached). After that the upgrade tool calls "eraseall" on the rootfs
partition, and then "dd" again to copy the image. At that point no critical
flash-read access should be requested since dd is already in cache (it's
busybox, so it's almost always in RAM anyway). When dd is finished, the only
other thing that's needed is to either be able to send some last html strings
to the web-server to complete the progress page and tell the user, that it's
ok to pull the plug, or reset the system, we don't care if the rest of the
system goes belly-up, since the fs was mounted read-only anyway, and the
upgrade is finished.
Of course this isn't failsafe, so there should always be a way to recover if
the rootfs gets trashed, but most of the time it's acceptable that service
personel is required in that situation. Until now, it has never been required
though.
> thing is what you suggested as second option but I have not so much ram. My
> CGI writes the downloaded new software in RAM and then I should directly
> jump to u-boot without leaving Linux the chance to mix things up and then
> u-boot should copy the RAM to the flash. It seems a strange procedure but
> what else could be done with 4MB flash and 16 MB ram?
Run from initrd? Maybe an uncomressed filesystem on a ramdisk to be able to do
XIP (execute in place)?
Greetings,
--
David Jander
^ permalink raw reply
* Re: Upgrading cramfs root file system
From: Wojciech Kromer @ 2006-04-21 6:42 UTC (permalink / raw)
To: linuxppc-embedded
In-Reply-To: <20060420221801.55331e15@White64>
Dnia 2006-04-20 22:18, Użytkownik White napisał:
> make it easy: if you start an application which do the flash and after
> this a reset.. nothing should happen. I do it that way.
> the application resist completly in RAM .. and all important libs are
> in RAm or in Filesystem Cache.
> It's only important that you pretend any Application from accessing
> Datafiles or start of new application ...
>
> Alternativly, you can put it in a reserved RAM Area ( mark it not
> usable by Linux) and put a Flash Code in your Bootloader (U-boot?)
> after a reset....
>
> But overwrite a cramfs works for me on >100 times without problems.
>
>
Problems with changing cramfs and reboot may vary depending on changes
made to filesystem.
You can't even call reboot, bacause it's not in the prevoius location
after changing flash.
> Am Thu, 20 Apr 2006 21:54:45 +0200 schrieb Antonio Di Bacco
> <antonio.dibacco@aruba.it> :
>
>
>> Yes you are right, it is not a good idea to overwrite working cramfs
>> filesystem. But what happens if I download the new cramfs plus kernel in RAM,
>> do a checksum and then, completely in kernel mode, disabling all the
>> interrupts, I write to flash? No process could complain that I am overwriting
>> because no one is executing.
>>
>>
Maybe such feature should be added to MTD code.
But disabling interrupts may cause watchdog reset in most embedded
platforms.
^ permalink raw reply
* Re: powermac S900 6 slot PCI broken in 2.6.16
From: Benjamin Herrenschmidt @ 2006-04-21 6:21 UTC (permalink / raw)
To: Robert Brose; +Cc: linuxppc-dev
In-Reply-To: <20060420152250.17407.qmail@kunk.qbjnet.com>
On Thu, 2006-04-20 at 15:22 +0000, Robert Brose wrote:
> The s900 has a 6 slot PCI bus. It works fine in 2.6.15. In 2.6.16 it
> appears the interrupts are being assigned in an incorrect way, I get
> an interrupt assignment failure on the device on the 21052 bridge.
> (notice in the interrupt assignment at the end that the stuff on the
> bridge is on irq 25 when it works in 2.6.15 and irq 1 when it doesn't
> work in 2.6.16).
can you send me a tarball of /proc/device-tree along with a complete
dmesg output and lspci -vv log as root for both 2.6.15 and .16 ? Thanks.
Ben.
^ permalink raw reply
* Re: Upgrading cramfs root file system while running (DENX wrote that is not possible)
From: antonio.dibacco @ 2006-04-21 5:51 UTC (permalink / raw)
To: Tolunay Orkun; +Cc: White, linuxppc-embedded
In-Reply-To: <44485B3F.8080308@orkun.us>
No, I have a cramfs on flash and the kernel uses it directly from flash,
extracting what it needs to execute. I'm not using initrd then, I have to
update in situ, while running.
Bye,
Antonio.
Tolunay Orkun Scrive:
> If your bootloader is U-Boot and you are using standard bootm command to
> boot, U-Boot decompresses the initrd image to RAM before passing the file
> system to Linux. So, you are not working with flash copy and updating the
> flash copy is not a problem at all. This applies to ext2, cramfs or
> squashfs based initrd.
>
> You can keep working as long as you like until it is time to reboot.
>
> Antonio Di Bacco wrote:
>> Yes, I also thought this too. Anything important should stay already in
>> RAM but there is a chance that something bad could happen. Probably the
>> best thing is what you suggested as second option but I have not so much
>> ram. My CGI writes the downloaded new software in RAM and then I should
>> directly jump to u-boot without leaving Linux the chance to mix things up
>> and then u-boot should copy the RAM to the flash. It seems a strange
>> procedure but what else could be done with 4MB flash and 16 MB ram?
>>
>> Bye,
>> Antonio.
>>
>> On Thursday 20 April 2006 22:18, White wrote:
>>> make it easy: if you start an application which do the flash and after
>>> this a reset.. nothing should happen. I do it that way.
>>> the application resist completly in RAM .. and all important libs are
>>> in RAm or in Filesystem Cache.
>>> It's only important that you pretend any Application from accessing
>>> Datafiles or start of new application ...
>>>
>>> Alternativly, you can put it in a reserved RAM Area ( mark it not
>>> usable by Linux) and put a Flash Code in your Bootloader (U-boot?)
>>> after a reset....
>>>
>>> But overwrite a cramfs works for me on >100 times without problems.
>>>
>>> Am Thu, 20 Apr 2006 21:54:45 +0200 schrieb Antonio Di Bacco
>>>
>>> <antonio.dibacco@aruba.it> :
>>>> Yes you are right, it is not a good idea to overwrite working cramfs
>>>> filesystem. But what happens if I download the new cramfs plus kernel
>>>> in
>>>> RAM, do a checksum and then, completely in kernel mode, disabling all
>>>> the
>>>> interrupts, I write to flash? No process could complain that I am
>>>> overwriting because no one is executing.
>>>>
>>>> Bye,
>>>> Antonio.
>>>>
>>>> On Wednesday 19 April 2006 09:42, Wojciech Kromer wrote:
>>>>> Dnia 2006-04-06 22:38, Użytkownik Antonio Di Bacco napisał:
>>>>>> Hi,
>>>>>>
>>>>>> how could I upgrade my cramfs rootfs? I have a CGI in the rootfs that
>>>>>> receives the new rootfs from a web interface and then tries to write
>>>>>> it in the flash. While overwriting the old cramfs, the CGI will
>>>>>> continue to work? something weird could happen?
>>>>> Generally it's not a good idea to override working filesystem ( I've
>>>>> tried to do it once).
>>>>>
>>>>> You can have two separate copies of filesystem, one to work with, and
>>>>> another to overwrite, it requires more flash.
>>>>> Another way is working in initrd, it requires more RAM.
>>>>> You can also use jffs2 or jffs3 (experimental) to have read-write
>>>>> filesystem, and change applications only, not whole filesystem (be
>>>>> carefull with changing busybox or libraries!)
>>>> _______________________________________________
>>>> Linuxppc-embedded mailing list
>>>> Linuxppc-embedded@ozlabs.org
>>>> https://ozlabs.org/mailman/listinfo/linuxppc-embedded
>>> _______________________________________________
>>> Linuxppc-embedded mailing list
>>> Linuxppc-embedded@ozlabs.org
>>> https://ozlabs.org/mailman/listinfo/linuxppc-embedded
>> _______________________________________________
>> Linuxppc-embedded mailing list
>> Linuxppc-embedded@ozlabs.org
>> https://ozlabs.org/mailman/listinfo/linuxppc-embedded
>
^ permalink raw reply
* Re: Not coherent cache DMA for G3/G4 CPUs: clarification needed
From: Benjamin Herrenschmidt @ 2006-04-21 4:38 UTC (permalink / raw)
To: Eugene Surovegin; +Cc: debian-powerpc, linuxppc-dev
In-Reply-To: <20060420215514.GE25755@gate.ebshome.net>
On Thu, 2006-04-20 at 14:55 -0700, Eugene Surovegin wrote:
> On Thu, Apr 20, 2006 at 11:10:55PM +0200, Gerhard Pircher wrote:
> > Well, Freescale's PPC programming environment manual clearly states that
> > this will not work on G4 CPUs (74xx). Also Benjamin Herrenschmidt told me,
> > that this implementation will not work for the reasons I mentioned before.
> > The approach I'm trying to implement was his idea, so I have to trust in
> > him.
>
> Well, you aren't the first person who tries to run G4 with
> CONFIG_NOT_COHERENT_CACHE. This was done before and I don't remember
> that those people had to implement anything as complex as you are
> trying to do.
>
> You can try asking on #mklinux. It always better to ask people who
> actually _did_ this :).
>
> In fact, I just grepped 2.6 and found
> #ifdef(CONFIG_NOT_COHERENT_CACHE) in syslib/mv64x60.c. Guess what
> systems usually have this type of bridge? Not 4xx/8xx, that's for sure.
I think some folks tried ... and failed.
Ben.
^ permalink raw reply
* Re: Upgrading cramfs root file system while running (DENX wrote that is not possible)
From: Tolunay Orkun @ 2006-04-21 4:10 UTC (permalink / raw)
To: Antonio Di Bacco; +Cc: White, linuxppc-embedded
In-Reply-To: <200604202303.21680.antonio.dibacco@aruba.it>
If your bootloader is U-Boot and you are using standard bootm command to
boot, U-Boot decompresses the initrd image to RAM before passing the
file system to Linux. So, you are not working with flash copy and
updating the flash copy is not a problem at all. This applies to ext2,
cramfs or squashfs based initrd.
You can keep working as long as you like until it is time to reboot.
Antonio Di Bacco wrote:
> Yes, I also thought this too. Anything important should stay already in RAM
> but there is a chance that something bad could happen. Probably the best
> thing is what you suggested as second option but I have not so much ram. My
> CGI writes the downloaded new software in RAM and then I should directly jump
> to u-boot without leaving Linux the chance to mix things up and then u-boot
> should copy the RAM to the flash. It seems a strange procedure but what else
> could be done with 4MB flash and 16 MB ram?
>
> Bye,
> Antonio.
>
> On Thursday 20 April 2006 22:18, White wrote:
>> make it easy: if you start an application which do the flash and after
>> this a reset.. nothing should happen. I do it that way.
>> the application resist completly in RAM .. and all important libs are
>> in RAm or in Filesystem Cache.
>> It's only important that you pretend any Application from accessing
>> Datafiles or start of new application ...
>>
>> Alternativly, you can put it in a reserved RAM Area ( mark it not
>> usable by Linux) and put a Flash Code in your Bootloader (U-boot?)
>> after a reset....
>>
>> But overwrite a cramfs works for me on >100 times without problems.
>>
>> Am Thu, 20 Apr 2006 21:54:45 +0200 schrieb Antonio Di Bacco
>>
>> <antonio.dibacco@aruba.it> :
>>> Yes you are right, it is not a good idea to overwrite working cramfs
>>> filesystem. But what happens if I download the new cramfs plus kernel in
>>> RAM, do a checksum and then, completely in kernel mode, disabling all the
>>> interrupts, I write to flash? No process could complain that I am
>>> overwriting because no one is executing.
>>>
>>> Bye,
>>> Antonio.
>>>
>>> On Wednesday 19 April 2006 09:42, Wojciech Kromer wrote:
>>>> Dnia 2006-04-06 22:38, Użytkownik Antonio Di Bacco napisał:
>>>>> Hi,
>>>>>
>>>>> how could I upgrade my cramfs rootfs? I have a CGI in the rootfs that
>>>>> receives the new rootfs from a web interface and then tries to write
>>>>> it in the flash. While overwriting the old cramfs, the CGI will
>>>>> continue to work? something weird could happen?
>>>> Generally it's not a good idea to override working filesystem ( I've
>>>> tried to do it once).
>>>>
>>>> You can have two separate copies of filesystem, one to work with, and
>>>> another to overwrite, it requires more flash.
>>>> Another way is working in initrd, it requires more RAM.
>>>> You can also use jffs2 or jffs3 (experimental) to have read-write
>>>> filesystem, and change applications only, not whole filesystem (be
>>>> carefull with changing busybox or libraries!)
>>> _______________________________________________
>>> Linuxppc-embedded mailing list
>>> Linuxppc-embedded@ozlabs.org
>>> https://ozlabs.org/mailman/listinfo/linuxppc-embedded
>> _______________________________________________
>> Linuxppc-embedded mailing list
>> Linuxppc-embedded@ozlabs.org
>> https://ozlabs.org/mailman/listinfo/linuxppc-embedded
> _______________________________________________
> Linuxppc-embedded mailing list
> Linuxppc-embedded@ozlabs.org
> https://ozlabs.org/mailman/listinfo/linuxppc-embedded
^ permalink raw reply
* [PATCH] powerpc/pseries: avoid crash in PCI code if mem system not up.
From: Linas Vepstas @ 2006-04-21 0:40 UTC (permalink / raw)
To: Paul Mackerras; +Cc: linuxppc-dev, linux-kernel
Paul,
Please apply and forward upstream. And a question: once
upon a time, the arch PCI subsystem was inited after mem init
was done; currently, it seems to be happening before mem init.
Is this intentional?
--linas
[PATCH] powerpc/pseries: avoid crash in PCI code if mem system not up.
The powerpc code is currently performing PCI setup before
memory initialization. PCI setup touches PCI config space
registers. If the PCI card is bad, this will evoke an error,
which currrently can't be handled, as the PCI error recovery
code expects kmalloc() to be functional. This patch will
cause the system to punt instead of crashing with
cpu 0x0: Vector: 300 (Data Access) at [c0000000004434d0]
pc: c0000000000c06b4: .kmem_cache_alloc+0x8c/0xf4
lr: c00000000004ad6c: .eeh_send_failure_event+0x48/0xfc
This patch Will also print name of the offending pci device.
Signed-off-by: Linas Vepstas <linas@austin.ibm.com>
----
arch/powerpc/platforms/pseries/eeh_event.c | 9 +++++++++
1 files changed, 9 insertions(+)
Index: linux-2.6.17-rc1/arch/powerpc/platforms/pseries/eeh_event.c
===================================================================
--- linux-2.6.17-rc1.orig/arch/powerpc/platforms/pseries/eeh_event.c 2006-04-05 09:56:38.000000000 -0500
+++ linux-2.6.17-rc1/arch/powerpc/platforms/pseries/eeh_event.c 2006-04-20 19:29:48.000000000 -0500
@@ -124,7 +124,16 @@ int eeh_send_failure_event (struct devic
{
unsigned long flags;
struct eeh_event *event;
+ char *location;
+ if (!mem_init_done)
+ {
+ printk(KERN_ERR "EEH: event during early boot not handled\n");
+ location = (char *) get_property(dn, "ibm,loc-code", NULL);
+ printk(KERN_ERR "EEH: device node = %s\n", dn->full_name);
+ printk(KERN_ERR "EEH: PCI location = %s\n", location);
+ return 1;
+ }
event = kmalloc(sizeof(*event), GFP_ATOMIC);
if (event == NULL) {
printk (KERN_ERR "EEH: out of memory, event not handled\n");
^ permalink raw reply
* Re: Not coherent cache DMA for G3/G4 CPUs: clarification needed
From: Benjamin Herrenschmidt @ 2006-04-21 0:09 UTC (permalink / raw)
To: Gabriel Paubert; +Cc: debian-powerpc, linuxppc-dev
In-Reply-To: <20060420234640.GA20059@iram.es>
> In this case the problem is double mapping with inconsistent attributes
> (through BAT and page tables I assume).
Yes.
> > On POWER4, 970 and later, the chip guys confirmed that the problem is
> > real though. Not only bcs of prefetch but also speculative execution
> > which can cause the chip to do a load that will never actually be
> > executed. Imagine for example a loop walking an array, the chip might
> > speculatively load elements beyond the array by speculatively executing
> > beyond the branch that ends the loop.
>
> Even if the page has the guarded bit set?
The BAT mapping doesn't have G set.
Ben.
^ permalink raw reply
* Re: Not coherent cache DMA for G3/G4 CPUs: clarification needed
From: Gabriel Paubert @ 2006-04-20 23:46 UTC (permalink / raw)
To: Benjamin Herrenschmidt; +Cc: debian-powerpc, linuxppc-dev
In-Reply-To: <1145572770.4517.8.camel@localhost.localdomain>
On Fri, Apr 21, 2006 at 08:39:29AM +1000, Benjamin Herrenschmidt wrote:
> On Thu, 2006-04-20 at 14:13 -0700, Eugene Surovegin wrote:
> > On Fri, Apr 21, 2006 at 07:06:13AM +1000, Benjamin Herrenschmidt wrote:
> > > Unfortunately, he has to do things a bit differently. He can't afford to
> > > have the kernel BAT mapping cover his non-cacheable pages. Thus he needs
> > > a reserved pool. Last I looked at our coherent code, it didn't reserve
> > > memory at all, just address space, thus assuming the CPU can handle
> > > having both a caheable and a non-cacheable mapping of the same pages...
> > > (On 6xx this is deadly even if you don't access those cacheable pages
> > > because the CPU prefetch may do it for you).
> >
> > Ben, is this _real_ problem on 6xx or just a theory? Does 6xx actually
> > prefetch beyond page boundary?
> >
> > So far, all "prefetching" I saw which broke non-coherent DMA was not
> > due to the CPU doing prefetching, but _software_ prefetching being
> > too aggressive.
>
> Not 100% certain... we definitely have a bug with AGP on macs currently
> for that reason though I yet have to isolate a crash caused by it ;)
> (That is, we map AGP pages non-cacheable but they stay in the linear
> mapping).
In this case the problem is double mapping with inconsistent attributes
(through BAT and page tables I assume).
>
> On POWER4, 970 and later, the chip guys confirmed that the problem is
> real though. Not only bcs of prefetch but also speculative execution
> which can cause the chip to do a load that will never actually be
> executed. Imagine for example a loop walking an array, the chip might
> speculatively load elements beyond the array by speculatively executing
> beyond the branch that ends the loop.
Even if the page has the guarded bit set?
Gabriel
^ permalink raw reply
* Re: [patch] powerpc: move PAGE_SIZE outside #ifdef __KERNEL__
From: Paul Mackerras @ 2006-04-20 23:07 UTC (permalink / raw)
To: Geoff Levand; +Cc: linuxppc-dev
In-Reply-To: <4447C3BA.6050709@am.sony.com>
Geoff Levand writes:
> The procps package needs PAGE_SIZE, defined in asm-powerpc/page.h,
Ummm... why? If procps is assuming that the page size of the kernel
it's running on is the same as the page size of the kernel it was
compiled on, it's broken. It should be using sysconf(PAGE_SIZE) at
runtime.
Paul.
^ permalink raw reply
* Re: Not coherent cache DMA for G3/G4 CPUs: clarification needed
From: Benjamin Herrenschmidt @ 2006-04-20 22:41 UTC (permalink / raw)
To: Eugene Surovegin; +Cc: debian-powerpc, linuxppc-dev
In-Reply-To: <20060420213316.GD25755@gate.ebshome.net>
On Thu, 2006-04-20 at 14:33 -0700, Eugene Surovegin wrote:
> On Fri, Apr 21, 2006 at 07:06:13AM +1000, Benjamin Herrenschmidt wrote:
> > (On 6xx this is deadly even if you don't access those cacheable pages
> > because the CPU prefetch may do it for you).
>
> Here is another thought if this "prefetch" theory is correct.
>
> You guys seems to focus on
> dma_alloc_coherent()/pci_alloc_consistent(), but forgeting about so
> called "streaming" mappings.
>
> You cannot just flush/invalidate cache any more, because "CPU can
> prefetch this data back". So, to be completely correct (if you insist
> on "6xx can prefetch"-theory), you have to actually _copy_ data to
> your consistent memory on dma_map_single(). You can imagine
> performance implications. I suspect even 440 will be faster in this
> case than G4 :).
Yes.
Ben.
^ permalink raw reply
* Re: Not coherent cache DMA for G3/G4 CPUs: clarification needed
From: Benjamin Herrenschmidt @ 2006-04-20 22:40 UTC (permalink / raw)
To: Eugene Surovegin; +Cc: debian-powerpc, linuxppc-dev
In-Reply-To: <20060420211908.GC25755@gate.ebshome.net>
On Thu, 2006-04-20 at 14:19 -0700, Eugene Surovegin wrote:
> On Thu, Apr 20, 2006 at 02:13:21PM -0700, Eugene Surovegin wrote:
> > On Fri, Apr 21, 2006 at 07:06:13AM +1000, Benjamin Herrenschmidt wrote:
> > > Unfortunately, he has to do things a bit differently. He can't afford to
> > > have the kernel BAT mapping cover his non-cacheable pages. Thus he needs
> > > a reserved pool. Last I looked at our coherent code, it didn't reserve
> > > memory at all, just address space, thus assuming the CPU can handle
> > > having both a caheable and a non-cacheable mapping of the same pages...
> > > (On 6xx this is deadly even if you don't access those cacheable pages
> > > because the CPU prefetch may do it for you).
> >
> > Ben, is this _real_ problem on 6xx or just a theory? Does 6xx actually
> > prefetch beyond page boundary?
> >
> > So far, all "prefetching" I saw which broke non-coherent DMA was not
> > due to the CPU doing prefetching, but _software_ prefetching being
> > too aggressive.
>
> Even if this "prefetching" problem is real, instead of implementing
> separate pool for allocations which will be quite rare at best, just
> allocate guard space before your consistent memory and stop worrying
> about it.
Won't necessarily help with the speculative execution problem and in
fact, how do you do that in practice ?
Ben.
^ permalink raw reply
* Re: Not coherent cache DMA for G3/G4 CPUs: clarification needed
From: Benjamin Herrenschmidt @ 2006-04-20 22:39 UTC (permalink / raw)
To: Eugene Surovegin; +Cc: debian-powerpc, linuxppc-dev
In-Reply-To: <20060420211321.GB25755@gate.ebshome.net>
On Thu, 2006-04-20 at 14:13 -0700, Eugene Surovegin wrote:
> On Fri, Apr 21, 2006 at 07:06:13AM +1000, Benjamin Herrenschmidt wrote:
> > Unfortunately, he has to do things a bit differently. He can't afford to
> > have the kernel BAT mapping cover his non-cacheable pages. Thus he needs
> > a reserved pool. Last I looked at our coherent code, it didn't reserve
> > memory at all, just address space, thus assuming the CPU can handle
> > having both a caheable and a non-cacheable mapping of the same pages...
> > (On 6xx this is deadly even if you don't access those cacheable pages
> > because the CPU prefetch may do it for you).
>
> Ben, is this _real_ problem on 6xx or just a theory? Does 6xx actually
> prefetch beyond page boundary?
>
> So far, all "prefetching" I saw which broke non-coherent DMA was not
> due to the CPU doing prefetching, but _software_ prefetching being
> too aggressive.
Not 100% certain... we definitely have a bug with AGP on macs currently
for that reason though I yet have to isolate a crash caused by it ;)
(That is, we map AGP pages non-cacheable but they stay in the linear
mapping).
On POWER4, 970 and later, the chip guys confirmed that the problem is
real though. Not only bcs of prefetch but also speculative execution
which can cause the chip to do a load that will never actually be
executed. Imagine for example a loop walking an array, the chip might
speculatively load elements beyond the array by speculatively executing
beyond the branch that ends the loop.
Ben.
^ permalink raw reply
* Re: Not coherent cache DMA for G3/G4 CPUs: clarification needed
From: Gerhard Pircher @ 2006-04-20 22:26 UTC (permalink / raw)
To: Gabriel Paubert; +Cc: linuxppc-dev, debian-powerpc
In-Reply-To: <20060420220707.GA17593@iram.es>
> --- Ursprüngliche Nachricht ---
> Von: Gabriel Paubert <paubert@iram.es>
> An: Gerhard Pircher <gerhard_pircher@gmx.net>
> Kopie: linuxppc-dev@ozlabs.org, debian-powerpc@lists.debian.org
> Betreff: Re: Not coherent cache DMA for G3/G4 CPUs: clarification needed
> Datum: Fri, 21 Apr 2006 00:07:08 +0200
>
> More details please what are the exact capabilities of the south and
> host bridges?
The southbridge is a VIA82C686B, which supports ISA DMA in the first 16MB.
The host bridge is a MAI ArticiaS. The ArticiaS has a bug in the snoop
signal logic and therefore does not support cache coherent DMA.
> I've never needed (and therefore) used floppy on my PreP boards (Motorola
> MVME2[467]xx series), but they have a south bridge (WinBond) that has 32
> bit DMA capability. This was specified in the PreP spec.
Oh, I thought PReP specifies only 24bit DMA. Okay, so the AmigaOne is more
like the i386 platform, just with a PPC cpu. ;-)
> This may also depend on the host bridge since RAM appears at 2GB on
> default PreP machines, which is an area that you can't access with
> normal ISA DMA anyway. On the MVME machines, you could map PCI addresses
> 0-16 MB anywhere in RAM by reprogramming the host bridge.
This is not the case for the AmigaOne. The RAM starts at physical address 0
(similar to CHRP). AFAIK the host bridge does not allow the remapping of the
address space. Maybe the southbridge can do this for DMA operation. I have
to investigate this. Thanks for the hint!
> > 3. How are DMA buffers used outside the kernel? Do user programs get a
> > pointer to the DMA buffer (in theory) from the device driver or is the
> > data copied to another buffer allocated by an user program?
>
> If your memory is uncacheable, you are better off copying it to
> cacheable memory. At least you are sure that you only access it
> once (trying to copy with FP registers to halve the number of
> accesses might be a big win, but you need to be careful).
Sounds like a big performance loss. I hope this is not necessary.
Thanks,
Gerhard
--
GMX Produkte empfehlen und ganz einfach Geld verdienen!
Satte Provisionen für GMX Partner: http://www.gmx.net/de/go/partner
^ permalink raw reply
* Re: Not coherent cache DMA for G3/G4 CPUs: clarification needed
From: Gabriel Paubert @ 2006-04-20 22:07 UTC (permalink / raw)
To: Gerhard Pircher; +Cc: linuxppc-dev, debian-powerpc
In-Reply-To: <28892.1145559466@www088.gmx.net>
On Thu, Apr 20, 2006 at 08:57:46PM +0200, Gerhard Pircher wrote:
> Hi,
>
> I try to implement not coherent cache/DMA support for G3/G4 processors, by
> reserving some physical memory for DMA operations. The memory used for
> consistent allocations (removed from the top of the physical memory below
> 896MB) is excluded from the BAT mapping and the pages are marked as
> reserved. This seems to work just fine, although I still have to mark the
> pages as cache inhibited.
>
> Whilst working on this workaround for the AmigaOne and reading some articles
> about the Linux kernel page tables and memory management, I came to the
> conclusion that there may be some problems with this approach for not
> coherent DMA:
>
> 1. The AmigaOne is similar to the PREP platform, i.e. DMA can only be
> performed in the first 16MB for ISA devices (there's only a VIA southbridge,
> no other SuperI/O IC with 32bit capable DMA controller).
More details please what are the exact capabilities of the south and
host bridges?
I've never needed (and therefore) used floppy on my PreP boards (Motorola
MVME2[467]xx series), but they have a south bridge (WinBond) that has 32
bit DMA capability. This was specified in the PreP spec.
This may also depend on the host bridge since RAM appears at 2GB on
default PreP machines, which is an area that you can't access with
normal ISA DMA anyway. On the MVME machines, you could map PCI addresses
0-16 MB anywhere in RAM by reprogramming the host bridge.
However I actually reprogram the chipset to look like CHRP, i.e.,
1:1 mapping of RAM to PCI. This caused problems a long time
ago since sometimes DMA went to VGA video memory instead of RAM.
This was when kernels were not bloated enough to at least occupy
768kB of RAM, nowadays there is strictly no risk.
> 3. How are DMA buffers used outside the kernel? Do user programs get a
> pointer to the DMA buffer (in theory) from the device driver or is the data
> copied to another buffer allocated by an user program?
If your memory is uncacheable, you are better off copying it to
cacheable memory. At least you are sure that you only access it
once (trying to copy with FP registers to halve the number of
accesses might be a big win, but you need to be careful).
Regards,
Gabriel
^ permalink raw reply
* Re: Not coherent cache DMA for G3/G4 CPUs: clarification needed
From: Gerhard Pircher @ 2006-04-20 22:08 UTC (permalink / raw)
To: Eugene Surovegin; +Cc: linuxppc-dev, debian-powerpc
In-Reply-To: <20060420215514.GE25755@gate.ebshome.net>
> --- Ursprüngliche Nachricht ---
> Von: Eugene Surovegin <ebs@ebshome.net>
> An: Gerhard Pircher <gerhard_pircher@gmx.net>
> Kopie: linuxppc-dev@ozlabs.org, debian-powerpc@lists.debian.org
> Betreff: Re: Not coherent cache DMA for G3/G4 CPUs: clarification needed
> Datum: Thu, 20 Apr 2006 14:55:14 -0700
>
> Well, you aren't the first person who tries to run G4 with
> CONFIG_NOT_COHERENT_CACHE. This was done before and I don't remember
> that those people had to implement anything as complex as you are
> trying to do.
Maybe these systems have cache coherent northbridges, which is not the case
for the AmigaOne and its "famous" ArticiaS northbridge.
> You can try asking on #mklinux. It always better to ask people who
> actually _did_ this :).
>
> In fact, I just grepped 2.6 and found
> #ifdef(CONFIG_NOT_COHERENT_CACHE) in syslib/mv64x60.c. Guess what
> systems usually have this type of bridge? Not 4xx/8xx, that's for sure.
Hmm, strange. AFAIK the NOT_COHERENT_CACHE config option is available only
for the 4xx and 8xx platforms. Wouldn't the config option depend on
CONFIG_6XX too, if there are not cache coherent systems with G4 cpus?
At least I could not compile in the dma-mapping.c file without modifying the
Kconfig file.
> Good luck.
Thanks!
Gerhard
--
Analog-/ISDN-Nutzer sparen mit GMX SmartSurfer bis zu 70%!
Kostenlos downloaden: http://www.gmx.net/de/go/smartsurfer
^ permalink raw reply
* Re: Not coherent cache DMA for G3/G4 CPUs: clarification needed
From: Eugene Surovegin @ 2006-04-20 21:55 UTC (permalink / raw)
To: Gerhard Pircher; +Cc: linuxppc-dev, debian-powerpc
In-Reply-To: <10026.1145567455@www088.gmx.net>
On Thu, Apr 20, 2006 at 11:10:55PM +0200, Gerhard Pircher wrote:
> Well, Freescale's PPC programming environment manual clearly states that
> this will not work on G4 CPUs (74xx). Also Benjamin Herrenschmidt told me,
> that this implementation will not work for the reasons I mentioned before.
> The approach I'm trying to implement was his idea, so I have to trust in
> him.
Well, you aren't the first person who tries to run G4 with
CONFIG_NOT_COHERENT_CACHE. This was done before and I don't remember
that those people had to implement anything as complex as you are
trying to do.
You can try asking on #mklinux. It always better to ask people who
actually _did_ this :).
In fact, I just grepped 2.6 and found
#ifdef(CONFIG_NOT_COHERENT_CACHE) in syslib/mv64x60.c. Guess what
systems usually have this type of bridge? Not 4xx/8xx, that's for sure.
Good luck.
--
Eugene
^ permalink raw reply
* Re: Upgrading cramfs root file system while running (DENX wrote that is not possible)
From: Antonio Di Bacco @ 2006-04-20 21:37 UTC (permalink / raw)
To: Alessandro Rubini; +Cc: linuxppc-embedded
In-Reply-To: <20060420211120.GA3546@mail.gnudd.com>
Flashing by myself means that I cannot use the mtd drivers? Then I cannot
erase the flash as usual but I should reproduce in the application the
algorithm to erase it, isn't it?
Bye,
Antonio.
P.S: I cannot believe that the author of the best book I've ever read,
answered my question.
On Thursday 20 April 2006 23:11, Alessandro Rubini wrote:
> > but there is a chance that something bad could happen.
>
> If you mlockall(2) and sched_setscheduler(SCHED_FIFO) you can be
> somewhat confident that nothing bad can happen. However, you should flash
> by yourself (in the application, without resorting to the kernel.
>
> But please remember that Denk is right most of the time.
>
> /alessandro
^ permalink raw reply
* Re: Not coherent cache DMA for G3/G4 CPUs: clarification needed
From: Gerhard Pircher @ 2006-04-20 21:33 UTC (permalink / raw)
To: Benjamin Herrenschmidt; +Cc: linuxppc-dev, debian-powerpc
In-Reply-To: <1145567025.17087.77.camel@localhost.localdomain>
> --- Ursprüngliche Nachricht ---
> Von: Benjamin Herrenschmidt <benh@kernel.crashing.org>
> An: Gerhard Pircher <gerhard_pircher@gmx.net>
> Kopie: linuxppc-dev@ozlabs.org, debian-powerpc@lists.debian.org
> Betreff: Re: Not coherent cache DMA for G3/G4 CPUs: clarification needed
> Datum: Fri, 21 Apr 2006 07:03:45 +1000
>
> On Thu, 2006-04-20 at 20:57 +0200, Gerhard Pircher wrote:
>
> > 1. The AmigaOne is similar to the PREP platform, i.e. DMA can only be
> > performed in the first 16MB for ISA devices (there's only a VIA
> > southbridge, no other SuperI/O IC with 32bit capable DMA controller).
> > I guess the first 16MB cannot be reserved for not coherent DMA
> > operation, because this memory area is occupied by kernel data? (not to
> > talk about the performance loss, if the kernel data area would be
> > excluded from the BAT mapping).
>
> Yeah that would suck. Are you sure you need ISA DMA ? You can do pseudo
> DMA like pegasos for the floppy. Anything else should be able to do 32
> bits DMA unless you have a very broken chipset.
I hope not. But I think the AmigaOne floppy driver is copied from the i386
architecture and this one uses DMA IIRC. On the side: who uses floppy drives
these days?
> > 2. I'm not sure how to allocate memory for DMA operation. I think
> > alloc_pages() will not do the job for me, as the page tables for not
> > coherent DMA are reserved (SetPageReserved()) and removed from the
> > available lowmem.
>
> alloc_pages() ? How so ? No, you want to allocate from your reserved
> pool, you'll have to implement your own allocator. Easiest is a running
> bitmap, look at ppc64 iommu code for an example.
Thanks! I was searching a while for an example, but couldn't found one.
> > Also memory fragmentation may be a problem, if a lot DMA operations
> > with different buffer sizes are performed. Therefore a system could
> > quickly run out of memory for not coherent DMA operation, right?
> > Is there a way to minimize fragmentation?
>
> Best you can do is have separate pools for small and big allocations I
> suppose.
Okay. Or is there a general slab allocator implementation in the Linux
kernel?
> > 3. How are DMA buffers used outside the kernel? Do user programs get a
> > pointer to the DMA buffer (in theory) from the device driver or is the
> > data copied to another buffer allocated by an user program?
>
> There are cases where some drivers try that but you should ignore it for
> the moment. Won't happen in most cases.
That's good to hear. Makes my life a little bit easier. ;-)
Thanks,
Gerhard
--
Analog-/ISDN-Nutzer sparen mit GMX SmartSurfer bis zu 70%!
Kostenlos downloaden: http://www.gmx.net/de/go/smartsurfer
^ permalink raw reply
* Re: Not coherent cache DMA for G3/G4 CPUs: clarification needed
From: Eugene Surovegin @ 2006-04-20 21:33 UTC (permalink / raw)
To: Benjamin Herrenschmidt; +Cc: debian-powerpc, linuxppc-dev
In-Reply-To: <1145567174.4517.1.camel@localhost.localdomain>
On Fri, Apr 21, 2006 at 07:06:13AM +1000, Benjamin Herrenschmidt wrote:
> (On 6xx this is deadly even if you don't access those cacheable pages
> because the CPU prefetch may do it for you).
Here is another thought if this "prefetch" theory is correct.
You guys seems to focus on
dma_alloc_coherent()/pci_alloc_consistent(), but forgeting about so
called "streaming" mappings.
You cannot just flush/invalidate cache any more, because "CPU can
prefetch this data back". So, to be completely correct (if you insist
on "6xx can prefetch"-theory), you have to actually _copy_ data to
your consistent memory on dma_map_single(). You can imagine
performance implications. I suspect even 440 will be faster in this
case than G4 :).
--
Eugene
^ permalink raw reply
* Re: Not coherent cache DMA for G3/G4 CPUs: clarification needed
From: Eugene Surovegin @ 2006-04-20 21:19 UTC (permalink / raw)
To: Benjamin Herrenschmidt, Gerhard Pircher, linuxppc-dev,
debian-powerpc
In-Reply-To: <20060420211321.GB25755@gate.ebshome.net>
On Thu, Apr 20, 2006 at 02:13:21PM -0700, Eugene Surovegin wrote:
> On Fri, Apr 21, 2006 at 07:06:13AM +1000, Benjamin Herrenschmidt wrote:
> > Unfortunately, he has to do things a bit differently. He can't afford to
> > have the kernel BAT mapping cover his non-cacheable pages. Thus he needs
> > a reserved pool. Last I looked at our coherent code, it didn't reserve
> > memory at all, just address space, thus assuming the CPU can handle
> > having both a caheable and a non-cacheable mapping of the same pages...
> > (On 6xx this is deadly even if you don't access those cacheable pages
> > because the CPU prefetch may do it for you).
>
> Ben, is this _real_ problem on 6xx or just a theory? Does 6xx actually
> prefetch beyond page boundary?
>
> So far, all "prefetching" I saw which broke non-coherent DMA was not
> due to the CPU doing prefetching, but _software_ prefetching being
> too aggressive.
Even if this "prefetching" problem is real, instead of implementing
separate pool for allocations which will be quite rare at best, just
allocate guard space before your consistent memory and stop worrying
about it.
--
Eugene
^ permalink raw reply
* Re: Not coherent cache DMA for G3/G4 CPUs: clarification needed
From: Eugene Surovegin @ 2006-04-20 21:13 UTC (permalink / raw)
To: Benjamin Herrenschmidt; +Cc: debian-powerpc, linuxppc-dev
In-Reply-To: <1145567174.4517.1.camel@localhost.localdomain>
On Fri, Apr 21, 2006 at 07:06:13AM +1000, Benjamin Herrenschmidt wrote:
> Unfortunately, he has to do things a bit differently. He can't afford to
> have the kernel BAT mapping cover his non-cacheable pages. Thus he needs
> a reserved pool. Last I looked at our coherent code, it didn't reserve
> memory at all, just address space, thus assuming the CPU can handle
> having both a caheable and a non-cacheable mapping of the same pages...
> (On 6xx this is deadly even if you don't access those cacheable pages
> because the CPU prefetch may do it for you).
Ben, is this _real_ problem on 6xx or just a theory? Does 6xx actually
prefetch beyond page boundary?
So far, all "prefetching" I saw which broke non-coherent DMA was not
due to the CPU doing prefetching, but _software_ prefetching being
too aggressive.
--
Eugene
^ permalink raw reply
* Re: Not coherent cache DMA for G3/G4 CPUs: clarification needed
From: Gerhard Pircher @ 2006-04-20 21:10 UTC (permalink / raw)
To: Eugene Surovegin; +Cc: linuxppc-dev, debian-powerpc
In-Reply-To: <20060420210201.GA25755@gate.ebshome.net>
> --- Ursprüngliche Nachricht ---
> Von: Eugene Surovegin <ebs@ebshome.net>
> An: Gerhard Pircher <gerhard_pircher@gmx.net>
> Kopie: linuxppc-dev@ozlabs.org, debian-powerpc@lists.debian.org
> Betreff: Re: Not coherent cache DMA for G3/G4 CPUs: clarification needed
> Datum: Thu, 20 Apr 2006 14:02:01 -0700
>
> On Thu, Apr 20, 2006 at 10:56:33PM +0200, Gerhard Pircher wrote:
> > > --- Urspr?ngliche Nachricht ---
> > > Von: Eugene Surovegin <ebs@ebshome.net>
> > > An: Gerhard Pircher <gerhard_pircher@gmx.net>
> > > Kopie: linuxppc-dev@ozlabs.org, debian-powerpc@lists.debian.org
> > > Betreff: Re: Not coherent cache DMA for G3/G4 CPUs: clarification
> > > needed
> > > Datum: Thu, 20 Apr 2006 13:38:48 -0700
> > >
> > > There are already non-coherent cache PPC archs (8xx, 4xx) just look
> > > how all this implemented there, don't reinvent the wheel.
> > >
> > > Also, read Documentation/DMA-API.txt and DMA-mapping.txt
> > I know! Unfortunately this implementation does not work at all with
> > G3/G4 PPC desktop CPUs for various reasons (for example due to the BAT
> > mapping, page tables with different access attributes for the same
> > physical memory area allocated by the consistent DMA functions, etc.).
>
> We have the same situation on 44x (all kernel memory is mapped
> through several big TLBs and consistent functions allocate additional
> cache-inhibited mappings for the same physical pages).
Well, Freescale's PPC programming environment manual clearly states that
this will not work on G4 CPUs (74xx). Also Benjamin Herrenschmidt told me,
that this implementation will not work for the reasons I mentioned before.
The approach I'm trying to implement was his idea, so I have to trust in
him.
regards,
Gerhard
--
GMX Produkte empfehlen und ganz einfach Geld verdienen!
Satte Provisionen für GMX Partner: http://www.gmx.net/de/go/partner
^ permalink raw reply
* Re: Not coherent cache DMA for G3/G4 CPUs: clarification needed
From: Benjamin Herrenschmidt @ 2006-04-20 21:06 UTC (permalink / raw)
To: Eugene Surovegin; +Cc: debian-powerpc, linuxppc-dev
In-Reply-To: <20060420203848.GA23192@gate.ebshome.net>
> > 3. How are DMA buffers used outside the kernel? Do user programs get a
> > pointer to the DMA buffer (in theory) from the device driver or is the data
> > copied to another buffer allocated by an user program?
>
>
> There are already non-coherent cache PPC archs (8xx, 4xx) just look
> how all this implemented there, don't reinvent the wheel.
Unfortunately, he has to do things a bit differently. He can't afford to
have the kernel BAT mapping cover his non-cacheable pages. Thus he needs
a reserved pool. Last I looked at our coherent code, it didn't reserve
memory at all, just address space, thus assuming the CPU can handle
having both a caheable and a non-cacheable mapping of the same pages...
(On 6xx this is deadly even if you don't access those cacheable pages
because the CPU prefetch may do it for you).
Ben.
^ permalink raw reply
* Re: Not coherent cache DMA for G3/G4 CPUs: clarification needed
From: Benjamin Herrenschmidt @ 2006-04-20 21:03 UTC (permalink / raw)
To: Gerhard Pircher; +Cc: linuxppc-dev, debian-powerpc
In-Reply-To: <28892.1145559466@www088.gmx.net>
On Thu, 2006-04-20 at 20:57 +0200, Gerhard Pircher wrote:
> 1. The AmigaOne is similar to the PREP platform, i.e. DMA can only be
> performed in the first 16MB for ISA devices (there's only a VIA southbridge,
> no other SuperI/O IC with 32bit capable DMA controller). I guess the first
> 16MB cannot be reserved for not coherent DMA operation, because this memory
> area is occupied by kernel data? (not to talk about the performance loss, if
> the kernel data area would be excluded from the BAT mapping).
Yeah that would suck. Are you sure you need ISA DMA ? You can do pseudo
DMA like pegasos for the floppy. Anything else should be able to do 32
bits DMA unless you have a very broken chipset.
> 2. I'm not sure how to allocate memory for DMA operation. I think
> alloc_pages() will not do the job for me, as the page tables for not
> coherent DMA are reserved (SetPageReserved()) and removed from the available
> lowmem.
alloc_pages() ? How so ? No, you want to allocate from your reserved
pool, you'll have to implement your own allocator. Easiest is a running
bitmap, look at ppc64 iommu code for an example.
> Also memory fragmentation may be a problem, if a lot DMA operations
> with different buffer sizes are performed. Therefore a system could quickly
> run out of memory for not coherent DMA operation, right?
> Is there a way to minimize fragmentation?
Best you can do is have separate pools for small and big allocations I
suppose.
> 3. How are DMA buffers used outside the kernel? Do user programs get a
> pointer to the DMA buffer (in theory) from the device driver or is the data
> copied to another buffer allocated by an user program?
There are cases where some drivers try that but you should ignore it for
the moment. Won't happen in most cases.
Ben.
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox