XIP kernel + MTD polling interest

XIP kernel + MTD polling interest

[Shane Nay]

I've been working on getting MTD working with the XIP kernel in the Agenda
(dev.agendacomputing.com), and I was wondering if the MTD project was
interested in this.  It's a litte hacky because we can't compile PIC...,
well, I'll explain below.

When the kernel is running in XIP mode, aka, Execute in Place, in flash, in
order to write/query the flash, you need to copy all the stuff you need
while in that mode into memory, and execute it from there.  Which means a
couple of things..., you have to fool the compiler into doing PIC code
without compiling pic and you can't use _any_ kernel functions while the
flash is in query or write mode..., which changes the resulting code into
something rather messy.  This wasn't too bad with the version that's
presently in 2.4, but from the more recent snapshots (which I'm working off
of now), it's quite messy.

In any event, I've gotten successful flash probes via cfi_probe, and gotten
it to dump all the debugging info via print_cfi_ident.  I'm now moving to
working on the intel command set for our chips, and will have an XIPable
version of that towards the beginning of this week, hopefully tomorrow
night.  What I'm really wondering is whether I should move to make patches
off of the present versions of cfi_probe.c, cfi.h, cfi_cmdset_0001.c, or
whether I should just fork them?  (Basically, it follows the same logic, I
just use macros to build copies of functions in memory also the code gets
messy because I can't use else, can't use switch, and can't use return
until the bottom of the function.  Also can't use printk, or any kernel
function while the flash is in query, or write mode)

Anyway, I just wanted to see if this was a goal of MTD, and get a vibe for
the interest on that.

Thanks,
Shane Nay.



To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org

Re: XIP kernel + MTD polling interest

[Shane Nay]

On Tue, 28 Nov 2000 14:08:22 David Woodhouse wrote:
> 
> shane@agendacomputing.com said:
> >   What I'm really wondering is whether I should move to make patches
> > off of the present versions of cfi_probe.c, cfi.h, cfi_cmdset_0001.c,
> > or whether I should just fork them?  (Basically, it follows the same
> > logic, I just use macros to build copies of functions in memory also
> > the code gets messy because I can't use else, can't use switch, and
> > can't use return until the bottom of the function.  Also can't use
> > printk, or any kernel function while the flash is in query, or write
> > mode)
> 
> If it can be cleanly merged with macros then it's probably worth doing. 
> Otherwise, it's best to fork it. The former is preferable.

Agreed if it's possible without blowing the design of mtd itself.  (Which
is debatable)


> Show me the code. Do you have a filesystem working XIP from flash yet?
> I'm about to start looking at hacking romfs to prepopulate the page cache
> 
> with pages directly from the flash chip.

I don't have a filesystem working just yet, as mentioned in the message I
haven't done the intel command set, just finished probing.  I'll attach
that code to you to give you an idea of how it is.  (Has the macros for
command copying, etc.)  Right now though it's very very messy, would have
to be cleaned up before inclusion anyhow, but I'll just send you snap as it
sits.  (Feel free to share obviously, not attaching to list because don't
want to bog everyones mailer down.)

Thanks,
Shane.



To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org

Re: XIP kernel + MTD polling interest

[Shane Nay]

> 
> How do you deal with calls to schedule()?  If you can't schedule you must
> poll the flash for termination and block all interrupts and say goodbye
> to
> low latency response from the kernel.  If that's not a problem for you
> then
> you might as well get rid of all the state machine code which is there
> only
> for concurent flash access which is impossible in your case since you
> need
> atomic flash operations.  Thus the code can become extremely simple.

Schedule? :-) Obviously can't do it, as you noted.  Yes, latency goes down
the drain, the flash chips are actually pretty fast on write, and okay on
erase.  Basically the plan is to kill any erasing of blocks, and have a
cleanup program that re-does the entire thing in one locked up motion.

> IMHO you are better to fork all relevant functions and have the chance to
> simplify them rather than trying to stretch the existing code which is
> really not designed for such purpose.

Well, we've come up with a better way (well, Mike Klar did from linux-vr)
to take care of weird c code, and allow everything to be normal.  (Special
elf sections, and twiddling with the linker script)  But in terms of
skipping around schedule.., it's not really a big deal.  You're right
though, it is an opportunity to slim down the code, and keep weird macros
out of MTD.  I was just thinking XIP kernel with flash stuff might be
usefull for MTD as a general thing, because people might want to take this
tact with various types of chips, etc.  I'm just going to finish the code,
and let you guys decide whether it's worth it to have it in MTD.  If it's
not worth it to have it as part of the files that are already in MTD, then
I can submit a seperate set of XIP versions of probing, etc.

Thanks,
Shane.



To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org

Re: XIP kernel + MTD polling interest

[Shane Nay]

> Cute, and not too horrible because you already had to copy the data
> section
> into RAM at startup anyway. Doing so with an extra section isn't too much
> of
> a problem, and there's enough other 'put this code in a different
> section' 
> magic in the kernel already that that bit isn't too troublesome either.

Hehe, yes :).


> In general, XIP isn't as interesting as I first thought. XIP is mutually 
> exclusive with compression, and RAM is cheaper than flash. That's even 
> _before_ you try to deal with the horrible problems that you're trying to

> work round right now.

Accurate because of the cost differential at this time :).

> But having said that, some of our customers do seem to be on crack so I'm
> looking at XIP anyway :) But I'm trying to get away with declaring 
> that the XIP flash chip is _strictly_ read-only, and that the writable 
> filesystem is to be kept on an entirely different flash chip.
> 
> I should be putting together a version of romfs designed to work on 
> read-only memory chips fairly shortly.

Our device has the flash memory mapped into the memory map of the device. 
So we use some /dev/rom stuff that's floating around on Brads ftp site, or
more recently I threw together a linear mapped version of cramfs which
works rather nicely, and has a slight boost in performance.  (Hair
splitting slight because of cache churning)

Our quandry is we're doing developers units right now, which are all flash,
and the PCB really can't work with two flash chips, so we have to have the
write-ability of the consumer units which have flash and ROM on the dev
units.  In any event, I don't expect it to be that bad on latency since the
flash chips write is really fast, and we can always call into schedule
between buffered writes or whatever.  (10 words at a time?, can't remember,
but those are imperceptable, it's the erase that's a killer)

Thanks,
Shane.



To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org

Re: XIP kernel + MTD polling interest

[David Woodhouse]

shane@agendacomputing.com said:
>   What I'm really wondering is whether I should move to make patches
> off of the present versions of cfi_probe.c, cfi.h, cfi_cmdset_0001.c,
> or whether I should just fork them?  (Basically, it follows the same
> logic, I just use macros to build copies of functions in memory also
> the code gets messy because I can't use else, can't use switch, and
> can't use return until the bottom of the function.  Also can't use
> printk, or any kernel function while the flash is in query, or write
> mode)

If it can be cleanly merged with macros then it's probably worth doing. 
Otherwise, it's best to fork it. The former is preferable.

Show me the code. Do you have a filesystem working XIP from flash yet?
I'm about to start looking at hacking romfs to prepopulate the page cache 
with pages directly from the flash chip.


--
dwmw2




To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org

Re: XIP kernel + MTD polling interest

[David Woodhouse]

shane@agendacomputing.com said:
> Well, we've come up with a better way (well, Mike Klar did from
> linux-vr) to take care of weird c code, and allow everything to be
> normal.  (Special elf sections, and twiddling with the linker script) 

Cute, and not too horrible because you already had to copy the data section
into RAM at startup anyway. Doing so with an extra section isn't too much of
a problem, and there's enough other 'put this code in a different section' 
magic in the kernel already that that bit isn't too troublesome either.

shane@agendacomputing.com said:
>   I was just thinking XIP kernel with flash stuff might be usefull for
> MTD as a general thing, because people might want to take this tact
> with various types of chips, etc

In general, XIP isn't as interesting as I first thought. XIP is mutually 
exclusive with compression, and RAM is cheaper than flash. That's even 
_before_ you try to deal with the horrible problems that you're trying to 
work round right now.

But having said that, some of our customers do seem to be on crack so I'm 
looking at XIP anyway :) But I'm trying to get away with declaring 
that the XIP flash chip is _strictly_ read-only, and that the writable 
filesystem is to be kept on an entirely different flash chip.

I should be putting together a version of romfs designed to work on 
read-only memory chips fairly shortly.

--
dwmw2




To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org

Re: XIP kernel + MTD polling interest

[Nicolas Pitre]

On Tue, 28 Nov 2000, Shane Nay wrote:

> I've been working on getting MTD working with the XIP kernel in the Agenda
> (dev.agendacomputing.com), and I was wondering if the MTD project was
> interested in this.  It's a litte hacky because we can't compile PIC...,
> well, I'll explain below.

[...]

> What I'm really wondering is whether I should move to make patches
> off of the present versions of cfi_probe.c, cfi.h, cfi_cmdset_0001.c, or
> whether I should just fork them?  (Basically, it follows the same logic, I
> just use macros to build copies of functions in memory also the code gets
> messy because I can't use else, can't use switch, and can't use return
> until the bottom of the function.  Also can't use printk, or any kernel
> function while the flash is in query, or write mode)

How do you deal with calls to schedule()?  If you can't schedule you must
poll the flash for termination and block all interrupts and say goodbye to
low latency response from the kernel.  If that's not a problem for you then
you might as well get rid of all the state machine code which is there only
for concurent flash access which is impossible in your case since you need
atomic flash operations.  Thus the code can become extremely simple.

IMHO you are better to fork all relevant functions and have the chance to
simplify them rather than trying to stretch the existing code which is
really not designed for such purpose.


Nicolas



To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org

Re: XIP kernel + MTD polling interest

[Shane Nay]

MTD folk,

Okay, I've redone my stuff with the ELF section.  It's actually pretty
clean, and does full intialization up through the Intel command set.  I'm
still trying to figure out the best way to deal with calls to schedule. 
What I'm thinking is we already have the wrapper code for cfi_udelay, if we
were to replace all calls to schedule to this wrapper guy in
cfi_cmdset_0001.c, then it would be really clean, just get rid of that
if(current->need_resched) schedule(), to just go to udelay(us) #if defined
MTD_XIP.  That would be the only real change that would need to be made,
except flipping off interrupts when switching modes in MTD_XIP.

What do you guys think of doing this change?..., Any negatives?

Thanks,
Shane Nay.



To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org

Re: XIP kernel + MTD polling interest

[Shane Nay]

> 
> Sounds reasonable to me. I'd like to see the section magic too.

It's in:
ftp://shane.dnsalias.net/pub/mtd-xip-snap-0.1.tar.gz

Included is the linker script, and the files I've changed thus far.  When
I'm finished I'll make a real diff, and do the kernel configuration stuff.



To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org

Re: XIP kernel + MTD polling interest

[David Woodhouse]

shane@agendacomputing.com said:
> What I'm thinking is we already have the wrapper code for cfi_udelay,
> if we were to replace all calls to schedule to this wrapper guy in
> cfi_cmdset_0001.c, then it would be really clean, just get rid of that
> if(current->need_resched) schedule(), to just go to udelay(us) #if
> defined MTD_XIP.  That would be the only real change that would need
> to be made, except flipping off interrupts when switching modes in
> MTD_XIP.

> What do you guys think of doing this change?..., Any negatives? 

Sounds reasonable to me. I'd like to see the section magic too.

--
dwmw2




To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org