Re: Kexec, DMA, and SMP

Corey Minyard
2003-02-12T15:56:21Z

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Eric W. Biederman wrote:

|Corey Minyard <cminyard@mvista.com> writes:
|
|>Eric W. Biederman wrote:
|>
|>|Corey Minyard <cminyard@mvista.com> writes:
|>|
|>|>
|>|>You don't understand. You don't *want* to set aside a block of
memory that's
|>|>reserved for DMA. You want to be able to DMA directly into any user
address.
|>|>Consider demand paging. The performance would suck if you DMA into some
|>|>fixed region then copied to the user address. Plus you then have
another
|>|>resource you have to manage in the kernel. And you still have to
change all
|>|>the drivers, buffer management, etc. to add a flag that says "I'm
going to use
|>
|>|>this for DMA" to allocations. You might as well add the quiesce
function,
|>it's
|>
|>|>probably easier to do. And it doesn't help if you DMA to static memory
|>|>addresses.
|>|>
|>|>I, too, would like a simpler solution. I just don't think this is it.
|>|
|>|
|>|You have it backwards. It is not about reserving a block of memory
|>|for DMA. It is about reserving a block of memory to not do DMA in.
|>|Something like 4MB or so. |
|>|The idea is not to let the original kernel touch the reserved block
at all.
|>|We just put the kernel that kexec will start in that block of memory.
|>|
|>|Eric
|>|
|>Ah, it makes much more sense now. Thank you. I still don't think
it's as easy
|>as you think, though.
|>Because there's no designation on most memory allocations to give you this
|>information. There's
|>GFP_DMA, but according to the docs that's just for x86 ISA DMA
devices. You
|>would
|>have to hunt down all the memory allocations, figure out of they are
DMA targets
|>
|>or not, and add a
|>flag for that. I still say it's easier to just add the function to
the drivers.
|
|
|It is trivial if you don't let alloc_pages give the memory to anyone for
|any purpose.

Ok, agreed, if you reserve a section of physical memory just for kexec
to copy it's kernel into, it will
prevent DMA from clobbering something from the time kexec copies the
kernel there to the time
decompressing starts.

Another thought. If you add a delay with all other processors and
interrupts off, the disk devices
will run out of things to do.

Once you add all the necessary quiesce functions, these can go away.

I do doubt these will make a big difference, though. The problem we
were seeing was with the
shared control structures in memory. The new kernel laid memory out a
little differently and
things like buffer pointers were overwritten with new data. This, in
turn, cause the device to
do random things. I would guess this is the most likely scenario, since
the time you are protecting
against with the memory layout is small compared to the time spent booting.

- -Corey
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQE+SnDpmUvlb4BhfF4RAruzAJ9Wts5ovfYf4Ncl+trn755L6JCc6QCfZffG
xGMlv58qX1v3ue0iLwxMRaw=
=aH5T
-----END PGP SIGNATURE-----