Does pci_alloc_consisent really need to zero memory?

Does pci_alloc_consisent really need to zero memory?

[Adam J. Richter]

	Is it really necessary for pci_alloc_consistent() to
fill the memory that it returns with all zeroes?  I don't
see anything in Documentation/DMA-mapping.txt that specifies
it.  I have been on the lookout for drivers that rely on it
for the past couple of months, and I haven't seen any.  It's
only one line of code in arch/i386/kernel/pci-dma.c, but it
is potentially a lot of cycles, even if only zeroes the
space you requested (rather than the full pages that it
actually allocates).

	If nobody objects in the next half day or so (or asks
me to follow some other course of action), I'll submit a patch for
2.5 for all architectures.

Adam J. Richter     __     ______________   575 Oroville Road
adam@yggdrasil.com     \ /                  Milpitas, California 95035
+1 408 309-6081         | g g d r a s i l   United States of America
                         "Free Software For The Rest Of Us."

Re: Does pci_alloc_consisent really need to zero memory?

[David S. Miller]

   From: "Adam J. Richter" <adam@yggdrasil.com>
   Date: Thu, 30 May 2002 02:11:12 -0700

   	Is it really necessary for pci_alloc_consistent() to
   fill the memory that it returns with all zeroes?  I don't
   see anything in Documentation/DMA-mapping.txt that specifies
   it.  I have been on the lookout for drivers that rely on it
   for the past couple of months, and I haven't seen any.  It's
   only one line of code in arch/i386/kernel/pci-dma.c, but it
   is potentially a lot of cycles, even if only zeroes the
   space you requested (rather than the full pages that it
   actually allocates).

pci_alloc_consistent is so rare, I doubt it matters performance
wise.

I'd rather see a patch to DMA-mapping.txt that specifies the memory
returned is zeroed out, as this is what every implementation appears
to do.

Re: Does pci_alloc_consisent really need to zero memory?

[David S. Miller]

   From: Emmanuel Michon <emmanuel_michon@realmagic.fr>
   Date: 30 May 2002 13:05:26 +0200

   "David S. Miller" <davem@redhat.com> writes:
   
   > I'd rather see a patch to DMA-mapping.txt that specifies the memory
   > returned is zeroed out, as this is what every implementation appears
   > to do.
   
   What was the idea when writing the code that zeroes memory? It seems
   so useless.
   
Because 9 out of 10 drivers where I had to change virt_to_bus
into the portable pci_alloc_consistent were using get_free_pages()
which zeros things out for you.

I do actually remember that many of those instances in fact did
depend on the memory being zero'd for them, they weren't using the
"zero the pages too" variant instead of __get_free_pages()
gratuitously.

Re: Does pci_alloc_consisent really need to zero memory?

[Emmanuel Michon]

"David S. Miller" <davem@redhat.com> writes:

> I'd rather see a patch to DMA-mapping.txt that specifies the memory
> returned is zeroed out, as this is what every implementation appears
> to do.

What was the idea when writing the code that zeroes memory? It seems
so useless.

Sincerely yours,

-- 
Emmanuel Michon
Chef de projet
REALmagic France SAS
Mobile: 0614372733 GPGkeyID: D2997E42  

Re: Does pci_alloc_consisent really need to zero memory?

[David S. Miller]

   From: "Adam J. Richter" <adam@yggdrasil.com>
   Date: Thu, 30 May 2002 04:41:06 -0700

   	In my efforts to port almost all of the scsi drivers to your
   DMA-mapping interface, I have converted some kmalloc's that
   are frequently called with pci_alloc_consistent (I have not
   submitted these changes, because I think there is an unrelated
   bug in my changes).  Come to think of it, you advised me to go
   that route, as opposed to using pci_map_single(), when I asked about
   it with respect to advansys.c.  I'd like to have as little
   performance penalty for this as possible.  That also makes it
   marginally easier to encourge movement to your DMA-mapping interface.
   
The only advansys.c should be using consistent memory for are
it's command descriptor blocks that it sends to/from the card.
It should allocate these at device probe time, not while SCSI
commands are being issued.

Maybe for rare things like scsi status blocks, but not for normal scsi
command blocks.  Therein lies your problem, nothing to do with
pci_alloc_consistent zero'ing out memory or not.

   	While it is normally good programming practice for
   routines to always return deterministic results (same initial
   values in memory), I think that in a performance-oriented
   software component like the kernel, it's better programming practice
   to have primitives that do no more than they need to, that run as
   fast as possible, and behave consistently (no other remaining memory
   allocators in linux-2.5 do this, right?).
   
pci_alloc_consistent is not defined to go fast, period.  This allows
considerable flexibility in implementation.  On several systems,
setting up consistent mappings are expensive.  There is simply no way
around it.  But that's ok if the drivers aren't using it in hot paths,
which as I describe above they should not.

So trying to "speed up" pci_alloc_consistent is a misguided adventure.

Franks a lot,
David S. Miller
davem@redhat.com

Re: Does pci_alloc_consisent really need to zero memory?

[Adam J. Richter]

Dave S. Miller writes:
>pci_alloc_consistent is so rare, I doubt it [clearing the memory it
>returns] matters performance wise.

	In my efforts to port almost all of the scsi drivers to your
DMA-mapping interface, I have converted some kmalloc's that
are frequently called with pci_alloc_consistent (I have not
submitted these changes, because I think there is an unrelated
bug in my changes).  Come to think of it, you advised me to go
that route, as opposed to using pci_map_single(), when I asked about
it with respect to advansys.c.  I'd like to have as little
performance penalty for this as possible.  That also makes it
marginally easier to encourge movement to your DMA-mapping interface.

	While it is normally good programming practice for
routines to always return deterministic results (same initial
values in memory), I think that in a performance-oriented
software component like the kernel, it's better programming practice
to have primitives that do no more than they need to, that run as
fast as possible, and behave consistently (no other remaining memory
allocators in linux-2.5 do this, right?).

	Also as a general practice, I especially would like to
be paranoid about unnecessary memory copying or filling.  I like
to grep for memcpy's and memset's and ask "is this call really
necessary?" Core CPU speed improvements have out pace memory bus
improvements.  Today, a 2GHz CPU and with a 2GB/sec memory channel
(typical new computer) can only sustain clearing 1 byte per CPU
cycle (although much of that can accumulate in a write-behind
cache, but then you've just flushed a lot of your cache
unnecessarily, and that generates some memory traffic, etc.).
Plus, CPU's often execute more than one instruction per cycle,
so each byte might burn the time in which 1.4 instructions
could be executed, and other devices may want that memory
bandwidth too (I/O devices, another CPU, built-in video).

	Maybe if you could point out an example of some
code that actually depends on pci_alloc_consistent return
zeroes, I be able to see what I missed and be able to
track down similar cases.  There are plenty of drivers
that do pci_alloc_consistent and then memset.  So, whichever
way the semantics of pci_alloc_consistent are defined, some
drivers ought to be modified.

Adam J. Richter     __     ______________   575 Oroville Road
adam@yggdrasil.com     \ /                  Milpitas, California 95035
+1 408 309-6081         | g g d r a s i l   United States of America
                         "Free Software For The Rest Of Us."

Re: Does pci_alloc_consisent really need to zero memory?

[Pavel Machek]

Hi!

> >pci_alloc_consistent is so rare, I doubt it [clearing the memory it
> >returns] matters performance wise.
> 
> 	In my efforts to port almost all of the scsi drivers to your
> DMA-mapping interface, I have converted some kmalloc's that
> are frequently called with pci_alloc_consistent (I have not

...

maybe invent __pci_alloc_consistent which does no zeroing?
									Pavel
-- 
(about SSSCA) "I don't say this lightly.  However, I really think that the U.S.
no longer is classifiable as a democracy, but rather as a plutocracy." --hpa