how does one disable processor cache on memory allocated with get_free_pages?

how does one disable processor cache on memory allocated with get_free_pages?

[Ross Tyler]

I am writing a kernel module that is allocating memory using get_free_pages.
It is important that access to/from these pages not use the processor 
(Pentium 4) cache.
How can I do this?

I have looked at drivers/char/mem.c which uses a pgprot_noncached 
function to modify the vm_page_prot attribute of a vma during a mmap call.
If I understand this correctly, this would make sure that all access to 
the pages that are done through the mapping will not be cached.

This will not work for me, however, because I need to access this memory 
from the device driver independent of any user code that might mmap it.
I thought of the device driver open'ing and mmap'ing itself but it 
doesn't have a special file in the file system (I am not using DEVFS) 
and this seems kludgy.

I am hoping that there is a way to simply modify the page table entries 
for the allocated pages.
Is there?
If so, what is the mechanism for modifying the entries and synchronizing 
the configuration with the processor cache?

Please CC any response directly to me (retyler@raytheon.com).

Thanks!

Re: how does one disable processor cache on memory allocated with get_free_pages?

[Ross Tyler]

Andrew,

Thank you for taking the time to reply. I really appreciate your help.

My understanding of ioremap_nocache is that it falls short of what I 
need to do.
It is appropriate for, say, mapping physical memory on a PCI device that 
is marked prefetchable (and otherwise subject to caching when mapped 
with ioremap) as non-caching.
Can you confirm my understanding?

If so, it will not work for me as I am not mapping physical memory but 
memory allocated by get_free_pages.
Do you concur?

AFAIK, the only way to access this memory without using processor cache 
is to have a device driver memory map it for a process like 
drivers/char/mem.c does.
When the memory is accessed through these memory mapped pages, the 
access will not be cached.
When the memory is accessed through the get_free_pages pages, the access 
will be cached.
Concur?

In order to access this process mapped memory from outside the context 
of the process it was mapped for, one either needs to independently 
remap it for the current process (what to do in interrupt code?) or set 
up a kiobuf to the memory.
It has been my experience, however, that the pages referenced by the 
kiobuf are the same pages returned by get_free_pages.
I expect these same pages have the same (caching) attributes associated 
with them which would not work.
?

Again, I thank you for your support.

Andrew Morton wrote:

>Ross Tyler <retyler@raytheon.com> wrote:
>  
>
>>If so, what is the mechanism for modifying the entries and synchronizing 
>>the configuration with the processor cache?
>>    
>>
>
>The only thing we have is ioremap_nocache()
>  
>

Re: how does one disable processor cache on memory allocated with get_free_pages?

[Benjamin Herrenschmidt]

On Fri, 2004-02-27 at 03:25, Ross Tyler wrote:
> Andrew,
> 
> Thank you for taking the time to reply. I really appreciate your help.
> 
> My understanding of ioremap_nocache is that it falls short of what I 
> need to do.
> It is appropriate for, say, mapping physical memory on a PCI device that 
> is marked prefetchable (and otherwise subject to caching when mapped 
> with ioremap) as non-caching.
> Can you confirm my understanding?

No, ioremap always maps non-cacheable, this has nothing to do with
the prefetching attribute

> If so, it will not work for me as I am not mapping physical memory but 
> memory allocated by get_free_pages.
> Do you concur?

get_free_pages() returns you physical memory...

> AFAIK, the only way to access this memory without using processor cache 
> is to have a device driver memory map it for a process like 
> drivers/char/mem.c does.
> When the memory is accessed through these memory mapped pages, the 
> access will not be cached.
> When the memory is accessed through the get_free_pages pages, the access 
> will be cached.
> Concur?

get_free_page allocates cacheable physical memory, nothing to do
with your PCI device... ioremap maps your device non-cacheable
into the kernel address space.
 
> In order to access this process mapped memory from outside the context 
> of the process it was mapped for, one either needs to independently 
> remap it for the current process (what to do in interrupt code?) or set 
> up a kiobuf to the memory.
> It has been my experience, however, that the pages referenced by the 
> kiobuf are the same pages returned by get_free_pages.
> I expect these same pages have the same (caching) attributes associated 
> with them which would not work.

What are you trying to do exactly ?

Ben.

Re: how does one disable processor cache on memory allocated with get_free_pages?

[Ross Tyler]

Benjamin Herrenschmidt wrote (more below):

>What are you trying to do exactly ?
>  
>
Thanks Ben.

Sorry, I can't tell you ... exactly.
Suffice to say, it is something very spooky.
My customer is very concerned about what I am writing to and reading 
from memory and where it ends up on the way.
To address these concerns, I need to make sure that processor cache is 
not used for some sensitive data.
I have sensitive data in (prefetchable and nonprefetchable) memory on 
some PCI devices and some in conventional memory.

My understanding so far (somebody, __please__ correct me if I am wrong) ...

O'Reilly Linux Device Drivers wrote (regarding ioremap_nocache):

> Quoting from one of the kernel headers: "It's useful if some control 
> registers are in such an area and write combining or read caching is 
> not desirable." Actually, the function's implementation is identical 
> to ioremap on most computer platforms: in situations in which all of 
> I/O memory is already visible throught noncacheable addresses, there's 
> no reason to implement a separate, noncaching version of ioremap.

O'Reilly Linux Device Drivers wrote (regarding PCI (non)prefetchable 
memory):

> The PCI device that implements I/O registers as a memory region marks 
> the difference by settings a "memory-is-prefetchable" bit in its 
> configuration register. If the memory region is marked as 
> prefetchable, the CPU can cache its contents and do all sorts of 
> optimization with it; nonprefetchable memory access, on the other 
> hand, can't be optimized because each access can have side effects, 
> exactly like I/O ports usually have.

Even though the name might suggest it, ioremap and ioremap_nocache 
methods are the same in function, when mapping prefetchable or 
nonprefetchable PCI memory, with regard to (Pentium 4) processor caching 
of this memory.
When mapping PCI memory that needs to be accessed without using (Pentium 
4) processor cache, it is sufficient to use ioremap - but using 
ioremap_nocache will not hurt.

This all confuses me, but that's OK - I have been confused before.
The safest thing seems to be to use ioremap_nocache.
This is what I will do.

Both kmalloc and get_free_pages return a kernel logical address to 
conventional physical memory.

O'Reilly Linux Device Drivers wrote (regarding a Kernel logical address):

> On most architectures, logical addresses and their associated physical 
> address differ only by a constant offset.

I believet this to be true.
I can use the __pa() macro to turn my logical address in to a physical 
address.

When accessing conventional memory without involving the processor 
cache, the only way that I (think I) know how to do this is to allocate 
the memory in the kernel (get_free_pages) and memory map (mmap) it to a 
process in a vma with the non-caching attribute set (as in 
drivers/char/mem.c).
The non-caching access must be done by the process that has done the 
mmap using the mapped address.
Access done using the address returned by the memory allocator 
(get_free_pages) will be subject to processor caching.

I know of no way to perform non-cached access to conventional memory 
other than this.
Remapping the mmap'ed memory back into the kernel in a kiobuf results in 
the same pages that were originally allocated and, I believe that, 
access through the kiobuf is no different than access to the originally 
allocated memory.

This means that there is no way (that I know of) to make non-cached 
access to conventional memory from an interrupt handler.

Benjamin Herrenschmidt wrote:

>On Fri, 2004-02-27 at 03:25, Ross Tyler wrote:
>  
>
>>Andrew,
>>
>>Thank you for taking the time to reply. I really appreciate your help.
>>
>>My understanding of ioremap_nocache is that it falls short of what I 
>>need to do.
>>It is appropriate for, say, mapping physical memory on a PCI device that 
>>is marked prefetchable (and otherwise subject to caching when mapped 
>>with ioremap) as non-caching.
>>Can you confirm my understanding?
>>    
>>
>
>No, ioremap always maps non-cacheable, this has nothing to do with
>the prefetching attribute
>
>  
>
>>If so, it will not work for me as I am not mapping physical memory but 
>>memory allocated by get_free_pages.
>>Do you concur?
>>    
>>
>
>get_free_pages() returns you physical memory...
>
>  
>
>>AFAIK, the only way to access this memory without using processor cache 
>>is to have a device driver memory map it for a process like 
>>drivers/char/mem.c does.
>>When the memory is accessed through these memory mapped pages, the 
>>access will not be cached.
>>When the memory is accessed through the get_free_pages pages, the access 
>>will be cached.
>>Concur?
>>    
>>
>
>get_free_page allocates cacheable physical memory, nothing to do
>with your PCI device... ioremap maps your device non-cacheable
>into the kernel address space.
> 
>  
>
>>In order to access this process mapped memory from outside the context 
>>of the process it was mapped for, one either needs to independently 
>>remap it for the current process (what to do in interrupt code?) or set 
>>up a kiobuf to the memory.
>>It has been my experience, however, that the pages referenced by the 
>>kiobuf are the same pages returned by get_free_pages.
>>I expect these same pages have the same (caching) attributes associated 
>>with them which would not work.
>>    
>>
>
>What are you trying to do exactly ?
>
>Ben.
>
>
>  
>

Re: how does one disable processor cache on memory allocated with get_free_pages?

[Benjamin Herrenschmidt]

On Sat, 2004-02-28 at 02:36, Ross Tyler wrote:
> Benjamin Herrenschmidt wrote (more below):
> 
> >What are you trying to do exactly ?
> >  
> >
> Thanks Ben.
> 
> Sorry, I can't tell you ... exactly.
> Suffice to say, it is something very spooky.
> My customer is very concerned about what I am writing to and reading 
> from memory and where it ends up on the way.
> To address these concerns, I need to make sure that processor cache is 
> not used for some sensitive data.

That's plain wrong. It's even a total nonsense imho. And you cannot
map memory non-cacheable just "like that", it will be mapped cacheable
via the linear kernel mapping and on a lot of CPUs, that double mapping
will lead into interesting things, like CPU checkstops.

> I have sensitive data in (prefetchable and nonprefetchable) memory on 
> some PCI devices and some in conventional memory.

Disabling cache because of "data sensitivity" is a no-no. If somebody
can "peek" at your cache, that person can even more easily peek at
your main memory.

> My understanding so far (somebody, __please__ correct me if I am wrong) ...
> 
> O'Reilly Linux Device Drivers wrote (regarding ioremap_nocache):
> 
> > Quoting from one of the kernel headers: "It's useful if some control 
> > registers are in such an area and write combining or read caching is 
> > not desirable." Actually, the function's implementation is identical 
> > to ioremap on most computer platforms: in situations in which all of 
> > I/O memory is already visible throught noncacheable addresses, there's 
> > no reason to implement a separate, noncaching version of ioremap.

This is partially incorrect.

ioremap maps IO devices in non-cacheable mappings. ioremap_nocache is
something that should die. It was introduced to allow non-cacheable
remapping of some portions of RAM, but that cannot be done reliably
in an architecture neutral way and triggers the problems I mentioned
above.

> O'Reilly Linux Device Drivers wrote (regarding PCI (non)prefetchable 
> memory):
> 
> > The PCI device that implements I/O registers as a memory region marks 
> > the difference by settings a "memory-is-prefetchable" bit in its 
> > configuration register. If the memory region is marked as 
> > prefetchable, the CPU can cache its contents and do all sorts of 
> > optimization with it; nonprefetchable memory access, on the other 
> > hand, can't be optimized because each access can have side effects, 
> > exactly like I/O ports usually have.

This is also misleading. The CPU will not "cache" the content of
PCI prefetchable memory because it has the prefetchable bit. The CPU
is _allowed_ to cache it in that sense that you are allowed, if you
explicitely do so, to create a cacheable mapping on top of a
prefetchable memory region, but we usually don't do that and ioremap
certainly doesn't know if the address you are passing it is for a
prefetchable region or not :)

What can happen usually is that prefetchable space of PCI devices
is gathered into a separate decoding region of the north bridge wich
is allowed to do read-ahead and byte coalescing and such things...

> Even though the name might suggest it, ioremap and ioremap_nocache 
> methods are the same in function, when mapping prefetchable or 
> nonprefetchable PCI memory, with regard to (Pentium 4) processor caching 
> of this memory.
> When mapping PCI memory that needs to be accessed without using (Pentium 
> 4) processor cache, it is sufficient to use ioremap - but using 
> ioremap_nocache will not hurt.

ioremap_nocache is non portable and should die. Don't use it.

> This all confuses me, but that's OK - I have been confused before.
> The safest thing seems to be to use ioremap_nocache.
> This is what I will do.

No, just use ioremap.

> Both kmalloc and get_free_pages return a kernel logical address to 
> conventional physical memory.

Yes.

> O'Reilly Linux Device Drivers wrote (regarding a Kernel logical address):
> 
> > On most architectures, logical addresses and their associated physical 
> > address differ only by a constant offset.
> 
> I believet this to be true.

This is only true for kernel addresses allocated in low memory, this is
_not_ true for pages obtained by vmalloc.

> I can use the __pa() macro to turn my logical address in to a physical 
> address.

What for ? You are not supposed to use ioremap on a memory address.

If you intend to pass the address to some devive doing DMA, then you
need to use the pci_dma operations to get a dma_addr_t suitable
for passsing to a PCI device, which may be different than a
phsycial address, depending on the platform.

> When accessing conventional memory without involving the processor 
> cache, the only way that I (think I) know how to do this is to allocate 
> the memory in the kernel (get_free_pages) and memory map (mmap) it to a 
> process in a vma with the non-caching attribute set (as in 
> drivers/char/mem.c).

If you do that, you end up with 2 mappings, one cacheable, one non
cacheable, of the same memory. Even if you do not actually use the
cacheable one, processor prefetch may well cause lines of cache for
this mapping to get in the cache. Concurrent cacheable and non cacheable
access to the same memory area will result at best in data corruption,
at worst into CPU checkstops.

Do not do that. There is no point in making memory uncacheable. If
you are really _that_ concerned about leaving your data in the CPU
cache (which doesn't make sense to me), you can always make sure you
1) disable interrupts when playing with your critical data, 2)
flush the CPU cache, 3) re-enable interrupts. But that won't buy you
any additional security imho.

> The non-caching access must be done by the process that has done the 
> mmap using the mapped address.

mmap is for userland processes, yes. You can also use vmap for in-kernel
mappings. But as I said above, this is evil. 

> Access done using the address returned by the memory allocator 
> (get_free_pages) will be subject to processor caching.
> 
> I know of no way to perform non-cached access to conventional memory 
> other than this.
> Remapping the mmap'ed memory back into the kernel in a kiobuf results in 
> the same pages that were originally allocated and, I believe that, 
> access through the kiobuf is no different than access to the originally 
> allocated memory.
> 
> This means that there is no way (that I know of) to make non-cached 
> access to conventional memory from an interrupt handler.

Well... you could use vmap instead of mmap to create a kernel mapping
as I explained above. But I also told you why you should not do that
anyway.

Ben.

Re: how does one disable processor cache on memory allocated with get_free_pages?

[Ross Tyler]

Ben,

Thank you for your response - it is very enlightening.
With this information I think I will be able to make a case not to worry 
about the processor cache.
If I win the argument, it will make my life much easier and my code much 
cleaner.
Thank you.

Regarding my usage of __pa() ...
My hope was that by allocating memory early enough in the boot process, 
I could guarantee that the same physical memory is allocated on every boot.
My experiments have shown that this is the case.
I just use __pa to report this result.

Thanks again!