ioremap_nocache problem?

ioremap_nocache problem?

[Mark Mokryn]

ioremap_nocache does the following:
	return __ioremap(offset, size, _PAGE_PCD);

However, in drivers/char/mem.c (2.4.0), we see the following:

	/* On PPro and successors, PCD alone doesn't always mean 
	    uncached because of interactions with the MTRRs. PCD | PWT
	    means definitely uncached. */ 
	if (boot_cpu_data.x86 > 3)
		prot |= _PAGE_PCD | _PAGE_PWT;

Does this mean ioremap_nocache() may not do the job?

-mark
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

ioremap_nocache problem?

[Mark Mokryn]

ioremap_nocache does the following:
	return __ioremap(offset, size, _PAGE_PCD);

However, in drivers/char/mem.c (2.4.0), we see the following:

	/* On PPro and successors, PCD alone doesn't always mean 
	    uncached because of interactions with the MTRRs. PCD | PWT
	    means definitely uncached. */ 
	if (boot_cpu_data.x86 > 3)
		prot |= _PAGE_PCD | _PAGE_PWT;

Does this mean ioremap_nocache() may not do the job?

-mark
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: ioremap_nocache problem?

[Timur Tabi]

** Reply to message from Mark Mokryn <mark@sangate.com> on Tue, 23 Jan 2001
12:30:00 +0200


> Does this mean ioremap_nocache() may not do the job?

Good luck trying to get an answer.  I've been asking questions on ioremap for
months, but no one's ever been able to tell me anything.

According to the comments, mem.c provides /dev/zero support, whatever that is.
It doesn't appear to be connected to ioremap in any way, so I understand your
question.

I can tell you that I have written a driver that depends on ioremap_nocache,
and it does work, so it appears that ioremap_nocache is doing something.

My problem is that it's very easy to map memory with ioremap_nocache, but if
you use iounmap() the un-map it, the entire system will crash.  No one has been
able to explain that one to me, either.


-- 
Timur Tabi - ttabi@interactivesi.com
Interactive Silicon - http://www.interactivesi.com

When replying to a mailing-list message, please direct the reply to the mailing list only.  Don't send another copy to me.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ioremap_nocache problem?

[Timur Tabi]

** Reply to message from Mark Mokryn <mark@sangate.com> on Tue, 23 Jan 2001
12:30:00 +0200


> Does this mean ioremap_nocache() may not do the job?

Good luck trying to get an answer.  I've been asking questions on ioremap for
months, but no one's ever been able to tell me anything.

According to the comments, mem.c provides /dev/zero support, whatever that is.
It doesn't appear to be connected to ioremap in any way, so I understand your
question.

I can tell you that I have written a driver that depends on ioremap_nocache,
and it does work, so it appears that ioremap_nocache is doing something.

My problem is that it's very easy to map memory with ioremap_nocache, but if
you use iounmap() the un-map it, the entire system will crash.  No one has been
able to explain that one to me, either.


-- 
Timur Tabi - ttabi@interactivesi.com
Interactive Silicon - http://www.interactivesi.com

When replying to a mailing-list message, please direct the reply to the mailing list only.  Don't send another copy to me.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: ioremap_nocache problem?

[Stephen C. Tweedie]

Hi,

On Tue, Jan 23, 2001 at 10:53:51AM -0600, Timur Tabi wrote:
> 
> My problem is that it's very easy to map memory with ioremap_nocache, but if
> you use iounmap() the un-map it, the entire system will crash.  No one has been
> able to explain that one to me, either.

ioremap*() is only supposed to be used on IO regions or reserved
pages.  If you haven't marked the pages as reserved, then iounmap will
do the wrong thing, so it's up to you to reserve the pages.

Cheers,
 Stephen
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ioremap_nocache problem?

[Stephen C. Tweedie]

Hi,

On Tue, Jan 23, 2001 at 10:53:51AM -0600, Timur Tabi wrote:
> 
> My problem is that it's very easy to map memory with ioremap_nocache, but if
> you use iounmap() the un-map it, the entire system will crash.  No one has been
> able to explain that one to me, either.

ioremap*() is only supposed to be used on IO regions or reserved
pages.  If you haven't marked the pages as reserved, then iounmap will
do the wrong thing, so it's up to you to reserve the pages.

Cheers,
 Stephen
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: ioremap_nocache problem?

[Timur Tabi]

** Reply to message from "Stephen C. Tweedie" <sct@redhat.com> on Thu, 25 Jan
2001 15:16:55 +0000


> ioremap*() is only supposed to be used on IO regions or reserved
> pages.  If you haven't marked the pages as reserved, then iounmap will
> do the wrong thing, so it's up to you to reserve the pages.

Au contraire!

I mark the page as reserved when I ioremap() it.  However, if I leave it marked
reserved, then iounmap() will not unmap it.  If I mark it "unreserved" (i.e.
reset the reserved bit), then iounmap will unmap it, but it will decrement the
page counter to -1 and the whole system will crash soon thereafter.

I've been asking about this problem for months, but no one has bothered to help
me out.


-- 
Timur Tabi - ttabi@interactivesi.com
Interactive Silicon - http://www.interactivesi.com

When replying to a mailing-list message, please direct the reply to the mailing list only.  Don't send another copy to me.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: ioremap_nocache problem?

[Roman Zippel]

Hi,

Timur Tabi wrote:

> I mark the page as reserved when I ioremap() it.  However, if I leave it marked
> reserved, then iounmap() will not unmap it.  If I mark it "unreserved" (i.e.
> reset the reserved bit), then iounmap will unmap it, but it will decrement the
> page counter to -1 and the whole system will crash soon thereafter.
> 
> I've been asking about this problem for months, but no one has bothered to help
> me out.

The order is important:

	get_free_page();
	set_bit(PG_reserved, &page->flags);
	ioremap();
	...
	iounmap();
	clear_bit(PG_reserved, &page->flags);
	free_page();

Alternatively something like this should also be possible:

	get_free_page();
	ioremap();
	...
	iounmap();

nopage() {
	...
	atomic_inc(&page->count);
	return page;
}

But I never tried this version, so I can't guarantee anything. :)

bye, Roman
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ioremap_nocache problem?

[Roman Zippel]

Hi,

Timur Tabi wrote:

> I mark the page as reserved when I ioremap() it.  However, if I leave it marked
> reserved, then iounmap() will not unmap it.  If I mark it "unreserved" (i.e.
> reset the reserved bit), then iounmap will unmap it, but it will decrement the
> page counter to -1 and the whole system will crash soon thereafter.
> 
> I've been asking about this problem for months, but no one has bothered to help
> me out.

The order is important:

	get_free_page();
	set_bit(PG_reserved, &page->flags);
	ioremap();
	...
	iounmap();
	clear_bit(PG_reserved, &page->flags);
	free_page();

Alternatively something like this should also be possible:

	get_free_page();
	ioremap();
	...
	iounmap();

nopage() {
	...
	atomic_inc(&page->count);
	return page;
}

But I never tried this version, so I can't guarantee anything. :)

bye, Roman
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: ioremap_nocache problem?

[Timur Tabi]

** Reply to message from Roman Zippel <roman@augan.com> on Thu, 25 Jan 2001
17:44:51 +0100


> set_bit(PG_reserved, &page->flags);
> 	ioremap();
> 	...
> 	iounmap();
> 	clear_bit(PG_reserved, &page->flags);

The problem with this is that between the ioremap and iounmap, the page is
reserved.  What happens if that page belongs to some disk buffer or user
process, and some other process tries to free it.  Won't that cause a problem?


-- 
Timur Tabi - ttabi@interactivesi.com
Interactive Silicon - http://www.interactivesi.com

When replying to a mailing-list message, please direct the reply to the mailing list only.  Don't send another copy to me.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: ioremap_nocache problem?

[Stephen C. Tweedie]

Hi,

On Thu, Jan 25, 2001 at 10:49:50AM -0600, Timur Tabi wrote:
> 
> > set_bit(PG_reserved, &page->flags);
> > 	ioremap();
> > 	...
> > 	iounmap();
> > 	clear_bit(PG_reserved, &page->flags);
> 
> The problem with this is that between the ioremap and iounmap, the page is
> reserved.  What happens if that page belongs to some disk buffer or user
> process, and some other process tries to free it.  Won't that cause a problem?

It depends on how it is being used, but yes, it is likely to --- page
reference counts won't be updated properly on reserved pages, for
example.  Why on earth do you want to do ioremap of something like a
page cache page, though?  That is _not_ what ioremap is designed for!

--Stephen
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ioremap_nocache problem?

[Stephen C. Tweedie]

Hi,

On Thu, Jan 25, 2001 at 10:49:50AM -0600, Timur Tabi wrote:
> 
> > set_bit(PG_reserved, &page->flags);
> > 	ioremap();
> > 	...
> > 	iounmap();
> > 	clear_bit(PG_reserved, &page->flags);
> 
> The problem with this is that between the ioremap and iounmap, the page is
> reserved.  What happens if that page belongs to some disk buffer or user
> process, and some other process tries to free it.  Won't that cause a problem?

It depends on how it is being used, but yes, it is likely to --- page
reference counts won't be updated properly on reserved pages, for
example.  Why on earth do you want to do ioremap of something like a
page cache page, though?  That is _not_ what ioremap is designed for!

--Stephen
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: ioremap_nocache problem?

[Timur Tabi]

** Reply to message from Roman Zippel <roman@augan.com> on Thu, 25 Jan 2001
17:44:51 +0100


> set_bit(PG_reserved, &page->flags);
> 	ioremap();
> 	...
> 	iounmap();
> 	clear_bit(PG_reserved, &page->flags);

The problem with this is that between the ioremap and iounmap, the page is
reserved.  What happens if that page belongs to some disk buffer or user
process, and some other process tries to free it.  Won't that cause a problem?


-- 
Timur Tabi - ttabi@interactivesi.com
Interactive Silicon - http://www.interactivesi.com

When replying to a mailing-list message, please direct the reply to the mailing list only.  Don't send another copy to me.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ioremap_nocache problem?

[Jeff Hartmann]

Timur Tabi wrote:

> ** Reply to message from Roman Zippel <roman@augan.com> on Thu, 25 Jan 2001
> 17:44:51 +0100
> 
> 
> 
>> set_bit(PG_reserved, &page->flags);
>> 	ioremap();
>> 	...
>> 	iounmap();
>> 	clear_bit(PG_reserved, &page->flags);
> 
> 
> The problem with this is that between the ioremap and iounmap, the page is
> reserved.  What happens if that page belongs to some disk buffer or user
> process, and some other process tries to free it.  Won't that cause a problem?

	The page can't belong to some other process/kernel component.  You own 
the page if you allocated it.  The kernel will only muck with memory you 
allocated if its GFP_HIGHMEM, or under certain circumstances if you map 
it into a user process (There are several rules here and I won't go into 
them, look at the DRM mmap setup for a start if your interested.)  This 
is the correct ordering of the calls (I was the one who added support to 
the kernel to ioremap real ram, trust me.)

-Jeff

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ioremap_nocache problem?

[Jeff Hartmann]

Timur Tabi wrote:

> ** Reply to message from Roman Zippel <roman@augan.com> on Thu, 25 Jan 2001
> 17:44:51 +0100
> 
> 
> 
>> set_bit(PG_reserved, &page->flags);
>> 	ioremap();
>> 	...
>> 	iounmap();
>> 	clear_bit(PG_reserved, &page->flags);
> 
> 
> The problem with this is that between the ioremap and iounmap, the page is
> reserved.  What happens if that page belongs to some disk buffer or user
> process, and some other process tries to free it.  Won't that cause a problem?

	The page can't belong to some other process/kernel component.  You own 
the page if you allocated it.  The kernel will only muck with memory you 
allocated if its GFP_HIGHMEM, or under certain circumstances if you map 
it into a user process (There are several rules here and I won't go into 
them, look at the DRM mmap setup for a start if your interested.)  This 
is the correct ordering of the calls (I was the one who added support to 
the kernel to ioremap real ram, trust me.)

-Jeff

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: ioremap_nocache problem?

[Timur Tabi]

** Reply to message from Jeff Hartmann <jhartmann@valinux.com> on Thu, 25 Jan
2001 10:04:47 -0700


> > The problem with this is that between the ioremap and iounmap, the page is
> > reserved.  What happens if that page belongs to some disk buffer or user
> > process, and some other process tries to free it.  Won't that cause a problem?
> 
> 	The page can't belong to some other process/kernel component.  You own 
> the page if you allocated it.  

Ok, my mistake.  I wasn't paying attention to the "get_free_pages" call.  My
problem is that I'm ioremap'ing someone else's page, but my hardware sits on the
memory bus disguised as real memory, and so I need to poke around the 4GB space
trying to find it.


> (I was the one who added support to 
> the kernel to ioremap real ram, trust me.)

I really appreciate that feature, because it helps me a lot.  Any
recommendations on how I can do what I do without causing any problems?  Right
now, my driver never calls iounmap on memory that's in real RAM, even when it
exits.  Fortunately, the driver isn't supposed to exit, so all it does is waste
a few KB of virtual memory.


-- 
Timur Tabi - ttabi@interactivesi.com
Interactive Silicon - http://www.interactivesi.com

When replying to a mailing-list message, please direct the reply to the mailing list only.  Don't send another copy to me.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: ioremap_nocache problem?

[Timur Tabi]

** Reply to message from Jeff Hartmann <jhartmann@valinux.com> on Thu, 25 Jan
2001 10:04:47 -0700


> > The problem with this is that between the ioremap and iounmap, the page is
> > reserved.  What happens if that page belongs to some disk buffer or user
> > process, and some other process tries to free it.  Won't that cause a problem?
> 
> 	The page can't belong to some other process/kernel component.  You own 
> the page if you allocated it.  

Ok, my mistake.  I wasn't paying attention to the "get_free_pages" call.  My
problem is that I'm ioremap'ing someone else's page, but my hardware sits on the
memory bus disguised as real memory, and so I need to poke around the 4GB space
trying to find it.


> (I was the one who added support to 
> the kernel to ioremap real ram, trust me.)

I really appreciate that feature, because it helps me a lot.  Any
recommendations on how I can do what I do without causing any problems?  Right
now, my driver never calls iounmap on memory that's in real RAM, even when it
exits.  Fortunately, the driver isn't supposed to exit, so all it does is waste
a few KB of virtual memory.


-- 
Timur Tabi - ttabi@interactivesi.com
Interactive Silicon - http://www.interactivesi.com

When replying to a mailing-list message, please direct the reply to the mailing list only.  Don't send another copy to me.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ioremap_nocache problem?

[Jeff Hartmann]

Timur Tabi wrote:

> ** Reply to message from Jeff Hartmann <jhartmann@valinux.com> on Thu, 25 Jan
> 2001 10:04:47 -0700
> 
> 
> 
>>> The problem with this is that between the ioremap and iounmap, the page is
>>> reserved.  What happens if that page belongs to some disk buffer or user
>>> process, and some other process tries to free it.  Won't that cause a problem?
>> 
>> 	The page can't belong to some other process/kernel component.  You own 
>> the page if you allocated it.  
> 
> 
> Ok, my mistake.  I wasn't paying attention to the "get_free_pages" call.  My
> problem is that I'm ioremap'ing someone else's page, but my hardware sits on the
> memory bus disguised as real memory, and so I need to poke around the 4GB space
> trying to find it.

As in an MMIO aperture?  If its MMIO on the bus you should be able to 
just call ioremap with the bus address.  By nature of it being outside 
of real ram, it should automatically be uncached (unless you've set an 
MTRR over that region saying otherwise).

> 
> 
> 
>> (I was the one who added support to 
>> the kernel to ioremap real ram, trust me.)
> 
> 
> I really appreciate that feature, because it helps me a lot.  Any
> recommendations on how I can do what I do without causing any problems?  Right
> now, my driver never calls iounmap on memory that's in real RAM, even when it
> exits.  Fortunately, the driver isn't supposed to exit, so all it does is waste
> a few KB of virtual memory.

Look at the functions agp_generic_free_gatt_table and 
agp_generic_create_gatt_table in agpgart_be.c (drivers/char/agp).  They 
do the ioremap_nocache on real ram for the GATT/GART table.  Heres some 
quick pseudo code as well.

I_want_a_no_cached_page() {
alloc a page
reserve the page
flush every cpu's cache
ioremap_nocache the page
}

I_want_to_free_a_no_cached_page() {
iounmap the page
unreserve the page
free the page
}

-Jeff

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ioremap_nocache problem?

[Jeff Hartmann]

Timur Tabi wrote:

> ** Reply to message from Jeff Hartmann <jhartmann@valinux.com> on Thu, 25 Jan
> 2001 10:04:47 -0700
> 
> 
> 
>>> The problem with this is that between the ioremap and iounmap, the page is
>>> reserved.  What happens if that page belongs to some disk buffer or user
>>> process, and some other process tries to free it.  Won't that cause a problem?
>> 
>> 	The page can't belong to some other process/kernel component.  You own 
>> the page if you allocated it.  
> 
> 
> Ok, my mistake.  I wasn't paying attention to the "get_free_pages" call.  My
> problem is that I'm ioremap'ing someone else's page, but my hardware sits on the
> memory bus disguised as real memory, and so I need to poke around the 4GB space
> trying to find it.

As in an MMIO aperture?  If its MMIO on the bus you should be able to 
just call ioremap with the bus address.  By nature of it being outside 
of real ram, it should automatically be uncached (unless you've set an 
MTRR over that region saying otherwise).

> 
> 
> 
>> (I was the one who added support to 
>> the kernel to ioremap real ram, trust me.)
> 
> 
> I really appreciate that feature, because it helps me a lot.  Any
> recommendations on how I can do what I do without causing any problems?  Right
> now, my driver never calls iounmap on memory that's in real RAM, even when it
> exits.  Fortunately, the driver isn't supposed to exit, so all it does is waste
> a few KB of virtual memory.

Look at the functions agp_generic_free_gatt_table and 
agp_generic_create_gatt_table in agpgart_be.c (drivers/char/agp).  They 
do the ioremap_nocache on real ram for the GATT/GART table.  Heres some 
quick pseudo code as well.

I_want_a_no_cached_page() {
alloc a page
reserve the page
flush every cpu's cache
ioremap_nocache the page
}

I_want_to_free_a_no_cached_page() {
iounmap the page
unreserve the page
free the page
}

-Jeff

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: ioremap_nocache problem?

[Timur Tabi]

** Reply to message from Jeff Hartmann <jhartmann@valinux.com> on Thu, 25 Jan
2001 10:47:13 -0700


> As in an MMIO aperture?  If its MMIO on the bus you should be able to 
> just call ioremap with the bus address.  By nature of it being outside 
> of real ram, it should automatically be uncached (unless you've set an 
> MTRR over that region saying otherwise).

It's not outside of real RAM.  The device is inside real RAM (it sits on the
DIMM itself), but I need to poke through the entire 4GB range to see how it
responds.

> Look at the functions agp_generic_free_gatt_table and 
> agp_generic_create_gatt_table in agpgart_be.c (drivers/char/agp).  They 
> do the ioremap_nocache on real ram for the GATT/GART table.

Unfortunately, the memory they remap is allocated:

table = (char *) __get_free_pages(GFP_KERNEL, page_order);

...

CACHE_FLUSH();
agp_bridge.gatt_table = ioremap_nocache(virt_to_phys(table), (PAGE_SIZE * (1 <<
page_order)));
CACHE_FLUSH();

I've searched high and low for examples of code that does what I do, and I
can't find any.


-- 
Timur Tabi - ttabi@interactivesi.com
Interactive Silicon - http://www.interactivesi.com

When replying to a mailing-list message, please direct the reply to the mailing list only.  Don't send another copy to me.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: ioremap_nocache problem?

[Stephen C. Tweedie]

Hi,

On Thu, Jan 25, 2001 at 11:53:01AM -0600, Timur Tabi wrote:
> 
> > As in an MMIO aperture?  If its MMIO on the bus you should be able to 
> > just call ioremap with the bus address.  By nature of it being outside 
> > of real ram, it should automatically be uncached (unless you've set an 
> > MTRR over that region saying otherwise).
> 
> It's not outside of real RAM.  The device is inside real RAM (it sits on the
> DIMM itself), but I need to poke through the entire 4GB range to see how it
> responds.

kmap() is designed for that, not ioremap().  Is it absolutely
essential that your mapping is uncached?  If so, extending kmap() to
support kmap_nocache() would seem to make a lot more sense than using
ioremap(): kmap is there for temporarily poking around in high memory,
whereas ioremap is really intended to be used for persistent maps.

--Stephen
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ioremap_nocache problem?

[Stephen C. Tweedie]

Hi,

On Thu, Jan 25, 2001 at 11:53:01AM -0600, Timur Tabi wrote:
> 
> > As in an MMIO aperture?  If its MMIO on the bus you should be able to 
> > just call ioremap with the bus address.  By nature of it being outside 
> > of real ram, it should automatically be uncached (unless you've set an 
> > MTRR over that region saying otherwise).
> 
> It's not outside of real RAM.  The device is inside real RAM (it sits on the
> DIMM itself), but I need to poke through the entire 4GB range to see how it
> responds.

kmap() is designed for that, not ioremap().  Is it absolutely
essential that your mapping is uncached?  If so, extending kmap() to
support kmap_nocache() would seem to make a lot more sense than using
ioremap(): kmap is there for temporarily poking around in high memory,
whereas ioremap is really intended to be used for persistent maps.

--Stephen
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: ioremap_nocache problem?

[Eric W. Biederman]

Timur Tabi <ttabi@interactivesi.com> writes:

> ** Reply to message from Jeff Hartmann <jhartmann@valinux.com> on Thu, 25 Jan
> 2001 10:47:13 -0700
> 
> 
> > As in an MMIO aperture?  If its MMIO on the bus you should be able to 
> > just call ioremap with the bus address.  By nature of it being outside 
> > of real ram, it should automatically be uncached (unless you've set an 
> > MTRR over that region saying otherwise).
> 
> It's not outside of real RAM.  The device is inside real RAM (it sits on the
> DIMM itself), but I need to poke through the entire 4GB range to see how it
> responds.

The architecture makes some difference.  If the device is inside of RAM
there are two moderately simple ways on x86 to make it work.
1) set mem=yyy where yyy = real_ram but is smaller than your device.
   make certain your device isn't on any mtrr.
2) Disable SPD on your device.
   Do the setup of the pseudo dimm yourself.

In either case that will leave you with a device on the memory bus,
but for all intents and purposes it is then just an i/o device you can
treat like any other.

Eric
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: ioremap_nocache problem?

[Timur Tabi]

** Reply to message from Eric W. Biederman <ebiederm@xmission.com> on 26 Jan
2001 09:32:58 -0700


> 1) set mem=yyy where yyy = real_ram but is smaller than your device.
>    make certain your device isn't on any mtrr.

That won't work, because the BIOS will enable MTRR regardless if it sees the
RAM.  Besides, I can't set mem=yyy because the device could sit in the LOWER
memory areas, not the higher ones.  I could have two DIMMs in the computer, and
the first one could have our device.  Or, both could have our device.  Not only
that, but the interleaving will determine exactly where the device sits.

> 2) Disable SPD on your device.
>    Do the setup of the pseudo dimm yourself.

That won't work either, because I'd have to know how to program the memory
controller, and each machine is different.

Thanks for the ideas, though.


-- 
Timur Tabi - ttabi@interactivesi.com
Interactive Silicon - http://www.interactivesi.com

When replying to a mailing-list message, please direct the reply to the mailing list only.  Don't send another copy to me.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: ioremap_nocache problem?

[Timur Tabi]

** Reply to message from Jeff Hartmann <jhartmann@valinux.com> on Thu, 25 Jan
2001 10:47:13 -0700


> As in an MMIO aperture?  If its MMIO on the bus you should be able to 
> just call ioremap with the bus address.  By nature of it being outside 
> of real ram, it should automatically be uncached (unless you've set an 
> MTRR over that region saying otherwise).

It's not outside of real RAM.  The device is inside real RAM (it sits on the
DIMM itself), but I need to poke through the entire 4GB range to see how it
responds.

> Look at the functions agp_generic_free_gatt_table and 
> agp_generic_create_gatt_table in agpgart_be.c (drivers/char/agp).  They 
> do the ioremap_nocache on real ram for the GATT/GART table.

Unfortunately, the memory they remap is allocated:

table = (char *) __get_free_pages(GFP_KERNEL, page_order);

...

CACHE_FLUSH();
agp_bridge.gatt_table = ioremap_nocache(virt_to_phys(table), (PAGE_SIZE * (1 <<
page_order)));
CACHE_FLUSH();

I've searched high and low for examples of code that does what I do, and I
can't find any.


-- 
Timur Tabi - ttabi@interactivesi.com
Interactive Silicon - http://www.interactivesi.com

When replying to a mailing-list message, please direct the reply to the mailing list only.  Don't send another copy to me.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ioremap_nocache problem?

[Jeff Hartmann]

Timur Tabi wrote:

> ** Reply to message from Jeff Hartmann <jhartmann@valinux.com> on Thu, 25 Jan
> 2001 10:47:13 -0700
> 
> 
> 
>> As in an MMIO aperture?  If its MMIO on the bus you should be able to 
>> just call ioremap with the bus address.  By nature of it being outside 
>> of real ram, it should automatically be uncached (unless you've set an 
>> MTRR over that region saying otherwise).
> 
> 
> It's not outside of real RAM.  The device is inside real RAM (it sits on the
> DIMM itself), but I need to poke through the entire 4GB range to see how it
> responds.
> 
> 
>> Look at the functions agp_generic_free_gatt_table and 
>> agp_generic_create_gatt_table in agpgart_be.c (drivers/char/agp).  They 
>> do the ioremap_nocache on real ram for the GATT/GART table.
> 
> 
> Unfortunately, the memory they remap is allocated:
> 
> table = (char *) __get_free_pages(GFP_KERNEL, page_order);
> 
> ...
> 
> CACHE_FLUSH();
> agp_bridge.gatt_table = ioremap_nocache(virt_to_phys(table), (PAGE_SIZE * (1 <<
> page_order)));
> CACHE_FLUSH();
> 
> I've searched high and low for examples of code that does what I do, and I
> can't find any.

You need to have your driver in the early bootup process then.  When 
memory is being detected (but before the free lists are created.), you 
can set your page as being reserved.  Then the kernel will leave it 
alone when it creates its free lists.  This does mean that this driver 
can not be a module and that it must run at least part of itself in the 
early bootup process.  I don't remember exactly where you need to do 
this, you might try looking at arch/i386/mm/init.c (Just an educated guess.)

-Jeff

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ioremap_nocache problem?

[Jeff Hartmann]

Timur Tabi wrote:

> ** Reply to message from Jeff Hartmann <jhartmann@valinux.com> on Thu, 25 Jan
> 2001 10:47:13 -0700
> 
> 
> 
>> As in an MMIO aperture?  If its MMIO on the bus you should be able to 
>> just call ioremap with the bus address.  By nature of it being outside 
>> of real ram, it should automatically be uncached (unless you've set an 
>> MTRR over that region saying otherwise).
> 
> 
> It's not outside of real RAM.  The device is inside real RAM (it sits on the
> DIMM itself), but I need to poke through the entire 4GB range to see how it
> responds.
> 
> 
>> Look at the functions agp_generic_free_gatt_table and 
>> agp_generic_create_gatt_table in agpgart_be.c (drivers/char/agp).  They 
>> do the ioremap_nocache on real ram for the GATT/GART table.
> 
> 
> Unfortunately, the memory they remap is allocated:
> 
> table = (char *) __get_free_pages(GFP_KERNEL, page_order);
> 
> ...
> 
> CACHE_FLUSH();
> agp_bridge.gatt_table = ioremap_nocache(virt_to_phys(table), (PAGE_SIZE * (1 <<
> page_order)));
> CACHE_FLUSH();
> 
> I've searched high and low for examples of code that does what I do, and I
> can't find any.

You need to have your driver in the early bootup process then.  When 
memory is being detected (but before the free lists are created.), you 
can set your page as being reserved.  Then the kernel will leave it 
alone when it creates its free lists.  This does mean that this driver 
can not be a module and that it must run at least part of itself in the 
early bootup process.  I don't remember exactly where you need to do 
this, you might try looking at arch/i386/mm/init.c (Just an educated guess.)

-Jeff

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: ioremap_nocache problem?

[Timur Tabi]

** Reply to message from Jeff Hartmann <jhartmann@valinux.com> on Thu, 25 Jan
2001 11:13:35 -0700


> You need to have your driver in the early bootup process then.  When 
> memory is being detected (but before the free lists are created.), you 
> can set your page as being reserved. 

But doesn't this mean that my driver has to be built as part of the kernel?
The end-user won't have the source code, so he won't be able to compile it, only
link it.  As it stands now, our driver is a binary that can be shipped
separately.


-- 
Timur Tabi - ttabi@interactivesi.com
Interactive Silicon - http://www.interactivesi.com

When replying to a mailing-list message, please direct the reply to the mailing list only.  Don't send another copy to me.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: ioremap_nocache problem?

[Timur Tabi]

** Reply to message from Jeff Hartmann <jhartmann@valinux.com> on Thu, 25 Jan
2001 11:13:35 -0700


> You need to have your driver in the early bootup process then.  When 
> memory is being detected (but before the free lists are created.), you 
> can set your page as being reserved. 

But doesn't this mean that my driver has to be built as part of the kernel?
The end-user won't have the source code, so he won't be able to compile it, only
link it.  As it stands now, our driver is a binary that can be shipped
separately.


-- 
Timur Tabi - ttabi@interactivesi.com
Interactive Silicon - http://www.interactivesi.com

When replying to a mailing-list message, please direct the reply to the mailing list only.  Don't send another copy to me.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ioremap_nocache problem?

[Jeff Hartmann]

Timur Tabi wrote:

> ** Reply to message from Jeff Hartmann <jhartmann@valinux.com> on Thu, 25 Jan
> 2001 11:13:35 -0700
> 
> 
> 
>> You need to have your driver in the early bootup process then.  When 
>> memory is being detected (but before the free lists are created.), you 
>> can set your page as being reserved. 
> 
> 
> But doesn't this mean that my driver has to be built as part of the kernel?
> The end-user won't have the source code, so he won't be able to compile it, only
> link it.  As it stands now, our driver is a binary that can be shipped
> separately.

Sorry, this is the only way to do it properly.  Binary kernel drivers 
are intensely evil. ;)  Open the driver and you have no problems.  You 
also do know that binary kernel drivers mean you'll be chasing every 
kernel release, having to provide several different flavors of your 
binary depending on the users kernel configuration.  It also means that 
when kernel interfaces change, people won't be nice and change your code 
over to the new interfaces for you.  For instance if a function 
depreciates, your code might be automatically moved to use the 
replacement function if your in the standard kernel.  If your a binary 
module, you have to do all that maintaining yourself.

(There are several other reasons to have open kernel modules.  I won't 
go into the entire argument, since that could take all day.)

You might be able to get away with making detection of this page open, 
and keep the rest of the driver closed.  However that is something for 
Linus to decided, not I.  I believe he doesn't like putting in hooks in 
the kernel for binary modules.  Since all you really want to do is 
reserve the page during early bootup, perhaps he might let you get away 
with it.  Not my call though.

-Jeff

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ioremap_nocache problem?

[Jeff Hartmann]

Timur Tabi wrote:

> ** Reply to message from Jeff Hartmann <jhartmann@valinux.com> on Thu, 25 Jan
> 2001 11:13:35 -0700
> 
> 
> 
>> You need to have your driver in the early bootup process then.  When 
>> memory is being detected (but before the free lists are created.), you 
>> can set your page as being reserved. 
> 
> 
> But doesn't this mean that my driver has to be built as part of the kernel?
> The end-user won't have the source code, so he won't be able to compile it, only
> link it.  As it stands now, our driver is a binary that can be shipped
> separately.

Sorry, this is the only way to do it properly.  Binary kernel drivers 
are intensely evil. ;)  Open the driver and you have no problems.  You 
also do know that binary kernel drivers mean you'll be chasing every 
kernel release, having to provide several different flavors of your 
binary depending on the users kernel configuration.  It also means that 
when kernel interfaces change, people won't be nice and change your code 
over to the new interfaces for you.  For instance if a function 
depreciates, your code might be automatically moved to use the 
replacement function if your in the standard kernel.  If your a binary 
module, you have to do all that maintaining yourself.

(There are several other reasons to have open kernel modules.  I won't 
go into the entire argument, since that could take all day.)

You might be able to get away with making detection of this page open, 
and keep the rest of the driver closed.  However that is something for 
Linus to decided, not I.  I believe he doesn't like putting in hooks in 
the kernel for binary modules.  Since all you really want to do is 
reserve the page during early bootup, perhaps he might let you get away 
with it.  Not my call though.

-Jeff

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: ioremap_nocache problem?

[Stephen C. Tweedie]

Hi,

On Thu, Jan 25, 2001 at 09:56:32AM -0600, Timur Tabi wrote:
> > ioremap*() is only supposed to be used on IO regions or reserved
> > pages.  If you haven't marked the pages as reserved, then iounmap will
> > do the wrong thing, so it's up to you to reserve the pages.
> 
> Au contraire!
> 
> I mark the page as reserved when I ioremap() it.  However, if I leave it marked
> reserved, then iounmap() will not unmap it.  

It certainly should do, and the 2.4 source certainly looks as if it
does.  At least on i386, iounmap calls vfree, which ends up in
free_area_pte(), which will unconditionally clear the pte (hence
unmapping the page).

Cheers,
 Stephen
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ioremap_nocache problem?

[Stephen C. Tweedie]

Hi,

On Thu, Jan 25, 2001 at 09:56:32AM -0600, Timur Tabi wrote:
> > ioremap*() is only supposed to be used on IO regions or reserved
> > pages.  If you haven't marked the pages as reserved, then iounmap will
> > do the wrong thing, so it's up to you to reserve the pages.
> 
> Au contraire!
> 
> I mark the page as reserved when I ioremap() it.  However, if I leave it marked
> reserved, then iounmap() will not unmap it.  

It certainly should do, and the 2.4 source certainly looks as if it
does.  At least on i386, iounmap calls vfree, which ends up in
free_area_pte(), which will unconditionally clear the pte (hence
unmapping the page).

Cheers,
 Stephen
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: ioremap_nocache problem?

[Timur Tabi]

** Reply to message from "Stephen C. Tweedie" <sct@redhat.com> on Thu, 25 Jan
2001 15:16:55 +0000


> ioremap*() is only supposed to be used on IO regions or reserved
> pages.  If you haven't marked the pages as reserved, then iounmap will
> do the wrong thing, so it's up to you to reserve the pages.

Au contraire!

I mark the page as reserved when I ioremap() it.  However, if I leave it marked
reserved, then iounmap() will not unmap it.  If I mark it "unreserved" (i.e.
reset the reserved bit), then iounmap will unmap it, but it will decrement the
page counter to -1 and the whole system will crash soon thereafter.

I've been asking about this problem for months, but no one has bothered to help
me out.


-- 
Timur Tabi - ttabi@interactivesi.com
Interactive Silicon - http://www.interactivesi.com

When replying to a mailing-list message, please direct the reply to the mailing list only.  Don't send another copy to me.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ioremap_nocache problem?

[Roman Zippel]

Hi,

On Tue, 23 Jan 2001, Mark Mokryn wrote:

> ioremap_nocache does the following:
> 	return __ioremap(offset, size, _PAGE_PCD);
> 
> However, in drivers/char/mem.c (2.4.0), we see the following:
> 
> 	/* On PPro and successors, PCD alone doesn't always mean 
> 	    uncached because of interactions with the MTRRs. PCD | PWT
> 	    means definitely uncached. */ 
> 	if (boot_cpu_data.x86 > 3)
> 		prot |= _PAGE_PCD | _PAGE_PWT;
> 
> Does this mean ioremap_nocache() may not do the job?

ioremap creates a new mapping that shouldn't interfere with MTRR, whereas
you can map a MTRR mapped area into userspace. But I'm not sure if it's
correct that no flag is set for boot_cpu_data.x86 <= 3...

bye, Roman

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ioremap_nocache problem?

[Roman Zippel]

Hi,

On Tue, 23 Jan 2001, Mark Mokryn wrote:

> ioremap_nocache does the following:
> 	return __ioremap(offset, size, _PAGE_PCD);
> 
> However, in drivers/char/mem.c (2.4.0), we see the following:
> 
> 	/* On PPro and successors, PCD alone doesn't always mean 
> 	    uncached because of interactions with the MTRRs. PCD | PWT
> 	    means definitely uncached. */ 
> 	if (boot_cpu_data.x86 > 3)
> 		prot |= _PAGE_PCD | _PAGE_PWT;
> 
> Does this mean ioremap_nocache() may not do the job?

ioremap creates a new mapping that shouldn't interfere with MTRR, whereas
you can map a MTRR mapped area into userspace. But I'm not sure if it's
correct that no flag is set for boot_cpu_data.x86 <= 3...

bye, Roman

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: ioremap_nocache problem?

[Timur Tabi]

** Reply to message from Roman Zippel <zippel@fh-brandenburg.de> on Tue, 23 Jan
2001 19:12:36 +0100 (MET)


> ioremap creates a new mapping that shouldn't interfere with MTRR, whereas
> you can map a MTRR mapped area into userspace. But I'm not sure if it's
> correct that no flag is set for boot_cpu_data.x86 <= 3...

I was under the impression that the "don't cache" bit that ioremap_nocache sets
overrides any MTRR.


-- 
Timur Tabi - ttabi@interactivesi.com
Interactive Silicon - http://www.interactivesi.com

When replying to a mailing-list message, please direct the reply to the mailing list only.  Don't send another copy to me.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: ioremap_nocache problem?

[David Wragg]

From: David Wragg <dpw@doc.ic.ac.uk>
Gcc: nnfolder:mail.sent
--text follows this line--
Roman Zippel <zippel@fh-brandenburg.de> writes:
> On Tue, 23 Jan 2001, Mark Mokryn wrote:
> > ioremap_nocache does the following:
> >     return __ioremap(offset, size, _PAGE_PCD);

You have a point.

It would be nice if ioremap took a argument indicating the desired
memory type -- normal, nocache, write-through, write-combining, etc.
Then it could look in an architecture-specific table to get the
appropriate page flags for that type.

(x86 processors with PAT and IA64 can set write-combining through page
flags.  x86 processors with MTRRs but not PAT would need a more
elaborate implementation for write-combining.)

> > 
> > However, in drivers/char/mem.c (2.4.0), we see the following:
> > 
> >     /* On PPro and successors, PCD alone doesn't always mean 
> >         uncached because of interactions with the MTRRs. PCD | PWT
> >         means definitely uncached. */ 
> >     if (boot_cpu_data.x86 > 3)
> >             prot |= _PAGE_PCD | _PAGE_PWT;
> > 
> > Does this mean ioremap_nocache() may not do the job?
> 
> ioremap creates a new mapping that shouldn't interfere with MTRR, whereas
> you can map a MTRR mapped area into userspace. But I'm not sure if it's
> correct that no flag is set for boot_cpu_data.x86 <= 3...

The boot_cpu_data.x86 > 3 test is there because the 386 doesn't have
PWT.


David Wragg
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ioremap_nocache problem?

[David Wragg, David Wragg]

--text follows this line--
Roman Zippel <zippel@fh-brandenburg.de> writes:
> On Tue, 23 Jan 2001, Mark Mokryn wrote:
> > ioremap_nocache does the following:
> >     return __ioremap(offset, size, _PAGE_PCD);

You have a point.

It would be nice if ioremap took a argument indicating the desired
memory type -- normal, nocache, write-through, write-combining, etc.
Then it could look in an architecture-specific table to get the
appropriate page flags for that type.

(x86 processors with PAT and IA64 can set write-combining through page
flags.  x86 processors with MTRRs but not PAT would need a more
elaborate implementation for write-combining.)

> > 
> > However, in drivers/char/mem.c (2.4.0), we see the following:
> > 
> >     /* On PPro and successors, PCD alone doesn't always mean 
> >         uncached because of interactions with the MTRRs. PCD | PWT
> >         means definitely uncached. */ 
> >     if (boot_cpu_data.x86 > 3)
> >             prot |= _PAGE_PCD | _PAGE_PWT;
> > 
> > Does this mean ioremap_nocache() may not do the job?
> 
> ioremap creates a new mapping that shouldn't interfere with MTRR, whereas
> you can map a MTRR mapped area into userspace. But I'm not sure if it's
> correct that no flag is set for boot_cpu_data.x86 <= 3...

The boot_cpu_data.x86 > 3 test is there because the 386 doesn't have
PWT.


David Wragg
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: ioremap_nocache problem?

[Timur Tabi]

** Reply to message from David Wragg <dpw@doc.ic.ac.uk> on 24 Jan 2001 00:50:20
+0000


> (x86 processors with PAT and IA64 can set write-combining through page
> flags.  x86 processors with MTRRs but not PAT would need a more
> elaborate implementation for write-combining.)

What is PAT?  I desperately need to figure out how to turn on write combining
on a per-page level.  I thought I had to use MTRRs, but now you're saying I can
use this "PAT" thing instead.  Please explain!


-- 
Timur Tabi - ttabi@interactivesi.com
Interactive Silicon - http://www.interactivesi.com

When replying to a mailing-list message, please direct the reply to the mailing list only.  Don't send another copy to me.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: ioremap_nocache problem?

[Timur Tabi]

** Reply to message from David Wragg <dpw@doc.ic.ac.uk> on 24 Jan 2001 00:50:20
+0000


> (x86 processors with PAT and IA64 can set write-combining through page
> flags.  x86 processors with MTRRs but not PAT would need a more
> elaborate implementation for write-combining.)

What is PAT?  I desperately need to figure out how to turn on write combining
on a per-page level.  I thought I had to use MTRRs, but now you're saying I can
use this "PAT" thing instead.  Please explain!


-- 
Timur Tabi - ttabi@interactivesi.com
Interactive Silicon - http://www.interactivesi.com

When replying to a mailing-list message, please direct the reply to the mailing list only.  Don't send another copy to me.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ioremap_nocache problem?

[David Wragg]

Timur Tabi <ttabi@interactivesi.com> writes:
> ** Reply to message from David Wragg <dpw@doc.ic.ac.uk> on 24 Jan 2001
> 00:50:20 +0000
> > (x86 processors with PAT and IA64 can set write-combining through
> >page flags.  x86 processors with MTRRs but not PAT would need a more
> >elaborate implementation for write-combining.)
> 
> What is PAT?  I desperately need to figure out how to turn on write
> combining on a per-page level.  I thought I had to use MTRRs, but now
> you're saying I can use this "PAT" thing instead.  Please explain!

PAT is basically the MTRR memory types on a per-page basis.  It adds a
new flag bit to the x86 page table entry, then that bit together with
the PCD and PWT bits is used to do a look-up in an 8-entry table that
gives the effective memory type (the table is set through an MSR).
All the details are in the Intel x86 manual, volume 3
<URL:http://developer.intel.com/design/pentium4/manuals/> (at the end
of chapter 9).

Quite a lot of the x86 CPUs out there support PAT: The PII except the
first couple of models, the Celeron except the first model, the PIII,
all PII and PIII Xeons, the P4, all AMD K7 models.  I'm guessing, but
I suspect that the majority of x86 CPUs supporting write combining in
any form that have been made also support PAT.

I wish Intel had put PAT in the PPro, rather than messing everyone
around with MTRRs (MTRRs are good for BIOS writers, but a pain for
everyone else).


David Wragg
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ioremap_nocache problem?

[David Wragg]

Timur Tabi <ttabi@interactivesi.com> writes:
> ** Reply to message from David Wragg <dpw@doc.ic.ac.uk> on 24 Jan 2001
> 00:50:20 +0000
> > (x86 processors with PAT and IA64 can set write-combining through
> >page flags.  x86 processors with MTRRs but not PAT would need a more
> >elaborate implementation for write-combining.)
> 
> What is PAT?  I desperately need to figure out how to turn on write
> combining on a per-page level.  I thought I had to use MTRRs, but now
> you're saying I can use this "PAT" thing instead.  Please explain!

PAT is basically the MTRR memory types on a per-page basis.  It adds a
new flag bit to the x86 page table entry, then that bit together with
the PCD and PWT bits is used to do a look-up in an 8-entry table that
gives the effective memory type (the table is set through an MSR).
All the details are in the Intel x86 manual, volume 3
<URL:http://developer.intel.com/design/pentium4/manuals/> (at the end
of chapter 9).

Quite a lot of the x86 CPUs out there support PAT: The PII except the
first couple of models, the Celeron except the first model, the PIII,
all PII and PIII Xeons, the P4, all AMD K7 models.  I'm guessing, but
I suspect that the majority of x86 CPUs supporting write combining in
any form that have been made also support PAT.

I wish Intel had put PAT in the PPro, rather than messing everyone
around with MTRRs (MTRRs are good for BIOS writers, but a pain for
everyone else).


David Wragg
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: ioremap_nocache problem?

[Timur Tabi]

** Reply to message from Roman Zippel <zippel@fh-brandenburg.de> on Tue, 23 Jan
2001 19:12:36 +0100 (MET)


> ioremap creates a new mapping that shouldn't interfere with MTRR, whereas
> you can map a MTRR mapped area into userspace. But I'm not sure if it's
> correct that no flag is set for boot_cpu_data.x86 <= 3...

I was under the impression that the "don't cache" bit that ioremap_nocache sets
overrides any MTRR.


-- 
Timur Tabi - ttabi@interactivesi.com
Interactive Silicon - http://www.interactivesi.com

When replying to a mailing-list message, please direct the reply to the mailing list only.  Don't send another copy to me.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ioremap_nocache problem?

[David Wragg]

Timur Tabi <ttabi@interactivesi.com> writes:
> ** Reply to message from Roman Zippel <zippel@fh-brandenburg.de> on
> Tue, 23 Jan 2001 19:12:36 +0100 (MET)
> > ioremap creates a new mapping that shouldn't interfere with MTRR,
> >whereas you can map a MTRR mapped area into userspace. But I'm not
> >sure if it's correct that no flag is set for boot_cpu_data.x86 <=
> >3...
> 
> I was under the impression that the "don't cache" bit that
> ioremap_nocache sets overrides any MTRR.

Nope.  There's a table explaining how page flags and MTRRs interact in
the Intel x86 manual, volume 3 (it's in section 9.5.1 "Precedence of
Cache Controls" in the fairly recent edition I have here).

For example, with PCD set, PWT clear, and the MTRRs saying WC, the
effective memory type is WC.  In addition, there's a note saying this
may change in future models.  So you have to set PCD | PWT if you want
to get uncached in all cases.


David Wragg
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: ioremap_nocache problem?

[David Wragg]

Timur Tabi <ttabi@interactivesi.com> writes:
> ** Reply to message from Roman Zippel <zippel@fh-brandenburg.de> on
> Tue, 23 Jan 2001 19:12:36 +0100 (MET)
> > ioremap creates a new mapping that shouldn't interfere with MTRR,
> >whereas you can map a MTRR mapped area into userspace. But I'm not
> >sure if it's correct that no flag is set for boot_cpu_data.x86 <=
> >3...
> 
> I was under the impression that the "don't cache" bit that
> ioremap_nocache sets overrides any MTRR.

Nope.  There's a table explaining how page flags and MTRRs interact in
the Intel x86 manual, volume 3 (it's in section 9.5.1 "Precedence of
Cache Controls" in the fairly recent edition I have here).

For example, with PCD set, PWT clear, and the MTRRs saying WC, the
effective memory type is WC.  In addition, there's a note saying this
may change in future models.  So you have to set PCD | PWT if you want
to get uncached in all cases.


David Wragg
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/