* Re: [PATCH v4 2/2] IB/qib: use arch_phys_wc_add()
From: Luis R. Rodriguez @ 2015-04-22 15:33 UTC (permalink / raw)
To: Doug Ledford
Cc: Luis R. Rodriguez, infinipath-ral2JQCrhuEAvxtiuMwx3w,
roland-DgEjT+Ai2ygdnm+yROfE0A, sean.hefty-ral2JQCrhuEAvxtiuMwx3w,
hal.rosenstock-Re5JQEeQqe8AvxtiuMwx3w,
linux-rdma-u79uwXL29TY76Z2rM5mHXA, luto-kltTT9wpgjJwATOyAt5JVQ,
mst-H+wXaHxf7aLQT0dZR+AlfA, linux-kernel-u79uwXL29TY76Z2rM5mHXA,
cocci-/FJkirnvOdkvYVN+rsErww, Toshi Kani, Rickard Strandqvist,
Mike Marciniszyn, Roland Dreier, Dennis Dalessandro,
Suresh Siddha, Ingo Molnar, Thomas Gleixner, Juergen Gross,
Daniel Vetter, Dave Airlie, Bjorn Helgaas, Antonino Daplas,
Jean-Christophe Plagniol-Villard, Tomi Valkeinen
In-Reply-To: <1429710878.45956.94.camel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
On Wed, Apr 22, 2015 at 09:54:38AM -0400, Doug Ledford wrote:
> On Tue, 2015-04-21 at 14:50 -0700, Luis R. Rodriguez wrote:
>
> This:
> > + /* MTRR was used if this is non-zero */
> > + if (!dd->wc_cookie)
> > vma->vm_page_prot = pgprot_writecombine(vma->vm_page_prot);
>
> And this:
> > + dd->wc_cookie = arch_phys_wc_add(pioaddr, piolen);
> > + if (dd->wc_cookie < 0)
> > + ret = -EINVAL;
>
> don't agree on what wc_cookie will be on error.
Can you elaborate? The one below is the one that starts things,
and arch_phys_wc_add() will return 0 on PAT systems. For non-PAT
systems it will return a number > 0 *iff* a valid MTRR was added.
It will return negative onloy on error then.
The change above is meant to replace a check put in place to see
if PAT was enabled. The way we replace this is to ensure that
arch_phys_wc_add() returned 0.
If you disagree it'd be great if you can elaborate why.
Luis
^ permalink raw reply
* Re: ioremap_uc() followed by set_memory_wc() - burrying MTRR
From: Luis R. Rodriguez @ 2015-04-22 15:54 UTC (permalink / raw)
To: Jason Gunthorpe, Andy Walls
Cc: Andy Lutomirski, mike.marciniszyn, infinipath, linux-rdma,
Toshi Kani, H. Peter Anvin, Ingo Molnar,
linux-kernel@vger.kernel.org, Hal Rosenstock, Sean Hefty,
Suresh Siddha, Rickard Strandqvist, Roland Dreier, Juergen Gross,
Mauro Carvalho Chehab, Borislav Petkov, Mel Gorman,
Vlastimil Babka, Davidlohr Bueso, Dave Hansen,
Jean-Christophe Plagniol-Villard, Thomas Gleixner, Ville Syrj?l?,
Linux Fbdev development list, linux-media, X86 ML, mcgrof
In-Reply-To: <20150422152328.GB5622@wotan.suse.de>
On Wed, Apr 22, 2015 at 05:23:28PM +0200, Luis R. Rodriguez wrote:
> On Tue, Apr 21, 2015 at 11:39:39PM -0600, Jason Gunthorpe wrote:
> > On Wed, Apr 22, 2015 at 01:39:07AM +0200, Luis R. Rodriguez wrote:
> > > > Mike, do you think the time is right to just remove the iPath driver?
> > >
> > > With PAT now being default the driver effectively won't work
> > > with write-combining on modern kernels. Even if systems are old
> > > they likely had PAT support, when upgrading kernels PAT will work
> > > but write-combing won't on ipath.
> >
> > Sorry, do you mean the driver already doesn't get WC? Or do you mean
> > after some more pending patches are applied?
>
> No, you have to consider the system used and the effects of calls used
> on the driver in light of this table:
>
> ----------------------------------------------------------------------
> MTRR Non-PAT PAT Linux ioremap value Effective memory type
> ----------------------------------------------------------------------
> Non-PAT | PAT
> PAT
> |PCD
> ||PWT
> |||
> WC 000 WB _PAGE_CACHE_MODE_WB WC | WC
> WC 001 WC _PAGE_CACHE_MODE_WC WC* | WC
> WC 010 UC- _PAGE_CACHE_MODE_UC_MINUS WC* | UC
> WC 011 UC _PAGE_CACHE_MODE_UC UC | UC
> ----------------------------------------------------------------------
>
> (*) denotes implementation defined and is discouraged
>
> ioremap_nocache() will use _PAGE_CACHE_MODE_UC_MINUS by default today,
> in the future we want to flip the switch and make _PAGE_CACHE_MODE_UC
> the default. When that flip occurs it will mean ipath cannot get
> write-combining on both non-PAT and PAT systems. Now that is for
> the future, lets review the current situation for ipath.
>
> For PAT capable systems if mtrr_add() is used today on a Linux system on a
> region mapped with ioremap_nocache() that will mean you effectively nullify the
> mtrr_add() effect as the combinatorial effect above yields an effective memory
> type of UC. For PAT systems you want to use ioremap_wc() on the region in
> which you need write-combining followed by arch_phys_wc_add() which will *only*
> call mtrr_add() *iff* PAT was not enabled. This also means we need to split
> the ioremap'd areas so that the area that is using ioremap_nocache() can never
> get write-combining (_PAGE_CACHE_MODE_UC). The ipath driver needs the regions
> split just as was done for the qib driver.
>
> Now we could just say that leaving things as-is is a non-issue if you are OK
> with non-write-combining effects being the default behaviour left on the ipath
> driver for PAT systems. In that case we can just use arch_phys_wc_add() on the
> driver and while it won't trigger the mtrr_add() on PAT systems it sill won't
> have any effect. We just typically don't want to see use of ioremap_nocache()
> paired with arch_phys_wc_add(), grammatically the correct thing to do is pair
> ioremap_wc() areas with a arch_phys_wc_add() to make the write-combining effects
> on non-PAT systems. If the ipath driver is not going to get he work required
> to split the regions though perhaps we can live with a corner case driver that
> annotates PAT must be disabled on the systems that use it and convert it to
> arch_phys_wc_add() to just help with phasing out of direct use of mtrr_add().
> With this strategy if and when ipath driver gets a split done it would gain WC
> on both PAT and non-PAT.
Folks, after some thought I do believe the above temporary strategy would
avoid issues and would not have to stir people up to go and make code
changes. We can use the same strategy for both ivtv and ipath:
* Annotate via Kconfig for the driver that it depends on !X86_PAT
that will ensure that PAT systems won't use it, and convert it
to use arch_phys_wc_add() to help phase out direct access to mtrr_add()
This would be correct given that the current situation on the driver
makes write-combining non-effective on PAT systems, we in fact gain
avoiding these type of use-cases, and annotate this as a big TODO item
for folks who do want it for PAT systems.
Thoughts?
Luis
^ permalink raw reply
* Re: ioremap_uc() followed by set_memory_wc() - burrying MTRR
From: Luis R. Rodriguez @ 2015-04-22 15:59 UTC (permalink / raw)
To: Jason Gunthorpe, Andy Walls
Cc: Andy Lutomirski, Mike Marciniszyn, Mike Marciniszyn, linux-rdma,
Toshi Kani, H. Peter Anvin, Ingo Molnar,
linux-kernel@vger.kernel.org, Hal Rosenstock, Sean Hefty,
Suresh Siddha, Rickard Strandqvist, Roland Dreier, Juergen Gross,
Mauro Carvalho Chehab, Borislav Petkov, Mel Gorman,
Vlastimil Babka, Davidlohr Bueso, Dave Hansen,
Jean-Christophe Plagniol-Villard, Thomas Gleixner, Ville Syrj?l?,
Linux Fbdev development list, linux-media, X86 ML,
Luis R. Rodriguez
In-Reply-To: <20150422155446.GF5622@wotan.suse.de>
On Wed, Apr 22, 2015 at 8:54 AM, Luis R. Rodriguez <mcgrof@suse.com> wrote:
> On Wed, Apr 22, 2015 at 05:23:28PM +0200, Luis R. Rodriguez wrote:
>> On Tue, Apr 21, 2015 at 11:39:39PM -0600, Jason Gunthorpe wrote:
>> > On Wed, Apr 22, 2015 at 01:39:07AM +0200, Luis R. Rodriguez wrote:
>> > > > Mike, do you think the time is right to just remove the iPath driver?
>> > >
>> > > With PAT now being default the driver effectively won't work
>> > > with write-combining on modern kernels. Even if systems are old
>> > > they likely had PAT support, when upgrading kernels PAT will work
>> > > but write-combing won't on ipath.
>> >
>> > Sorry, do you mean the driver already doesn't get WC? Or do you mean
>> > after some more pending patches are applied?
>>
>> No, you have to consider the system used and the effects of calls used
>> on the driver in light of this table:
>>
>> ----------------------------------------------------------------------
>> MTRR Non-PAT PAT Linux ioremap value Effective memory type
>> ----------------------------------------------------------------------
>> Non-PAT | PAT
>> PAT
>> |PCD
>> ||PWT
>> |||
>> WC 000 WB _PAGE_CACHE_MODE_WB WC | WC
>> WC 001 WC _PAGE_CACHE_MODE_WC WC* | WC
>> WC 010 UC- _PAGE_CACHE_MODE_UC_MINUS WC* | UC
>> WC 011 UC _PAGE_CACHE_MODE_UC UC | UC
>> ----------------------------------------------------------------------
>>
>> (*) denotes implementation defined and is discouraged
>>
>> ioremap_nocache() will use _PAGE_CACHE_MODE_UC_MINUS by default today,
>> in the future we want to flip the switch and make _PAGE_CACHE_MODE_UC
>> the default. When that flip occurs it will mean ipath cannot get
>> write-combining on both non-PAT and PAT systems. Now that is for
>> the future, lets review the current situation for ipath.
>>
>> For PAT capable systems if mtrr_add() is used today on a Linux system on a
>> region mapped with ioremap_nocache() that will mean you effectively nullify the
>> mtrr_add() effect as the combinatorial effect above yields an effective memory
>> type of UC. For PAT systems you want to use ioremap_wc() on the region in
>> which you need write-combining followed by arch_phys_wc_add() which will *only*
>> call mtrr_add() *iff* PAT was not enabled. This also means we need to split
>> the ioremap'd areas so that the area that is using ioremap_nocache() can never
>> get write-combining (_PAGE_CACHE_MODE_UC). The ipath driver needs the regions
>> split just as was done for the qib driver.
>>
>> Now we could just say that leaving things as-is is a non-issue if you are OK
>> with non-write-combining effects being the default behaviour left on the ipath
>> driver for PAT systems. In that case we can just use arch_phys_wc_add() on the
>> driver and while it won't trigger the mtrr_add() on PAT systems it sill won't
>> have any effect. We just typically don't want to see use of ioremap_nocache()
>> paired with arch_phys_wc_add(), grammatically the correct thing to do is pair
>> ioremap_wc() areas with a arch_phys_wc_add() to make the write-combining effects
>> on non-PAT systems. If the ipath driver is not going to get he work required
>> to split the regions though perhaps we can live with a corner case driver that
>> annotates PAT must be disabled on the systems that use it and convert it to
>> arch_phys_wc_add() to just help with phasing out of direct use of mtrr_add().
>> With this strategy if and when ipath driver gets a split done it would gain WC
>> on both PAT and non-PAT.
>
> Folks, after some thought I do believe the above temporary strategy would
> avoid issues and would not have to stir people up to go and make code
> changes. We can use the same strategy for both ivtv and ipath:
>
> * Annotate via Kconfig for the driver that it depends on !X86_PAT
> that will ensure that PAT systems won't use it, and convert it
> to use arch_phys_wc_add() to help phase out direct access to mtrr_add()
>
> This would be correct given that the current situation on the driver
> makes write-combining non-effective on PAT systems, we in fact gain
> avoiding these type of use-cases, and annotate this as a big TODO item
> for folks who do want it for PAT systems.
>
> Thoughts?
Another option in order to enable this type of checks at run time and
still be able to build the driver on standard distributions and just
prevent if from loading on PAT systems is to have some code in place
which would prevent the driver from loading if PAT was enabled, this
would enable folks to disable PAT via a kernel command line option,
and if that was used then the driver probe would complete.
Luis
^ permalink raw reply
* Re: ioremap_uc() followed by set_memory_wc() - burrying MTRR
From: Jason Gunthorpe @ 2015-04-22 16:17 UTC (permalink / raw)
To: Luis R. Rodriguez
Cc: Andy Lutomirski, mike.marciniszyn, infinipath, linux-rdma, awalls,
Toshi Kani, H. Peter Anvin, Ingo Molnar,
linux-kernel@vger.kernel.org, Hal Rosenstock, Sean Hefty,
Suresh Siddha, Rickard Strandqvist, Roland Dreier, Juergen Gross,
Mauro Carvalho Chehab, Borislav Petkov, Mel Gorman,
Vlastimil Babka, Davidlohr Bueso, Dave Hansen,
Jean-Christophe Plagniol-Villard, Thomas Gleixner, Ville Syrj?l?,
Linux Fbdev development list, linux-media, X86 ML, mcgrof
In-Reply-To: <20150422152328.GB5622@wotan.suse.de>
On Wed, Apr 22, 2015 at 05:23:28PM +0200, Luis R. Rodriguez wrote:
> On Tue, Apr 21, 2015 at 11:39:39PM -0600, Jason Gunthorpe wrote:
> > On Wed, Apr 22, 2015 at 01:39:07AM +0200, Luis R. Rodriguez wrote:
> > > > Mike, do you think the time is right to just remove the iPath driver?
> > >
> > > With PAT now being default the driver effectively won't work
> > > with write-combining on modern kernels. Even if systems are old
> > > they likely had PAT support, when upgrading kernels PAT will work
> > > but write-combing won't on ipath.
> >
> > Sorry, do you mean the driver already doesn't get WC? Or do you mean
> > after some more pending patches are applied?
>
> No, you have to consider the system used and the effects of calls used
> on the driver in light of this table:
So, just to be clear:
At some point Linux started setting the PAT bits during
ioremap_nocache, which overrides MTRR, and at that point the driver
became broken on all PAT capable systems?
Not only that, but we've only just noticed it now, and no user ever
complained?
So that means either no users exist, or all users are on non-PAT
systems?
This driver only works on x86-64 systems. Are there any x86-64 systems
that are not PAT capable? IIRC even the first Opteron had PAT, but my
memory is fuzzy from back then :|
> Another option in order to enable this type of checks at run time
> and still be able to build the driver on standard distributions and
> just prevent if from loading on PAT systems is to have some code in
> place which would prevent the driver from loading if PAT was
> enabled, this would enable folks to disable PAT via a kernel command
> line option, and if that was used then the driver probe would
> complete.
This seems like a reasonble option to me. At the very least we might
learn if anyone is still using these cards.
I'd also love to remove the driver if it turns out there are actually
no users. qib substantially replaces it except for a few very old
cards.
Mike?
Jason
^ permalink raw reply
* Re: ioremap_uc() followed by set_memory_wc() - burrying MTRR
From: Luis R. Rodriguez @ 2015-04-22 16:51 UTC (permalink / raw)
To: Jason Gunthorpe, Juergen Gross, Andy Walls
Cc: Andy Lutomirski, mike.marciniszyn, infinipath, linux-rdma,
Toshi Kani, H. Peter Anvin, Ingo Molnar,
linux-kernel@vger.kernel.org, Hal Rosenstock, Sean Hefty,
Suresh Siddha, Rickard Strandqvist, Roland Dreier,
Mauro Carvalho Chehab, Borislav Petkov, Mel Gorman,
Vlastimil Babka, Davidlohr Bueso, Dave Hansen,
Jean-Christophe Plagniol-Villard, Thomas Gleixner, Ville Syrj?l?,
Linux Fbdev development list, linux-media, X86 ML, mcgrof
In-Reply-To: <20150422161755.GA19500@obsidianresearch.com>
On Wed, Apr 22, 2015 at 10:17:55AM -0600, Jason Gunthorpe wrote:
> On Wed, Apr 22, 2015 at 05:23:28PM +0200, Luis R. Rodriguez wrote:
> > On Tue, Apr 21, 2015 at 11:39:39PM -0600, Jason Gunthorpe wrote:
> > > On Wed, Apr 22, 2015 at 01:39:07AM +0200, Luis R. Rodriguez wrote:
> > > > > Mike, do you think the time is right to just remove the iPath driver?
> > > >
> > > > With PAT now being default the driver effectively won't work
> > > > with write-combining on modern kernels. Even if systems are old
> > > > they likely had PAT support, when upgrading kernels PAT will work
> > > > but write-combing won't on ipath.
> > >
> > > Sorry, do you mean the driver already doesn't get WC? Or do you mean
> > > after some more pending patches are applied?
> >
> > No, you have to consider the system used and the effects of calls used
> > on the driver in light of this table:
>
> So, just to be clear:
>
> At some point Linux started setting the PAT bits during
> ioremap_nocache, which overrides MTRR, and at that point the driver
> became broken on all PAT capable systems?
No, PAT code lacked quite a bit of love, and Juergen and some others have
been giving it some love and now we expect PAT to be enabled by default on
more systems. When and and on what systemes and as of what commits? Not
sure, there's quite a bit of PAT work but hoping Juergen might fill
in the details, CC'd.
> Not only that, but we've only just noticed it now, and no user ever
> complained?
No, well this is all recent, so we expect more PAT enabled systems now.
> So that means either no users exist, or all users are on non-PAT
> systems?
PAT was not being enabled before on most systems, it will now be on
most systems, for some systems there may be some errata that needs to
be addressed for PAT.
> This driver only works on x86-64 systems. Are there any x86-64 systems
> that are not PAT capable?
Not that I know of, but I've heard of some "PAT errata" systems, and
that may need some attention.
> IIRC even the first Opteron had PAT, but my
> memory is fuzzy from back then :|
>
> > Another option in order to enable this type of checks at run time
> > and still be able to build the driver on standard distributions and
> > just prevent if from loading on PAT systems is to have some code in
> > place which would prevent the driver from loading if PAT was
> > enabled, this would enable folks to disable PAT via a kernel command
> > line option, and if that was used then the driver probe would
> > complete.
>
> This seems like a reasonble option to me. At the very least we might
> learn if anyone is still using these cards.
OK great.
> I'd also love to remove the driver if it turns out there are actually
> no users. qib substantially replaces it except for a few very old
> cards.
>
> Mike?
By replacing do you mean same hardware? BTW below are the changes
which I describe above which would prevent both ipath and ivtv
to load on PAT enabled systems. I think its a reasonable compromise.
If this is OK I can proceed with a split of the patches and move
on with the last series that burries MTRR.
Andy Walls, please review too.
Luis
diff --git a/drivers/infiniband/hw/ipath/ipath_driver.c b/drivers/infiniband/hw/ipath/ipath_driver.c
index bd0caed..3ef592c 100644
--- a/drivers/infiniband/hw/ipath/ipath_driver.c
+++ b/drivers/infiniband/hw/ipath/ipath_driver.c
@@ -42,6 +42,9 @@
#include <linux/bitmap.h>
#include <linux/slab.h>
#include <linux/module.h>
+#ifdef CONFIG_X86_64
+#include <asm/pat.h>
+#endif
#include "ipath_kernel.h"
#include "ipath_verbs.h"
@@ -395,6 +398,14 @@ static int ipath_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
unsigned long long addr;
u32 bar0 = 0, bar1 = 0;
+#ifdef CONFIG_X86_64
+ if (WARN(pat_enabled,
+ "ipath needs PAT disabled, boot with nopat kernel parameter\n")) {
+ ret = EINVAL;
+ goto bail;
+ }
+#endif
+
dd = ipath_alloc_devdata(pdev);
if (IS_ERR(dd)) {
ret = PTR_ERR(dd);
@@ -542,6 +553,7 @@ static int ipath_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
dd->ipath_kregbase = __ioremap(addr, len,
(_PAGE_NO_CACHE|_PAGE_WRITETHRU));
#else
+ /* XXX: split this properly to enable on PAT */
dd->ipath_kregbase = ioremap_nocache(addr, len);
#endif
@@ -587,12 +599,8 @@ static int ipath_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
ret = ipath_enable_wc(dd);
- if (ret) {
- ipath_dev_err(dd, "Write combining not enabled "
- "(err %d): performance may be poor\n",
- -ret);
+ if (ret)
ret = 0;
- }
ipath_verify_pioperf(dd);
diff --git a/drivers/infiniband/hw/ipath/ipath_kernel.h b/drivers/infiniband/hw/ipath/ipath_kernel.h
index e08db70..f0f9471 100644
--- a/drivers/infiniband/hw/ipath/ipath_kernel.h
+++ b/drivers/infiniband/hw/ipath/ipath_kernel.h
@@ -463,9 +463,7 @@ struct ipath_devdata {
/* offset in HT config space of slave/primary interface block */
u8 ipath_ht_slave_off;
/* for write combining settings */
- unsigned long ipath_wc_cookie;
- unsigned long ipath_wc_base;
- unsigned long ipath_wc_len;
+ int wc_cookie;
/* ref count for each pkey */
atomic_t ipath_pkeyrefs[4];
/* shadow copy of struct page *'s for exp tid pages */
diff --git a/drivers/infiniband/hw/ipath/ipath_wc_x86_64.c b/drivers/infiniband/hw/ipath/ipath_wc_x86_64.c
index 70c1f3a..7b6e4c8 100644
--- a/drivers/infiniband/hw/ipath/ipath_wc_x86_64.c
+++ b/drivers/infiniband/hw/ipath/ipath_wc_x86_64.c
@@ -37,7 +37,6 @@
*/
#include <linux/pci.h>
-#include <asm/mtrr.h>
#include <asm/processor.h>
#include "ipath_kernel.h"
@@ -122,27 +121,14 @@ int ipath_enable_wc(struct ipath_devdata *dd)
}
if (!ret) {
- int cookie;
- ipath_cdbg(VERBOSE, "Setting mtrr for chip to WC "
- "(addr %llx, len=0x%llx)\n",
- (unsigned long long) pioaddr,
- (unsigned long long) piolen);
- cookie = mtrr_add(pioaddr, piolen, MTRR_TYPE_WRCOMB, 1);
- if (cookie < 0) {
- {
- dev_info(&dd->pcidev->dev,
- "mtrr_add() WC for PIO bufs "
- "failed (%d)\n",
- cookie);
- ret = -EINVAL;
- }
- } else {
- ipath_cdbg(VERBOSE, "Set mtrr for chip to WC, "
- "cookie is %d\n", cookie);
- dd->ipath_wc_cookie = cookie;
- dd->ipath_wc_base = (unsigned long) pioaddr;
- dd->ipath_wc_len = (unsigned long) piolen;
- }
+ dd->wc_cookie = arch_phys_wc_add(pioaddr, piolen);
+ if (dd->wc_cookie < 0) {
+ ipath_dev_err(dd, "Seting mtrr failed on PIO buffers\n");
+ ret = -ENODEV;
+ } else if (dd->wc_cookie = 0)
+ ipath_cdbg(VERBOSE, "Set mtrr for chip to WC not needed\n");
+ else
+ ipath_cdbg(VERBOSE, "Set mtrr for chip to WC\n");
}
return ret;
@@ -154,16 +140,5 @@ int ipath_enable_wc(struct ipath_devdata *dd)
*/
void ipath_disable_wc(struct ipath_devdata *dd)
{
- if (dd->ipath_wc_cookie) {
- int r;
- ipath_cdbg(VERBOSE, "undoing WCCOMB on pio buffers\n");
- r = mtrr_del(dd->ipath_wc_cookie, dd->ipath_wc_base,
- dd->ipath_wc_len);
- if (r < 0)
- dev_info(&dd->pcidev->dev,
- "mtrr_del(%lx, %lx, %lx) failed: %d\n",
- dd->ipath_wc_cookie, dd->ipath_wc_base,
- dd->ipath_wc_len, r);
- dd->ipath_wc_cookie = 0; /* even on failure */
- }
+ arch_phys_wc_del(dd->wc_cookie);
}
diff --git a/drivers/media/pci/ivtv/ivtvfb.c b/drivers/media/pci/ivtv/ivtvfb.c
index 9ff1230..369598c 100644
--- a/drivers/media/pci/ivtv/ivtvfb.c
+++ b/drivers/media/pci/ivtv/ivtvfb.c
@@ -44,8 +44,8 @@
#include <linux/ivtvfb.h>
#include <linux/slab.h>
-#ifdef CONFIG_MTRR
-#include <asm/mtrr.h>
+#ifdef CONFIG_X86_64
+#include <asm/pat.h>
#endif
#include "ivtv-driver.h"
@@ -155,12 +155,11 @@ struct osd_info {
/* Buffer size */
u32 video_buffer_size;
-#ifdef CONFIG_MTRR
/* video_base rounded down as required by hardware MTRRs */
unsigned long fb_start_aligned_physaddr;
/* video_base rounded up as required by hardware MTRRs */
unsigned long fb_end_aligned_physaddr;
-#endif
+ int wc_cookie;
/* Store the buffer offset */
int set_osd_coords_x;
@@ -1099,6 +1098,8 @@ static int ivtvfb_init_vidmode(struct ivtv *itv)
static int ivtvfb_init_io(struct ivtv *itv)
{
struct osd_info *oi = itv->osd_info;
+ /* Find the largest power of two that maps the whole buffer */
+ int size_shift = 31;
mutex_lock(&itv->serialize_lock);
if (ivtv_init_on_first_open(itv)) {
@@ -1120,6 +1121,7 @@ static int ivtvfb_init_io(struct ivtv *itv)
oi->video_buffer_size = 1704960;
oi->video_pbase = itv->base_addr + IVTV_DECODER_OFFSET + oi->video_rbase;
+ /* XXX: split this for PAT */
oi->video_vbase = itv->dec_mem + oi->video_rbase;
if (!oi->video_vbase) {
@@ -1132,29 +1134,16 @@ static int ivtvfb_init_io(struct ivtv *itv)
oi->video_pbase, oi->video_vbase,
oi->video_buffer_size / 1024);
-#ifdef CONFIG_MTRR
- {
- /* Find the largest power of two that maps the whole buffer */
- int size_shift = 31;
-
- while (!(oi->video_buffer_size & (1 << size_shift))) {
- size_shift--;
- }
- size_shift++;
- oi->fb_start_aligned_physaddr = oi->video_pbase & ~((1 << size_shift) - 1);
- oi->fb_end_aligned_physaddr = oi->video_pbase + oi->video_buffer_size;
- oi->fb_end_aligned_physaddr += (1 << size_shift) - 1;
- oi->fb_end_aligned_physaddr &= ~((1 << size_shift) - 1);
- if (mtrr_add(oi->fb_start_aligned_physaddr,
- oi->fb_end_aligned_physaddr - oi->fb_start_aligned_physaddr,
- MTRR_TYPE_WRCOMB, 1) < 0) {
- IVTVFB_INFO("disabled mttr\n");
- oi->fb_start_aligned_physaddr = 0;
- oi->fb_end_aligned_physaddr = 0;
- }
- }
-#endif
-
+ while (!(oi->video_buffer_size & (1 << size_shift)))
+ size_shift--;
+ size_shift++;
+ oi->fb_start_aligned_physaddr = oi->video_pbase & ~((1 << size_shift) - 1);
+ oi->fb_end_aligned_physaddr = oi->video_pbase + oi->video_buffer_size;
+ oi->fb_end_aligned_physaddr += (1 << size_shift) - 1;
+ oi->fb_end_aligned_physaddr &= ~((1 << size_shift) - 1);
+ oi->wc_cookie = arch_phys_wc_add(oi->fb_start_aligned_physaddr,
+ oi->fb_end_aligned_physaddr -
+ oi->fb_start_aligned_physaddr);
/* Blank the entire osd. */
memset_io(oi->video_vbase, 0, oi->video_buffer_size);
@@ -1172,14 +1161,8 @@ static void ivtvfb_release_buffers (struct ivtv *itv)
/* Release pseudo palette */
kfree(oi->ivtvfb_info.pseudo_palette);
-
-#ifdef CONFIG_MTRR
- if (oi->fb_end_aligned_physaddr) {
- mtrr_del(-1, oi->fb_start_aligned_physaddr,
- oi->fb_end_aligned_physaddr - oi->fb_start_aligned_physaddr);
- }
-#endif
-
+ iounmap(oi->video_vbase);
+ arch_phys_wc_del(oi->wc_cookie);
kfree(oi);
itv->osd_info = NULL;
}
@@ -1190,6 +1173,13 @@ static int ivtvfb_init_card(struct ivtv *itv)
{
int rc;
+#ifdef CONFIG_X86_64
+ if (WARN(pat_enabled,
+ "ivtv needs PAT disabled, boot with nopat kernel parameter\n")) {
+ return EINVAL;
+ }
+#endif
+
if (itv->osd_info) {
IVTVFB_ERR("Card %d already initialised\n", ivtvfb_card_id);
return -EBUSY;
^ permalink raw reply related
* Re: ioremap_uc() followed by set_memory_wc() - burrying MTRR
From: Andy Lutomirski @ 2015-04-22 16:53 UTC (permalink / raw)
To: Luis R. Rodriguez
Cc: Jason Gunthorpe, Mike Marciniszyn, Mike Marciniszyn, linux-rdma,
Andy Walls, Toshi Kani, H. Peter Anvin, Ingo Molnar,
linux-kernel@vger.kernel.org, Hal Rosenstock, Sean Hefty,
Suresh Siddha, Rickard Strandqvist, Roland Dreier, Juergen Gross,
Mauro Carvalho Chehab, Borislav Petkov, Mel Gorman,
Vlastimil Babka, Davidlohr Bueso, Dave Hansen,
Jean-Christophe Plagniol-Villard, Thomas Gleixner, Ville Syrj?l?,
Linux Fbdev development list, linux-media, X86 ML,
Luis R. Rodriguez
In-Reply-To: <20150422152328.GB5622@wotan.suse.de>
On Wed, Apr 22, 2015 at 8:23 AM, Luis R. Rodriguez <mcgrof@suse.com> wrote:
> On Tue, Apr 21, 2015 at 11:39:39PM -0600, Jason Gunthorpe wrote:
>> On Wed, Apr 22, 2015 at 01:39:07AM +0200, Luis R. Rodriguez wrote:
>> > > Mike, do you think the time is right to just remove the iPath driver?
>> >
>> > With PAT now being default the driver effectively won't work
>> > with write-combining on modern kernels. Even if systems are old
>> > they likely had PAT support, when upgrading kernels PAT will work
>> > but write-combing won't on ipath.
>>
>> Sorry, do you mean the driver already doesn't get WC? Or do you mean
>> after some more pending patches are applied?
>
> No, you have to consider the system used and the effects of calls used
> on the driver in light of this table:
>
> ----------------------------------------------------------------------
> MTRR Non-PAT PAT Linux ioremap value Effective memory type
> ----------------------------------------------------------------------
> Non-PAT | PAT
> PAT
> |PCD
> ||PWT
> |||
> WC 000 WB _PAGE_CACHE_MODE_WB WC | WC
> WC 001 WC _PAGE_CACHE_MODE_WC WC* | WC
> WC 010 UC- _PAGE_CACHE_MODE_UC_MINUS WC* | UC
> WC 011 UC _PAGE_CACHE_MODE_UC UC | UC
> ----------------------------------------------------------------------
>
> (*) denotes implementation defined and is discouraged
>
> ioremap_nocache() will use _PAGE_CACHE_MODE_UC_MINUS by default today,
> in the future we want to flip the switch and make _PAGE_CACHE_MODE_UC
> the default. When that flip occurs it will mean ipath cannot get
> write-combining on both non-PAT and PAT systems. Now that is for
> the future, lets review the current situation for ipath.
>
> For PAT capable systems if mtrr_add() is used today on a Linux system on a
> region mapped with ioremap_nocache() that will mean you effectively nullify the
> mtrr_add() effect as the combinatorial effect above yields an effective memory
> type of UC.
Are you sure? I thought that ioremap_nocache currently is UC-, so
mtrr_add + ioremap_nocache gets WC even on PAT systems.
Going forward, when mtrr_add is gone, this will change, of course.
--Andy
^ permalink raw reply
* Re: [PATCH v4 2/2] IB/qib: use arch_phys_wc_add()
From: Doug Ledford @ 2015-04-22 16:57 UTC (permalink / raw)
To: Luis R. Rodriguez
Cc: Luis R. Rodriguez, infinipath-ral2JQCrhuEAvxtiuMwx3w,
roland-DgEjT+Ai2ygdnm+yROfE0A, sean.hefty-ral2JQCrhuEAvxtiuMwx3w,
hal.rosenstock-Re5JQEeQqe8AvxtiuMwx3w,
linux-rdma-u79uwXL29TY76Z2rM5mHXA, luto-kltTT9wpgjJwATOyAt5JVQ,
mst-H+wXaHxf7aLQT0dZR+AlfA, linux-kernel-u79uwXL29TY76Z2rM5mHXA,
cocci-/FJkirnvOdkvYVN+rsErww, Toshi Kani, Rickard Strandqvist,
Mike Marciniszyn, Roland Dreier, Dennis Dalessandro,
Suresh Siddha, Ingo Molnar, Thomas Gleixner, Juergen Gross,
Daniel Vetter, Dave Airlie, Bjorn Helgaas, Antonino Daplas,
Jean-Christophe Plagniol-Villard, Tomi Valkeinen
In-Reply-To: <20150422153348.GD5622-B4tOwbsTzaBolqkO4TVVkw@public.gmane.org>
[-- Attachment #1: Type: text/plain, Size: 1862 bytes --]
On Wed, 2015-04-22 at 17:33 +0200, Luis R. Rodriguez wrote:
> On Wed, Apr 22, 2015 at 09:54:38AM -0400, Doug Ledford wrote:
> > On Tue, 2015-04-21 at 14:50 -0700, Luis R. Rodriguez wrote:
> >
> > This:
> > > + /* MTRR was used if this is non-zero */
> > > + if (!dd->wc_cookie)
> > > vma->vm_page_prot = pgprot_writecombine(vma->vm_page_prot);
> >
> > And this:
> > > + dd->wc_cookie = arch_phys_wc_add(pioaddr, piolen);
> > > + if (dd->wc_cookie < 0)
> > > + ret = -EINVAL;
> >
> > don't agree on what wc_cookie will be on error.
>
> Can you elaborate? The one below is the one that starts things,
> and arch_phys_wc_add() will return 0 on PAT systems. For non-PAT
> systems it will return a number > 0 *iff* a valid MTRR was added.
> It will return negative onloy on error then.
>
> The change above is meant to replace a check put in place to see
> if PAT was enabled. The way we replace this is to ensure that
> arch_phys_wc_add() returned 0.
>
> If you disagree it'd be great if you can elaborate why.
Maybe I'm missing something, but in qib_enable_wc() you store the return
from arch_phys_wc_add into wc_cookie. That return is negative, so you
return from qib_enable_wc() to qib_init_one(), they see the ret value,
they print out a warning about bad performance, then they clear the
return value and continue with device initialization.
In all of this though, wc_cookie is never cleared and so it still has
the error condition in it. Then, much later at run time, you call
mmap_piobufs() and you check the contents of wc_cookie, and if it's
non-0 (which is still will be), you do the wrong thing, right? And what
about at shutdown when you call qib_disable_wc() and your cookie still
has an error code in it as well?
--
Doug Ledford <dledford@redhat.com>
GPG KeyID: 0E572FDD
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply
* Re: ioremap_uc() followed by set_memory_wc() - burrying MTRR
From: Luis R. Rodriguez @ 2015-04-22 17:07 UTC (permalink / raw)
To: Andy Lutomirski
Cc: Jason Gunthorpe, Mike Marciniszyn, Mike Marciniszyn, linux-rdma,
Andy Walls, Toshi Kani, H. Peter Anvin, Ingo Molnar,
linux-kernel@vger.kernel.org, Hal Rosenstock, Sean Hefty,
Suresh Siddha, Rickard Strandqvist, Roland Dreier, Juergen Gross,
Mauro Carvalho Chehab, Borislav Petkov, Mel Gorman,
Vlastimil Babka, Davidlohr Bueso, Dave Hansen,
Jean-Christophe Plagniol-Villard, Thomas Gleixner, Ville Syrj?l?,
Linux Fbdev development list, linux-media, X86 ML,
Luis R. Rodriguez
In-Reply-To: <CALCETrWYRazYgovguNEodVZUwO3sCmzvg9-q73nTfJ2ahNrBxw@mail.gmail.com>
On Wed, Apr 22, 2015 at 09:53:03AM -0700, Andy Lutomirski wrote:
> On Wed, Apr 22, 2015 at 8:23 AM, Luis R. Rodriguez <mcgrof@suse.com> wrote:
> > On Tue, Apr 21, 2015 at 11:39:39PM -0600, Jason Gunthorpe wrote:
> >> On Wed, Apr 22, 2015 at 01:39:07AM +0200, Luis R. Rodriguez wrote:
> >> > > Mike, do you think the time is right to just remove the iPath driver?
> >> >
> >> > With PAT now being default the driver effectively won't work
> >> > with write-combining on modern kernels. Even if systems are old
> >> > they likely had PAT support, when upgrading kernels PAT will work
> >> > but write-combing won't on ipath.
> >>
> >> Sorry, do you mean the driver already doesn't get WC? Or do you mean
> >> after some more pending patches are applied?
> >
> > No, you have to consider the system used and the effects of calls used
> > on the driver in light of this table:
> >
> > ----------------------------------------------------------------------
> > MTRR Non-PAT PAT Linux ioremap value Effective memory type
> > ----------------------------------------------------------------------
> > Non-PAT | PAT
> > PAT
> > |PCD
> > ||PWT
> > |||
> > WC 000 WB _PAGE_CACHE_MODE_WB WC | WC
> > WC 001 WC _PAGE_CACHE_MODE_WC WC* | WC
> > WC 010 UC- _PAGE_CACHE_MODE_UC_MINUS WC* | UC
> > WC 011 UC _PAGE_CACHE_MODE_UC UC | UC
> > ----------------------------------------------------------------------
> >
> > (*) denotes implementation defined and is discouraged
> >
> > ioremap_nocache() will use _PAGE_CACHE_MODE_UC_MINUS by default today,
> > in the future we want to flip the switch and make _PAGE_CACHE_MODE_UC
> > the default. When that flip occurs it will mean ipath cannot get
> > write-combining on both non-PAT and PAT systems. Now that is for
> > the future, lets review the current situation for ipath.
> >
> > For PAT capable systems if mtrr_add() is used today on a Linux system on a
> > region mapped with ioremap_nocache() that will mean you effectively nullify the
> > mtrr_add() effect as the combinatorial effect above yields an effective memory
> > type of UC.
>
> Are you sure?
Well lets double check.
> I thought that ioremap_nocache currently is UC-,
It is.
> so mtrr_add + ioremap_nocache gets WC even on PAT systems.
https://www-ssl.intel.com/content/dam/www/public/us/en/documents/manuals/64-ia-32-architectures-software-developer-manual-325462.pdf
As per Intel SDM "11.5.2.2 Selecting Memory Types for Pentium
III and More Recent Processor Families" the ffect of a WC MTRR
for a region with a PAT entry value of UC will be UC. The effect
of a WC MTRR on a region with a PAT entry UC- will be WC. The
effect of a WC MTRR on a regoin with PAT entry WC is WC.
And indeed as per table 11-7 mtrr WC on PAT UC- yields WC. So ineed the above
table needs adjustment for this. So for PAT systems write-combing would be
effective with mtrr_add(), but once strong UC (_PAGE_CACHE_MODE_UC) is used by
default for ioremap_nocache() what I mentioned will be true. Furhtermore if we
switch the drivers to use arch_phys_wc_add() then for sure write-combining will
also not be effective.
Jason, Andy, is the change still a reasonable compromise? We'd just be asking
users to boot with noat for users for ipath, ivtv until the drivers gets proper
PAT support with a split.
There are two motivations for this:
* help move to strong UC as default
* bury MTRR
> Going forward, when mtrr_add is gone, this will change, of course.
Indeed.
Luis
^ permalink raw reply
* Re: [PATCH v3] mtrr: avoid ifdef'ery with phys_wc_to_mtrr_index()
From: Luis R. Rodriguez @ 2015-04-22 17:09 UTC (permalink / raw)
To: Luis R. Rodriguez
Cc: mingo, tglx, hpa, plagnioj, tomi.valkeinen, linux-fbdev, luto,
mst, cocci, Toshi Kani, Ingo Molnar, Will Deacon, Thierry Reding,
Andrew Morton, Dave Hansen, Greg Kroah-Hartman, Catalin Marinas,
Abhilash Kesavan, Matthias Brugger, Cristian Stoica, dri-devel,
Suresh Siddha, Linus Torvalds, Juergen Gross, Daniel Vetter,
Dave Airlie, Antonino Daplas <ad>
In-Reply-To: <1429648971-18007-1-git-send-email-mcgrof@do-not-panic.com>
On Tue, Apr 21, 2015 at 01:42:51PM -0700, Luis R. Rodriguez wrote:
> From: "Luis R. Rodriguez" <mcgrof@suse.com>
>
> There is only one user but since we're going to bury
> MTRR next out of access to drivers expose this last
> piece of API to drivers in a general fashion only
> needing io.h for access to helpers.
>
> Cc: Toshi Kani <toshi.kani@hp.com>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: Ingo Molnar <mingo@redhat.com>
> Cc: "H. Peter Anvin" <hpa@zytor.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Cc: Thierry Reding <treding@nvidia.com>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Dave Hansen <dave.hansen@linux.intel.com>
> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Abhilash Kesavan <a.kesavan@samsung.com>
> Cc: Matthias Brugger <matthias.bgg@gmail.com>
> Cc: Cristian Stoica <cristian.stoica@freescale.com>
> Cc: dri-devel@lists.freedesktop.org
> Cc: Suresh Siddha <sbsiddha@gmail.com>
> Cc: Linus Torvalds <torvalds@linux-foundation.org>
> Cc: Juergen Gross <jgross@suse.com>
> Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
> Cc: Andy Lutomirski <luto@amacapital.net>
> Cc: Dave Airlie <airlied@redhat.com>
> Cc: Antonino Daplas <adaplas@gmail.com>
> Cc: Jean-Christophe Plagniol-Villard <plagnioj@jcrosoft.com>
> Cc: Tomi Valkeinen <tomi.valkeinen@ti.com>
> Cc: Ville Syrjälä <syrjala@sci.fi>
> Cc: Mel Gorman <mgorman@suse.de>
> Cc: Vlastimil Babka <vbabka@suse.cz>
> Cc: Borislav Petkov <bp@suse.de>
> Cc: Davidlohr Bueso <dbueso@suse.de>
> Cc: x86@kernel.org
> Cc: linux-fbdev@vger.kernel.org
> Cc: linux-kernel@vger.kernel.org
> Signed-off-by: Luis R. Rodriguez <mcgrof@suse.com>
> ---
> arch/x86/include/asm/io.h | 3 +++
> arch/x86/include/asm/mtrr.h | 5 -----
> arch/x86/kernel/cpu/mtrr/main.c | 6 +++---
> drivers/gpu/drm/drm_ioctl.c | 14 +-------------
> include/linux/io.h | 6 ++++++
> 5 files changed, 13 insertions(+), 21 deletions(-)
>
> diff --git a/arch/x86/include/asm/io.h b/arch/x86/include/asm/io.h
> index 4afc05f..a2b9740 100644
> --- a/arch/x86/include/asm/io.h
> +++ b/arch/x86/include/asm/io.h
> @@ -339,6 +339,9 @@ extern bool xen_biovec_phys_mergeable(const struct bio_vec *vec1,
> #define IO_SPACE_LIMIT 0xffff
>
> #ifdef CONFIG_MTRR
> +extern int __must_check arch_phys_wc_index(int handle);
> +#define arch_phys_wc_index arch_phys_wc_index
> +
> extern int __must_check arch_phys_wc_add(unsigned long base,
> unsigned long size);
> extern void arch_phys_wc_del(int handle);
> diff --git a/arch/x86/include/asm/mtrr.h b/arch/x86/include/asm/mtrr.h
> index da8dff1..27e3dc0 100644
> --- a/arch/x86/include/asm/mtrr.h
> +++ b/arch/x86/include/asm/mtrr.h
> @@ -48,7 +48,6 @@ extern void mtrr_aps_init(void);
> extern void mtrr_bp_restore(void);
> extern int mtrr_trim_uncached_memory(unsigned long end_pfn);
> extern int amd_special_default_mtrr(void);
> -extern int phys_wc_to_mtrr_index(int handle);
> # else
> static inline u8 mtrr_type_lookup(u64 addr, u64 end, u8 *uniform)
> {
> @@ -85,10 +84,6 @@ static inline int mtrr_trim_uncached_memory(unsigned long end_pfn)
> static inline void mtrr_centaur_report_mcr(int mcr, u32 lo, u32 hi)
> {
> }
> -static inline int phys_wc_to_mtrr_index(int handle)
> -{
> - return -1;
> -}
>
> #define mtrr_ap_init() do {} while (0)
> #define mtrr_bp_init() do {} while (0)
> diff --git a/arch/x86/kernel/cpu/mtrr/main.c b/arch/x86/kernel/cpu/mtrr/main.c
> index 12abdbe..d8c106c 100644
> --- a/arch/x86/kernel/cpu/mtrr/main.c
> +++ b/arch/x86/kernel/cpu/mtrr/main.c
> @@ -580,7 +580,7 @@ void arch_phys_wc_del(int handle)
> EXPORT_SYMBOL(arch_phys_wc_del);
>
> /*
> - * phys_wc_to_mtrr_index - translates arch_phys_wc_add's return value
> + * arch_phys_wc_index - translates arch_phys_wc_add's return value
> * @handle: Return value from arch_phys_wc_add
> *
> * This will turn the return value from arch_phys_wc_add into an mtrr
> @@ -590,14 +590,14 @@ EXPORT_SYMBOL(arch_phys_wc_del);
> * in printk line. Alas there is an illegitimate use in some ancient
> * drm ioctls.
> */
> -int phys_wc_to_mtrr_index(int handle)
> +int arch_phys_wc_index(int handle)
> {
> if (handle < MTRR_TO_PHYS_WC_OFFSET)
> return -1;
> else
> return handle - MTRR_TO_PHYS_WC_OFFSET;
> }
> -EXPORT_SYMBOL_GPL(phys_wc_to_mtrr_index);
> +EXPORT_SYMBOL_GPL(arch_phys_wc_index);
>
> /*
> * HACK ALERT!
> diff --git a/drivers/gpu/drm/drm_ioctl.c b/drivers/gpu/drm/drm_ioctl.c
> index 266dcd6..0a95782 100644
> --- a/drivers/gpu/drm/drm_ioctl.c
> +++ b/drivers/gpu/drm/drm_ioctl.c
> @@ -36,9 +36,6 @@
>
> #include <linux/pci.h>
> #include <linux/export.h>
> -#ifdef CONFIG_X86
> -#include <asm/mtrr.h>
> -#endif
>
> static int drm_version(struct drm_device *dev, void *data,
> struct drm_file *file_priv);
> @@ -197,16 +194,7 @@ static int drm_getmap(struct drm_device *dev, void *data,
> map->type = r_list->map->type;
> map->flags = r_list->map->flags;
> map->handle = (void *)(unsigned long) r_list->user_token;
> -
> -#ifdef CONFIG_X86
> - /*
> - * There appears to be exactly one user of the mtrr index: dritest.
> - * It's easy enough to keep it working on non-PAT systems.
> - */
> - map->mtrr = phys_wc_to_mtrr_index(r_list->map->mtrr);
> -#else
> - map->mtrr = -1;
> -#endif
> + map->mtrr = arch_phys_wc_index(r_list->map->mtrr);
>
> mutex_unlock(&dev->struct_mutex);
>
> diff --git a/include/linux/io.h b/include/linux/io.h
> index 986f2bf..6e29acf 100644
> --- a/include/linux/io.h
> +++ b/include/linux/io.h
> @@ -111,6 +111,12 @@ static inline void arch_phys_wc_del(int handle)
> }
>
> #define arch_phys_wc_add arch_phys_wc_add
> +#ifndef arch_phys_wc_index
> +static inline int arch_phys_wc_index(int handle)
> +{
> + return -1;
> +}
> +#define arch_phys_wc_index arch_phys_wc_index
> #endif
Sorry this is missing an #endif here, will send a v4.
Luis
^ permalink raw reply
* [PATCH v4] mtrr: avoid ifdef'ery with phys_wc_to_mtrr_index()
From: Luis R. Rodriguez @ 2015-04-22 17:12 UTC (permalink / raw)
To: mingo, tglx, hpa, plagnioj, tomi.valkeinen
Cc: linux-fbdev, luto, mst, linux-kernel, Luis R. Rodriguez,
Toshi Kani, Ingo Molnar, Will Deacon, Thierry Reding,
Andrew Morton, Dave Hansen, Greg Kroah-Hartman, Catalin Marinas,
Abhilash Kesavan, Matthias Brugger, Cristian Stoica, dri-devel,
Suresh Siddha, Linus Torvalds, Juergen Gross, Daniel Vetter,
Dave Airlie, Antonino Daplas
From: "Luis R. Rodriguez" <mcgrof@suse.com>
There is only one user but since we're going to bury
MTRR next out of access to drivers expose this last
piece of API to drivers in a general fashion only
needing io.h for access to helpers.
Cc: Toshi Kani <toshi.kani@hp.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Thierry Reding <treding@nvidia.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Abhilash Kesavan <a.kesavan@samsung.com>
Cc: Matthias Brugger <matthias.bgg@gmail.com>
Cc: Cristian Stoica <cristian.stoica@freescale.com>
Cc: dri-devel@lists.freedesktop.org
Cc: Suresh Siddha <sbsiddha@gmail.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Juergen Gross <jgross@suse.com>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Dave Airlie <airlied@redhat.com>
Cc: Antonino Daplas <adaplas@gmail.com>
Cc: Jean-Christophe Plagniol-Villard <plagnioj@jcrosoft.com>
Cc: Tomi Valkeinen <tomi.valkeinen@ti.com>
Cc: Ville Syrjälä <syrjala@sci.fi>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Borislav Petkov <bp@suse.de>
Cc: Davidlohr Bueso <dbueso@suse.de>
Cc: x86@kernel.org
Cc: linux-fbdev@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Luis R. Rodriguez <mcgrof@suse.com>
---
This v4 adds a missing #endif.
arch/x86/include/asm/io.h | 3 +++
arch/x86/include/asm/mtrr.h | 5 -----
arch/x86/kernel/cpu/mtrr/main.c | 6 +++---
drivers/gpu/drm/drm_ioctl.c | 14 +-------------
include/linux/io.h | 7 +++++++
5 files changed, 14 insertions(+), 21 deletions(-)
diff --git a/arch/x86/include/asm/io.h b/arch/x86/include/asm/io.h
index 4afc05f..a2b9740 100644
--- a/arch/x86/include/asm/io.h
+++ b/arch/x86/include/asm/io.h
@@ -339,6 +339,9 @@ extern bool xen_biovec_phys_mergeable(const struct bio_vec *vec1,
#define IO_SPACE_LIMIT 0xffff
#ifdef CONFIG_MTRR
+extern int __must_check arch_phys_wc_index(int handle);
+#define arch_phys_wc_index arch_phys_wc_index
+
extern int __must_check arch_phys_wc_add(unsigned long base,
unsigned long size);
extern void arch_phys_wc_del(int handle);
diff --git a/arch/x86/include/asm/mtrr.h b/arch/x86/include/asm/mtrr.h
index da8dff1..27e3dc0 100644
--- a/arch/x86/include/asm/mtrr.h
+++ b/arch/x86/include/asm/mtrr.h
@@ -48,7 +48,6 @@ extern void mtrr_aps_init(void);
extern void mtrr_bp_restore(void);
extern int mtrr_trim_uncached_memory(unsigned long end_pfn);
extern int amd_special_default_mtrr(void);
-extern int phys_wc_to_mtrr_index(int handle);
# else
static inline u8 mtrr_type_lookup(u64 addr, u64 end, u8 *uniform)
{
@@ -85,10 +84,6 @@ static inline int mtrr_trim_uncached_memory(unsigned long end_pfn)
static inline void mtrr_centaur_report_mcr(int mcr, u32 lo, u32 hi)
{
}
-static inline int phys_wc_to_mtrr_index(int handle)
-{
- return -1;
-}
#define mtrr_ap_init() do {} while (0)
#define mtrr_bp_init() do {} while (0)
diff --git a/arch/x86/kernel/cpu/mtrr/main.c b/arch/x86/kernel/cpu/mtrr/main.c
index 12abdbe..d8c106c 100644
--- a/arch/x86/kernel/cpu/mtrr/main.c
+++ b/arch/x86/kernel/cpu/mtrr/main.c
@@ -580,7 +580,7 @@ void arch_phys_wc_del(int handle)
EXPORT_SYMBOL(arch_phys_wc_del);
/*
- * phys_wc_to_mtrr_index - translates arch_phys_wc_add's return value
+ * arch_phys_wc_index - translates arch_phys_wc_add's return value
* @handle: Return value from arch_phys_wc_add
*
* This will turn the return value from arch_phys_wc_add into an mtrr
@@ -590,14 +590,14 @@ EXPORT_SYMBOL(arch_phys_wc_del);
* in printk line. Alas there is an illegitimate use in some ancient
* drm ioctls.
*/
-int phys_wc_to_mtrr_index(int handle)
+int arch_phys_wc_index(int handle)
{
if (handle < MTRR_TO_PHYS_WC_OFFSET)
return -1;
else
return handle - MTRR_TO_PHYS_WC_OFFSET;
}
-EXPORT_SYMBOL_GPL(phys_wc_to_mtrr_index);
+EXPORT_SYMBOL_GPL(arch_phys_wc_index);
/*
* HACK ALERT!
diff --git a/drivers/gpu/drm/drm_ioctl.c b/drivers/gpu/drm/drm_ioctl.c
index 266dcd6..0a95782 100644
--- a/drivers/gpu/drm/drm_ioctl.c
+++ b/drivers/gpu/drm/drm_ioctl.c
@@ -36,9 +36,6 @@
#include <linux/pci.h>
#include <linux/export.h>
-#ifdef CONFIG_X86
-#include <asm/mtrr.h>
-#endif
static int drm_version(struct drm_device *dev, void *data,
struct drm_file *file_priv);
@@ -197,16 +194,7 @@ static int drm_getmap(struct drm_device *dev, void *data,
map->type = r_list->map->type;
map->flags = r_list->map->flags;
map->handle = (void *)(unsigned long) r_list->user_token;
-
-#ifdef CONFIG_X86
- /*
- * There appears to be exactly one user of the mtrr index: dritest.
- * It's easy enough to keep it working on non-PAT systems.
- */
- map->mtrr = phys_wc_to_mtrr_index(r_list->map->mtrr);
-#else
- map->mtrr = -1;
-#endif
+ map->mtrr = arch_phys_wc_index(r_list->map->mtrr);
mutex_unlock(&dev->struct_mutex);
diff --git a/include/linux/io.h b/include/linux/io.h
index 986f2bf..04cce4d 100644
--- a/include/linux/io.h
+++ b/include/linux/io.h
@@ -111,6 +111,13 @@ static inline void arch_phys_wc_del(int handle)
}
#define arch_phys_wc_add arch_phys_wc_add
+#ifndef arch_phys_wc_index
+static inline int arch_phys_wc_index(int handle)
+{
+ return -1;
+}
+#define arch_phys_wc_index arch_phys_wc_index
+#endif
#endif
#endif /* _LINUX_IO_H */
--
2.3.2.209.gd67f9d5.dirty
^ permalink raw reply related
* Re: [PATCH v4 2/2] IB/qib: use arch_phys_wc_add()
From: Luis R. Rodriguez @ 2015-04-22 17:37 UTC (permalink / raw)
To: Doug Ledford
Cc: Luis R. Rodriguez, infinipath, roland, sean.hefty, hal.rosenstock,
linux-rdma, luto, mst, linux-kernel, cocci, Toshi Kani,
Rickard Strandqvist, Mike Marciniszyn, Roland Dreier,
Dennis Dalessandro, Suresh Siddha, Ingo Molnar, Thomas Gleixner,
Juergen Gross, Daniel Vetter, Dave Airlie, Bjorn Helgaas,
Antonino Daplas, Jean-Christophe Plagniol-Villard, Tomi Valkeinen
In-Reply-To: <1429721838.45956.142.camel@redhat.com>
On Wed, Apr 22, 2015 at 12:57:18PM -0400, Doug Ledford wrote:
> On Wed, 2015-04-22 at 17:33 +0200, Luis R. Rodriguez wrote:
> > On Wed, Apr 22, 2015 at 09:54:38AM -0400, Doug Ledford wrote:
> > > On Tue, 2015-04-21 at 14:50 -0700, Luis R. Rodriguez wrote:
> > >
> > > This:
> > > > + /* MTRR was used if this is non-zero */
> > > > + if (!dd->wc_cookie)
> > > > vma->vm_page_prot = pgprot_writecombine(vma->vm_page_prot);
> > >
> > > And this:
> > > > + dd->wc_cookie = arch_phys_wc_add(pioaddr, piolen);
> > > > + if (dd->wc_cookie < 0)
> > > > + ret = -EINVAL;
> > >
> > > don't agree on what wc_cookie will be on error.
> >
> > Can you elaborate? The one below is the one that starts things,
> > and arch_phys_wc_add() will return 0 on PAT systems. For non-PAT
> > systems it will return a number > 0 *iff* a valid MTRR was added.
> > It will return negative onloy on error then.
> >
> > The change above is meant to replace a check put in place to see
> > if PAT was enabled. The way we replace this is to ensure that
> > arch_phys_wc_add() returned 0.
> >
> > If you disagree it'd be great if you can elaborate why.
>
> Maybe I'm missing something, but in qib_enable_wc() you store the return
> from arch_phys_wc_add into wc_cookie. That return is negative,
If and only if the system was non-PAT and mtrr_add() failed.
> so you
> return from qib_enable_wc() to qib_init_one(), they see the ret value,
> they print out a warning about bad performance, then they clear the
> return value and continue with device initialization.
>
> In all of this though, wc_cookie is never cleared and so it still has
> the error condition in it. Then, much later at run time, you call
> mmap_piobufs() and you check the contents of wc_cookie, and if it's
> non-0 (which is still will be), you do the wrong thing, right?
Originally the code had it to run pgprot_writecombine() if PAT was going to be
used. After the code changes we check for !cookie which will be true when
cookie is 0 only. In case the cookie was an error, that is if mtrr_add()
failed, then this code would not run because (!negative) is false. The goal was
to trigger a run if the cookie was 0, which can only happen if PAT was enabled.
Please let me know, I'd like to get this right too.
> And what
> about at shutdown when you call qib_disable_wc() and your cookie still
> has an error code in it as well?
Well fortunately arch_phys_wc_del(negative) and arch_phys_wc_del(0) will be
a no-op. Its what helps us remove so much clutter.
Luis
^ permalink raw reply
* Re: [PATCH v4 2/2] IB/qib: use arch_phys_wc_add()
From: Doug Ledford @ 2015-04-22 17:48 UTC (permalink / raw)
To: Luis R. Rodriguez
Cc: Luis R. Rodriguez, infinipath, roland, sean.hefty, hal.rosenstock,
linux-rdma, luto, mst, linux-kernel, cocci, Toshi Kani,
Rickard Strandqvist, Mike Marciniszyn, Roland Dreier,
Dennis Dalessandro, Suresh Siddha, Ingo Molnar, Thomas Gleixner,
Juergen Gross, Daniel Vetter, Dave Airlie, Bjorn Helgaas,
Antonino Daplas, Jean-Christophe Plagniol-Villard, Tomi Valkeinen
In-Reply-To: <20150422173728.GJ5622@wotan.suse.de>
[-- Attachment #1: Type: text/plain, Size: 3240 bytes --]
On Wed, 2015-04-22 at 19:37 +0200, Luis R. Rodriguez wrote:
> On Wed, Apr 22, 2015 at 12:57:18PM -0400, Doug Ledford wrote:
> > On Wed, 2015-04-22 at 17:33 +0200, Luis R. Rodriguez wrote:
> > > On Wed, Apr 22, 2015 at 09:54:38AM -0400, Doug Ledford wrote:
> > > > On Tue, 2015-04-21 at 14:50 -0700, Luis R. Rodriguez wrote:
> > > >
> > > > This:
> > > > > + /* MTRR was used if this is non-zero */
> > > > > + if (!dd->wc_cookie)
> > > > > vma->vm_page_prot = pgprot_writecombine(vma->vm_page_prot);
> > > >
> > > > And this:
> > > > > + dd->wc_cookie = arch_phys_wc_add(pioaddr, piolen);
> > > > > + if (dd->wc_cookie < 0)
> > > > > + ret = -EINVAL;
> > > >
> > > > don't agree on what wc_cookie will be on error.
> > >
> > > Can you elaborate? The one below is the one that starts things,
> > > and arch_phys_wc_add() will return 0 on PAT systems. For non-PAT
> > > systems it will return a number > 0 *iff* a valid MTRR was added.
> > > It will return negative onloy on error then.
> > >
> > > The change above is meant to replace a check put in place to see
> > > if PAT was enabled. The way we replace this is to ensure that
> > > arch_phys_wc_add() returned 0.
> > >
> > > If you disagree it'd be great if you can elaborate why.
> >
> > Maybe I'm missing something, but in qib_enable_wc() you store the return
> > from arch_phys_wc_add into wc_cookie. That return is negative,
>
> If and only if the system was non-PAT and mtrr_add() failed.
>
> > so you
> > return from qib_enable_wc() to qib_init_one(), they see the ret value,
> > they print out a warning about bad performance, then they clear the
> > return value and continue with device initialization.
> >
> > In all of this though, wc_cookie is never cleared and so it still has
> > the error condition in it. Then, much later at run time, you call
> > mmap_piobufs() and you check the contents of wc_cookie, and if it's
> > non-0 (which is still will be), you do the wrong thing, right?
>
> Originally the code had it to run pgprot_writecombine() if PAT was going to be
> used. After the code changes we check for !cookie which will be true when
> cookie is 0 only. In case the cookie was an error, that is if mtrr_add()
> failed, then this code would not run because (!negative) is false. The goal was
> to trigger a run if the cookie was 0, which can only happen if PAT was enabled.
OK, the logic works, but as much as anything, it's the comment that's
misleading. The code would be clearer with a comment like this:
/* We used PAT if wc_cookie == 0 */
if (!dd->wc_cookie) {
That would be more accurate as well since the original comment didn't
account for the possible error code in wc_cookie, so it's possible you
didn't use either PAT or wc if you have that error code.
> Please let me know, I'd like to get this right too.
>
> > And what
> > about at shutdown when you call qib_disable_wc() and your cookie still
> > has an error code in it as well?
>
> Well fortunately arch_phys_wc_del(negative) and arch_phys_wc_del(0) will be
> a no-op. Its what helps us remove so much clutter.
OK.
--
Doug Ledford <dledford@redhat.com>
GPG KeyID: 0E572FDD
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply
* Re: [PATCH v4 2/2] IB/qib: use arch_phys_wc_add()
From: Luis R. Rodriguez @ 2015-04-22 18:32 UTC (permalink / raw)
To: Doug Ledford
Cc: Luis R. Rodriguez, infinipath-ral2JQCrhuEAvxtiuMwx3w,
roland-DgEjT+Ai2ygdnm+yROfE0A, sean.hefty-ral2JQCrhuEAvxtiuMwx3w,
hal.rosenstock-Re5JQEeQqe8AvxtiuMwx3w,
linux-rdma-u79uwXL29TY76Z2rM5mHXA, luto-kltTT9wpgjJwATOyAt5JVQ,
mst-H+wXaHxf7aLQT0dZR+AlfA, linux-kernel-u79uwXL29TY76Z2rM5mHXA,
cocci-/FJkirnvOdkvYVN+rsErww, Toshi Kani, Rickard Strandqvist,
Mike Marciniszyn, Roland Dreier, Dennis Dalessandro,
Suresh Siddha, Ingo Molnar, Thomas Gleixner, Juergen Gross,
Daniel Vetter, Dave Airlie, Bjorn Helgaas, Antonino Daplas,
Jean-Christophe Plagniol-Villard, Tomi Valkeinen
In-Reply-To: <1429724907.45956.165.camel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
On Wed, Apr 22, 2015 at 01:48:27PM -0400, Doug Ledford wrote:
> On Wed, 2015-04-22 at 19:37 +0200, Luis R. Rodriguez wrote:
> > On Wed, Apr 22, 2015 at 12:57:18PM -0400, Doug Ledford wrote:
> > > On Wed, 2015-04-22 at 17:33 +0200, Luis R. Rodriguez wrote:
> > > > On Wed, Apr 22, 2015 at 09:54:38AM -0400, Doug Ledford wrote:
> > > > > On Tue, 2015-04-21 at 14:50 -0700, Luis R. Rodriguez wrote:
> > > > >
> > > > > This:
> > > > > > + /* MTRR was used if this is non-zero */
> > > > > > + if (!dd->wc_cookie)
> > > > > > vma->vm_page_prot = pgprot_writecombine(vma->vm_page_prot);
> > > > >
> > > > > And this:
> > > > > > + dd->wc_cookie = arch_phys_wc_add(pioaddr, piolen);
> > > > > > + if (dd->wc_cookie < 0)
> > > > > > + ret = -EINVAL;
> > > > >
> > > > > don't agree on what wc_cookie will be on error.
> > > >
> > > > Can you elaborate? The one below is the one that starts things,
> > > > and arch_phys_wc_add() will return 0 on PAT systems. For non-PAT
> > > > systems it will return a number > 0 *iff* a valid MTRR was added.
> > > > It will return negative onloy on error then.
> > > >
> > > > The change above is meant to replace a check put in place to see
> > > > if PAT was enabled. The way we replace this is to ensure that
> > > > arch_phys_wc_add() returned 0.
> > > >
> > > > If you disagree it'd be great if you can elaborate why.
> > >
> > > Maybe I'm missing something, but in qib_enable_wc() you store the return
> > > from arch_phys_wc_add into wc_cookie. That return is negative,
> >
> > If and only if the system was non-PAT and mtrr_add() failed.
> >
> > > so you
> > > return from qib_enable_wc() to qib_init_one(), they see the ret value,
> > > they print out a warning about bad performance, then they clear the
> > > return value and continue with device initialization.
> > >
> > > In all of this though, wc_cookie is never cleared and so it still has
> > > the error condition in it. Then, much later at run time, you call
> > > mmap_piobufs() and you check the contents of wc_cookie, and if it's
> > > non-0 (which is still will be), you do the wrong thing, right?
> >
> > Originally the code had it to run pgprot_writecombine() if PAT was going to be
> > used. After the code changes we check for !cookie which will be true when
> > cookie is 0 only. In case the cookie was an error, that is if mtrr_add()
> > failed, then this code would not run because (!negative) is false. The goal was
> > to trigger a run if the cookie was 0, which can only happen if PAT was enabled.
>
> OK, the logic works, but as much as anything, it's the comment that's
> misleading. The code would be clearer with a comment like this:
>
> /* We used PAT if wc_cookie = 0 */
> if (!dd->wc_cookie) {
>
> That would be more accurate as well since the original comment didn't
> account for the possible error code in wc_cookie, so it's possible you
> didn't use either PAT or wc if you have that error code.
Fair enough, will send a v5 follow up with the comment enhanced,
but will leave the first patch in this series as-is.
Luis
^ permalink raw reply
* [PATCH v5 2/2] IB/qib: use arch_phys_wc_add()
From: Luis R. Rodriguez @ 2015-04-22 18:38 UTC (permalink / raw)
To: infinipath-ral2JQCrhuEAvxtiuMwx3w, roland-DgEjT+Ai2ygdnm+yROfE0A,
sean.hefty-ral2JQCrhuEAvxtiuMwx3w,
hal.rosenstock-Re5JQEeQqe8AvxtiuMwx3w,
dledford-H+wXaHxf7aLQT0dZR+AlfA,
linux-rdma-u79uwXL29TY76Z2rM5mHXA
Cc: luto-kltTT9wpgjJwATOyAt5JVQ, mst-H+wXaHxf7aLQT0dZR+AlfA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA, Luis R. Rodriguez,
Toshi Kani, Rickard Strandqvist, Mike Marciniszyn, Roland Dreier,
Dennis Dalessandro, Suresh Siddha, Ingo Molnar, Thomas Gleixner,
Juergen Gross, Daniel Vetter, Dave Airlie, Bjorn Helgaas,
Antonino Daplas, Jean-Christophe Plagniol-Villard, Tomi Valkeinen,
Dave Hansen, Arnd Bergmann, Stefan Bader,
konrad.wilk-QHcLZuEGTsvQT0dZR+AlfA, ville.syrjala
From: "Luis R. Rodriguez" <mcgrof@suse.com>
This driver already makes use of ioremap_wc() on PIO buffers,
so convert it to use arch_phys_wc_add().
The qib driver uses a mmap() special case for when PAT is
not used, this behaviour used to be determined with a
module parameter but since we have been asked to just
remove that module parameter this checks for the WC cookie,
if not set we can assume PAT was used. If its set we do
what we used to do for the mmap for when MTRR was enabled.
The removal of the module parameter is OK given that Andy
notes that even if users of module parameter are still around
it will not prevent loading of the module on recent kernels.
Cc: Doug Ledford <dledford@redhat.com>
Cc: Toshi Kani <toshi.kani@hp.com>
Cc: Rickard Strandqvist <rickard_strandqvist@spectrumdigital.se>
Cc: Mike Marciniszyn <mike.marciniszyn@intel.com>
Cc: Roland Dreier <roland@purestorage.com>
Cc: Sean Hefty <sean.hefty@intel.com>
Cc: Hal Rosenstock <hal.rosenstock@gmail.com>
Cc: Dennis Dalessandro <dennis.dalessandro@intel.com>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Suresh Siddha <sbsiddha@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Juergen Gross <jgross@suse.com>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Dave Airlie <airlied@redhat.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Antonino Daplas <adaplas@gmail.com>
Cc: Jean-Christophe Plagniol-Villard <plagnioj@jcrosoft.com>
Cc: Tomi Valkeinen <tomi.valkeinen@ti.com>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Michael S. Tsirkin <mst@redhat.com>
Cc: Stefan Bader <stefan.bader@canonical.com>
Cc: konrad.wilk@oracle.com
Cc: ville.syrjala@linux.intel.com
Cc: david.vrabel@citrix.com
Cc: jbeulich@suse.com
Cc: Roger Pau Monné <roger.pau@citrix.com>
Cc: infinipath@intel.com
Cc: linux-rdma@vger.kernel.org
Cc: linux-fbdev@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Cc: xen-devel@lists.xensource.com
Signed-off-by: Luis R. Rodriguez <mcgrof@suse.com>
---
This v5 changes the comment to make it clear when the mmap code
will be true as requested by Doug. This patch still depends on
the v4 1/2 qib patch which enables accounting for the mtrr_add()
call. I won't be resending that one.
drivers/infiniband/hw/qib/qib.h | 1 -
drivers/infiniband/hw/qib/qib_file_ops.c | 3 ++-
drivers/infiniband/hw/qib/qib_iba6120.c | 8 +++---
drivers/infiniband/hw/qib/qib_iba7220.c | 8 +++---
drivers/infiniband/hw/qib/qib_iba7322.c | 41 +++++++++++++++----------------
drivers/infiniband/hw/qib/qib_init.c | 26 ++++++--------------
drivers/infiniband/hw/qib/qib_wc_x86_64.c | 31 +++--------------------
7 files changed, 39 insertions(+), 79 deletions(-)
diff --git a/drivers/infiniband/hw/qib/qib.h b/drivers/infiniband/hw/qib/qib.h
index ffd48bf..ba5173e2 100644
--- a/drivers/infiniband/hw/qib/qib.h
+++ b/drivers/infiniband/hw/qib/qib.h
@@ -1136,7 +1136,6 @@ extern struct qib_devdata *qib_lookup(int unit);
extern u32 qib_cpulist_count;
extern unsigned long *qib_cpulist;
-extern unsigned qib_wc_pat;
extern unsigned qib_cc_table_size;
int qib_init(struct qib_devdata *, int);
int init_chip_wc_pat(struct qib_devdata *dd, u32);
diff --git a/drivers/infiniband/hw/qib/qib_file_ops.c b/drivers/infiniband/hw/qib/qib_file_ops.c
index 9ea6c44..7258818 100644
--- a/drivers/infiniband/hw/qib/qib_file_ops.c
+++ b/drivers/infiniband/hw/qib/qib_file_ops.c
@@ -835,7 +835,8 @@ static int mmap_piobufs(struct vm_area_struct *vma,
vma->vm_flags &= ~VM_MAYREAD;
vma->vm_flags |= VM_DONTCOPY | VM_DONTEXPAND;
- if (qib_wc_pat)
+ /* We used PAT if wc_cookie = 0 */
+ if (!dd->wc_cookie)
vma->vm_page_prot = pgprot_writecombine(vma->vm_page_prot);
ret = io_remap_pfn_range(vma, vma->vm_start, phys >> PAGE_SHIFT,
diff --git a/drivers/infiniband/hw/qib/qib_iba6120.c b/drivers/infiniband/hw/qib/qib_iba6120.c
index 0d2ba59..4b927809 100644
--- a/drivers/infiniband/hw/qib/qib_iba6120.c
+++ b/drivers/infiniband/hw/qib/qib_iba6120.c
@@ -3315,11 +3315,9 @@ static int init_6120_variables(struct qib_devdata *dd)
qib_6120_config_ctxts(dd);
qib_set_ctxtcnt(dd);
- if (qib_wc_pat) {
- ret = init_chip_wc_pat(dd, 0);
- if (ret)
- goto bail;
- }
+ ret = init_chip_wc_pat(dd, 0);
+ if (ret)
+ goto bail;
set_6120_baseaddrs(dd); /* set chip access pointers now */
ret = 0;
diff --git a/drivers/infiniband/hw/qib/qib_iba7220.c b/drivers/infiniband/hw/qib/qib_iba7220.c
index 22affda..00b2af2 100644
--- a/drivers/infiniband/hw/qib/qib_iba7220.c
+++ b/drivers/infiniband/hw/qib/qib_iba7220.c
@@ -4126,11 +4126,9 @@ static int qib_init_7220_variables(struct qib_devdata *dd)
qib_7220_config_ctxts(dd);
qib_set_ctxtcnt(dd); /* needed for PAT setup */
- if (qib_wc_pat) {
- ret = init_chip_wc_pat(dd, 0);
- if (ret)
- goto bail;
- }
+ ret = init_chip_wc_pat(dd, 0);
+ if (ret)
+ goto bail;
set_7220_baseaddrs(dd); /* set chip access pointers now */
ret = 0;
diff --git a/drivers/infiniband/hw/qib/qib_iba7322.c b/drivers/infiniband/hw/qib/qib_iba7322.c
index ef97b71..f32b462 100644
--- a/drivers/infiniband/hw/qib/qib_iba7322.c
+++ b/drivers/infiniband/hw/qib/qib_iba7322.c
@@ -6429,6 +6429,7 @@ static int qib_init_7322_variables(struct qib_devdata *dd)
unsigned features, pidx, sbufcnt;
int ret, mtu;
u32 sbufs, updthresh;
+ resource_size_t vl15off;
/* pport structs are contiguous, allocated after devdata */
ppd = (struct qib_pportdata *)(dd + 1);
@@ -6677,29 +6678,27 @@ static int qib_init_7322_variables(struct qib_devdata *dd)
qib_7322_config_ctxts(dd);
qib_set_ctxtcnt(dd);
- if (qib_wc_pat) {
- resource_size_t vl15off;
- /*
- * We do not set WC on the VL15 buffers to avoid
- * a rare problem with unaligned writes from
- * interrupt-flushed store buffers, so we need
- * to map those separately here. We can't solve
- * this for the rarely used mtrr case.
- */
- ret = init_chip_wc_pat(dd, 0);
- if (ret)
- goto bail;
+ /*
+ * We do not set WC on the VL15 buffers to avoid
+ * a rare problem with unaligned writes from
+ * interrupt-flushed store buffers, so we need
+ * to map those separately here. We can't solve
+ * this for the rarely used mtrr case.
+ */
+ ret = init_chip_wc_pat(dd, 0);
+ if (ret)
+ goto bail;
- /* vl15 buffers start just after the 4k buffers */
- vl15off = dd->physaddr + (dd->piobufbase >> 32) +
- dd->piobcnt4k * dd->align4k;
- dd->piovl15base = ioremap_nocache(vl15off,
- NUM_VL15_BUFS * dd->align4k);
- if (!dd->piovl15base) {
- ret = -ENOMEM;
- goto bail;
- }
+ /* vl15 buffers start just after the 4k buffers */
+ vl15off = dd->physaddr + (dd->piobufbase >> 32) +
+ dd->piobcnt4k * dd->align4k;
+ dd->piovl15base = ioremap_nocache(vl15off,
+ NUM_VL15_BUFS * dd->align4k);
+ if (!dd->piovl15base) {
+ ret = -ENOMEM;
+ goto bail;
}
+
qib_7322_set_baseaddrs(dd); /* set chip access pointers now */
ret = 0;
diff --git a/drivers/infiniband/hw/qib/qib_init.c b/drivers/infiniband/hw/qib/qib_init.c
index 2ee3695..7e00470 100644
--- a/drivers/infiniband/hw/qib/qib_init.c
+++ b/drivers/infiniband/hw/qib/qib_init.c
@@ -91,15 +91,6 @@ MODULE_PARM_DESC(krcvqs, "number of kernel receive queues per IB port");
unsigned qib_cc_table_size;
module_param_named(cc_table_size, qib_cc_table_size, uint, S_IRUGO);
MODULE_PARM_DESC(cc_table_size, "Congestion control table entries 0 (CCA disabled - default), min = 128, max = 1984");
-/*
- * qib_wc_pat parameter:
- * 0 is WC via MTRR
- * 1 is WC via PAT
- * If PAT initialization fails, code reverts back to MTRR
- */
-unsigned qib_wc_pat = 1; /* default (1) is to use PAT, not MTRR */
-module_param_named(wc_pat, qib_wc_pat, uint, S_IRUGO);
-MODULE_PARM_DESC(wc_pat, "enable write-combining via PAT mechanism");
static void verify_interrupt(unsigned long);
@@ -1377,8 +1368,7 @@ static void cleanup_device_data(struct qib_devdata *dd)
spin_unlock(&dd->pport[pidx].cc_shadow_lock);
}
- if (!qib_wc_pat)
- qib_disable_wc(dd);
+ qib_disable_wc(dd);
if (dd->pioavailregs_dma) {
dma_free_coherent(&dd->pcidev->dev, PAGE_SIZE,
@@ -1547,14 +1537,12 @@ static int qib_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
goto bail;
}
- if (!qib_wc_pat) {
- ret = qib_enable_wc(dd);
- if (ret) {
- qib_dev_err(dd,
- "Write combining not enabled (err %d): performance may be poor\n",
- -ret);
- ret = 0;
- }
+ ret = qib_enable_wc(dd);
+ if (ret) {
+ qib_dev_err(dd,
+ "Write combining not enabled (err %d): performance may be poor\n",
+ -ret);
+ ret = 0;
}
qib_verify_pioperf(dd);
diff --git a/drivers/infiniband/hw/qib/qib_wc_x86_64.c b/drivers/infiniband/hw/qib/qib_wc_x86_64.c
index fe0850a..6d61ef9 100644
--- a/drivers/infiniband/hw/qib/qib_wc_x86_64.c
+++ b/drivers/infiniband/hw/qib/qib_wc_x86_64.c
@@ -116,21 +116,9 @@ int qib_enable_wc(struct qib_devdata *dd)
}
if (!ret) {
- int cookie;
-
- cookie = mtrr_add(pioaddr, piolen, MTRR_TYPE_WRCOMB, 1);
- if (cookie < 0) {
- {
- qib_devinfo(dd->pcidev,
- "mtrr_add() WC for PIO bufs failed (%d)\n",
- cookie);
- ret = -EINVAL;
- }
- } else {
- dd->wc_cookie = cookie;
- dd->wc_base = (unsigned long) pioaddr;
- dd->wc_len = (unsigned long) piolen;
- }
+ dd->wc_cookie = arch_phys_wc_add(pioaddr, piolen);
+ if (dd->wc_cookie < 0)
+ ret = -EINVAL;
}
return ret;
@@ -142,18 +130,7 @@ int qib_enable_wc(struct qib_devdata *dd)
*/
void qib_disable_wc(struct qib_devdata *dd)
{
- if (dd->wc_cookie) {
- int r;
-
- r = mtrr_del(dd->wc_cookie, dd->wc_base,
- dd->wc_len);
- if (r < 0)
- qib_devinfo(dd->pcidev,
- "mtrr_del(%lx, %lx, %lx) failed: %d\n",
- dd->wc_cookie, dd->wc_base,
- dd->wc_len, r);
- dd->wc_cookie = 0; /* even on failure */
- }
+ arch_phys_wc_del(dd->wc_cookie);
}
/**
--
2.3.2.209.gd67f9d5.dirty
^ permalink raw reply related
* Re: ioremap_uc() followed by set_memory_wc() - burrying MTRR
From: Doug Ledford @ 2015-04-22 18:53 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Luis R. Rodriguez, Andy Lutomirski, mike.marciniszyn, infinipath,
linux-rdma, awalls, Toshi Kani, H. Peter Anvin, Ingo Molnar,
linux-kernel@vger.kernel.org, Hal Rosenstock, Sean Hefty,
Suresh Siddha, Rickard Strandqvist, Roland Dreier, Juergen Gross,
Mauro Carvalho Chehab, Borislav Petkov, Mel Gorman,
Vlastimil Babka, Davidlohr Bueso, Dave Hansen,
Jean-Christophe Plagniol-Villard, Thomas Gleixner, Ville Syrj?l?,
Linux Fbdev development list, linux-media, X86 ML, mcgrof
In-Reply-To: <20150422161755.GA19500@obsidianresearch.com>
[-- Attachment #1: Type: text/plain, Size: 2885 bytes --]
On Wed, 2015-04-22 at 10:17 -0600, Jason Gunthorpe wrote:
> On Wed, Apr 22, 2015 at 05:23:28PM +0200, Luis R. Rodriguez wrote:
> > On Tue, Apr 21, 2015 at 11:39:39PM -0600, Jason Gunthorpe wrote:
> > > On Wed, Apr 22, 2015 at 01:39:07AM +0200, Luis R. Rodriguez wrote:
> > > > > Mike, do you think the time is right to just remove the iPath driver?
> > > >
> > > > With PAT now being default the driver effectively won't work
> > > > with write-combining on modern kernels. Even if systems are old
> > > > they likely had PAT support, when upgrading kernels PAT will work
> > > > but write-combing won't on ipath.
> > >
> > > Sorry, do you mean the driver already doesn't get WC? Or do you mean
> > > after some more pending patches are applied?
> >
> > No, you have to consider the system used and the effects of calls used
> > on the driver in light of this table:
>
> So, just to be clear:
>
> At some point Linux started setting the PAT bits during
> ioremap_nocache, which overrides MTRR, and at that point the driver
> became broken on all PAT capable systems?
>
> Not only that, but we've only just noticed it now, and no user ever
> complained?
>
> So that means either no users exist, or all users are on non-PAT
> systems?
>
> This driver only works on x86-64 systems. Are there any x86-64 systems
> that are not PAT capable? IIRC even the first Opteron had PAT, but my
> memory is fuzzy from back then :|
>
> > Another option in order to enable this type of checks at run time
> > and still be able to build the driver on standard distributions and
> > just prevent if from loading on PAT systems is to have some code in
> > place which would prevent the driver from loading if PAT was
> > enabled, this would enable folks to disable PAT via a kernel command
> > line option, and if that was used then the driver probe would
> > complete.
>
> This seems like a reasonble option to me. At the very least we might
> learn if anyone is still using these cards.
>
> I'd also love to remove the driver if it turns out there are actually
> no users. qib substantially replaces it except for a few very old
> cards.
To be precise, the split is that ipath powers the old HTX bus cards that
only work in AMD systems, qib is all PCI-e cards. I still have a few
HTX cards, but I no longer have any systems with HTX slots, so we
haven't even used this driver in testing for 3 or 4 years now. And
these are all old SDR cards, where the performance numbers were 800MB/s
with WC enabled, 50MB/s without it.
> Mike?
>
> Jason
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Doug Ledford <dledford@redhat.com>
GPG KeyID: 0E572FDD
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply
* Re: ioremap_uc() followed by set_memory_wc() - burrying MTRR
From: Luis R. Rodriguez @ 2015-04-22 19:05 UTC (permalink / raw)
To: Doug Ledford
Cc: Jason Gunthorpe, Andy Lutomirski, mike.marciniszyn, infinipath,
linux-rdma, awalls, Toshi Kani, H. Peter Anvin, Ingo Molnar,
linux-kernel@vger.kernel.org, Hal Rosenstock, Sean Hefty,
Suresh Siddha, Rickard Strandqvist, Roland Dreier, Juergen Gross,
Mauro Carvalho Chehab, Borislav Petkov, Mel Gorman,
Vlastimil Babka, Davidlohr Bueso, Dave Hansen,
Jean-Christophe Plagniol-Villard, Thomas Gleixner, Ville Syrj?l?,
Linux Fbdev development list, linux-media, X86 ML, mcgrof
In-Reply-To: <1429728791.121496.10.camel@redhat.com>
On Wed, Apr 22, 2015 at 02:53:11PM -0400, Doug Ledford wrote:
> On Wed, 2015-04-22 at 10:17 -0600, Jason Gunthorpe wrote:
> > On Wed, Apr 22, 2015 at 05:23:28PM +0200, Luis R. Rodriguez wrote:
> > > On Tue, Apr 21, 2015 at 11:39:39PM -0600, Jason Gunthorpe wrote:
> > > > On Wed, Apr 22, 2015 at 01:39:07AM +0200, Luis R. Rodriguez wrote:
> > > > > > Mike, do you think the time is right to just remove the iPath driver?
> > > > >
> > > > > With PAT now being default the driver effectively won't work
> > > > > with write-combining on modern kernels. Even if systems are old
> > > > > they likely had PAT support, when upgrading kernels PAT will work
> > > > > but write-combing won't on ipath.
> > > >
> > > > Sorry, do you mean the driver already doesn't get WC? Or do you mean
> > > > after some more pending patches are applied?
> > >
> > > No, you have to consider the system used and the effects of calls used
> > > on the driver in light of this table:
> >
> > So, just to be clear:
> >
> > At some point Linux started setting the PAT bits during
> > ioremap_nocache, which overrides MTRR, and at that point the driver
> > became broken on all PAT capable systems?
> >
> > Not only that, but we've only just noticed it now, and no user ever
> > complained?
> >
> > So that means either no users exist, or all users are on non-PAT
> > systems?
> >
> > This driver only works on x86-64 systems. Are there any x86-64 systems
> > that are not PAT capable? IIRC even the first Opteron had PAT, but my
> > memory is fuzzy from back then :|
> >
> > > Another option in order to enable this type of checks at run time
> > > and still be able to build the driver on standard distributions and
> > > just prevent if from loading on PAT systems is to have some code in
> > > place which would prevent the driver from loading if PAT was
> > > enabled, this would enable folks to disable PAT via a kernel command
> > > line option, and if that was used then the driver probe would
> > > complete.
> >
> > This seems like a reasonble option to me. At the very least we might
> > learn if anyone is still using these cards.
> >
> > I'd also love to remove the driver if it turns out there are actually
> > no users. qib substantially replaces it except for a few very old
> > cards.
>
> To be precise, the split is that ipath powers the old HTX bus cards that
> only work in AMD systems,
Do those systems have PAT support? CAn anyone check if PAT is enabled
if booted on a recent kernel?
Luis
^ permalink raw reply
* Re: ioremap_uc() followed by set_memory_wc() - burrying MTRR
From: Doug Ledford @ 2015-04-22 19:10 UTC (permalink / raw)
To: Luis R. Rodriguez
Cc: Jason Gunthorpe, Andy Lutomirski, mike.marciniszyn, infinipath,
linux-rdma, awalls, Toshi Kani, H. Peter Anvin, Ingo Molnar,
linux-kernel@vger.kernel.org, Hal Rosenstock, Sean Hefty,
Suresh Siddha, Rickard Strandqvist, Roland Dreier, Juergen Gross,
Mauro Carvalho Chehab, Borislav Petkov, Mel Gorman,
Vlastimil Babka, Davidlohr Bueso, Dave Hansen,
Jean-Christophe Plagniol-Villard, Thomas Gleixner, Ville Syrj?l?,
Linux Fbdev development list, linux-media, X86 ML, mcgrof
In-Reply-To: <20150422190520.GL5622@wotan.suse.de>
[-- Attachment #1: Type: text/plain, Size: 679 bytes --]
On Wed, 2015-04-22 at 21:05 +0200, Luis R. Rodriguez wrote:
> > > I'd also love to remove the driver if it turns out there are actually
> > > no users. qib substantially replaces it except for a few very old
> > > cards.
> >
> > To be precise, the split is that ipath powers the old HTX bus cards that
> > only work in AMD systems,
>
> Do those systems have PAT support? CAn anyone check if PAT is enabled
> if booted on a recent kernel?
I don't have one of these systems any more. The *only* one I ever had
was a monster IBM box...I can't even find a reference to it any more.
--
Doug Ledford <dledford@redhat.com>
GPG KeyID: 0E572FDD
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply
* Re: ioremap_uc() followed by set_memory_wc() - burrying MTRR
From: Luis R. Rodriguez @ 2015-04-22 19:14 UTC (permalink / raw)
To: Doug Ledford
Cc: Jason Gunthorpe, Andy Lutomirski, Mike Marciniszyn,
Mike Marciniszyn, linux-rdma, Andy Walls, Toshi Kani,
H. Peter Anvin, Ingo Molnar, linux-kernel@vger.kernel.org,
Hal Rosenstock, Sean Hefty, Suresh Siddha, Rickard Strandqvist,
Roland Dreier, Juergen Gross, Mauro Carvalho Chehab,
Borislav Petkov, Mel Gorman, Vlastimil Babka, Davidlohr Bueso,
Dave Hansen, Jean-Christophe Plagniol-Villard, Thomas Gleixner,
Ville Syrj?l?, Linux Fbdev development list, linux-media, X86 ML
In-Reply-To: <1429729841.121496.15.camel@redhat.com>
On Wed, Apr 22, 2015 at 12:10 PM, Doug Ledford <dledford@redhat.com> wrote:
> On Wed, 2015-04-22 at 21:05 +0200, Luis R. Rodriguez wrote:
>
>> > > I'd also love to remove the driver if it turns out there are actually
>> > > no users. qib substantially replaces it except for a few very old
>> > > cards.
>> >
>> > To be precise, the split is that ipath powers the old HTX bus cards that
>> > only work in AMD systems,
>>
>> Do those systems have PAT support? CAn anyone check if PAT is enabled
>> if booted on a recent kernel?
>
> I don't have one of these systems any more. The *only* one I ever had
> was a monster IBM box...I can't even find a reference to it any more.
Um, yeah if its so rare then I think the compromise proposed might
make sense, specially since folks were even *considering* seriously
removing this device driver. I'll send some patches to propose the
strategy explained to require booting with pat disabled.
Luis
^ permalink raw reply
* [PATCH] IB/ipath: use arch_phys_wc_add() and require PAT disabled
From: Luis R. Rodriguez @ 2015-04-22 19:26 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA,
infinipath-ral2JQCrhuEAvxtiuMwx3w, roland-DgEjT+Ai2ygdnm+yROfE0A,
sean.hefty-ral2JQCrhuEAvxtiuMwx3w,
hal.rosenstock-Re5JQEeQqe8AvxtiuMwx3w,
linux-rdma-u79uwXL29TY76Z2rM5mHXA
Cc: luto-kltTT9wpgjJwATOyAt5JVQ, mst-H+wXaHxf7aLQT0dZR+AlfA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA, Luis R. Rodriguez,
Suresh Siddha, Rickard Strandqvist, Mike Marciniszyn,
Roland Dreier, Ingo Molnar, Linus Torvalds, Thomas Gleixner,
Juergen Gross, Daniel Vetter, Dave Airlie, Bjorn Helgaas,
Antonino Daplas, Jean-Christophe Plagniol-Villard, Tomi Valkeinen,
Ville Syrjälä, Mel Gorman, Vlastimil Babka,
Borislav Petkov, Davidlohr Bueso
From: "Luis R. Rodriguez" <mcgrof@suse.com>
We are burrying direct access to MTRR code support on
x86 in order to take advantage of PAT. In the future we
also want to make the default behaviour of ioremap_nocache()
to use strong UC, use of mtrr_add() on those systems
would make write-combining void.
In order to help both enable us to later make strong
UC default and in order to phase out direct MTRR access
code port the driver over to arch_phys_wc_add() and
annotate that the device driver requires systems to
boot with PAT disabled, with the nopat kernel parameter.
This is a worthy compromise given that the ipath device
driver powers the old HTX bus cards that only work in
AMD systems, while the newer IB/qib device driver
powers all PCI-e cards. The ipath device driver is
obsolete, hardware hard to find and because of this
this its a reasonable compromise to make to require
users of ipath to boot with nopat.
Cc: Doug Ledford <dledford@redhat.com>
Cc: Hal Rosenstock <hal.rosenstock@gmail.com>
Cc: Sean Hefty <sean.hefty@intel.com>
Cc: Suresh Siddha <sbsiddha@gmail.com>
Cc: Rickard Strandqvist <rickard_strandqvist@spectrumdigital.se>
Cc: Mike Marciniszyn <mike.marciniszyn@intel.com>
Cc: Roland Dreier <roland@purestorage.com>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Juergen Gross <jgross@suse.com>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Dave Airlie <airlied@redhat.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Antonino Daplas <adaplas@gmail.com>
Cc: Jean-Christophe Plagniol-Villard <plagnioj@jcrosoft.com>
Cc: Tomi Valkeinen <tomi.valkeinen@ti.com>
Cc: Ville Syrjälä <syrjala@sci.fi>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Borislav Petkov <bp@suse.de>
Cc: Davidlohr Bueso <dbueso@suse.de>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Michael S. Tsirkin <mst@redhat.com>
Cc: Stefan Bader <stefan.bader@canonical.com>
Cc: konrad.wilk@oracle.com
Cc: ville.syrjala@linux.intel.com
Cc: david.vrabel@citrix.com
Cc: jbeulich@suse.com
Cc: toshi.kani@hp.com
Cc: Roger Pau Monné <roger.pau@citrix.com>
Cc: infinipath@intel.com
Cc: linux-rdma@vger.kernel.org
Cc: linux-fbdev@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Cc: xen-devel@lists.xensource.com
Signed-off-by: Luis R. Rodriguez <mcgrof@suse.com>
---
drivers/infiniband/hw/ipath/Kconfig | 3 ++
drivers/infiniband/hw/ipath/ipath_driver.c | 18 +++++++----
drivers/infiniband/hw/ipath/ipath_kernel.h | 4 +--
drivers/infiniband/hw/ipath/ipath_wc_x86_64.c | 43 ++++++---------------------
4 files changed, 26 insertions(+), 42 deletions(-)
diff --git a/drivers/infiniband/hw/ipath/Kconfig b/drivers/infiniband/hw/ipath/Kconfig
index 1d9bb11..8fe54ff 100644
--- a/drivers/infiniband/hw/ipath/Kconfig
+++ b/drivers/infiniband/hw/ipath/Kconfig
@@ -9,3 +9,6 @@ config INFINIBAND_IPATH
as IP-over-InfiniBand as well as with userspace applications
(in conjunction with InfiniBand userspace access).
For QLogic PCIe QLE based cards, use the QIB driver instead.
+
+ If you have this hardware you will need to boot with PAT disabled
+ on your x86-64 systems, use the nopat kernel parameter.
diff --git a/drivers/infiniband/hw/ipath/ipath_driver.c b/drivers/infiniband/hw/ipath/ipath_driver.c
index bd0caed..3ef592c 100644
--- a/drivers/infiniband/hw/ipath/ipath_driver.c
+++ b/drivers/infiniband/hw/ipath/ipath_driver.c
@@ -42,6 +42,9 @@
#include <linux/bitmap.h>
#include <linux/slab.h>
#include <linux/module.h>
+#ifdef CONFIG_X86_64
+#include <asm/pat.h>
+#endif
#include "ipath_kernel.h"
#include "ipath_verbs.h"
@@ -395,6 +398,14 @@ static int ipath_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
unsigned long long addr;
u32 bar0 = 0, bar1 = 0;
+#ifdef CONFIG_X86_64
+ if (WARN(pat_enabled,
+ "ipath needs PAT disabled, boot with nopat kernel parameter\n")) {
+ ret = EINVAL;
+ goto bail;
+ }
+#endif
+
dd = ipath_alloc_devdata(pdev);
if (IS_ERR(dd)) {
ret = PTR_ERR(dd);
@@ -542,6 +553,7 @@ static int ipath_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
dd->ipath_kregbase = __ioremap(addr, len,
(_PAGE_NO_CACHE|_PAGE_WRITETHRU));
#else
+ /* XXX: split this properly to enable on PAT */
dd->ipath_kregbase = ioremap_nocache(addr, len);
#endif
@@ -587,12 +599,8 @@ static int ipath_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
ret = ipath_enable_wc(dd);
- if (ret) {
- ipath_dev_err(dd, "Write combining not enabled "
- "(err %d): performance may be poor\n",
- -ret);
+ if (ret)
ret = 0;
- }
ipath_verify_pioperf(dd);
diff --git a/drivers/infiniband/hw/ipath/ipath_kernel.h b/drivers/infiniband/hw/ipath/ipath_kernel.h
index e08db70..f0f9471 100644
--- a/drivers/infiniband/hw/ipath/ipath_kernel.h
+++ b/drivers/infiniband/hw/ipath/ipath_kernel.h
@@ -463,9 +463,7 @@ struct ipath_devdata {
/* offset in HT config space of slave/primary interface block */
u8 ipath_ht_slave_off;
/* for write combining settings */
- unsigned long ipath_wc_cookie;
- unsigned long ipath_wc_base;
- unsigned long ipath_wc_len;
+ int wc_cookie;
/* ref count for each pkey */
atomic_t ipath_pkeyrefs[4];
/* shadow copy of struct page *'s for exp tid pages */
diff --git a/drivers/infiniband/hw/ipath/ipath_wc_x86_64.c b/drivers/infiniband/hw/ipath/ipath_wc_x86_64.c
index 70c1f3a..7b6e4c8 100644
--- a/drivers/infiniband/hw/ipath/ipath_wc_x86_64.c
+++ b/drivers/infiniband/hw/ipath/ipath_wc_x86_64.c
@@ -37,7 +37,6 @@
*/
#include <linux/pci.h>
-#include <asm/mtrr.h>
#include <asm/processor.h>
#include "ipath_kernel.h"
@@ -122,27 +121,14 @@ int ipath_enable_wc(struct ipath_devdata *dd)
}
if (!ret) {
- int cookie;
- ipath_cdbg(VERBOSE, "Setting mtrr for chip to WC "
- "(addr %llx, len=0x%llx)\n",
- (unsigned long long) pioaddr,
- (unsigned long long) piolen);
- cookie = mtrr_add(pioaddr, piolen, MTRR_TYPE_WRCOMB, 1);
- if (cookie < 0) {
- {
- dev_info(&dd->pcidev->dev,
- "mtrr_add() WC for PIO bufs "
- "failed (%d)\n",
- cookie);
- ret = -EINVAL;
- }
- } else {
- ipath_cdbg(VERBOSE, "Set mtrr for chip to WC, "
- "cookie is %d\n", cookie);
- dd->ipath_wc_cookie = cookie;
- dd->ipath_wc_base = (unsigned long) pioaddr;
- dd->ipath_wc_len = (unsigned long) piolen;
- }
+ dd->wc_cookie = arch_phys_wc_add(pioaddr, piolen);
+ if (dd->wc_cookie < 0) {
+ ipath_dev_err(dd, "Seting mtrr failed on PIO buffers\n");
+ ret = -ENODEV;
+ } else if (dd->wc_cookie = 0)
+ ipath_cdbg(VERBOSE, "Set mtrr for chip to WC not needed\n");
+ else
+ ipath_cdbg(VERBOSE, "Set mtrr for chip to WC\n");
}
return ret;
@@ -154,16 +140,5 @@ int ipath_enable_wc(struct ipath_devdata *dd)
*/
void ipath_disable_wc(struct ipath_devdata *dd)
{
- if (dd->ipath_wc_cookie) {
- int r;
- ipath_cdbg(VERBOSE, "undoing WCCOMB on pio buffers\n");
- r = mtrr_del(dd->ipath_wc_cookie, dd->ipath_wc_base,
- dd->ipath_wc_len);
- if (r < 0)
- dev_info(&dd->pcidev->dev,
- "mtrr_del(%lx, %lx, %lx) failed: %d\n",
- dd->ipath_wc_cookie, dd->ipath_wc_base,
- dd->ipath_wc_len, r);
- dd->ipath_wc_cookie = 0; /* even on failure */
- }
+ arch_phys_wc_del(dd->wc_cookie);
}
--
2.3.2.209.gd67f9d5.dirty
^ permalink raw reply related
* Re: [PATCH 4/5] ASoC: omap-hdmi-audio: Force channel allocation only for OMAP4
From: Mark Brown @ 2015-04-22 19:32 UTC (permalink / raw)
To: Jyri Sarha
Cc: alsa-devel, linux-fbdev, linux-omap, peter.ujfalusi,
liam.r.girdwood, tomi.valkeinen, misael.lopez
In-Reply-To: <24329adf0076ff2ac004817a602c12db2ccc477d.1429705991.git.jsarha@ti.com>
[-- Attachment #1: Type: text/plain, Size: 290 bytes --]
On Wed, Apr 22, 2015 at 04:23:00PM +0300, Jyri Sarha wrote:
> From: Misael Lopez Cruz <misael.lopez@ti.com>
>
> There is a constraint in the OMAP4 HDMI IP that requires to use
> the 8-channel code when transmitting more than two channels.
Acked-by: Mark Brown <broonie@kernel.org>
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 473 bytes --]
^ permalink raw reply
* [PATCH] [media] ivtv: use arch_phys_wc_add() and require PAT disabled
From: Luis R. Rodriguez @ 2015-04-22 19:33 UTC (permalink / raw)
To: andy, awalls, linux-media
Cc: luto, mst, linux-kernel, linux-fbdev, Luis R. Rodriguez,
Mauro Carvalho Chehab, Suresh Siddha, Ingo Molnar,
Thomas Gleixner, Juergen Gross, Daniel Vetter, Dave Airlie,
Bjorn Helgaas, Antonino Daplas, Jean-Christophe Plagniol-Villard,
Tomi Valkeinen, Dave Hansen, Arnd Bergmann, Stefan Bader,
Ville Syrjälä, Mel Gorman, Vlastimil Babka,
Borislav Petkov, Davidlohr Bueso, konrad.wilk, ville.syrjala,
david.vrabel, jbeulich, toshi.kani, Roger Pau Monné,
ivtv-devel, xen-devel
From: "Luis R. Rodriguez" <mcgrof@suse.com>
We are burrying direct access to MTRR code support on
x86 in order to take advantage of PAT. In the future we
also want to make the default behaviour of ioremap_nocache()
to use strong UC, use of mtrr_add() on those systems
would make write-combining void.
In order to help both enable us to later make strong
UC default and in order to phase out direct MTRR access
code port the driver over to arch_phys_wc_add() and
annotate that the device driver requires systems to
boot with PAT disabled, with the nopat kernel parameter.
This is a worthy comprmise given that the hardware is
really rare these days, and perhaps only some lost souls
in some third world country are expected to be using this
feature of the device driver.
Cc: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Andy Walls <awalls@md.metrocast.net>
Cc: Suresh Siddha <sbsiddha@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Juergen Gross <jgross@suse.com>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Dave Airlie <airlied@redhat.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Antonino Daplas <adaplas@gmail.com>
Cc: Jean-Christophe Plagniol-Villard <plagnioj@jcrosoft.com>
Cc: Tomi Valkeinen <tomi.valkeinen@ti.com>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Michael S. Tsirkin <mst@redhat.com>
Cc: Stefan Bader <stefan.bader@canonical.com>
Cc: Ville Syrjälä <syrjala@sci.fi>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Borislav Petkov <bp@suse.de>
Cc: Davidlohr Bueso <dbueso@suse.de>
Cc: konrad.wilk@oracle.com
Cc: ville.syrjala@linux.intel.com
Cc: david.vrabel@citrix.com
Cc: jbeulich@suse.com
Cc: toshi.kani@hp.com
Cc: Roger Pau Monné <roger.pau@citrix.com>
Cc: linux-fbdev@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Cc: ivtv-devel@ivtvdriver.org
Cc: linux-media@vger.kernel.org
Cc: xen-devel@lists.xensource.com
Signed-off-by: Luis R. Rodriguez <mcgrof@suse.com>
---
drivers/media/pci/ivtv/Kconfig | 3 +++
drivers/media/pci/ivtv/ivtvfb.c | 59 +++++++++++++++++------------------------
2 files changed, 27 insertions(+), 35 deletions(-)
diff --git a/drivers/media/pci/ivtv/Kconfig b/drivers/media/pci/ivtv/Kconfig
index dd6ee57e..b2a7f88 100644
--- a/drivers/media/pci/ivtv/Kconfig
+++ b/drivers/media/pci/ivtv/Kconfig
@@ -57,5 +57,8 @@ config VIDEO_FB_IVTV
This is used in the Hauppauge PVR-350 card. There is a driver
homepage at <http://www.ivtvdriver.org>.
+ If you have this hardware you will need to boot with PAT disabled
+ on your x86 systems, use the nopat kernel parameter.
+
To compile this driver as a module, choose M here: the
module will be called ivtvfb.
diff --git a/drivers/media/pci/ivtv/ivtvfb.c b/drivers/media/pci/ivtv/ivtvfb.c
index 9ff1230..552408b 100644
--- a/drivers/media/pci/ivtv/ivtvfb.c
+++ b/drivers/media/pci/ivtv/ivtvfb.c
@@ -44,8 +44,8 @@
#include <linux/ivtvfb.h>
#include <linux/slab.h>
-#ifdef CONFIG_MTRR
-#include <asm/mtrr.h>
+#ifdef CONFIG_X86_64
+#include <asm/pat.h>
#endif
#include "ivtv-driver.h"
@@ -155,12 +155,11 @@ struct osd_info {
/* Buffer size */
u32 video_buffer_size;
-#ifdef CONFIG_MTRR
/* video_base rounded down as required by hardware MTRRs */
unsigned long fb_start_aligned_physaddr;
/* video_base rounded up as required by hardware MTRRs */
unsigned long fb_end_aligned_physaddr;
-#endif
+ int wc_cookie;
/* Store the buffer offset */
int set_osd_coords_x;
@@ -1099,6 +1098,8 @@ static int ivtvfb_init_vidmode(struct ivtv *itv)
static int ivtvfb_init_io(struct ivtv *itv)
{
struct osd_info *oi = itv->osd_info;
+ /* Find the largest power of two that maps the whole buffer */
+ int size_shift = 31;
mutex_lock(&itv->serialize_lock);
if (ivtv_init_on_first_open(itv)) {
@@ -1120,6 +1121,7 @@ static int ivtvfb_init_io(struct ivtv *itv)
oi->video_buffer_size = 1704960;
oi->video_pbase = itv->base_addr + IVTV_DECODER_OFFSET + oi->video_rbase;
+ /* XXX: split this for PAT */
oi->video_vbase = itv->dec_mem + oi->video_rbase;
if (!oi->video_vbase) {
@@ -1132,29 +1134,16 @@ static int ivtvfb_init_io(struct ivtv *itv)
oi->video_pbase, oi->video_vbase,
oi->video_buffer_size / 1024);
-#ifdef CONFIG_MTRR
- {
- /* Find the largest power of two that maps the whole buffer */
- int size_shift = 31;
-
- while (!(oi->video_buffer_size & (1 << size_shift))) {
- size_shift--;
- }
- size_shift++;
- oi->fb_start_aligned_physaddr = oi->video_pbase & ~((1 << size_shift) - 1);
- oi->fb_end_aligned_physaddr = oi->video_pbase + oi->video_buffer_size;
- oi->fb_end_aligned_physaddr += (1 << size_shift) - 1;
- oi->fb_end_aligned_physaddr &= ~((1 << size_shift) - 1);
- if (mtrr_add(oi->fb_start_aligned_physaddr,
- oi->fb_end_aligned_physaddr - oi->fb_start_aligned_physaddr,
- MTRR_TYPE_WRCOMB, 1) < 0) {
- IVTVFB_INFO("disabled mttr\n");
- oi->fb_start_aligned_physaddr = 0;
- oi->fb_end_aligned_physaddr = 0;
- }
- }
-#endif
-
+ while (!(oi->video_buffer_size & (1 << size_shift)))
+ size_shift--;
+ size_shift++;
+ oi->fb_start_aligned_physaddr = oi->video_pbase & ~((1 << size_shift) - 1);
+ oi->fb_end_aligned_physaddr = oi->video_pbase + oi->video_buffer_size;
+ oi->fb_end_aligned_physaddr += (1 << size_shift) - 1;
+ oi->fb_end_aligned_physaddr &= ~((1 << size_shift) - 1);
+ oi->wc_cookie = arch_phys_wc_add(oi->fb_start_aligned_physaddr,
+ oi->fb_end_aligned_physaddr -
+ oi->fb_start_aligned_physaddr);
/* Blank the entire osd. */
memset_io(oi->video_vbase, 0, oi->video_buffer_size);
@@ -1172,14 +1161,7 @@ static void ivtvfb_release_buffers (struct ivtv *itv)
/* Release pseudo palette */
kfree(oi->ivtvfb_info.pseudo_palette);
-
-#ifdef CONFIG_MTRR
- if (oi->fb_end_aligned_physaddr) {
- mtrr_del(-1, oi->fb_start_aligned_physaddr,
- oi->fb_end_aligned_physaddr - oi->fb_start_aligned_physaddr);
- }
-#endif
-
+ arch_phys_wc_del(oi->wc_cookie);
kfree(oi);
itv->osd_info = NULL;
}
@@ -1190,6 +1172,13 @@ static int ivtvfb_init_card(struct ivtv *itv)
{
int rc;
+#ifdef CONFIG_X86_64
+ if (WARN(pat_enabled,
+ "ivtv needs PAT disabled, boot with nopat kernel parameter\n")) {
+ return EINVAL;
+ }
+#endif
+
if (itv->osd_info) {
IVTVFB_ERR("Card %d already initialised\n", ivtvfb_card_id);
return -EBUSY;
--
2.3.2.209.gd67f9d5.dirty
^ permalink raw reply related
* Re: [PATCH 5/5] ASoC: omap-hdmi-audio: Fix invalid combination of DM_INH and CA
From: Mark Brown @ 2015-04-22 19:33 UTC (permalink / raw)
To: Jyri Sarha
Cc: alsa-devel, linux-fbdev, linux-omap, peter.ujfalusi,
liam.r.girdwood, tomi.valkeinen, misael.lopez
In-Reply-To: <79f95cefe5f69f90698520adab90ce83b8539bf8.1429705991.git.jsarha@ti.com>
[-- Attachment #1: Type: text/plain, Size: 338 bytes --]
On Wed, Apr 22, 2015 at 04:23:01PM +0300, Jyri Sarha wrote:
> From: Misael Lopez Cruz <misael.lopez@ti.com>
>
> DM_INH = 1 (stereo downmix prohibited) and CA = 0x00 (Channel
> Allocation: FR, FL) is an invalid combination according to the
> HDMI Compliance Test 7.31 "Audio InfoFrame".
Acked-by: Mark Brown <broonie@kernel.org>
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 473 bytes --]
^ permalink raw reply
* Re: ioremap_uc() followed by set_memory_wc() - burrying MTRR
From: Jason Gunthorpe @ 2015-04-22 20:46 UTC (permalink / raw)
To: Doug Ledford
Cc: Luis R. Rodriguez, Andy Lutomirski, mike.marciniszyn, infinipath,
linux-rdma, awalls, Toshi Kani, H. Peter Anvin, Ingo Molnar,
linux-kernel@vger.kernel.org, Hal Rosenstock, Sean Hefty,
Suresh Siddha, Rickard Strandqvist, Roland Dreier, Juergen Gross,
Mauro Carvalho Chehab, Borislav Petkov, Mel Gorman,
Vlastimil Babka, Davidlohr Bueso, Dave Hansen,
Jean-Christophe Plagniol-Villard, Thomas Gleixner, Ville Syrj?l?,
Linux Fbdev development list, linux-media, X86 ML, mcgrof
In-Reply-To: <1429728791.121496.10.camel@redhat.com>
On Wed, Apr 22, 2015 at 02:53:11PM -0400, Doug Ledford wrote:
> To be precise, the split is that ipath powers the old HTX bus cards that
> only work in AMD systems, qib is all PCI-e cards. I still have a few
> HTX cards, but I no longer have any systems with HTX slots, so we
> haven't even used this driver in testing for 3 or 4 years now. And
> these are all old SDR cards, where the performance numbers were 800MB/s
> with WC enabled, 50MB/s without it.
Wow, I doubt any HTX systems are still in any kind of use.
It would be a nice clean up to drop the PPC support out of this driver
too. PPC never had HTX.
Jason
^ permalink raw reply
* Re: ioremap_uc() followed by set_memory_wc() - burrying MTRR
From: Doug Ledford @ 2015-04-22 20:58 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Luis R. Rodriguez, Andy Lutomirski, mike.marciniszyn, infinipath,
linux-rdma, awalls, Toshi Kani, H. Peter Anvin, Ingo Molnar,
linux-kernel@vger.kernel.org, Hal Rosenstock, Sean Hefty,
Suresh Siddha, Rickard Strandqvist, Roland Dreier, Juergen Gross,
Mauro Carvalho Chehab, Borislav Petkov, Mel Gorman,
Vlastimil Babka, Davidlohr Bueso, Dave Hansen,
Jean-Christophe Plagniol-Villard, Thomas Gleixner, Ville Syrj?l?,
Linux Fbdev development list, linux-media, X86 ML, mcgrof
In-Reply-To: <20150422204637.GA29491@obsidianresearch.com>
[-- Attachment #1: Type: text/plain, Size: 1548 bytes --]
On Wed, 2015-04-22 at 14:46 -0600, Jason Gunthorpe wrote:
> On Wed, Apr 22, 2015 at 02:53:11PM -0400, Doug Ledford wrote:
>
> > To be precise, the split is that ipath powers the old HTX bus cards that
> > only work in AMD systems, qib is all PCI-e cards. I still have a few
> > HTX cards, but I no longer have any systems with HTX slots, so we
> > haven't even used this driver in testing for 3 or 4 years now. And
> > these are all old SDR cards, where the performance numbers were 800MB/s
> > with WC enabled, 50MB/s without it.
>
> Wow, I doubt any HTX systems are still in any kind of use.
>
> It would be a nice clean up to drop the PPC support out of this driver
> too. PPC never had HTX.
commit f6d60848baf9f4015c76c665791875ed623cd5b7
Author: Ralph Campbell <ralph.campbell@qlogic.com>
Date: Thu May 6 17:03:19 2010 -0700
IB/ipath: Remove support for QLogic PCIe QLE devices
The ib_qib driver is taking over support for QLogic PCIe QLE
devices,
so remove support for them from ib_ipath. The ib_ipath driver now
supports only the obsolete QLogic Hyper-Transport IB host channel
adapter (model QHT7140).
Signed-off-by: Ralph Campbell <ralph.campbell@qlogic.com>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
There you go. It's been HTX only since 2010, and those cards were
already old then. I think we should seriously consider deprecating and
then removing the driver.
--
Doug Ledford <dledford@redhat.com>
GPG KeyID: 0E572FDD
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply
* [PATCHv3 0/4] add devm_of_phy_get_by_index and update platform drivers
From: Arun Ramamurthy @ 2015-04-22 23:04 UTC (permalink / raw)
To: linux-arm-kernel
This patch set adds a new API to get phy by index when multiple
phys are present. This patch is based on discussion with Arnd Bergmann
about dt bindings for multiple phys.
History:
v1:
- Removed null pointers on Dmitry's suggestion
- Improved documentation in commit messages
- Exported new phy api
v2:
- EHCI and OHCI platform Kconfigs select Generic Phy
to fix build errors in certain configs.
v3:
- Made GENERIC_PHY an invisible option so
that other configs can select it
- Added stubs for devm_of_phy_get_by_index
- Reformated code
Arun Ramamurthy (4):
phy: phy-core: Make GENERIC_PHY an invisible option
phy: core: Add devm_of_phy_get_by_index to phy-core
usb: ehci-platform: Use devm_of_phy_get_by_index
usb: ohci-platform: Use devm_of_phy_get_by_index
Documentation/phy.txt | 7 +++-
drivers/ata/Kconfig | 1 -
drivers/media/platform/exynos4-is/Kconfig | 2 +-
drivers/phy/Kconfig | 4 +-
drivers/phy/phy-core.c | 32 ++++++++++++++
drivers/usb/host/Kconfig | 4 +-
drivers/usb/host/ehci-platform.c | 69 +++++++++++--------------------
drivers/usb/host/ohci-platform.c | 69 +++++++++++--------------------
drivers/video/fbdev/exynos/Kconfig | 2 +-
include/linux/phy/phy.h | 8 ++++
10 files changed, 100 insertions(+), 98 deletions(-)
--
2.3.4
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox