* [PATCH] arch/i386/pci/mmconfig.c tlb flush fix
@ 2006-12-23 1:00 OGAWA Hirofumi
0 siblings, 0 replies; 6+ messages in thread
From: OGAWA Hirofumi @ 2006-12-23 1:00 UTC (permalink / raw)
To: Linus Torvalds, Andrew Morton; +Cc: linux-kernel, OGAWA Hirofumi
We use the fixmap for accessing pci config space in pci_mmcfg_read/write().
The problem is in pci_exp_set_dev_base(). It is caching a last
accessed address to avoid calling set_fixmap_nocache() whenever
pci_mmcfg_read/write() is used.
static inline void pci_exp_set_dev_base(int bus, int devfn)
{
u32 dev_base = base | (bus << 20) | (devfn << 12);
if (dev_base != mmcfg_last_accessed_device) {
mmcfg_last_accessed_device = dev_base;
set_fixmap_nocache(FIX_PCIE_MCFG, dev_base);
}
}
cpu0 cpu1
---------------------------------------------------------------------------
pci_mmcfg_read("device-A")
pci_exp_set_dev_base()
set_fixmap_nocache()
pci_mmcfg_read("device-B")
pci_exp_set_dev_base()
set_fixmap_nocache()
pci_mmcfg_read("device-B")
pci_exp_set_dev_base()
/* doesn't flush tlb */
But if cpus accessed the above order, the second pci_mmcfg_read() on
cpu0 doesn't flush the TLB, because "mmcfg_last_accessed_device" is
device-B. So, second pci_mmcfg_read() on cpu0 accesses a device-A via
a previous TLB cache. This problem became the cause of several strange
behavior.
This patches fixes this situation by adds "mmcfg_last_accessed_cpu" check.
Signed-off-by: OGAWA Hirofumi <hogawa@miraclelinux.com>
Signed-off-by: OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
---
arch/i386/pci/mmconfig.c | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff -puN arch/i386/pci/mmconfig.c~i386-mmconfig-flush arch/i386/pci/mmconfig.c
--- linux-2.6/arch/i386/pci/mmconfig.c~i386-mmconfig-flush 2006-10-06 08:38:33.000000000 +0900
+++ linux-2.6-hirofumi/arch/i386/pci/mmconfig.c 2006-10-06 08:38:33.000000000 +0900
@@ -26,6 +26,7 @@
/* The base address of the last MMCONFIG device accessed */
static u32 mmcfg_last_accessed_device;
+static int mmcfg_last_accessed_cpu;
static DECLARE_BITMAP(fallback_slots, MAX_CHECK_BUS*32);
@@ -73,8 +74,11 @@ static u32 get_base_addr(unsigned int se
static void pci_exp_set_dev_base(unsigned int base, int bus, int devfn)
{
u32 dev_base = base | (bus << 20) | (devfn << 12);
- if (dev_base != mmcfg_last_accessed_device) {
+ int cpu = smp_processor_id();
+ if (dev_base != mmcfg_last_accessed_device ||
+ cpu != mmcfg_last_accessed_cpu) {
mmcfg_last_accessed_device = dev_base;
+ mmcfg_last_accessed_cpu = cpu;
set_fixmap_nocache(FIX_PCIE_MCFG, dev_base);
}
}
_
--
OGAWA Hirofumi <hogawa@miraclelinux.com>
^ permalink raw reply [flat|nested] 6+ messages in thread* [PATCH] arch/i386/pci/mmconfig.c tlb flush fix
@ 2006-09-26 9:28 OGAWA Hirofumi
2006-09-26 12:22 ` Andi Kleen
0 siblings, 1 reply; 6+ messages in thread
From: OGAWA Hirofumi @ 2006-09-26 9:28 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-kernel, OGAWA Hirofumi
Hi,
We use the fixmap for accessing pci config space in pci_mmcfg_read/write().
The problem is in pci_exp_set_dev_base(). It is caching a last
accessed address to avoid calling set_fixmap_nocache() whenever
pci_mmcfg_read/write() is used.
static inline void pci_exp_set_dev_base(int bus, int devfn)
{
u32 dev_base = base | (bus << 20) | (devfn << 12);
if (dev_base != mmcfg_last_accessed_device) {
mmcfg_last_accessed_device = dev_base;
set_fixmap_nocache(FIX_PCIE_MCFG, dev_base);
}
}
cpu0 cpu1
---------------------------------------------------------------------------
pci_mmcfg_read("device-A")
pci_exp_set_dev_base()
set_fixmap_nocache()
pci_mmcfg_read("device-B")
pci_exp_set_dev_base()
set_fixmap_nocache()
pci_mmcfg_read("device-B")
pci_exp_set_dev_base()
/* doesn't flush tlb */
But if cpus accessed the above order, the second pci_mmcfg_read() on
cpu0 doesn't flush the TLB, because "mmcfg_last_accessed_device" is
device-B. So, second pci_mmcfg_read() on cpu0 accesses a device-A via
a previous TLB cache.
This patches fixes this situation by adds "mmcfg_last_accessed_cpu" check.
Please apply.
Signed-off-by: OGAWA Hirofumi <hogawa@miraclelinux.com>
---
arch/i386/pci/mmconfig.c | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff -puN arch/i386/pci/mmconfig.c~mmconfig-tlb-race-fix arch/i386/pci/mmconfig.c
--- linux-2.6/arch/i386/pci/mmconfig.c~mmconfig-tlb-race-fix 2006-09-26 11:35:54.000000000 +0900
+++ linux-2.6-hirofumi/arch/i386/pci/mmconfig.c 2006-09-26 11:36:54.000000000 +0900
@@ -26,6 +26,7 @@
/* The base address of the last MMCONFIG device accessed */
static u32 mmcfg_last_accessed_device;
+static int mmcfg_last_accessed_cpu;
static DECLARE_BITMAP(fallback_slots, MAX_CHECK_BUS*32);
@@ -70,8 +71,11 @@ static u32 get_base_addr(unsigned int se
static inline void pci_exp_set_dev_base(unsigned int base, int bus, int devfn)
{
u32 dev_base = base | (bus << 20) | (devfn << 12);
- if (dev_base != mmcfg_last_accessed_device) {
+ int cpu = smp_processor_id();
+ if (dev_base != mmcfg_last_accessed_device ||
+ cpu != mmcfg_last_accessed_cpu) {
mmcfg_last_accessed_device = dev_base;
+ mmcfg_last_accessed_cpu = cpu;
set_fixmap_nocache(FIX_PCIE_MCFG, dev_base);
}
}
_
--
OGAWA Hirofumi <hogawa@miraclelinux.com>
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [PATCH] arch/i386/pci/mmconfig.c tlb flush fix
2006-09-26 9:28 OGAWA Hirofumi
@ 2006-09-26 12:22 ` Andi Kleen
2006-09-26 15:45 ` OGAWA Hirofumi
0 siblings, 1 reply; 6+ messages in thread
From: Andi Kleen @ 2006-09-26 12:22 UTC (permalink / raw)
To: OGAWA Hirofumi; +Cc: linux-kernel, OGAWA Hirofumi
OGAWA Hirofumi <hogawa@miraclelinux.com> writes:
> Hi,
>
> We use the fixmap for accessing pci config space in pci_mmcfg_read/write().
> The problem is in pci_exp_set_dev_base(). It is caching a last
> accessed address to avoid calling set_fixmap_nocache() whenever
> pci_mmcfg_read/write() is used.
Good catch. I already had another report of mmconfig corruption on i386,
but didn't have time to look at it yet.
Will be definitely stable material once it hit mainline.
-Andi
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] arch/i386/pci/mmconfig.c tlb flush fix
2006-09-26 12:22 ` Andi Kleen
@ 2006-09-26 15:45 ` OGAWA Hirofumi
2006-09-26 15:47 ` Andi Kleen
0 siblings, 1 reply; 6+ messages in thread
From: OGAWA Hirofumi @ 2006-09-26 15:45 UTC (permalink / raw)
To: Greg KH, Andi Kleen; +Cc: OGAWA Hirofumi, linux-kernel
Andi Kleen <ak@suse.de> writes:
> OGAWA Hirofumi <hogawa@miraclelinux.com> writes:
>>
>> We use the fixmap for accessing pci config space in pci_mmcfg_read/write().
>> The problem is in pci_exp_set_dev_base(). It is caching a last
>> accessed address to avoid calling set_fixmap_nocache() whenever
>> pci_mmcfg_read/write() is used.
>
>
> Good catch. I already had another report of mmconfig corruption on i386,
> but didn't have time to look at it yet.
>
> Will be definitely stable material once it hit mainline.
Indeed.
If I found this in mainline, should I send something like a reminder
email with patch to you, Greg?
--
OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] arch/i386/pci/mmconfig.c tlb flush fix
2006-09-26 15:45 ` OGAWA Hirofumi
@ 2006-09-26 15:47 ` Andi Kleen
2006-09-26 16:31 ` OGAWA Hirofumi
0 siblings, 1 reply; 6+ messages in thread
From: Andi Kleen @ 2006-09-26 15:47 UTC (permalink / raw)
To: OGAWA Hirofumi; +Cc: Greg KH, OGAWA Hirofumi, linux-kernel
On Tuesday 26 September 2006 17:45, OGAWA Hirofumi wrote:
> Andi Kleen <ak@suse.de> writes:
>
> > OGAWA Hirofumi <hogawa@miraclelinux.com> writes:
> >>
> >> We use the fixmap for accessing pci config space in pci_mmcfg_read/write().
> >> The problem is in pci_exp_set_dev_base(). It is caching a last
> >> accessed address to avoid calling set_fixmap_nocache() whenever
> >> pci_mmcfg_read/write() is used.
> >
> >
> > Good catch. I already had another report of mmconfig corruption on i386,
> > but didn't have time to look at it yet.
> >
> > Will be definitely stable material once it hit mainline.
>
> Indeed.
>
> If I found this in mainline, should I send something like a reminder
> email with patch to you, Greg?
I have it already queued for both.
-Andi
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] arch/i386/pci/mmconfig.c tlb flush fix
2006-09-26 15:47 ` Andi Kleen
@ 2006-09-26 16:31 ` OGAWA Hirofumi
0 siblings, 0 replies; 6+ messages in thread
From: OGAWA Hirofumi @ 2006-09-26 16:31 UTC (permalink / raw)
To: Andi Kleen; +Cc: Greg KH, OGAWA Hirofumi, linux-kernel
Andi Kleen <ak@suse.de> writes:
> On Tuesday 26 September 2006 17:45, OGAWA Hirofumi wrote:
>> Andi Kleen <ak@suse.de> writes:
>>
>> > OGAWA Hirofumi <hogawa@miraclelinux.com> writes:
>> >>
>> >> We use the fixmap for accessing pci config space in pci_mmcfg_read/write().
>> >> The problem is in pci_exp_set_dev_base(). It is caching a last
>> >> accessed address to avoid calling set_fixmap_nocache() whenever
>> >> pci_mmcfg_read/write() is used.
>> >
>> >
>> > Good catch. I already had another report of mmconfig corruption on i386,
>> > but didn't have time to look at it yet.
>> >
>> > Will be definitely stable material once it hit mainline.
>>
>> Indeed.
>>
>> If I found this in mainline, should I send something like a reminder
>> email with patch to you, Greg?
>
> I have it already queued for both.
Oh, thanks.
--
OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2006-12-23 1:20 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2006-12-23 1:00 [PATCH] arch/i386/pci/mmconfig.c tlb flush fix OGAWA Hirofumi
-- strict thread matches above, loose matches on Subject: below --
2006-09-26 9:28 OGAWA Hirofumi
2006-09-26 12:22 ` Andi Kleen
2006-09-26 15:45 ` OGAWA Hirofumi
2006-09-26 15:47 ` Andi Kleen
2006-09-26 16:31 ` OGAWA Hirofumi
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox