* [PATCH v5 1/3] x86,fs/resctrl: Add resctrl_arch_preconvert_bw()
2026-07-09 9:31 [PATCH v5 0/3] x86,fs/resctrl,arm_mpam: Factor MBA parse-time conversion to be per-arch Ben Horgan
@ 2026-07-09 9:31 ` Ben Horgan
2026-07-09 9:31 ` [PATCH v5 2/3] arm_mpam: resctrl: Add pass-through resctrl_arch_preconvert_bw() Ben Horgan
` (3 subsequent siblings)
4 siblings, 0 replies; 9+ messages in thread
From: Ben Horgan @ 2026-07-09 9:31 UTC (permalink / raw)
To: ben.horgan
Cc: james.morse, reinette.chatre, fenghuay, linux-kernel,
linux-arm-kernel, tglx, mingo, bp, dave.hansen, hpa, corbet, x86,
linux-doc, dave.martin
From: Dave Martin <dave.martin@arm.com>
On MPAM systems the rounding behaviour of the MBA control would be improved
if the rounding in the fs/resctrl code is removed but this is not the
case for x86. To allow any rounding or conversion of the bandwidth value
provided by the user to be specified by the arch code a new arch hook is
required.
Introduce resctrl_arch_preconvert_bw(), and add its x86 implementation.
This is currently unused in resctrl but when plumbed in it will replace the
call to roundup() in bw_validate().
Signed-off-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Ben Horgan <ben.horgan@arm.com>
Reviewed-by: Reinette Chatre <reinette.chatre@intel.com>
---
Changes since Dave's v2:
Split from larger patch and add commit message
Update kernel-doc (Reinette)
Changes since v3:
Swap parameter order (Reinette)
Change summary prefix include fs/resctrl (Reinette)
val -> @val
Add Reinette's R-b
Changes since v4:
Set author to Dave as he wrote the patch
---
arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 6 ++++++
include/linux/resctrl.h | 19 +++++++++++++++++++
2 files changed, 25 insertions(+)
diff --git a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
index b20e705606b8..81a08526d3d1 100644
--- a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
+++ b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
@@ -16,9 +16,15 @@
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
#include <linux/cpu.h>
+#include <linux/math.h>
#include "internal.h"
+u32 resctrl_arch_preconvert_bw(const struct rdt_resource *r, u32 val)
+{
+ return roundup(val, (unsigned long)r->membw.bw_gran);
+}
+
int resctrl_arch_update_one(struct rdt_resource *r, struct rdt_ctrl_domain *d,
u32 closid, enum resctrl_conf_type t, u32 cfg_val)
{
diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
index 73ff522448a0..1ebd4b90043a 100644
--- a/include/linux/resctrl.h
+++ b/include/linux/resctrl.h
@@ -504,6 +504,25 @@ bool resctrl_arch_mbm_cntr_assign_enabled(struct rdt_resource *r);
*/
int resctrl_arch_mbm_cntr_assign_set(struct rdt_resource *r, bool enable);
+/**
+ * resctrl_arch_preconvert_bw() - Prepare bandwidth control value for arch use.
+ * @r: Resource whose schema was written.
+ * @val: Bandwidth control value written to the schemata file by userspace.
+ *
+ * Convert the user provided bandwidth control value to an appropriate form for
+ * consumption by the hardware driver for resource @r. Converted value is stored
+ * in rdt_ctrl_domain::staged_config[] for later consumption by
+ * resctrl_arch_update_domains(). Is not called when MBA software controller is
+ * enabled.
+ *
+ * Architectures for which this pre-conversion hook is not useful should supply
+ * an implementation of this function that just returns @val unmodified.
+ *
+ * Return:
+ * The converted value.
+ */
+u32 resctrl_arch_preconvert_bw(const struct rdt_resource *r, u32 val);
+
/*
* Update the ctrl_val and apply this config right now.
* Must be called on one of the domain's CPUs.
--
2.43.0
^ permalink raw reply related [flat|nested] 9+ messages in thread* [PATCH v5 2/3] arm_mpam: resctrl: Add pass-through resctrl_arch_preconvert_bw()
2026-07-09 9:31 [PATCH v5 0/3] x86,fs/resctrl,arm_mpam: Factor MBA parse-time conversion to be per-arch Ben Horgan
2026-07-09 9:31 ` [PATCH v5 1/3] x86,fs/resctrl: Add resctrl_arch_preconvert_bw() Ben Horgan
@ 2026-07-09 9:31 ` Ben Horgan
2026-07-09 9:31 ` [PATCH v5 3/3] fs/resctrl: Factor MBA parse-time conversion to be per-arch Ben Horgan
` (2 subsequent siblings)
4 siblings, 0 replies; 9+ messages in thread
From: Ben Horgan @ 2026-07-09 9:31 UTC (permalink / raw)
To: ben.horgan
Cc: james.morse, reinette.chatre, fenghuay, linux-kernel,
linux-arm-kernel, tglx, mingo, bp, dave.hansen, hpa, corbet, x86,
linux-doc, dave.martin
resctrl rounds up the percentage value of the MBA based on the bw_gran. As
MPAM uses a binary fixed point fraction format for MBA rather than a
decimal percentage, this introduces rounding errors.
Without this additional rounding, if the user reads the value in an MB
schema and then writes it back to the schema, the value in hardware won't
change. However, with this additional rounding, this guarantee is broken
for systems with mbw_wd < 7.
resctrl is introducing resctrl_arch_preconvert_bw() to allow the arch code
to specify the conversion resctrl does to the user-provided bandwidth
value. Add the MPAM version of resctrl_arch_preconvert_bw(). This does no
conversion.
Signed-off-by: Ben Horgan <ben.horgan@arm.com>
Reviewed-by: Reinette Chatre <reinette.chatre@intel.com>
---
Changes since v3:
Parameter order switch (Reinette)
Add Reinette's R-b
Changes since v4:
Add missing tearoff divider, ---
---
drivers/resctrl/mpam_resctrl.c | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/drivers/resctrl/mpam_resctrl.c b/drivers/resctrl/mpam_resctrl.c
index 226ff6f532fa..e9dea8c40265 100644
--- a/drivers/resctrl/mpam_resctrl.c
+++ b/drivers/resctrl/mpam_resctrl.c
@@ -167,6 +167,11 @@ bool resctrl_arch_get_cdp_enabled(enum resctrl_res_level rid)
return mpam_resctrl_controls[rid].cdp_enabled;
}
+u32 resctrl_arch_preconvert_bw(const struct rdt_resource *r, u32 val)
+{
+ return val;
+}
+
/**
* resctrl_reset_task_closids() - Reset the PARTID/PMG values for all tasks.
*
--
2.43.0
^ permalink raw reply related [flat|nested] 9+ messages in thread* [PATCH v5 3/3] fs/resctrl: Factor MBA parse-time conversion to be per-arch
2026-07-09 9:31 [PATCH v5 0/3] x86,fs/resctrl,arm_mpam: Factor MBA parse-time conversion to be per-arch Ben Horgan
2026-07-09 9:31 ` [PATCH v5 1/3] x86,fs/resctrl: Add resctrl_arch_preconvert_bw() Ben Horgan
2026-07-09 9:31 ` [PATCH v5 2/3] arm_mpam: resctrl: Add pass-through resctrl_arch_preconvert_bw() Ben Horgan
@ 2026-07-09 9:31 ` Ben Horgan
2026-07-14 17:45 ` [PATCH v5 0/3] x86,fs/resctrl,arm_mpam: " Reinette Chatre
2026-07-15 0:13 ` Gavin Shan
4 siblings, 0 replies; 9+ messages in thread
From: Ben Horgan @ 2026-07-09 9:31 UTC (permalink / raw)
To: ben.horgan
Cc: james.morse, reinette.chatre, fenghuay, linux-kernel,
linux-arm-kernel, tglx, mingo, bp, dave.hansen, hpa, corbet, x86,
linux-doc, dave.martin, Dave Martin, Ben Horgan
From: Dave Martin <dave.martin@arm.com>
The control value parser for the MB resource currently coerces the memory
bandwidth percentage value from userspace to be an exact multiple of the
rdt_resource::resctrl_membw::bw_gran parameter.
On MPAM systems, this results in somewhat worse-than-worst-case rounding, since
the bandwidth granularity advertised to resctrl by the MPAM driver is in general
only an approximation to the actual hardware granularity on these systems, and
the hardware bandwidth allocation control value is not natively a percentage --
necessitating a further conversion in the resctrl_arch_update_domains() path,
regardless of the conversion done at parse time.
For MPAM and x86 use their custom pre-prepared parse-time conversion,
resctrl_arch_preconvert_bw(). This will avoid accumulated error from rounding
the value twice on MPAM systems. For x86 systems there is no functional change.
Clarify the documentation, but avoid overly exact promises.
Clamping to bw_min and bw_max still feels generic: leave it in the core code,
for now.
[ BH: Split out x86 specific changes ]
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Signed-off-by: Ben Horgan <Ben.Horgan@arm.com>
Reviewed-by: Ben Horgan <ben.horgan@arm.com>
Reviewed-by: Reinette Chatre <reinette.chatre@intel.com>
---
Changes since v3:
Parameter order swap (Reinette)
Reflow commit message to use 80 characters (Reinette)
Adjust comment line length (Reinette)
Add Reinette's R-b
Changes since v4:
Make Dave the author as he wrote the patch
---
Documentation/filesystems/resctrl.rst | 17 +++++++++--------
fs/resctrl/ctrlmondata.c | 6 +++---
2 files changed, 12 insertions(+), 11 deletions(-)
diff --git a/Documentation/filesystems/resctrl.rst b/Documentation/filesystems/resctrl.rst
index e4b66af55ffb..e4448c20c72e 100644
--- a/Documentation/filesystems/resctrl.rst
+++ b/Documentation/filesystems/resctrl.rst
@@ -236,12 +236,11 @@ with respect to allocation:
user can request.
"bandwidth_gran":
- The granularity in which the memory bandwidth
- percentage is allocated. The allocated
- b/w percentage is rounded off to the next
- control step available on the hardware. The
- available bandwidth control steps are:
- min_bandwidth + N * bandwidth_gran.
+ The approximate granularity in which the memory bandwidth
+ percentage is allocated. The allocated bandwidth percentage
+ is rounded up to the next control step available on the
+ hardware. The available hardware steps are no larger than
+ this value.
"delay_linear":
Indicates if the delay scale is linear or
@@ -881,8 +880,10 @@ The minimum bandwidth percentage value for each cpu model is predefined
and can be looked up through "info/MB/min_bandwidth". The bandwidth
granularity that is allocated is also dependent on the cpu model and can
be looked up at "info/MB/bandwidth_gran". The available bandwidth
-control steps are: min_bw + N * bw_gran. Intermediate values are rounded
-to the next control step available on the hardware.
+control steps are, approximately, min_bw + N * bw_gran. The steps may
+appear irregular due to rounding to an exact percentage: bw_gran is the
+maximum interval between the percentage values corresponding to any two
+adjacent steps in the hardware.
The bandwidth throttling is a core specific mechanism on some of Intel
SKUs. Using a high bandwidth and a low bandwidth setting on two threads
diff --git a/fs/resctrl/ctrlmondata.c b/fs/resctrl/ctrlmondata.c
index 9a7dfc48cb2e..62c9c6b24d54 100644
--- a/fs/resctrl/ctrlmondata.c
+++ b/fs/resctrl/ctrlmondata.c
@@ -37,8 +37,8 @@ typedef int (ctrlval_parser_t)(struct rdt_parse_data *data,
/*
* Check whether MBA bandwidth percentage value is correct. The value is
* checked against the minimum and max bandwidth values specified by the
- * hardware. The allocated bandwidth percentage is rounded to the next
- * control step available on the hardware.
+ * hardware. The allocated bandwidth percentage is converted as appropriate
+ * for consumption by the specific hardware driver.
*/
static bool bw_validate(char *buf, u32 *data, struct rdt_resource *r)
{
@@ -71,7 +71,7 @@ static bool bw_validate(char *buf, u32 *data, struct rdt_resource *r)
return false;
}
- *data = roundup(bw, (unsigned long)r->membw.bw_gran);
+ *data = resctrl_arch_preconvert_bw(r, bw);
return true;
}
--
2.43.0
^ permalink raw reply related [flat|nested] 9+ messages in thread* Re: [PATCH v5 0/3] x86,fs/resctrl,arm_mpam: Factor MBA parse-time conversion to be per-arch
2026-07-09 9:31 [PATCH v5 0/3] x86,fs/resctrl,arm_mpam: Factor MBA parse-time conversion to be per-arch Ben Horgan
` (2 preceding siblings ...)
2026-07-09 9:31 ` [PATCH v5 3/3] fs/resctrl: Factor MBA parse-time conversion to be per-arch Ben Horgan
@ 2026-07-14 17:45 ` Reinette Chatre
2026-07-14 22:48 ` Moger, Babu
2026-07-15 0:13 ` Gavin Shan
4 siblings, 1 reply; 9+ messages in thread
From: Reinette Chatre @ 2026-07-14 17:45 UTC (permalink / raw)
To: Ben Horgan, bp, x86
Cc: james.morse, fenghuay, linux-kernel, linux-arm-kernel, tglx,
mingo, dave.hansen, hpa, corbet, linux-doc, dave.martin
Dear x86 maintainers,
Could you please consider this series for inclusion? It applies cleanly on top
of x86/cache with HEAD at
2566b5cd6a27 ("fs/resctrl: Fix UAF from worker threads when domains are removed")
Please note that this is the first instance of a series that touches resctrl fs, x86, and
Arm. Since this is a resctrl fs API change these patches should stay together. You will
find in Ben's message below that patch routing via tip is supported by Arm.
Thank you very much
Reinette
On 7/9/26 2:31 AM, Ben Horgan wrote:
> This version fixes a couple of non-functional mistakes in v4 pointed out by Reinette.
>
> The patches should stay together so please could this all go via tip.
>
> Changelogs in patches.
>
> From cover letter of v3:
>
> This is a new version of Dave Martin's patch [1] to delegate rounding of
> bandwidth control user values to the arch code. As there is now more than one
> architecture using resctrl, I split the original patch into two, a core resctrl
> patch and an x86 patch, and added an MPAM patch. Please let me know if the patch
> break down and ordering is sensible and whether the pattern should be followed
> for any future similar changes.
>
> This does have a user visible effect on MB schema when using MPAM hardware
> with 'bandwidth_gran' greater than 1. I'm not sure if MPAM hardware with such
> coarse controls exists in the wild but it is spec compliant and I've tested it
> on a model.
>
> [1] https://lore.kernel.org/lkml/20251031154225.14799-1-Dave.Martin@arm.com/
>
> v3: https://lore.kernel.org/lkml/20260515140612.1205251-1-ben.horgan@arm.com/
> v4: https://lore.kernel.org/lkml/20260706160639.2136674-1-ben.horgan@arm.com/
>
> Based on v7.2-rc2
>
> Ben Horgan (1):
> arm_mpam: resctrl: Add pass-through resctrl_arch_preconvert_bw()
>
> Dave Martin (2):
> x86,fs/resctrl: Add resctrl_arch_preconvert_bw()
> fs/resctrl: Factor MBA parse-time conversion to be per-arch
>
> Documentation/filesystems/resctrl.rst | 17 +++++++++--------
> arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 6 ++++++
> drivers/resctrl/mpam_resctrl.c | 5 +++++
> fs/resctrl/ctrlmondata.c | 6 +++---
> include/linux/resctrl.h | 19 +++++++++++++++++++
> 5 files changed, 42 insertions(+), 11 deletions(-)
>
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: [PATCH v5 0/3] x86,fs/resctrl,arm_mpam: Factor MBA parse-time conversion to be per-arch
2026-07-14 17:45 ` [PATCH v5 0/3] x86,fs/resctrl,arm_mpam: " Reinette Chatre
@ 2026-07-14 22:48 ` Moger, Babu
2026-07-14 23:59 ` Reinette Chatre
0 siblings, 1 reply; 9+ messages in thread
From: Moger, Babu @ 2026-07-14 22:48 UTC (permalink / raw)
To: Reinette Chatre, Ben Horgan, bp, x86
Cc: james.morse, fenghuay, linux-kernel, linux-arm-kernel, tglx,
mingo, dave.hansen, hpa, corbet, linux-doc, dave.martin
Ran few tests on the series. Everything looks good.
Tested-by: Babu Moger <babu.moger@amd.com>
On 7/14/2026 12:45 PM, Reinette Chatre wrote:
> Dear x86 maintainers,
>
> Could you please consider this series for inclusion? It applies cleanly on top
> of x86/cache with HEAD at
> 2566b5cd6a27 ("fs/resctrl: Fix UAF from worker threads when domains are removed")
>
> Please note that this is the first instance of a series that touches resctrl fs, x86, and
> Arm. Since this is a resctrl fs API change these patches should stay together. You will
> find in Ben's message below that patch routing via tip is supported by Arm.
>
> Thank you very much
>
> Reinette
>
> On 7/9/26 2:31 AM, Ben Horgan wrote:
>> This version fixes a couple of non-functional mistakes in v4 pointed out by Reinette.
>>
>> The patches should stay together so please could this all go via tip.
>>
>> Changelogs in patches.
>>
>> From cover letter of v3:
>>
>> This is a new version of Dave Martin's patch [1] to delegate rounding of
>> bandwidth control user values to the arch code. As there is now more than one
>> architecture using resctrl, I split the original patch into two, a core resctrl
>> patch and an x86 patch, and added an MPAM patch. Please let me know if the patch
>> break down and ordering is sensible and whether the pattern should be followed
>> for any future similar changes.
>>
>> This does have a user visible effect on MB schema when using MPAM hardware
>> with 'bandwidth_gran' greater than 1. I'm not sure if MPAM hardware with such
>> coarse controls exists in the wild but it is spec compliant and I've tested it
>> on a model.
>>
>> [1] https://lore.kernel.org/lkml/20251031154225.14799-1-Dave.Martin@arm.com/
>>
>> v3: https://lore.kernel.org/lkml/20260515140612.1205251-1-ben.horgan@arm.com/
>> v4: https://lore.kernel.org/lkml/20260706160639.2136674-1-ben.horgan@arm.com/
>>
>> Based on v7.2-rc2
>>
>> Ben Horgan (1):
>> arm_mpam: resctrl: Add pass-through resctrl_arch_preconvert_bw()
>>
>> Dave Martin (2):
>> x86,fs/resctrl: Add resctrl_arch_preconvert_bw()
>> fs/resctrl: Factor MBA parse-time conversion to be per-arch
>>
>> Documentation/filesystems/resctrl.rst | 17 +++++++++--------
>> arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 6 ++++++
>> drivers/resctrl/mpam_resctrl.c | 5 +++++
>> fs/resctrl/ctrlmondata.c | 6 +++---
>> include/linux/resctrl.h | 19 +++++++++++++++++++
>> 5 files changed, 42 insertions(+), 11 deletions(-)
>>
>
>
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: [PATCH v5 0/3] x86,fs/resctrl,arm_mpam: Factor MBA parse-time conversion to be per-arch
2026-07-14 22:48 ` Moger, Babu
@ 2026-07-14 23:59 ` Reinette Chatre
0 siblings, 0 replies; 9+ messages in thread
From: Reinette Chatre @ 2026-07-14 23:59 UTC (permalink / raw)
To: Moger, Babu, Ben Horgan, bp, x86
Cc: james.morse, fenghuay, linux-kernel, linux-arm-kernel, tglx,
mingo, dave.hansen, hpa, corbet, linux-doc, dave.martin
On 7/14/26 3:48 PM, Moger, Babu wrote:
> Ran few tests on the series. Everything looks good.
>
> Tested-by: Babu Moger <babu.moger@amd.com>
>
Thank you very much Babu.
Reinette
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH v5 0/3] x86,fs/resctrl,arm_mpam: Factor MBA parse-time conversion to be per-arch
2026-07-09 9:31 [PATCH v5 0/3] x86,fs/resctrl,arm_mpam: Factor MBA parse-time conversion to be per-arch Ben Horgan
` (3 preceding siblings ...)
2026-07-14 17:45 ` [PATCH v5 0/3] x86,fs/resctrl,arm_mpam: " Reinette Chatre
@ 2026-07-15 0:13 ` Gavin Shan
2026-07-15 16:02 ` Reinette Chatre
4 siblings, 1 reply; 9+ messages in thread
From: Gavin Shan @ 2026-07-15 0:13 UTC (permalink / raw)
To: Ben Horgan
Cc: james.morse, reinette.chatre, fenghuay, linux-kernel,
linux-arm-kernel, tglx, mingo, bp, dave.hansen, hpa, corbet, x86,
linux-doc, dave.martin
On 7/9/26 7:31 PM, Ben Horgan wrote:
> This version fixes a couple of non-functional mistakes in v4 pointed out by Reinette.
>
> The patches should stay together so please could this all go via tip.
>
> Changelogs in patches.
>
> From cover letter of v3:
>
> This is a new version of Dave Martin's patch [1] to delegate rounding of
> bandwidth control user values to the arch code. As there is now more than one
> architecture using resctrl, I split the original patch into two, a core resctrl
> patch and an x86 patch, and added an MPAM patch. Please let me know if the patch
> break down and ordering is sensible and whether the pattern should be followed
> for any future similar changes.
>
> This does have a user visible effect on MB schema when using MPAM hardware
> with 'bandwidth_gran' greater than 1. I'm not sure if MPAM hardware with such
> coarse controls exists in the wild but it is spec compliant and I've tested it
> on a model.
>
> [1] https://lore.kernel.org/lkml/20251031154225.14799-1-Dave.Martin@arm.com/
>
> v3: https://lore.kernel.org/lkml/20260515140612.1205251-1-ben.horgan@arm.com/
> v4: https://lore.kernel.org/lkml/20260706160639.2136674-1-ben.horgan@arm.com/
>
> Based on v7.2-rc2
>
> Ben Horgan (1):
> arm_mpam: resctrl: Add pass-through resctrl_arch_preconvert_bw()
>
> Dave Martin (2):
> x86,fs/resctrl: Add resctrl_arch_preconvert_bw()
> fs/resctrl: Factor MBA parse-time conversion to be per-arch
>
> Documentation/filesystems/resctrl.rst | 17 +++++++++--------
> arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 6 ++++++
> drivers/resctrl/mpam_resctrl.c | 5 +++++
> fs/resctrl/ctrlmondata.c | 6 +++---
> include/linux/resctrl.h | 19 +++++++++++++++++++
> 5 files changed, 42 insertions(+), 11 deletions(-)
>
Looks good in my tests on NVidia's grace-hopper machine. No errors found
from the kunit-tests and the MBW limiting works with more precise granularity.
The code changes look good to me either.
Tested-by: Gavin Shan <gshan@redhat.com>
Reviewed-by: Gavin Shan <gshan@redhat.com>
Thanks,
Gavin
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: [PATCH v5 0/3] x86,fs/resctrl,arm_mpam: Factor MBA parse-time conversion to be per-arch
2026-07-15 0:13 ` Gavin Shan
@ 2026-07-15 16:02 ` Reinette Chatre
0 siblings, 0 replies; 9+ messages in thread
From: Reinette Chatre @ 2026-07-15 16:02 UTC (permalink / raw)
To: Gavin Shan, Ben Horgan
Cc: james.morse, fenghuay, linux-kernel, linux-arm-kernel, tglx,
mingo, bp, dave.hansen, hpa, corbet, x86, linux-doc, dave.martin
On 7/14/26 5:13 PM, Gavin Shan wrote:
> Looks good in my tests on NVidia's grace-hopper machine. No errors found
> from the kunit-tests and the MBW limiting works with more precise granularity.
> The code changes look good to me either.
>
> Tested-by: Gavin Shan <gshan@redhat.com>
> Reviewed-by: Gavin Shan <gshan@redhat.com>
Thank you very much Gavin.
Reinette
^ permalink raw reply [flat|nested] 9+ messages in thread