[PATCH 0/2] Hotplug fixes

[PATCH 0/2] Hotplug fixes

[Boris Ostrovsky]

Two patches fixing regressions introduced by new CPU hotplug framework

Boris Ostrovsky (2):
  xen/apic: Provide Xen-specific version of cpu_present_to_apicid APIC
    op
  xen/x86: Call cpu_startup_entry(CPUHP_AP_ONLINE_IDLE) from
    xen_play_dead()

 arch/x86/xen/apic.c |   12 ++++++++++--
 arch/x86/xen/smp.c  |    2 ++
 2 files changed, 12 insertions(+), 2 deletions(-)

[PATCH 1/2] xen/apic: Provide Xen-specific version of cpu_present_to_apicid APIC op

[Boris Ostrovsky]

Currently Xen uses default_cpu_present_to_apicid() which will always
report BAD_APICID for PV guests since x86_bios_cpu_apic_id is initialised
to that value and is never updated.

With commit 1f12e32f4cd5 ("x86/topology: Create logical package id"), this
op is now called by smp_init_package_map() when deciding whether to call
topology_update_package_map() which sets cpu_data(cpu).logical_proc_id.
The latter (as topology_logical_package_id(cpu)) may be used, for example,
by cpu_to_rapl_pmu() as an array index. Since uninitialized
logical_package_id is set to -1, the index will become 64K which is
obviously problematic.

While RAPL code (and any other users of logical_package_id) should be
careful in their assumptions about id's validity, Xen's
cpu_present_to_apicid op should still provide value consistent with its
own xen_apic_read(APIC_ID).

Signed-off-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
---
 arch/x86/xen/apic.c |   12 ++++++++++--
 1 files changed, 10 insertions(+), 2 deletions(-)

diff --git a/arch/x86/xen/apic.c b/arch/x86/xen/apic.c
index abf4901..db52a7f 100644
--- a/arch/x86/xen/apic.c
+++ b/arch/x86/xen/apic.c
@@ -66,7 +66,7 @@ static u32 xen_apic_read(u32 reg)
 
 	ret = HYPERVISOR_platform_op(&op);
 	if (ret)
-		return 0;
+		op.u.pcpu_info.apic_id = BAD_APICID;
 
 	return op.u.pcpu_info.apic_id << 24;
 }
@@ -142,6 +142,14 @@ static void xen_silent_inquire(int apicid)
 {
 }
 
+static int xen_cpu_present_to_apicid(int cpu)
+{
+	if (cpu_present(cpu))
+		return xen_get_apic_id(xen_apic_read(APIC_ID));
+	else
+		return BAD_APICID;
+}
+
 static struct apic xen_pv_apic = {
 	.name 				= "Xen PV",
 	.probe 				= xen_apic_probe_pv,
@@ -162,7 +170,7 @@ static struct apic xen_pv_apic = {
 
 	.ioapic_phys_id_map		= default_ioapic_phys_id_map, /* Used on 32-bit */
 	.setup_apic_routing		= NULL,
-	.cpu_present_to_apicid		= default_cpu_present_to_apicid,
+	.cpu_present_to_apicid		= xen_cpu_present_to_apicid,
 	.apicid_to_cpu_present		= physid_set_mask_of_physid, /* Used on 32-bit */
 	.check_phys_apicid_present	= default_check_phys_apicid_present, /* smp_sanity_check needs it */
 	.phys_pkg_id			= xen_phys_pkg_id, /* detect_ht */
-- 
1.7.1

Re: [PATCH 1/2] xen/apic: Provide Xen-specific version of cpu_present_to_apicid APIC op

[Konrad Rzeszutek Wilk]

On Thu, Mar 17, 2016 at 09:03:24AM -0400, Boris Ostrovsky wrote:
> Currently Xen uses default_cpu_present_to_apicid() which will always
> report BAD_APICID for PV guests since x86_bios_cpu_apic_id is initialised
> to that value and is never updated.
> 
> With commit 1f12e32f4cd5 ("x86/topology: Create logical package id"), this
> op is now called by smp_init_package_map() when deciding whether to call
> topology_update_package_map() which sets cpu_data(cpu).logical_proc_id.
> The latter (as topology_logical_package_id(cpu)) may be used, for example,
> by cpu_to_rapl_pmu() as an array index. Since uninitialized
> logical_package_id is set to -1, the index will become 64K which is
> obviously problematic.
> 
> While RAPL code (and any other users of logical_package_id) should be
> careful in their assumptions about id's validity, Xen's
> cpu_present_to_apicid op should still provide value consistent with its
> own xen_apic_read(APIC_ID).
> 
> Signed-off-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>

Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>

> ---
>  arch/x86/xen/apic.c |   12 ++++++++++--
>  1 files changed, 10 insertions(+), 2 deletions(-)
> 
> diff --git a/arch/x86/xen/apic.c b/arch/x86/xen/apic.c
> index abf4901..db52a7f 100644
> --- a/arch/x86/xen/apic.c
> +++ b/arch/x86/xen/apic.c
> @@ -66,7 +66,7 @@ static u32 xen_apic_read(u32 reg)
>  
>  	ret = HYPERVISOR_platform_op(&op);
>  	if (ret)
> -		return 0;
> +		op.u.pcpu_info.apic_id = BAD_APICID;
>  
>  	return op.u.pcpu_info.apic_id << 24;
>  }
> @@ -142,6 +142,14 @@ static void xen_silent_inquire(int apicid)
>  {
>  }
>  
> +static int xen_cpu_present_to_apicid(int cpu)
> +{
> +	if (cpu_present(cpu))
> +		return xen_get_apic_id(xen_apic_read(APIC_ID));
> +	else
> +		return BAD_APICID;
> +}
> +
>  static struct apic xen_pv_apic = {
>  	.name 				= "Xen PV",
>  	.probe 				= xen_apic_probe_pv,
> @@ -162,7 +170,7 @@ static struct apic xen_pv_apic = {
>  
>  	.ioapic_phys_id_map		= default_ioapic_phys_id_map, /* Used on 32-bit */
>  	.setup_apic_routing		= NULL,
> -	.cpu_present_to_apicid		= default_cpu_present_to_apicid,
> +	.cpu_present_to_apicid		= xen_cpu_present_to_apicid,
>  	.apicid_to_cpu_present		= physid_set_mask_of_physid, /* Used on 32-bit */
>  	.check_phys_apicid_present	= default_check_phys_apicid_present, /* smp_sanity_check needs it */
>  	.phys_pkg_id			= xen_phys_pkg_id, /* detect_ht */
> -- 
> 1.7.1
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH 1/2] xen/apic: Provide Xen-specific version of cpu_present_to_apicid APIC op

[Boris Ostrovsky]

Currently Xen uses default_cpu_present_to_apicid() which will always
report BAD_APICID for PV guests since x86_bios_cpu_apic_id is initialised
to that value and is never updated.

With commit 1f12e32f4cd5 ("x86/topology: Create logical package id"), this
op is now called by smp_init_package_map() when deciding whether to call
topology_update_package_map() which sets cpu_data(cpu).logical_proc_id.
The latter (as topology_logical_package_id(cpu)) may be used, for example,
by cpu_to_rapl_pmu() as an array index. Since uninitialized
logical_package_id is set to -1, the index will become 64K which is
obviously problematic.

While RAPL code (and any other users of logical_package_id) should be
careful in their assumptions about id's validity, Xen's
cpu_present_to_apicid op should still provide value consistent with its
own xen_apic_read(APIC_ID).

Signed-off-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
---
 arch/x86/xen/apic.c |   12 ++++++++++--
 1 files changed, 10 insertions(+), 2 deletions(-)

diff --git a/arch/x86/xen/apic.c b/arch/x86/xen/apic.c
index abf4901..db52a7f 100644
--- a/arch/x86/xen/apic.c
+++ b/arch/x86/xen/apic.c
@@ -66,7 +66,7 @@ static u32 xen_apic_read(u32 reg)
 
 	ret = HYPERVISOR_platform_op(&op);
 	if (ret)
-		return 0;
+		op.u.pcpu_info.apic_id = BAD_APICID;
 
 	return op.u.pcpu_info.apic_id << 24;
 }
@@ -142,6 +142,14 @@ static void xen_silent_inquire(int apicid)
 {
 }
 
+static int xen_cpu_present_to_apicid(int cpu)
+{
+	if (cpu_present(cpu))
+		return xen_get_apic_id(xen_apic_read(APIC_ID));
+	else
+		return BAD_APICID;
+}
+
 static struct apic xen_pv_apic = {
 	.name 				= "Xen PV",
 	.probe 				= xen_apic_probe_pv,
@@ -162,7 +170,7 @@ static struct apic xen_pv_apic = {
 
 	.ioapic_phys_id_map		= default_ioapic_phys_id_map, /* Used on 32-bit */
 	.setup_apic_routing		= NULL,
-	.cpu_present_to_apicid		= default_cpu_present_to_apicid,
+	.cpu_present_to_apicid		= xen_cpu_present_to_apicid,
 	.apicid_to_cpu_present		= physid_set_mask_of_physid, /* Used on 32-bit */
 	.check_phys_apicid_present	= default_check_phys_apicid_present, /* smp_sanity_check needs it */
 	.phys_pkg_id			= xen_phys_pkg_id, /* detect_ht */
-- 
1.7.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH 2/2] xen/x86: Call cpu_startup_entry(CPUHP_AP_ONLINE_IDLE) from xen_play_dead()

[Boris Ostrovsky]

This call has always been missing from xen_play dead() but until
recently this was rather benign. With new cpu hotplug framework
however this call is required, otherwise a hot-plugged CPU will not
be properly brough up (by never calling cpuhp_online_idle())

Signed-off-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
---
 arch/x86/xen/smp.c |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/arch/x86/xen/smp.c b/arch/x86/xen/smp.c
index 3c6d17f..719cf29 100644
--- a/arch/x86/xen/smp.c
+++ b/arch/x86/xen/smp.c
@@ -545,6 +545,8 @@ static void xen_play_dead(void) /* used only with HOTPLUG_CPU */
 	 * data back is to call:
 	 */
 	tick_nohz_idle_enter();
+
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 #else /* !CONFIG_HOTPLUG_CPU */
-- 
1.7.1

Re: [PATCH 2/2] xen/x86: Call cpu_startup_entry(CPUHP_AP_ONLINE_IDLE) from xen_play_dead()

[Konrad Rzeszutek Wilk]

On Thu, Mar 17, 2016 at 09:03:25AM -0400, Boris Ostrovsky wrote:
> This call has always been missing from xen_play dead() but until
> recently this was rather benign. With new cpu hotplug framework
> however this call is required, otherwise a hot-plugged CPU will not

Could you include the commit id of the 'new cpu hotplug' in case
anybody wants to backport this?

Thanks!
> be properly brough up (by never calling cpuhp_online_idle())
> 
> Signed-off-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
> ---
>  arch/x86/xen/smp.c |    2 ++
>  1 files changed, 2 insertions(+), 0 deletions(-)
> 
> diff --git a/arch/x86/xen/smp.c b/arch/x86/xen/smp.c
> index 3c6d17f..719cf29 100644
> --- a/arch/x86/xen/smp.c
> +++ b/arch/x86/xen/smp.c
> @@ -545,6 +545,8 @@ static void xen_play_dead(void) /* used only with HOTPLUG_CPU */
>  	 * data back is to call:
>  	 */
>  	tick_nohz_idle_enter();
> +
> +	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
>  }
>  
>  #else /* !CONFIG_HOTPLUG_CPU */
> -- 
> 1.7.1
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH 2/2] xen/x86: Call cpu_startup_entry(CPUHP_AP_ONLINE_IDLE) from xen_play_dead()

[Boris Ostrovsky]

On 03/25/2016 10:53 AM, Konrad Rzeszutek Wilk wrote:
> On Thu, Mar 17, 2016 at 09:03:25AM -0400, Boris Ostrovsky wrote:
>> This call has always been missing from xen_play dead() but until
>> recently this was rather benign. With new cpu hotplug framework
>> however this call is required, otherwise a hot-plugged CPU will not
> Could you include the commit id of the 'new cpu hotplug' in case
> anybody wants to backport this?

Sure.

It's commit 8df3e07e7f21 ("cpu/hotplug: Let upcoming cpu bring itself 
fully up").

Do you (or David) want me to re-send it?

-boris



>
> Thanks!
>> be properly brough up (by never calling cpuhp_online_idle())
>>
>> Signed-off-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
>> ---
>>   arch/x86/xen/smp.c |    2 ++
>>   1 files changed, 2 insertions(+), 0 deletions(-)
>>
>> diff --git a/arch/x86/xen/smp.c b/arch/x86/xen/smp.c
>> index 3c6d17f..719cf29 100644
>> --- a/arch/x86/xen/smp.c
>> +++ b/arch/x86/xen/smp.c
>> @@ -545,6 +545,8 @@ static void xen_play_dead(void) /* used only with HOTPLUG_CPU */
>>   	 * data back is to call:
>>   	 */
>>   	tick_nohz_idle_enter();
>> +
>> +	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
>>   }
>>   
>>   #else /* !CONFIG_HOTPLUG_CPU */
>> -- 
>> 1.7.1
>>

Re: [PATCH 2/2] xen/x86: Call cpu_startup_entry(CPUHP_AP_ONLINE_IDLE) from xen_play_dead()

[Boris Ostrovsky]

On 03/25/2016 10:53 AM, Konrad Rzeszutek Wilk wrote:
> On Thu, Mar 17, 2016 at 09:03:25AM -0400, Boris Ostrovsky wrote:
>> This call has always been missing from xen_play dead() but until
>> recently this was rather benign. With new cpu hotplug framework
>> however this call is required, otherwise a hot-plugged CPU will not
> Could you include the commit id of the 'new cpu hotplug' in case
> anybody wants to backport this?

Sure.

It's commit 8df3e07e7f21 ("cpu/hotplug: Let upcoming cpu bring itself 
fully up").

Do you (or David) want me to re-send it?

-boris



>
> Thanks!
>> be properly brough up (by never calling cpuhp_online_idle())
>>
>> Signed-off-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
>> ---
>>   arch/x86/xen/smp.c |    2 ++
>>   1 files changed, 2 insertions(+), 0 deletions(-)
>>
>> diff --git a/arch/x86/xen/smp.c b/arch/x86/xen/smp.c
>> index 3c6d17f..719cf29 100644
>> --- a/arch/x86/xen/smp.c
>> +++ b/arch/x86/xen/smp.c
>> @@ -545,6 +545,8 @@ static void xen_play_dead(void) /* used only with HOTPLUG_CPU */
>>   	 * data back is to call:
>>   	 */
>>   	tick_nohz_idle_enter();
>> +
>> +	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
>>   }
>>   
>>   #else /* !CONFIG_HOTPLUG_CPU */
>> -- 
>> 1.7.1
>>


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH 2/2] xen/x86: Call cpu_startup_entry(CPUHP_AP_ONLINE_IDLE) from xen_play_dead()

[Konrad Rzeszutek Wilk]

On Fri, Mar 25, 2016 at 11:08:32AM -0400, Boris Ostrovsky wrote:
> On 03/25/2016 10:53 AM, Konrad Rzeszutek Wilk wrote:
> >On Thu, Mar 17, 2016 at 09:03:25AM -0400, Boris Ostrovsky wrote:
> >>This call has always been missing from xen_play dead() but until
> >>recently this was rather benign. With new cpu hotplug framework
> >>however this call is required, otherwise a hot-plugged CPU will not
> >Could you include the commit id of the 'new cpu hotplug' in case
> >anybody wants to backport this?
> 
> Sure.
> 
> It's commit 8df3e07e7f21 ("cpu/hotplug: Let upcoming cpu bring itself fully
> up").
> 
> Do you (or David) want me to re-send it?

That is OK. I've updated the patch and committed both of them in for-linus-4.6.

Thanks!
> 
> -boris
> 
> 
> 
> >
> >Thanks!
> >>be properly brough up (by never calling cpuhp_online_idle())
> >>
> >>Signed-off-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
> >>---
> >>  arch/x86/xen/smp.c |    2 ++
> >>  1 files changed, 2 insertions(+), 0 deletions(-)
> >>
> >>diff --git a/arch/x86/xen/smp.c b/arch/x86/xen/smp.c
> >>index 3c6d17f..719cf29 100644
> >>--- a/arch/x86/xen/smp.c
> >>+++ b/arch/x86/xen/smp.c
> >>@@ -545,6 +545,8 @@ static void xen_play_dead(void) /* used only with HOTPLUG_CPU */
> >>  	 * data back is to call:
> >>  	 */
> >>  	tick_nohz_idle_enter();
> >>+
> >>+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
> >>  }
> >>  #else /* !CONFIG_HOTPLUG_CPU */
> >>-- 
> >>1.7.1
> >>
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH 2/2] xen/x86: Call cpu_startup_entry(CPUHP_AP_ONLINE_IDLE) from xen_play_dead()

[Boris Ostrovsky]

This call has always been missing from xen_play dead() but until
recently this was rather benign. With new cpu hotplug framework
however this call is required, otherwise a hot-plugged CPU will not
be properly brough up (by never calling cpuhp_online_idle())

Signed-off-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
---
 arch/x86/xen/smp.c |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/arch/x86/xen/smp.c b/arch/x86/xen/smp.c
index 3c6d17f..719cf29 100644
--- a/arch/x86/xen/smp.c
+++ b/arch/x86/xen/smp.c
@@ -545,6 +545,8 @@ static void xen_play_dead(void) /* used only with HOTPLUG_CPU */
 	 * data back is to call:
 	 */
 	tick_nohz_idle_enter();
+
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 #else /* !CONFIG_HOTPLUG_CPU */
-- 
1.7.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel