cpu_down() but no cpu_up() in drivers/xen/cpu_hotplug.c ?

cpu_down() but no cpu_up() in drivers/xen/cpu_hotplug.c ?

[Jan Beulich]

Jeremy,

how is pv-ops bringing up CPUs set to online in xenstore? Is this not
an automatic process (i.e. requires writing to respective online node
in sysfs), like in the traditional tree?

Thanks, Jan

Re: cpu_down() but no cpu_up() in drivers/xen/cpu_hotplug.c ?

[Ian Campbell]

On Tue, 2010-05-11 at 10:34 +0100, Jan Beulich wrote:
> Jeremy,
> 
> how is pv-ops bringing up CPUs set to online in xenstore? Is this not
> an automatic process (i.e. requires writing to respective online node
> in sysfs), like in the traditional tree?

The original commit which added CPU hotplug to pvops says:

    xen: implement CPU hotplugging
    
    Note the changes from 2.6.18-xen CPU hotplugging:
    
    A vcpu_down request from the remote admin via Xenbus both hotunplugs the
    CPU, and disables it by removing it from the cpu_present map, and removing
    its entry in /sys.
    
    A vcpu_up request from the remote admin only re-enables the CPU, and does
    not immediately bring the CPU up. A udev event is emitted, which can be
    caught by the user if he wishes to automatically re-up CPUs when available,
    or implement a more complex policy.
    
    Signed-off-by: Alex Nixon <alex.nixon@citrix.com>
    Acked-by: Jeremy Fitzhardinge <jeremy@goop.org>
    Signed-off-by: Ingo Molnar <mingo@elte.hu>

I'm not sure how the decision was reached to implement it this way,
perhaps for consistency with CPU hotplug on other
platforms/architectures?

FWIW I use a udev rule to bring up CPUs as they are added, which is
equivalent to the old behaviour:

        ACTION=="add", SUBSYSTEM=="cpu", RUN+="/bin/sh -c '[ ! -e /sys$devpath/online ] || echo 1 > /sys$devpath/online'"

Ian.

Re: cpu_down() but no cpu_up() in drivers/xen/cpu_hotplug.c ?

[Dietmar Hahn]

Am 11.05.2010 schrieb Ian Campbell:
> On Tue, 2010-05-11 at 10:34 +0100, Jan Beulich wrote:
> > Jeremy,
> > 
> > how is pv-ops bringing up CPUs set to online in xenstore? Is this not
> > an automatic process (i.e. requires writing to respective online node
> > in sysfs), like in the traditional tree?
> 
> The original commit which added CPU hotplug to pvops says:
> 
>     xen: implement CPU hotplugging
>     
>     Note the changes from 2.6.18-xen CPU hotplugging:
>     
>     A vcpu_down request from the remote admin via Xenbus both hotunplugs the
>     CPU, and disables it by removing it from the cpu_present map, and removing
>     its entry in /sys.
>     
>     A vcpu_up request from the remote admin only re-enables the CPU, and does
>     not immediately bring the CPU up. A udev event is emitted, which can be
>     caught by the user if he wishes to automatically re-up CPUs when available,
>     or implement a more complex policy.
>     
>     Signed-off-by: Alex Nixon <alex.nixon@citrix.com>
>     Acked-by: Jeremy Fitzhardinge <jeremy@goop.org>
>     Signed-off-by: Ingo Molnar <mingo@elte.hu>
> 
> I'm not sure how the decision was reached to implement it this way,
> perhaps for consistency with CPU hotplug on other
> platforms/architectures?
> 
> FWIW I use a udev rule to bring up CPUs as they are added, which is
> equivalent to the old behaviour:
> 
>         ACTION=="add", SUBSYSTEM=="cpu", RUN+="/bin/sh -c '[ ! -e /sys$devpath/online ] || echo 1 > /sys$devpath/online'"
> 
> Ian.

Maybe it would be good then to have a comment somewhere in the tree with
this udev rule as a hint?
Thanks.

Dietmar.

-- 
Company details: http://ts.fujitsu.com/imprint.html

Re: cpu_down() but no cpu_up() in drivers/xen/cpu_hotplug.c ?

[Jeremy Fitzhardinge]

On 05/11/2010 02:45 AM, Ian Campbell wrote:
> The original commit which added CPU hotplug to pvops says:
>
>     xen: implement CPU hotplugging
>     
>     Note the changes from 2.6.18-xen CPU hotplugging:
>     
>     A vcpu_down request from the remote admin via Xenbus both hotunplugs the
>     CPU, and disables it by removing it from the cpu_present map, and removing
>     its entry in /sys.
>     
>     A vcpu_up request from the remote admin only re-enables the CPU, and does
>     not immediately bring the CPU up. A udev event is emitted, which can be
>     caught by the user if he wishes to automatically re-up CPUs when available,
>     or implement a more complex policy.
>     
>     Signed-off-by: Alex Nixon <alex.nixon@citrix.com>
>     Acked-by: Jeremy Fitzhardinge <jeremy@goop.org>
>     Signed-off-by: Ingo Molnar <mingo@elte.hu>
>
> I'm not sure how the decision was reached to implement it this way,
> perhaps for consistency with CPU hotplug on other
> platforms/architectures?
>   

Yes, it was to make it consistent with native physical CPU hotplug.  It
also replaced some other xen-specific mechanism to allow the domain to
control when the cpu was actually added (I forget the details; something
like "cpus allowed" vs "cpus active" or something?).

> FWIW I use a udev rule to bring up CPUs as they are added, which is
> equivalent to the old behaviour:
>
>         ACTION=="add", SUBSYSTEM=="cpu", RUN+="/bin/sh -c '[ ! -e /sys$devpath/online ] || echo 1 > /sys$devpath/online'"
>   

Fedora and RHEL have been shipping with something like this for a while.

    J

RE: cpu_down() but no cpu_up() in drivers/xen/cpu_hotplug.c ?

[Jiang, Yunhong]

[-- Attachment #1: Type: text/plain, Size: 2389 bytes --]



>-----Original Message-----
>From: xen-devel-bounces@lists.xensource.com
>[mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Jeremy Fitzhardinge
>Sent: Wednesday, May 12, 2010 1:40 AM
>To: Ian Campbell
>Cc: xen-devel@lists.xensource.com; Jan Beulich
>Subject: Re: [Xen-devel] cpu_down() but no cpu_up() in drivers/xen/cpu_hotplug.c ?
>
>On 05/11/2010 02:45 AM, Ian Campbell wrote:
>> The original commit which added CPU hotplug to pvops says:
>>
>>     xen: implement CPU hotplugging
>>
>>     Note the changes from 2.6.18-xen CPU hotplugging:
>>
>>     A vcpu_down request from the remote admin via Xenbus both hotunplugs the
>>     CPU, and disables it by removing it from the cpu_present map, and removing
>>     its entry in /sys.
>>
>>     A vcpu_up request from the remote admin only re-enables the CPU, and does
>>     not immediately bring the CPU up. A udev event is emitted, which can be
>>     caught by the user if he wishes to automatically re-up CPUs when available,
>>     or implement a more complex policy.
>>
>>     Signed-off-by: Alex Nixon <alex.nixon@citrix.com>
>>     Acked-by: Jeremy Fitzhardinge <jeremy@goop.org>
>>     Signed-off-by: Ingo Molnar <mingo@elte.hu>
>>
>> I'm not sure how the decision was reached to implement it this way,
>> perhaps for consistency with CPU hotplug on other
>> platforms/architectures?
>>
>
>Yes, it was to make it consistent with native physical CPU hotplug.  It
>also replaced some other xen-specific mechanism to allow the domain to
>control when the cpu was actually added (I forget the details; something
>like "cpus allowed" vs "cpus active" or something?).

I remember for cpu remove, the xen's vcpu is different to native method. In native, it will only trigger a uevent to user space (at least in version like 2.6.31), while for xen vcpu, it will remove the vcpu directly.

--jyh

>
>> FWIW I use a udev rule to bring up CPUs as they are added, which is
>> equivalent to the old behaviour:
>>
>>         ACTION=="add", SUBSYSTEM=="cpu", RUN+="/bin/sh -c '[ ! -e
>/sys$devpath/online ] || echo 1 > /sys$devpath/online'"
>>
>
>Fedora and RHEL have been shipping with something like this for a while.
>
>    J
>
>_______________________________________________
>Xen-devel mailing list
>Xen-devel@lists.xensource.com
>http://lists.xensource.com/xen-devel

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: cpu_down() but no cpu_up() in drivers/xen/cpu_hotplug.c ?

[Jeremy Fitzhardinge]

On 05/11/2010 08:25 PM, Jiang, Yunhong wrote:
>> Yes, it was to make it consistent with native physical CPU hotplug.  It
>> also replaced some other xen-specific mechanism to allow the domain to
>> control when the cpu was actually added (I forget the details; something
>> like "cpus allowed" vs "cpus active" or something?).
>>     
> I remember for cpu remove, the xen's vcpu is different to native method. In native, it will only trigger a uevent to user space (at least in version like 2.6.31), while for xen vcpu, it will remove the vcpu directly.
>   

I would think that Xen and native are much the same; if you pull out a
physical CPU from the system, that's in no way advisory ;)  Similarly,
if you remove a vcpu from a guest, that's an external policy being
imposed onto the guest, and it doesn't get much say in the matter,
beyond being able to clean up before the vcpu goes away.

If the domain wants to stop using a vcpu, it can simply do that by
soft-downing the vcpu, but it remains attached to the domain (again,
analogous to the native case).

    J

RE: cpu_down() but no cpu_up() in drivers/xen/cpu_hotplug.c ?

[Jiang, Yunhong]

[-- Attachment #1: Type: text/plain, Size: 2702 bytes --]



>-----Original Message-----
>From: Jeremy Fitzhardinge [mailto:jeremy@goop.org]
>Sent: Thursday, May 13, 2010 12:27 AM
>To: Jiang, Yunhong
>Cc: Ian Campbell; xen-devel@lists.xensource.com; Jan Beulich
>Subject: Re: [Xen-devel] cpu_down() but no cpu_up() in drivers/xen/cpu_hotplug.c ?
>
>On 05/11/2010 08:25 PM, Jiang, Yunhong wrote:
>>> Yes, it was to make it consistent with native physical CPU hotplug.  It
>>> also replaced some other xen-specific mechanism to allow the domain to
>>> control when the cpu was actually added (I forget the details; something
>>> like "cpus allowed" vs "cpus active" or something?).
>>>
>> I remember for cpu remove, the xen's vcpu is different to native method. In native,
>it will only trigger a uevent to user space (at least in version like 2.6.31), while for
>xen vcpu, it will remove the vcpu directly.
>>
>
>I would think that Xen and native are much the same; if you pull out a

I suspect if native has tested the CPU hotmove. But at least from the code in container_notify_cb() at drivers/acpi/container.c, seems it will trigger a uevent to user space and the user space will take action (mostly through eiject entry under sysfs, I think). But still I don't know if this has been tested on native environment. So maybe the difference from xen vcpu does not matter.

    case ACPI_NOTIFY_DEVICE_CHECK:
        printk(KERN_WARNING "Container driver received %s event\n",
               (type == ACPI_NOTIFY_BUS_CHECK) ?
               "ACPI_NOTIFY_BUS_CHECK" : "ACPI_NOTIFY_DEVICE_CHECK");
        status = acpi_bus_get_device(handle, &device);
        if (present) {
            if (ACPI_FAILURE(status) || !device) {
                result = container_device_add(&device, handle);
                if (!result)
                    kobject_uevent(&device->dev.kobj,
                               KOBJ_ONLINE);
                else
                    printk(KERN_WARNING
                           "Failed to add container\n");
            }
        } else {
            if (ACPI_SUCCESS(status)) {
                /* device exist and this is a remove request */
                kobject_uevent(&device->dev.kobj, KOBJ_OFFLINE);
            }
        }

>physical CPU from the system, that's in no way advisory ;)  Similarly,
>if you remove a vcpu from a guest, that's an external policy being
>imposed onto the guest, and it doesn't get much say in the matter,
>beyond being able to clean up before the vcpu goes away.
>
>If the domain wants to stop using a vcpu, it can simply do that by
>soft-downing the vcpu, but it remains attached to the domain (again,
>analogous to the native case).
>
>    J

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel