cpuset trouble after hibernate

cpuset trouble after hibernate

[Nicolas Capit]

Hello,

This is my situation:
  - I mounted the pseudo cpuset filesystem into /dev/cpuset
  - I created a cpuset named oar with my 2 cpus

cat /dev/cpuset/oar/cpus 
0-1

  - Then I hibernate my computer with 'echo -n "disk" >/sys/power/state'
  - After reboot:

cat /dev/cpuset/oar/cpus 
0

Why did I lost a cpu?
Is this a normal behavior???

Thank you for your attention,
Nicolas Capit


Note on my system:
  - laptop HP dv2000 with a Turion64x2
  - kernel : Linux 2.6.22-1-686 #1 SMP Sun Jul 29 14:37:42 UTC
2007 i686 GNU/Linux

Re: cpuset trouble after hibernate

[Pavel Machek]

Hi!

> This is my situation:
>   - I mounted the pseudo cpuset filesystem into /dev/cpuset
>   - I created a cpuset named oar with my 2 cpus
> 
> cat /dev/cpuset/oar/cpus 
> 0-1
> 
>   - Then I hibernate my computer with 'echo -n "disk" >/sys/power/state'
>   - After reboot:
> 
> cat /dev/cpuset/oar/cpus 
> 0
> 
> Why did I lost a cpu?
> Is this a normal behavior???

One of the cpus was unplugged during suspend... perhaps some
save/restore is needed during hotplug/unplug?

							Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: cpuset trouble after hibernate

[Simon Derr]

On Sat, 8 Sep 2007, Nicolas Capit wrote:

> Hello,
> 
> This is my situation:
>   - I mounted the pseudo cpuset filesystem into /dev/cpuset
>   - I created a cpuset named oar with my 2 cpus
> 
> cat /dev/cpuset/oar/cpus 
> 0-1
> 
>   - Then I hibernate my computer with 'echo -n "disk" >/sys/power/state'
>   - After reboot:
> 
> cat /dev/cpuset/oar/cpus 
> 0
> 
> Why did I lost a cpu?
> Is this a normal behavior???

Hi Nicolas,

I believe this is related to the fact that hibernation uses the hotplug 
subsystem to disable all CPUs except the boot CPU.

Thus guarantee_online_cpus() is called on each cpuset and removes all 
CPUs, except CPU 0, from all cpusets.

I'm not quite sure about if/how this should be fixed in the kernel, 
though. Looks like a very simple user-land workaround would be enough.

	Simon.

Re: cpuset trouble after hibernate

[Nicolas Capit]

On Mon, 10 Sep 2007 11:45:10 +0200 (CEST)
Simon Derr <Simon.Derr@bull.net> wrote:

> On Sat, 8 Sep 2007, Nicolas Capit wrote:
> 
> > Hello,
> > 
> > This is my situation:
> >   - I mounted the pseudo cpuset filesystem into /dev/cpuset
> >   - I created a cpuset named oar with my 2 cpus
> > 
> > cat /dev/cpuset/oar/cpus 
> > 0-1
> > 
> >   - Then I hibernate my computer with 'echo -n "disk" >/sys/power/state'
> >   - After reboot:
> > 
> > cat /dev/cpuset/oar/cpus 
> > 0
> > 
> > Why did I lost a cpu?
> > Is this a normal behavior???
> 
> Hi Nicolas,
> 
> I believe this is related to the fact that hibernation uses the hotplug 
> subsystem to disable all CPUs except the boot CPU.
> 
> Thus guarantee_online_cpus() is called on each cpuset and removes all 
> CPUs, except CPU 0, from all cpusets.
> 
> I'm not quite sure about if/how this should be fixed in the kernel, 
> though. Looks like a very simple user-land workaround would be enough.
> 
> 	Simon.

Ok, it is not a big deal for me (you don't hibernate computers in a
cluster...) but I wanted to bring you this issue in case of a similar
behavior in another case...

Thanks for your response.

Nicolas Capit.

Re: cpuset trouble after hibernate

[Andrew Morton]

On Mon, 10 Sep 2007 11:45:10 +0200 (CEST) Simon Derr <Simon.Derr@bull.net> wrote:

> On Sat, 8 Sep 2007, Nicolas Capit wrote:
> 
> > Hello,
> > 
> > This is my situation:
> >   - I mounted the pseudo cpuset filesystem into /dev/cpuset
> >   - I created a cpuset named oar with my 2 cpus
> > 
> > cat /dev/cpuset/oar/cpus 
> > 0-1
> > 
> >   - Then I hibernate my computer with 'echo -n "disk" >/sys/power/state'
> >   - After reboot:
> > 
> > cat /dev/cpuset/oar/cpus 
> > 0
> > 
> > Why did I lost a cpu?
> > Is this a normal behavior???
> 
> Hi Nicolas,
> 
> I believe this is related to the fact that hibernation uses the hotplug 
> subsystem to disable all CPUs except the boot CPU.
> 
> Thus guarantee_online_cpus() is called on each cpuset and removes all 
> CPUs, except CPU 0, from all cpusets.
> 
> I'm not quite sure about if/how this should be fixed in the kernel, 
> though. Looks like a very simple user-land workaround would be enough.
> 

Yeah.  Bug, surely.  But I guess it's always been there.

What are the implications of this for cpusets-via-containers?

Re: cpuset trouble after hibernate

[Paul Menage]

On 9/15/07, Andrew Morton <akpm@linux-foundation.org> wrote:
>
> Yeah.  Bug, surely.  But I guess it's always been there.
>
> What are the implications of this for cpusets-via-containers?
>

I don't think it should be any different from the previous version - I
tried to avoid touching those bits of cpusets where possible.

Paul

Re: cpuset trouble after hibernate

[Balbir Singh]

Andrew Morton wrote:
> On Mon, 10 Sep 2007 11:45:10 +0200 (CEST) Simon Derr <Simon.Derr@bull.net> wrote:
> 
>> On Sat, 8 Sep 2007, Nicolas Capit wrote:
>>
>>> Hello,
>>>
>>> This is my situation:
>>>   - I mounted the pseudo cpuset filesystem into /dev/cpuset
>>>   - I created a cpuset named oar with my 2 cpus
>>>
>>> cat /dev/cpuset/oar/cpus 
>>> 0-1
>>>
>>>   - Then I hibernate my computer with 'echo -n "disk" >/sys/power/state'
>>>   - After reboot:
>>>
>>> cat /dev/cpuset/oar/cpus 
>>> 0
>>>
>>> Why did I lost a cpu?
>>> Is this a normal behavior???
>> Hi Nicolas,
>>
>> I believe this is related to the fact that hibernation uses the hotplug 
>> subsystem to disable all CPUs except the boot CPU.
>>
>> Thus guarantee_online_cpus() is called on each cpuset and removes all 
>> CPUs, except CPU 0, from all cpusets.
>>
>> I'm not quite sure about if/how this should be fixed in the kernel, 
>> though. Looks like a very simple user-land workaround would be enough.
>>
> 
> Yeah.  Bug, surely.  But I guess it's always been there.
> 
> What are the implications of this for cpusets-via-containers?
> 

I suspect the functionality of cpusets is not affected by containers.

I wonder if containers should become suspend/resume aware and pass
that option on to controllers. I think it's only the bus drivers
and device drivers that do that now.



-- 
	Warm Regards,
	Balbir Singh
	Linux Technology Center
	IBM, ISTL

Re: cpuset trouble after hibernate

[Paul Menage]

On 9/9/07, Pavel Machek <pavel@ucw.cz> wrote:
>
> One of the cpus was unplugged during suspend... perhaps some
> save/restore is needed during hotplug/unplug?

Or else keep track separately in cpusets of

- cpus that the cpuset can run on
- cpus that the admin has specified for the cpu to run on

hotplug/hotunplug events would only affect the former; userspace would
only see/modify the latter. Then when hibernate is over and the CPUs
are hotplugged back in, things would be back as before.

Paul