BUG: sleeping function called from invalid context

BUG: sleeping function called from invalid context

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 2756 bytes --]

Hello everyone,

Since I really plan to spend some time here, let me introduce myself
first. My name is Dario Faggioli and I just joined the Citrix Platform
Team in Cambridge (although I'll be working from Italy). I've some
experience in Linux kernel (mainly scheduling) and not that much
experience in Xen or virtualization in general, but I really want to
learn and be able to contribute ASAP!

In fact, while "doing my homework", I stumbled against the following
BUG(). I'm able to reproduce it with xen-unstable and by just by
enabling spinlock and mutex debug checks in linus' Linux
(65112dccf8a113737684366349d7f9ec373ddc47) _iff_ using 'tap:qcow2' disk
image, while no such thing happens with 'phy:'.

[  996.282544] BUG: sleeping function called from invalid context at /local/scratch/dariof/linux/kernel/mutex.c:271
[  996.282570] in_atomic(): 1, irqs_disabled(): 0, pid: 3256, name: qemu-dm
[  996.282581] 1 lock held by qemu-dm/3256:
[  996.282589]  #0:  (&(&priv->lock)->rlock){......}, at: [<ffffffff813223da>] gntdev_ioctl+0x2bd/0x4d5
[  996.282628] Pid: 3256, comm: qemu-dm Tainted: G        W   3.1.0-rc8+ #5
[  996.282638] Call Trace:
[  996.282687]  [<ffffffff81054594>] __might_sleep+0x131/0x135
[  996.282704]  [<ffffffff816bd64f>] mutex_lock_nested+0x25/0x45
[  996.282721]  [<ffffffff8131c7c8>] free_xenballooned_pages+0x20/0xb1
[  996.282735]  [<ffffffff8132194d>] gntdev_put_map+0xa8/0xdb
[  996.282749]  [<ffffffff816be546>] ? _raw_spin_lock+0x71/0x7a
[  996.282763]  [<ffffffff813223da>] ? gntdev_ioctl+0x2bd/0x4d5
[  996.282776]  [<ffffffff8132243c>] gntdev_ioctl+0x31f/0x4d5
[  996.282790]  [<ffffffff81007d62>] ? check_events+0x12/0x20
[  996.282804]  [<ffffffff811433bc>] do_vfs_ioctl+0x488/0x4d7
[  996.282818]  [<ffffffff81007d4f>] ? xen_restore_fl_direct_reloc+0x4/0x4
[  996.282832]  [<ffffffff8109168b>] ? lock_release+0x21c/0x229
[  996.282847]  [<ffffffff81135cdd>] ? rcu_read_unlock+0x21/0x32
[  996.282860]  [<ffffffff81143452>] sys_ioctl+0x47/0x6a
[  996.282873]  [<ffffffff816bfd82>] system_call_fastpath+0x16/0x1b

This seems to be due to free_xenballooned_pages(), called by
gntdev_put_map(), taking balloon_mutex, with the latter that can be
called within a spin_lock() (e.g., in gntdev_release()).

I'm not enough confident with the code do attempt fixing it, but I
thought it was worth to at least point it out!

Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
----------------------------------------------------------------------
Dario Faggioli, http://retis.sssup.it/people/faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
PhD Candidate, ReTiS Lab, Scuola Superiore Sant'Anna, Pisa  (Italy)

[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: BUG: sleeping function called from invalid context

[Konrad Rzeszutek Wilk]

On Tue, Oct 11, 2011 at 07:13:38PM +0200, Dario Faggioli wrote:
> Hello everyone,
> 
> Since I really plan to spend some time here, let me introduce myself
> first. My name is Dario Faggioli and I just joined the Citrix Platform
> Team in Cambridge (although I'll be working from Italy). I've some
> experience in Linux kernel (mainly scheduling) and not that much
> experience in Xen or virtualization in general, but I really want to
> learn and be able to contribute ASAP!
> 
> In fact, while "doing my homework", I stumbled against the following
> BUG(). I'm able to reproduce it with xen-unstable and by just by
> enabling spinlock and mutex debug checks in linus' Linux
> (65112dccf8a113737684366349d7f9ec373ddc47) _iff_ using 'tap:qcow2' disk
> image, while no such thing happens with 'phy:'.

Aha.. I saw that at some point but never narrowed it down to the
right combination. Thank you for tracking this down.

I am CC Daniel who might have some patches for this or ideas.
> 
> [  996.282544] BUG: sleeping function called from invalid context at /local/scratch/dariof/linux/kernel/mutex.c:271
> [  996.282570] in_atomic(): 1, irqs_disabled(): 0, pid: 3256, name: qemu-dm
> [  996.282581] 1 lock held by qemu-dm/3256:
> [  996.282589]  #0:  (&(&priv->lock)->rlock){......}, at: [<ffffffff813223da>] gntdev_ioctl+0x2bd/0x4d5
> [  996.282628] Pid: 3256, comm: qemu-dm Tainted: G        W   3.1.0-rc8+ #5
> [  996.282638] Call Trace:
> [  996.282687]  [<ffffffff81054594>] __might_sleep+0x131/0x135
> [  996.282704]  [<ffffffff816bd64f>] mutex_lock_nested+0x25/0x45
> [  996.282721]  [<ffffffff8131c7c8>] free_xenballooned_pages+0x20/0xb1
> [  996.282735]  [<ffffffff8132194d>] gntdev_put_map+0xa8/0xdb
> [  996.282749]  [<ffffffff816be546>] ? _raw_spin_lock+0x71/0x7a
> [  996.282763]  [<ffffffff813223da>] ? gntdev_ioctl+0x2bd/0x4d5
> [  996.282776]  [<ffffffff8132243c>] gntdev_ioctl+0x31f/0x4d5
> [  996.282790]  [<ffffffff81007d62>] ? check_events+0x12/0x20
> [  996.282804]  [<ffffffff811433bc>] do_vfs_ioctl+0x488/0x4d7
> [  996.282818]  [<ffffffff81007d4f>] ? xen_restore_fl_direct_reloc+0x4/0x4
> [  996.282832]  [<ffffffff8109168b>] ? lock_release+0x21c/0x229
> [  996.282847]  [<ffffffff81135cdd>] ? rcu_read_unlock+0x21/0x32
> [  996.282860]  [<ffffffff81143452>] sys_ioctl+0x47/0x6a
> [  996.282873]  [<ffffffff816bfd82>] system_call_fastpath+0x16/0x1b
> 
> This seems to be due to free_xenballooned_pages(), called by
> gntdev_put_map(), taking balloon_mutex, with the latter that can be
> called within a spin_lock() (e.g., in gntdev_release()).
> 
> I'm not enough confident with the code do attempt fixing it, but I
> thought it was worth to at least point it out!

[PATCH] xen/gntdev: Fix sleep-inside-spinlock

[Daniel De Graaf]

On 10/11/2011 01:32 PM, Konrad Rzeszutek Wilk wrote:
> On Tue, Oct 11, 2011 at 07:13:38PM +0200, Dario Faggioli wrote:
>> Hello everyone,
>>
>> Since I really plan to spend some time here, let me introduce myself
>> first. My name is Dario Faggioli and I just joined the Citrix Platform
>> Team in Cambridge (although I'll be working from Italy). I've some
>> experience in Linux kernel (mainly scheduling) and not that much
>> experience in Xen or virtualization in general, but I really want to
>> learn and be able to contribute ASAP!
>>
>> In fact, while "doing my homework", I stumbled against the following
>> BUG(). I'm able to reproduce it with xen-unstable and by just by
>> enabling spinlock and mutex debug checks in linus' Linux
>> (65112dccf8a113737684366349d7f9ec373ddc47) _iff_ using 'tap:qcow2' disk
>> image, while no such thing happens with 'phy:'.
> 
> Aha.. I saw that at some point but never narrowed it down to the
> right combination. Thank you for tracking this down.
> 
> I am CC Daniel who might have some patches for this or ideas.
>>
>> [  996.282544] BUG: sleeping function called from invalid context at /local/scratch/dariof/linux/kernel/mutex.c:271
>> [  996.282570] in_atomic(): 1, irqs_disabled(): 0, pid: 3256, name: qemu-dm
>> [  996.282581] 1 lock held by qemu-dm/3256:
>> [  996.282589]  #0:  (&(&priv->lock)->rlock){......}, at: [<ffffffff813223da>] gntdev_ioctl+0x2bd/0x4d5
>> [  996.282628] Pid: 3256, comm: qemu-dm Tainted: G        W   3.1.0-rc8+ #5
>> [  996.282638] Call Trace:
>> [  996.282687]  [<ffffffff81054594>] __might_sleep+0x131/0x135
>> [  996.282704]  [<ffffffff816bd64f>] mutex_lock_nested+0x25/0x45
>> [  996.282721]  [<ffffffff8131c7c8>] free_xenballooned_pages+0x20/0xb1
>> [  996.282735]  [<ffffffff8132194d>] gntdev_put_map+0xa8/0xdb
>> [  996.282749]  [<ffffffff816be546>] ? _raw_spin_lock+0x71/0x7a
>> [  996.282763]  [<ffffffff813223da>] ? gntdev_ioctl+0x2bd/0x4d5
>> [  996.282776]  [<ffffffff8132243c>] gntdev_ioctl+0x31f/0x4d5
>> [  996.282790]  [<ffffffff81007d62>] ? check_events+0x12/0x20
>> [  996.282804]  [<ffffffff811433bc>] do_vfs_ioctl+0x488/0x4d7
>> [  996.282818]  [<ffffffff81007d4f>] ? xen_restore_fl_direct_reloc+0x4/0x4
>> [  996.282832]  [<ffffffff8109168b>] ? lock_release+0x21c/0x229
>> [  996.282847]  [<ffffffff81135cdd>] ? rcu_read_unlock+0x21/0x32
>> [  996.282860]  [<ffffffff81143452>] sys_ioctl+0x47/0x6a
>> [  996.282873]  [<ffffffff816bfd82>] system_call_fastpath+0x16/0x1b
>>
>> This seems to be due to free_xenballooned_pages(), called by
>> gntdev_put_map(), taking balloon_mutex, with the latter that can be
>> called within a spin_lock() (e.g., in gntdev_release()).
>>
>> I'm not enough confident with the code do attempt fixing it, but I
>> thought it was worth to at least point it out!
> 

It looks like gntdev_release doesn't need to take the spinlock at all, since
it's a per-file lock on the file's release path. The unmap ioctl looks like
it'll also trigger this message; that's also fixed here. Compile tested only.

-------------------------------------------------------->8

gntdev_put_map tries to acquire a mutex when freeing pages back to the
xenballoon pool, so it cannot be called with a spinlock held. In
gntdev_release, the spinlock is not needed as we are freeing the
structure later; in the ioctl, only the list manipulation needs to be
under the lock.

Reported-By: Dario Faggioli <raistlin@linux.it>
Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov>
---
 drivers/xen/gntdev.c |    5 ++---
 1 files changed, 2 insertions(+), 3 deletions(-)

diff --git a/drivers/xen/gntdev.c b/drivers/xen/gntdev.c
index f914b26..23b1c83 100644
--- a/drivers/xen/gntdev.c
+++ b/drivers/xen/gntdev.c
@@ -462,13 +462,11 @@ static int gntdev_release(struct inode *inode, struct file *flip)
 
 	pr_debug("priv %p\n", priv);
 
-	spin_lock(&priv->lock);
 	while (!list_empty(&priv->maps)) {
 		map = list_entry(priv->maps.next, struct grant_map, next);
 		list_del(&map->next);
 		gntdev_put_map(map);
 	}
-	spin_unlock(&priv->lock);
 
 	if (use_ptemod)
 		mmu_notifier_unregister(&priv->mn, priv->mm);
@@ -532,10 +530,11 @@ static long gntdev_ioctl_unmap_grant_ref(struct gntdev_priv *priv,
 	map = gntdev_find_map_index(priv, op.index >> PAGE_SHIFT, op.count);
 	if (map) {
 		list_del(&map->next);
-		gntdev_put_map(map);
 		err = 0;
 	}
 	spin_unlock(&priv->lock);
+	if (map)
+		gntdev_put_map(map);
 	return err;
 }
 
-- 
1.7.6.4

Re: [PATCH] xen/gntdev: Fix sleep-inside-spinlock

[Konrad Rzeszutek Wilk]

On Tue, Oct 11, 2011 at 03:16:06PM -0400, Daniel De Graaf wrote:
> On 10/11/2011 01:32 PM, Konrad Rzeszutek Wilk wrote:
> > On Tue, Oct 11, 2011 at 07:13:38PM +0200, Dario Faggioli wrote:
> >> Hello everyone,

Dario, does the patch fix the problem for you?

.. snip..
> it'll also trigger this message; that's also fixed here. Compile tested only.
> 
> -------------------------------------------------------->8
> 
> gntdev_put_map tries to acquire a mutex when freeing pages back to the
> xenballoon pool, so it cannot be called with a spinlock held. In
> gntdev_release, the spinlock is not needed as we are freeing the
> structure later; in the ioctl, only the list manipulation needs to be
> under the lock.
> 
> Reported-By: Dario Faggioli <raistlin@linux.it>
> Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov>
> ---
>  drivers/xen/gntdev.c |    5 ++---
>  1 files changed, 2 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/xen/gntdev.c b/drivers/xen/gntdev.c
> index f914b26..23b1c83 100644
> --- a/drivers/xen/gntdev.c
> +++ b/drivers/xen/gntdev.c
> @@ -462,13 +462,11 @@ static int gntdev_release(struct inode *inode, struct file *flip)
>  
>  	pr_debug("priv %p\n", priv);
>  
> -	spin_lock(&priv->lock);
>  	while (!list_empty(&priv->maps)) {
>  		map = list_entry(priv->maps.next, struct grant_map, next);
>  		list_del(&map->next);
>  		gntdev_put_map(map);
>  	}
> -	spin_unlock(&priv->lock);
>  
>  	if (use_ptemod)
>  		mmu_notifier_unregister(&priv->mn, priv->mm);
> @@ -532,10 +530,11 @@ static long gntdev_ioctl_unmap_grant_ref(struct gntdev_priv *priv,
>  	map = gntdev_find_map_index(priv, op.index >> PAGE_SHIFT, op.count);
>  	if (map) {
>  		list_del(&map->next);
> -		gntdev_put_map(map);
>  		err = 0;
>  	}
>  	spin_unlock(&priv->lock);
> +	if (map)
> +		gntdev_put_map(map);
>  	return err;
>  }
>  
> -- 
> 1.7.6.4

Re: [PATCH] xen/gntdev: Fix sleep-inside-spinlock

[Dario Faggioli]

[-- Attachment #1.1.1: Type: text/plain, Size: 3248 bytes --]

On Thu, 2011-10-13 at 11:40 -0400, Konrad Rzeszutek Wilk wrote:
> Dario, does the patch fix the problem for you?
> 
Here I am, and sorry for being late, I had issues with a testbox and
needed to setup a new one.

Actually, I was just trying to check that, but I'm now hitting this when
trying to boot a PV-guest:
[    6.129478] BUG: scheduling while atomic: swapper/0/0x10000002
[    6.129483] no locks held by swapper/0.
[    6.129487] Pid: 0, comm: swapper Not tainted 3.1.0-rc9+ #14
[    6.129492] Call Trace:
[    6.129504]  [<ffffffff81054550>] __schedule_bug+0x80/0x85
[    6.129513]  [<ffffffff81b7fc73>] __schedule+0xa5/0x6ad
[    6.129520]  [<ffffffff8105aed1>] __cond_resched+0x2a/0x35
[    6.129525]  [<ffffffff81b802f6>] _cond_resched+0x2c/0x37
[    6.129533]  [<ffffffff810dedce>] __alloc_pages_nodemask+0xa5/0x73a
[    6.129542]  [<ffffffff810082c5>] ? p2m_top_index+0x9/0x1f
[    6.129547]  [<ffffffff8100849b>] ? get_phys_to_machine+0x1f/0x62
[    6.129556]  [<ffffffff8143ee6d>] ? trace_hardirqs_off_thunk+0x3a/0x3c
[    6.129562]  [<ffffffff8143ee2e>] ? trace_hardirqs_on_thunk+0x3a/0x3f
[    6.129568]  [<ffffffff81b831b4>] ? retint_restore_args+0x13/0x13
[    6.129573]  [<ffffffff810082c5>] ? p2m_top_index+0x9/0x1f
[    6.129581]  [<ffffffff8110d445>] alloc_page_interleave+0x39/0x86
[    6.129587]  [<ffffffff81004e0d>] ? xen_make_pte+0x8a/0x8e
[    6.129592]  [<ffffffff8110d501>] alloc_pages_current+0x6f/0xbf
[    6.129598]  [<ffffffff810dd92a>] __get_free_pages+0xe/0x3d
[    6.129605]  [<ffffffff8103acc0>] pte_alloc_one_kernel+0x15/0x17
[    6.129611]  [<ffffffff810f56d9>] __pte_alloc_kernel+0x1b/0xf4
[    6.129617]  [<ffffffff8110292a>] vmap_page_range_noflush+0x1c5/0x317
[    6.129623]  [<ffffffff81102aae>] ? map_vm_area+0x32/0x46
[    6.129629]  [<ffffffff8100763d>] ? xen_force_evtchn_callback+0xd/0xf
[    6.129634]  [<ffffffff81102aae>] map_vm_area+0x32/0x46
[    6.129640]  [<ffffffff81103a94>] __vmalloc_node_range+0x194/0x1d6
[    6.129648]  [<ffffffff8254a0a5>] ? alloc_large_system_hash+0x14c/0x203
[    6.129654]  [<ffffffff81007d4f>] ? xen_restore_fl_direct_reloc+0x4/0x4
[    6.129660]  [<ffffffff81103b0b>] __vmalloc_node+0x35/0x37
[    6.129666]  [<ffffffff8254a0a5>] ? alloc_large_system_hash+0x14c/0x203
[    6.129672]  [<ffffffff81103cd3>] __vmalloc+0x20/0x22
[    6.129677]  [<ffffffff8254a0a5>] alloc_large_system_hash+0x14c/0x203
[    6.129683]  [<ffffffff8254e4bb>] vfs_caches_init+0x9d/0x10f
[    6.129690]  [<ffffffff82529da9>] start_kernel+0x378/0x3b5
[    6.129696]  [<ffffffff825292cd>] x86_64_start_reservations+0xb8/0xbc
[    6.129702]  [<ffffffff8252cfc7>] xen_start_kernel+0x5ab/0x5b2

So, am I doing something stupid (e.g., VM config file, which is
attached) or this is a different thing as a whole? :-O

I have to say that I updated both Linux and xen-unstable in these days.

Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
----------------------------------------------------------------------
Dario Faggioli, http://retis.sssup.it/people/faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
PhD Candidate, ReTiS Lab, Scuola Superiore Sant'Anna, Pisa  (Italy)

[-- Attachment #1.1.2: Debian-squeeze.pv --]
[-- Type: text/plain, Size: 14918 bytes --]

#----------------------------------------------------------------------------
# Kernel image file.
kernel = "/boot/vmlinuz-3.1.0-rc9"
extra = "root=/dev/sda1 ro"

# Optional ramdisk.
#ramdisk = "/boot/initrd.gz"

# The domain build function. Default is 'linux'.
#builder='linux'

# Initial memory allocation (in megabytes) for the new domain.
#
# WARNING: Creating a domain with insufficient memory may cause out of
#          memory errors. The domain needs enough memory to boot kernel
#          and modules. Allocating less than 32MBs is not recommended.
memory = 128

# Shadow pagetable memory for the domain, in MB.
# If not explicictly set, xend will pick an appropriate value.  
# Should be at least 2KB per MB of domain memory, plus a few MB per vcpu.
# shadow_memory = 8

# Whether to transparently share this domain's memory with other domains.
# default = 0
# memory_sharing = 0

# A name for your domain. All domains must have different names.
name = "Debian-suqeeze_pv"

# 128-bit UUID for the domain.  The default behavior is to generate a new UUID
# on each call to 'xm create'.
#uuid = "06ed00fe-1162-4fc4-b5d8-11993ee4a8b9"

#-----------------------------------------------------------------------------
# The number of cpus guest platform has, default=1
vcpus=1

# Enable/disable HVM guest PAE, default=1 (enabled)
#pae=1

# Enable/disable HVM guest ACPI, default=1 (enabled)
#acpi=1

# Enable/disable HVM APIC mode, default=1 (enabled)
# Note that this option is ignored if vcpus > 1
#apic=1

# Enable/disable extended power management support within HVM guest, i.e., beyond
# S3, S4, S5 within guest like exposing battery meter.
# 0 (default option, extended power management support disabled)
# 1 (pass-through mode; uses pass-through as needed; efficient but limited in scope)
# 2 (non pass-through mode; extended scope, likely to work on all applicable environment
#    but comparitively less efficient than pass-through mode)
# xen_extended_power_mgmt=0

# List of which CPUS this domain is allowed to use, default Xen picks
#cpus = ""         # leave to Xen to pick
#cpus = "0"        # all vcpus run on CPU0
#cpus = "0-3,5,^1" # all vcpus run on cpus 0,2,3,5
#cpus = ["2", "3"] # VCPU0 runs on CPU2, VCPU1 runs on CPU3

# Optionally define mac and/or bridge for the network interfaces.
# Random MACs are assigned if not given.
#vif = [ 'type=ioemu, mac=00:16:3e:00:00:11, bridge=xenbr0, model=ne2k_pci' ]
# type=ioemu specify the NIC is an ioemu device not netfront
vif = [ 'type=ioemu, bridge=xenbr0' ]

#----------------------------------------------------------------------------
# Define the disk devices you want the domain to have access to, and
# what you want them accessible as.
# Each disk entry is of the form phy:UNAME,DEV,MODE
# where UNAME is the device, DEV is the device name the domain will see,
# and MODE is r for read-only, w for read-write.

#disk = [ 'phy:hda1,hda1,r' ]
#disk = [ 'vdev=xvda, access=rw, target=/dev/loop0' ]
disk = [ 'format=qcow2, vdev=hda, access=rw, target=/root/xen/VMs/debian_squeeze_amd64_standard.qcow2' ]

#----------------------------------------------------------------------------
# Configure the behaviour when a domain exits.  There are three 'reasons'
# for a domain to stop: poweroff, reboot, and crash.  For each of these you
# may specify:
#
#   "destroy",        meaning that the domain is cleaned up as normal;
#   "restart",        meaning that a new domain is started in place of the old
#                     one;
#   "preserve",       meaning that no clean-up is done until the domain is
#                     manually destroyed (using xm destroy, for example); or
#   "rename-restart", meaning that the old domain is not cleaned up, but is
#                     renamed and a new domain started in its place.
#
# In the event a domain stops due to a crash, you have the additional options:
#
#   "coredump-destroy", meaning dump the crashed domain's core and then destroy;
#   "coredump-restart', meaning dump the crashed domain's core and the restart.
#
# The default is
#
#   on_poweroff = 'destroy'
#   on_reboot   = 'restart'
#   on_crash    = 'restart'
#
# For backwards compatibility we also support the deprecated option restart
#
# restart = 'onreboot' means on_poweroff = 'destroy'
#                            on_reboot   = 'restart'
#                            on_crash    = 'destroy'
#
# restart = 'always'   means on_poweroff = 'restart'
#                            on_reboot   = 'restart'
#                            on_crash    = 'restart'
#
# restart = 'never'    means on_poweroff = 'destroy'
#                            on_reboot   = 'destroy'
#                            on_crash    = 'destroy'

#on_poweroff = 'destroy'
#on_reboot   = 'restart'
#on_crash    = 'restart'

#============================================================================

# Device Model to be used
device_model = 'qemu-dm'

#-----------------------------------------------------------------------------
# boot on floppy (a), hard disk (c), Network (n) or CD-ROM (d) 
# default: hard disk, cd-rom, floppy
#boot="cda"

#-----------------------------------------------------------------------------
#  write to temporary files instead of disk image files
#snapshot=1

#----------------------------------------------------------------------------
# enable SDL library for graphics, default = 0
sdl=0

#----------------------------------------------------------------------------
# enable OpenGL for texture rendering inside the SDL window, default = 1
# valid only if sdl is enabled.
opengl=1

#----------------------------------------------------------------------------
# enable VNC library for graphics, default = 1
vnc=1

#----------------------------------------------------------------------------
# address that should be listened on for the VNC server if vnc is set.
# default is to use 'vnc-listen' setting from
# auxbin.xen_configdir() + /xend-config.sxp
#vnclisten="127.0.0.1"

#----------------------------------------------------------------------------
# set VNC display number, default = domid
#vncdisplay=1

#----------------------------------------------------------------------------
# try to find an unused port for the VNC server, default = 1
#vncunused=1

#----------------------------------------------------------------------------
# set password for domain's VNC console
# default is depents on vncpasswd in xend-config.sxp
vncpasswd=''

#----------------------------------------------------------------------------
# no graphics, use serial port
#nographic=0

#----------------------------------------------------------------------------
# enable stdvga, default = 0 (use cirrus logic device model)
stdvga=0

#-----------------------------------------------------------------------------
#   serial port re-direct to pty deivce, /dev/pts/n 
#   then xm console or minicom can connect
serial='pty'

#----------------------------------------------------------------------------
#   tsc_mode : TSC mode (0=default, 1=native TSC, 2=never emulate, 3=pvrdtscp)
#   emulate TSC provides synced TSC for all vcpus, but lose perfomrance.
#   native TSC leverages hardware's TSC(no perf loss), but vcpu's TSC may lose
#    sync due to hardware's unreliable/unsynced TSC between CPUs.
#   default intelligently uses native TSC on machines where it is safe, but
#    switches to emulated if necessary after save/restore/migration
#   pvrdtscp is for intelligent apps that use special Xen-only paravirtualized
#    cpuid instructions to obtain offset/scaling/migration info and maximize
#    performance within pools of machines that support the rdtscp instruction
tsc_mode=0

#-----------------------------------------------------------------------------
#   Qemu Monitor, default is disable
#   Use ctrl-alt-2 to connect
#monitor=1


#-----------------------------------------------------------------------------
#   enable sound card support, [sb16|es1370|all|..,..], default none
#soundhw='sb16'


#-----------------------------------------------------------------------------
#    set the real time clock to local time [default=0 i.e. set to utc]
#localtime=1


#-----------------------------------------------------------------------------
#    set the real time clock offset in seconds [default=0 i.e. same as dom0]
#rtc_timeoffset=3600

#-----------------------------------------------------------------------------
#    start in full screen
#full-screen=1   


#-----------------------------------------------------------------------------
#   Enable USB support (specific devices specified at runtime through the
#			monitor window)
#usb=1

#   Enable USB mouse support (only enable one of the following, `mouse' for
#			      PS/2 protocol relative mouse, `tablet' for
#			      absolute mouse)
#usbdevice='mouse'
#usbdevice='tablet'

#-----------------------------------------------------------------------------
#   Set keyboard layout, default is en-us keyboard. 
#keymap='ja'

#-----------------------------------------------------------------------------
#   Enable/disable xen platform PCI device, default=1 (enabled)
#xen_platform_pci=1

#-----------------------------------------------------------------------------
#   Configure guest CPUID responses:
#
#cpuid=[ '1:ecx=xxxxxxxxxxx00xxxxxxxxxxxxxxxxxxx,
#           eax=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx' ]
# - Unset the SSE4 features (CPUID.1[ECX][20-19])
# - Default behaviour for all other bits in ECX And EAX registers.
# 
# Each successive character represent a lesser-significant bit:
#  '1' -> force the corresponding bit to 1
#  '0' -> force to 0
#  'x' -> Get a safe value (pass through and mask with the default policy)
#  'k' -> pass through the host bit value
#  's' -> as 'k' but preserve across save/restore and migration
# 
#   Expose to the guest multi-core cpu instead of multiple processors
# Example for intel, expose a 8-core processor :
#cpuid=['1:edx=xxx1xxxxxxxxxxxxxxxxxxxxxxxxxxxx,
#          ebx=xxxxxxxx00010000xxxxxxxxxxxxxxxx',
#     '4,0:eax=001111xxxxxxxxxxxxxxxxxxxxxxxxxx']
#  - CPUID.1[EDX][HT] : Enable HT
#  - CPUID.1[EBX] : Number of vcpus * 2
#  - CPUID.4,0[EAX] : Number of vcpus * 2 - 1
#vcpus=8
#
# Example for amd, expose a 5-core processor :
# cpuid = ['1:ebx=xxxxxxxx00001010xxxxxxxxxxxxxxxx,
#             edx=xxx1xxxxxxxxxxxxxxxxxxxxxxxxxxxx',
# '0x80000001:ecx=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx1x',
# '0x80000008:ecx=xxxxxxxxxxxxxxxxxxxxxxxxxx001001']
#   - CPUID.1[EBX] : Threads per Core * Cores per Socket (2 * #vcpus)
#   - CPUID.1[EDX][HT] : Enable HT
#   - CPUID.0x80000001[CmpLegacy] : Use legacy method
#   - CPUID.0x80000008[ECX] : #vcpus * 2 - 1
#vcpus=5
#
#  Downgrade the cpuid to make a better compatibility for migration :
# Look like a generic 686 :
# cpuid = [ '0:eax=0x3,ebx=0x0,ecx=0x0,edx=0x0',
#           '1:eax=0x06b1,
#              ecx=xxxxxxxxxxx0000xx00xxx0000000xx0,
#              edx=xxx00000xxxxxxx0xxxxxxxxx0xxxxxx',
#           '4:eax=0x3,ebx=0x0,ecx=0x0,edx=0x0',
#  '0x80000000:eax=0x3,ebx=0x0,ecx=0x0,edx=0x0']
#  with the highest leaf
#  - CPUID.0[EAX] : Set the highest leaf
#  - CPUID.1[EAX] : 686 
#  - CPUID.1[ECX] : Mask some features
#  - CPUID.1[EDX] : Mask some features
#  - CPUID.4 : Reply like the highest leaf, in our case CPUID.3
#  - CPUID.0x80000000 : No extension we are on a Pentium III, reply like the
#  highest leaf (CPUID.3).
#
#   Configure host CPUID consistency checks, which must be satisfied for this
#   VM to be allowed to run on this host's processor type:
#cpuid_check=[ '1:ecx=xxxxxxxxxxxxxxxxxxxxxxxxxx1xxxxx' ]
# - Host must have VMX feature flag set
#
# The format is similar to the above for 'cpuid':
#  '1' -> the bit must be '1'
#  '0' -> the bit must be '0'
#  'x' -> we don't care (do not check)
#  's' -> the bit must be the same as on the host that started this VM

#-----------------------------------------------------------------------------
#   Configure passthrough PCI{,-X,e} devices:
#
#   pci=[ '[SSSS:]BB:DD.F[,option1[,option2[...]]]', ... ]
#
#   [SSSS]:BB:DD.F  "bus segment:bus:device.function"(1) of the device to
#                   be assigned, bus segment is optional. All fields are
#                   in hexadecimal and no field should be longer than that
#                   as shown in the pattern. Successful assignment may need
#                   certain hardware support and additional configurations
#                   (e.g. VT-d, see docs/misc/vtd.txt for more details).
#
#       (1) bus segment is sometimes also referred to as the PCI "domain",
#           not to be confused with Xen domain.
#
#
#   optionN         per-device options in "key=val" format. Current
#                   available options are:
#                   - msitranslate=0|1
#                      per-device overriden of pci_msitranslate, see below
#                   - power_mgmt=0|1
#                      per-device overriden of pci_power_mgmt, see below
#
#pci=[ '07:00.0', '07:00.1' ]

#   MSI-INTx translation for MSI capable devices:
#
#   If it's set, Xen will enable MSI for the device that supports it even
# if the guest don't use MSI. In the case, an IO-APIC type interrupt will
# be injected to the guest every time a corresponding MSI message is
# received.
#   If the guest enables MSI or MSI-X, the translation is automatically
# turned off.
# 
#pci_msitranslate=1

#   PCI Power Management:
#
#   If it's set, the guest OS will be able to program D0-D3hot states of the
# PCI device for the purpose of low power consumption.
# 
#pci_power_mgmt=0

#   Enable graphics passthrough:
#
#   If it's set, and specify grapchis device BDF in pci passthrough option,
# like pci=['xx:xx.x'], it enables graphics passthrough, default=0 (disabled)
#gfx_passthru=0

#-----------------------------------------------------------------------------
#   Configure PVSCSI devices:
#
#vscsi=[ 'PDEV, VDEV' ]
#
#   PDEV   gives physical SCSI device to be attached to specified guest
#          domain by one of the following identifier format.
#          - XX:XX:XX:XX (4-tuples with decimal notation which shows
#                          "host:channel:target:lun")
#          - /dev/sdxx or sdx
#          - /dev/stxx or stx
#          - /dev/sgxx or sgx
#          - result of 'scsi_id -gu -s'.
#            ex. # scsi_id -gu -s /block/sdb
#                  36000b5d0006a0000006a0257004c0000
#
#   VDEV   gives virtual SCSI device by 4-tuples (XX:XX:XX:XX) as 
#          which the specified guest domain recognize.
#

#vscsi = [ '/dev/sdx, 0:0:0:0' ]

[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Re: [PATCH] xen/gntdev: Fix sleep-inside-spinlock

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 1163 bytes --]

On Thu, 2011-10-13 at 18:26 +0200, Dario Faggioli wrote:
> On Thu, 2011-10-13 at 11:40 -0400, Konrad Rzeszutek Wilk wrote:
> > Dario, does the patch fix the problem for you?
> > 
> Here I am, and sorry for being late, I had issues with a testbox and
> needed to setup a new one.
>
> [    6.129478] BUG: scheduling while atomic: swapper/0/0x10000002
> [    6.129483] no locks held by swapper/0.
> [    6.129487] Pid: 0, comm: swapper Not tainted 3.1.0-rc9+ #14
> [    6.129492] Call Trace:
> [ ... ]
>
For the records, this is still there for baremetal and dom0 kernel, but
it doesn't bother any longer my pv-domU. Actually, it is complaining
about something different now, but it is able to boot and run.

I'll dig it a bit more and start a new thread if that will be the case.

Thanks and Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
----------------------------------------------------------------------
Dario Faggioli, http://retis.sssup.it/people/faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
PhD Candidate, ReTiS Lab, Scuola Superiore Sant'Anna, Pisa  (Italy)

[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: [PATCH] xen/gntdev: Fix sleep-inside-spinlock

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 1053 bytes --]

On Tue, 2011-10-11 at 15:16 -0400, Daniel De Graaf wrote:
> gntdev_put_map tries to acquire a mutex when freeing pages back to the
> xenballoon pool, so it cannot be called with a spinlock held. In
> gntdev_release, the spinlock is not needed as we are freeing the
> structure later; in the ioctl, only the list manipulation needs to be
> under the lock.
> 
> Reported-By: Dario Faggioli <raistlin@linux.it>
> Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov>
>
Ok, I finally manage in testing this, and it actually cures the issue
here!

You can add a tested-by tag if you like. Just, if you don't mind, use my
corp. e-mail for (both?) the tag(s): <dario.faggioli@citrix.com>

Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
----------------------------------------------------------------------
Dario Faggioli, http://retis.sssup.it/people/faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
PhD Candidate, ReTiS Lab, Scuola Superiore Sant'Anna, Pisa  (Italy)

[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Re: [PATCH] xen/gntdev: Fix sleep-inside-spinlock

[Konrad Rzeszutek Wilk]

On Fri, Oct 14, 2011 at 11:57:05AM +0200, Dario Faggioli wrote:
> On Tue, 2011-10-11 at 15:16 -0400, Daniel De Graaf wrote:
> > gntdev_put_map tries to acquire a mutex when freeing pages back to the
> > xenballoon pool, so it cannot be called with a spinlock held. In
> > gntdev_release, the spinlock is not needed as we are freeing the
> > structure later; in the ioctl, only the list manipulation needs to be
> > under the lock.
> > 
> > Reported-By: Dario Faggioli <raistlin@linux.it>
> > Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov>
> >
> Ok, I finally manage in testing this, and it actually cures the issue
> here!
> 
> You can add a tested-by tag if you like. Just, if you don't mind, use my
> corp. e-mail for (both?) the tag(s): <dario.faggioli@citrix.com>

Sure. Thx for checking it.