follow up to a pciback "pv pci-passthrough co-assigned problem"

follow up to a pciback "pv pci-passthrough co-assigned problem"

[mail ignored]

i've

 rpm -qa | grep -i xen-3
  xen-3.4.1_19718_05-1.1.x86_64

i'm trying to passthrough a PCI Eth NIC to a DomU

@ grub,

  module /vmlinuz-xen ... guestdev=0000:04:06.0 reassign_resources ...

and.

  test.cfg
	...
	pci          = [ '04:06.0' ]
	...

where,

lspci -vvv | grep -A20 04:06.0
	04:06.0 Ethernet controller: Intel Corporation 82541PI Gigabit
Ethernet Controller (rev 05)
	        Subsystem: Intel Corporation PRO/1000 GT Desktop Adapter
	        Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop-
ParErr- Stepping- SERR- FastB2B- DisINTx-
	        Status: Cap+ 66MHz+ UDF- FastB2B- ParErr- DEVSEL=medium
>TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	        Interrupt: pin A routed to IRQ 20
	        Region 0: Memory at fbf00000 (32-bit, non-prefetchable)
[disabled] [size=128K]
	        Region 1: Memory at fbf20000 (32-bit, non-prefetchable)
[disabled] [size=128K]
	        Region 2: I/O ports at ec00 [disabled] [size=64]
	        Expansion ROM at f0180000 [disabled] [size=128K]
	        Capabilities: [dc] Power Management version 2
	                Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA
PME(D0+,D1-,D2-,D3hot+,D3cold+)
	                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME-
	        Capabilities: [e4] PCI-X non-bridge device
	                Command: DPERE- ERO+ RBC=512 OST=1
	                Status: Dev=00:00.0 64bit- 133MHz- SCD- USC-
DC=simple DMMRBC=2048 DMOST=1 DMCRS=8 RSCEM- 266MHz- 533MHz-
	        Kernel driver in use: pciback
	...

@

 xm create -c test.cfg

I get,

	Error: pci: 0000:04:07.0 must be co-assigned to the same guest with
0000:04:06.0

fyi,

 lspci -vvv | grep -A20 04:07.0
	04:07.0 RAID bus controller: Silicon Image, Inc. SiI 3124 PCI-X
Serial ATA Controller (rev 02)
	        Subsystem: Silicon Image, Inc. Device 7124
	        Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop-
ParErr- Stepping+ SERR- FastB2B- DisINTx-
	        Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium
>TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	        Interrupt: pin A routed to IRQ 21
	        Region 0: Memory at fbf48000 (64-bit, non-prefetchable)
[disabled] [size=128]
	        Region 2: Memory at fbf40000 (64-bit, non-prefetchable)
[disabled] [size=32K]
	        Region 4: I/O ports at e880 [disabled] [size=16]
	        Expansion ROM at f0100000 [disabled] [size=512K]
	        Capabilities: [64] Power Management version 2
	                Flags: PMEClk- DSI+ D1+ D2+ AuxCurrent=0mA
PME(D0-,D1-,D2-,D3hot-,D3cold-)
	                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME-
	        Capabilities: [40] PCI-X non-bridge device
	                Command: DPERE- ERO+ RBC=512 OST=12
	                Status: Dev=ff:1f.0 64bit+ 133MHz+ SCD- USC-
DC=simple DMMRBC=2048 DMOST=12 DMCRS=128 RSCEM- 266MHz- 533MHz-
	        Capabilities: [54] MSI: Enable- Count=1/1 Maskable- 64bit+
	                Address: 0000000000000000  Data: 0000
	        Kernel driver in use: pciback


searching, i find a patch
(http://old.nabble.com/attachment/20011206/0/disable_co-assignment.patch)
@,

	http://old.nabble.com/xen-3.3.0-pv-pci-passthrough-co-assigned-problem-td20008460.html

that doesn't make any difference.

i did find, @

	http://article.gmane.org/gmane.comp.emulators.xen.devel/72618
weidong.han <at> intel.com
	"I guess 07:00.0 and 07:00.1 are two PCI (non-PCIe) devices in your
system. PCI devices under the same bridge must be assigned to one
guest. Pls hide both of them by pci-stub, then you can assign both of
them to a guest, or just assign one
."

and,

	[Xen-devel] [PATCH] Xend: allow pci-stub to hide devices for assignment
    http://lists.xensource.com/archives/html/xen-devel/2009-07/msg00114.html

but, iiuc, pc-sub hiding is a pvops, not 'legacy' xen, capability ...

at this point, i'm unclear whether this is fixable :-/

Is the requirement to coassign PCI cards on the same bridge a *bug*
that's been fixed, or can be?

Re: follow up to a pciback "pv pci-passthrough co-assigned problem"

[mail ignored]

iiuc, this may be the same issue (?),

 http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1340

 "With 3.2.0 I passed through two PCI cards to a linux domU, now with 3.3.0 I am
 unable to pass them, I receive this error on creating the domU

 pci: 0000:08:03.0 must be co-assigned to the same guest with 0000:08:00.0"

as yet unresolved ...

Re: Re: follow up to a pciback "pv pci-passthrough co-assigned problem"

[Pasi Kärkkäinen]

On Fri, Jan 22, 2010 at 08:09:50PM -0800, mail ignored wrote:
> iiuc, this may be the same issue (?),
> 
>  http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1340
> 
>  "With 3.2.0 I passed through two PCI cards to a linux domU, now with 3.3.0 I am
>  unable to pass them, I receive this error on creating the domU
> 
>  pci: 0000:08:03.0 must be co-assigned to the same guest with 0000:08:00.0"
> 
> as yet unresolved ...
> 

This has been discussed many times here on xen-devel and on xen-users..

with xen-unstable (upcoming Xen 4.0) take a look at 
/etc/xen/xend-config.sxp:

	(pci-passthrough-strict-check yes)

set that to no, restart xend, and it'll work.

With Xen 3.3, or 3.4, you can apply a "disable FLR" patch, which also makes it work:
http://xen.markmail.org/download.xqy?id=7dbm675e4x4dawec&number=1

See this thread:
http://xen.markmail.org/search/?q=disable+flr#query:disableflr+page:1+mid:7dbm675e4x4dawec+state:results

Hopefully that helps.

-- Pasi

Re: Re: follow up to a pciback "pv pci-passthrough co-assigned problem"

[mail ignored]

hi

On Sat, Jan 23, 2010 at 4:30 AM, Pasi Kärkkäinen <pasik@iki.fi> wrote:
> This has been discussed many times here on xen-devel and on xen-users..

You're right -- there sure has.  That's been my problem -- there's not
a consistent solution to be found. :-/

> with xen-unstable (upcoming Xen 4.0) take a look at /etc/xen/xend-config.sxp:
>        (pci-passthrough-strict-check yes)
> set that to no, restart xend, and it'll work.

Yep, I'm aware of that.  But 4.0.x fails miserably on my box, atm :-/
I'm hoping that there's a fix for 'stable' 3.4.x.

> With Xen 3.3, or 3.4, you can apply a "disable FLR" patch, which also makes it work:
> http://xen.markmail.org/download.xqy?id=7dbm675e4x4dawec&number=1

I applied the patch at that link. At VM launch, I still get the

  Error: pci: 0000:04:07.0 must be co-assigned to the same guest with
0000:04:06.0

message ...

Thanks.

Re: Re: follow up to a pciback "pv pci-passthrough co-assigned problem"

[Pasi Kärkkäinen]

On Sat, Jan 23, 2010 at 08:40:28AM -0800, mail ignored wrote:
> hi
> 
> On Sat, Jan 23, 2010 at 4:30 AM, Pasi Kärkkäinen <pasik@iki.fi> wrote:
> > This has been discussed many times here on xen-devel and on xen-users..
> 
> You're right -- there sure has.  That's been my problem -- there's not
> a consistent solution to be found. :-/
> 
> > with xen-unstable (upcoming Xen 4.0) take a look at /etc/xen/xend-config.sxp:
> >        (pci-passthrough-strict-check yes)
> > set that to no, restart xend, and it'll work.
> 
> Yep, I'm aware of that.  But 4.0.x fails miserably on my box, atm :-/
> I'm hoping that there's a fix for 'stable' 3.4.x.
> 

How does Xen 4.0 (xen-unstable) fail for you? 

> > With Xen 3.3, or 3.4, you can apply a "disable FLR" patch, which also makes it work:
> > http://xen.markmail.org/download.xqy?id=7dbm675e4x4dawec&number=1
> 
> I applied the patch at that link. At VM launch, I still get the
> 
>   Error: pci: 0000:04:07.0 must be co-assigned to the same guest with
> 0000:04:06.0
> 

Are you sure you installed the new patched xen tools after applying the patch?
And restarted xend? 

-- Pasi

Re: Re: follow up to a pciback "pv pci-passthrough co-assigned problem"

[mail ignored]

On Sat, Jan 23, 2010 at 8:48 AM, Pasi Kärkkäinen <pasik@iki.fi> wrote:
> How does Xen 4.0 (xen-unstable) fail for you?

well, wasn't going to make this an issue -- as I'm not yet committed
to Xen 4 (*suse's flavor, anyway) until it's a bit more widely-used &
stable, but -- since you asked,

zypper dup -r XEN4
cp /etc/xend-config.sxp.XEN4 /etc/xend-config.sxp
reboot

uname -rp
  2.6.31.8-0.1-xen x86_64
rpm -qa | grep -i xen-4
  xen-4.0.0_20809_01-28.2.x86_64
xm dmesg
  ...
  (XEN) Xen version 4.0.0_20809_01-28.2 (abuild@) (gcc version 4.4.1
[gcc-4_4-branch revision 150839] (SUSE Linux) ) Wed Jan 20 01:09:45
UTC 2010
  (XEN) Latest ChangeSet: 20809
  (XEN) Command line: loglvl=all loglvl_guest=all apic_verbosity=debug
dom0_mem=1024M vga=gfx-1280x1024x32 console=vga,com1 com1=57600,8n1
iommu=1 dom0_max_vcpus=4 dom0_vcpus_pin=true sched=credit cpuidle
cpufreq=xen clocksource=acpi numa=on
  ...

@ any/all vm launch, i'd get some message

xm create -c /home/xen/vm/test.cfg
  Using config file "/home/xen/vm/test.cfg".
  Error: Domain 'test' does not exist.

xend log -> http://pastebin.com/d6c5d3100.  if something 'in there' is
informative, hth!

drop back to xen3x, all's OK again.

>> > With Xen 3.3, or 3.4, you can apply a "disable FLR" patch, which also makes it work:
> Are you sure you installed the new patched xen tools after applying the patch?
> And restarted xend?

i backed up my production xen/util tree, and applied the patch
directly there.  after reboot, @

  xm create -c test.cfg

i get

  Error: pci: 0000:04:07.0 must be co-assigned to the same guest with
0000:04:06.0

Re: Re: follow up to a pciback "pv pci-passthrough co-assigned problem"

[Pasi Kärkkäinen]

On Sat, Jan 23, 2010 at 09:36:53AM -0800, mail ignored wrote:
> On Sat, Jan 23, 2010 at 8:48 AM, Pasi Kärkkäinen <pasik@iki.fi> wrote:
> > How does Xen 4.0 (xen-unstable) fail for you?
> 
> well, wasn't going to make this an issue -- as I'm not yet committed
> to Xen 4 (*suse's flavor, anyway) until it's a bit more widely-used &
> stable, but -- since you asked,
> 
> zypper dup -r XEN4
> cp /etc/xend-config.sxp.XEN4 /etc/xend-config.sxp
> reboot
> 
> uname -rp
>   2.6.31.8-0.1-xen x86_64
> rpm -qa | grep -i xen-4
>   xen-4.0.0_20809_01-28.2.x86_64
> xm dmesg
>   ...
>   (XEN) Xen version 4.0.0_20809_01-28.2 (abuild@) (gcc version 4.4.1
> [gcc-4_4-branch revision 150839] (SUSE Linux) ) Wed Jan 20 01:09:45
> UTC 2010
>   (XEN) Latest ChangeSet: 20809
>   (XEN) Command line: loglvl=all loglvl_guest=all apic_verbosity=debug
> dom0_mem=1024M vga=gfx-1280x1024x32 console=vga,com1 com1=57600,8n1
> iommu=1 dom0_max_vcpus=4 dom0_vcpus_pin=true sched=credit cpuidle
> cpufreq=xen clocksource=acpi numa=on
>   ...
> 
> @ any/all vm launch, i'd get some message
> 
> xm create -c /home/xen/vm/test.cfg
>   Using config file "/home/xen/vm/test.cfg".
>   Error: Domain 'test' does not exist.
> 
> xend log -> http://pastebin.com/d6c5d3100.  if something 'in there' is
> informative, hth!
> 

It seems the guest crashes:
[2010-01-23 09:23:11 4595] WARNING (XendDomainInfo:2072) Domain has crashed: name=test id=6.



> drop back to xen3x, all's OK again.
> 
> >> > With Xen 3.3, or 3.4, you can apply a "disable FLR" patch, which also makes it work:
> > Are you sure you installed the new patched xen tools after applying the patch?
> > And restarted xend?
> 
> i backed up my production xen/util tree, and applied the patch
> directly there.  after reboot, @
> 
>   xm create -c test.cfg
> 
> i get
> 
>   Error: pci: 0000:04:07.0 must be co-assigned to the same guest with
> 0000:04:06.0
>

Hmm.. this patched worked OK for others. 

Search for that error from the sources, and try to read what causes it.. 
and patch the code to bypass it? 

That's what the patch is supposed to do..

-- Pasi

Re: Re: follow up to a pciback "pv pci-passthrough co-assigned problem"

[mail ignored]

On Sat, Jan 23, 2010 at 9:43 AM, Pasi Kärkkäinen <pasik@iki.fi> wrote:
> It seems the guest crashes:
> [2010-01-23 09:23:11 4595] WARNING (XendDomainInfo:2072) Domain has crashed: name=test id=6.

fwiw, the guest is fine when launched from Dom0 Xen 3x ...

> Hmm.. this patched worked OK for others.

welcome to my world  ;-)

Re: Re: follow up to a pciback "pv pci-passthrough co-assigned problem"

[mail ignored]

On Sat, Jan 23, 2010 at 9:45 AM, mail ignored <0.bugs.only.0@gmail.com> wrote:
>> Hmm.. this patched worked OK for others.

i'd tried/failed to get this to work 5+ times, warm-rebooting each time.

i just did a full shutdown, then cold start ... and now it works.

after patch, @ DomU,

lspci
  00:00.0 Ethernet controller: Intel Corporation 82541PI Gigabit
Ethernet Controller (rev 05)

gremlins.  sigh.

Re: Re: follow up to a pciback "pv pci-passthrough co-assigned problem"

[Pasi Kärkkäinen]

On Sat, Jan 23, 2010 at 09:54:30AM -0800, mail ignored wrote:
> On Sat, Jan 23, 2010 at 9:45 AM, mail ignored <0.bugs.only.0@gmail.com> wrote:
> >> Hmm.. this patched worked OK for others.
> 
> i'd tried/failed to get this to work 5+ times, warm-rebooting each time.
> 
> i just did a full shutdown, then cold start ... and now it works.
> 
> after patch, @ DomU,
> 
> lspci
>   00:00.0 Ethernet controller: Intel Corporation 82541PI Gigabit
> Ethernet Controller (rev 05)
> 
> gremlins.  sigh.
>

Heh. Well good to hear it works now :)

-- Pasi

Re: Re: follow up to a pciback "pv pci-passthrough co-assigned problem"

[mail ignored]

On Sat, Jan 23, 2010 at 9:57 AM, Pasi Kärkkäinen <pasik@iki.fi> wrote:
> Heh. Well good to hear it works now :)

thanks.  i suppose i'll file a bug @ opensuse abt this, and see if
they'll put something into code so these changes will 'survive'
xen(-tools) updates ...

Re: Re: follow up to a pciback "pv pci-passthrough co-assigned problem"

[mail ignored]

On Sat, Jan 23, 2010 at 9:58 AM, mail ignored <0.bugs.only.0@gmail.com> wrote:
> thanks.  i suppose i'll file a bug @ opensuse abt this, and see if
> they'll put something into code so these changes will 'survive'
> xen(-tools) updates ...

cref -> https://bugzilla.novell.com/show_bug.cgi?id=573311

Re: Re: follow up to a pciback "pv pci-passthrough co-assigned problem"

[Weidong Han]

mail ignored wrote:
> On Sat, Jan 23, 2010 at 9:58 AM, mail ignored <0.bugs.only.0@gmail.com> wrote:
>   
>> thanks.  i suppose i'll file a bug @ opensuse abt this, and see if
>> they'll put something into code so these changes will 'survive'
>> xen(-tools) updates ...
>>     
>
> cref -> https://bugzilla.novell.com/show_bug.cgi?id=573311
>   

I think this is not a bug. pls note that the devices behind 
PCIe-to-PCI/PCI-x bridge or Conventional PCI bridge can only be 
collectively assigned to a single domain, because the source-id in DMA 
requests is not device's bdf. Pls refer to section 3.6.1 in VT-d spec. 
What's more, FLR is required to make sure the device in a correct status 
before assignment, and some devices need to reset secondary bus to reset 
the device which results in all the devices behind same 
PCIe-to-PCI/PCI-x bridge or Conventional PCI bridge be reset. In your 
case, you need to hide all NICs (04:06.0 and 04:07.0) behind the same 
bridge, and assign them together to a guest. or you can set 
"pci-passthrough-strict-check no" in /etc/xen/xend-config.sxp and 
restart xend to loose the check in xend, but in this way you should be 
aware of potential issues (e.g. assigned device doesn't work).

Regards,
Weidong

Re: Re: follow up to a pciback "pv pci-passthrough co-assigned problem"

[0bo0]

hi,

On Sun, Jan 24, 2010 at 5:44 PM, Weidong Han <weidong.han@intel.com> wrote:
> I think this is not a bug. pls note ...

(snip)

Ack'd, even if the limitations are not fully understood:-/

> or you
> can set "pci-passthrough-strict-check no" in /etc/xen/xend-config.sxp and
> restart xend to loose the check in xend, but in this way you should be aware
> of potential issues (e.g. assigned device doesn't work).

sure.  would it not be reasonable to have a similar config-file option
in xen 3.x -- with similar warnings etc -- as is done in 4.x? if all
that's required for a workaround is a patch similar to that above ...

thanks

Re: Re: follow up to a pciback "pv pci-passthrough co-assigned problem"

[Weidong Han]

0bo0 wrote:
> hi,
>
> On Sun, Jan 24, 2010 at 5:44 PM, Weidong Han <weidong.han@intel.com> wrote:
>   
>> I think this is not a bug. pls note ...
>>     
>
> (snip)
>
> Ack'd, even if the limitations are not fully understood:-/
>
>   
>> or you
>> can set "pci-passthrough-strict-check no" in /etc/xen/xend-config.sxp and
>> restart xend to loose the check in xend, but in this way you should be aware
>> of potential issues (e.g. assigned device doesn't work).
>>     
>
> sure.  would it not be reasonable to have a similar config-file option
> in xen 3.x -- with similar warnings etc -- as is done in 4.x? if all
> that's required for a workaround is a patch similar to that above ...
>
> thanks
>   
The option was introduced by changeset 20179. It's small. It can be back 
ported to Xen 3.x if need.

Regards,
Weidong

Re: Re: follow up to a pciback "pv pci-passthrough co-assigned problem"

[Pasi Kärkkäinen]

On Tue, Jan 26, 2010 at 09:42:00AM +0800, Weidong Han wrote:
> 0bo0 wrote:
>> hi,
>>
>> On Sun, Jan 24, 2010 at 5:44 PM, Weidong Han <weidong.han@intel.com> wrote:
>>   
>>> I think this is not a bug. pls note ...
>>>     
>>
>> (snip)
>>
>> Ack'd, even if the limitations are not fully understood:-/
>>
>>   
>>> or you
>>> can set "pci-passthrough-strict-check no" in /etc/xen/xend-config.sxp and
>>> restart xend to loose the check in xend, but in this way you should be aware
>>> of potential issues (e.g. assigned device doesn't work).
>>>     
>>
>> sure.  would it not be reasonable to have a similar config-file option
>> in xen 3.x -- with similar warnings etc -- as is done in 4.x? if all
>> that's required for a workaround is a patch similar to that above ...
>>
>> thanks
>>   
> The option was introduced by changeset 20179. It's small. It can be back  
> ported to Xen 3.x if need.
>

It would be good to backport this.

The point here is, I think, that before the VT-d stuff you could passthrough
for example each port from a dual-port NIC to different PV guests, even
if it was not the recommended way.

Now at some point it stopped working for many people, and Xen started complaining
that the devices need to be passed to the same guest.

I know some people are also using another port from a dual-port NIC in the dom0, 
and another port passthroughed to a PV guest. And they've been doing this for a long time.

-- Pasi

Re: Re: follow up to a pciback "pv pci-passthrough co-assigned problem"

[Keir Fraser]

On 23/01/2010 17:43, "Pasi Kärkkäinen" <pasik@iki.fi> wrote:

>> xm create -c /home/xen/vm/test.cfg
>>   Using config file "/home/xen/vm/test.cfg".
>>   Error: Domain 'test' does not exist.
>> 
>> xend log -> http://pastebin.com/d6c5d3100.  if something 'in there' is
>> informative, hth!
>> 
> 
> It seems the guest crashes:
> [2010-01-23 09:23:11 4595] WARNING (XendDomainInfo:2072) Domain has crashed:
> name=test id=6.

Qemu-dm dies on SIGSEGV.

 -- Keir

Re: Re: follow up to a pciback "pv pci-passthrough co-assigned problem"

[0bo0]

On Sat, Jan 23, 2010 at 10:19 AM, Keir Fraser <keir.fraser@eu.citrix.com> wrote:
> On 23/01/2010 17:43, "Pasi Kärkkäinen" <pasik@iki.fi> wrote:
>
>>> xm create -c /home/xen/vm/test.cfg
>>>   Using config file "/home/xen/vm/test.cfg".
>>>   Error: Domain 'test' does not exist.
>>>
>>> xend log -> http://pastebin.com/d6c5d3100.  if something 'in there' is
>>> informative, hth!
>>>
>>
>> It seems the guest crashes:
>> [2010-01-23 09:23:11 4595] WARNING (XendDomainInfo:2072) Domain has crashed:
>> name=test id=6.
>
> Qemu-dm dies on SIGSEGV.


Seems like I may have not much choice but to start diving into 4.0.
That siad, where do I look to get a handle on this?

...
[2010-01-25 22:41:11 4493] DEBUG (XendDomainInfo:1892)
XendDomainInfo.handleShutdownWatch
[2010-01-25 22:41:11 4493] WARNING (image:561) domain test: device
model failure: pid 5288: malfunctioning (closed sentinel), killed; see
/var/log/xen/qemu-dm-nas.log
[2010-01-25 22:41:11 4493] WARNING (XendDomainInfo:2072) Domain has
crashed: name=test id=1.
[2010-01-25 22:41:11 4493] DEBUG (XendDomainInfo:3061)
XendDomainInfo.destroy: domid=1
[2010-01-25 22:41:15 4493] DEBUG (XendDomainInfo:2416) Destroying device model
...

checking,

cat /var/log/xen/qemu-dm-nas.log
 domid: 1
 Warning: vlan 0 is not connected to host network
 Using xvda for guest's hda

doesn't tell me much of what to do.  any suggestions, or further info
I can provide here?

thanks.

Re: Re: follow up to a pciback "pv pci-passthrough co-assigned problem"

[Keir Fraser]

On 26/01/2010 06:50, "0bo0" <0.bugs.only.0@gmail.com> wrote:

> cat /var/log/xen/qemu-dm-nas.log
>  domid: 1
>  Warning: vlan 0 is not connected to host network
>  Using xvda for guest's hda
> 
> doesn't tell me much of what to do.  any suggestions, or further info
> I can provide here?

If qemu-dm died without logging anything, it probably crashed again. If it
produced a core file you could load that in gdb and get a symbolic
backtrace.

My guess, since qemu-dm does seem to be working in most people's tests, is
that either you have managed to install and not replace a mismatching
version, somehow, or the set of options getting passed to it from you domain
configuration have not been tested and do not work.

 -- Keir