no udev events in netback domU driver domain 2.6.32.14

no udev events in netback domU driver domain 2.6.32.14

[philip tricca]

I've spent a bit trying to configure and unprivileged network driver 
domain using the current 2.6.32.14 pvops kernel (haven't up'd to .15 due 
to incompatibility with 4.0.0 release).  I've been partially successful 
but am failing in what I'd think would be the last step: getting udev 
rules to fire when attaching network devices using xen-netback & 
xen-netfront drivers.  From my reading of the pvops wiki page there's a 
possibility that the wiring between the drivers and the udev events may 
not have been forward ported completely.  In fact, using 'udevadm 
monitor' I don't see any events at all when the vif is created in the 
driver domain and when eth0 is created in the client (both are created 
when xm network-attach is run).

Is anyone familiar enough with these portions of the driver to comment 
on this?  If this "should" be working I can post the details of my setup 
and debug information if necessary but I don't want to flood the list 
with a huge email if it isn't necessary.

TIA,
- Philip

Re: no udev events in netback domU driver domain 2.6.32.14

[Konrad Rzeszutek Wilk]

On Tue, Jul 06, 2010 at 10:04:03PM -0400, philip tricca wrote:
> I've spent a bit trying to configure and unprivileged network driver
> domain using the current 2.6.32.14 pvops kernel (haven't up'd to .15
> due to incompatibility with 4.0.0 release).  I've been partially
> successful but am failing in what I'd think would be the last step:
> getting udev rules to fire when attaching network devices using
> xen-netback & xen-netfront drivers.  From my reading of the pvops
> wiki page there's a possibility that the wiring between the drivers
> and the udev events may not have been forward ported completely.  In
> fact, using 'udevadm monitor' I don't see any events at all when the
> vif is created in the driver domain and when eth0 is created in the
> client (both are created when xm network-attach is run).

You won't see the 'eth0' being created from Dom0 side. But you should
see the rest:
> 
> Is anyone familiar enough with these portions of the driver to
> comment on this?  If this "should" be working I can post the details
> of my setup and debug information if necessary but I don't want to
> flood the list with a huge email if it isn't necessary.

This is what I get:

[root@phenom ~]#  udevadm monitor --kernel --env --property
monitor will print the received events for:
KERNEL - the kernel uevent

KERNEL[1278510109.977326] add      /devices/vbd-2-51712 (xen-backend)
UDEV_LOG=3
ACTION=add
DEVPATH=/devices/vbd-2-51712
SUBSYSTEM=xen-backend
XENBUS_TYPE=vbd
XENBUS_PATH=backend/vbd/2/51712
XENBUS_BASE_PATH=backend
SEQNUM=2530

KERNEL[1278510110.042021] add      /devices/vbd-2-51744 (xen-backend)
UDEV_LOG=3
ACTION=add
DEVPATH=/devices/vbd-2-51744
SUBSYSTEM=xen-backend
XENBUS_TYPE=vbd
XENBUS_PATH=backend/vbd/2/51744
XENBUS_BASE_PATH=backend
SEQNUM=2531

KERNEL[1278510110.112319] add      /devices/vif-2-0 (xen-backend)
UDEV_LOG=3
ACTION=add
DEVPATH=/devices/vif-2-0
SUBSYSTEM=xen-backend
XENBUS_TYPE=vif
XENBUS_PATH=backend/vif/2/0
XENBUS_BASE_PATH=backend
SEQNUM=2532

KERNEL[1278510110.134121] add      /devices/vif-2-0/net/vif2.0 (net)
UDEV_LOG=3
ACTION=add
DEVPATH=/devices/vif-2-0/net/vif2.0
SUBSYSTEM=net
INTERFACE=vif2.0
IFINDEX=6
SEQNUM=2533

KERNEL[1278510110.134562] online   /devices/vif-2-0 (xen-backend)
UDEV_LOG=3
ACTION=online
DEVPATH=/devices/vif-2-0
SUBSYSTEM=xen-backend
DRIVER=vif
XENBUS_TYPE=vif
XENBUS_PATH=backend/vif/2/0
XENBUS_BASE_PATH=backend
script=/etc/xen/scripts/vif-bridge
vif=vif2.0
SEQNUM=2534

KERNEL[1278510110.343838] add      /devices/console-2-0 (xen-backend)
UDEV_LOG=3
ACTION=add
DEVPATH=/devices/console-2-0
SUBSYSTEM=xen-backend
XENBUS_TYPE=console
XENBUS_PATH=backend/console/2/0
XENBUS_BASE_PATH=backend
SEQNUM=2535


and then when I shut the guest:

5

KERNEL[1278510216.746063] offline  /devices/vif-2-0 (xen-backend)
UDEV_LOG=3
ACTION=offline
DEVPATH=/devices/vif-2-0
SUBSYSTEM=xen-backend
DRIVER=vif
XENBUS_TYPE=vif
XENBUS_PATH=backend/vif/2/0
XENBUS_BASE_PATH=backend
script=/etc/xen/scripts/vif-bridge
vif=vif2.0
SEQNUM=2536

KERNEL[1278510216.785420] remove   /devices/vif-2-0/net/vif2.0 (net)
UDEV_LOG=3
ACTION=remove
DEVPATH=/devices/vif-2-0/net/vif2.0
SUBSYSTEM=net
INTERFACE=vif2.0
IFINDEX=6
SEQNUM=2537


KERNEL[1278510217.514812] remove   /devices/console-2-0 (xen-backend)
UDEV_LOG=3
ACTION=remove
DEVPATH=/devices/console-2-0
SUBSYSTEM=xen-backend
XENBUS_TYPE=console
XENBUS_PATH=backend/console/2/0
XENBUS_BASE_PATH=backend
SEQNUM=2538

KERNEL[1278510217.588877] remove   /devices/vbd-2-51712 (xen-backend)
UDEV_LOG=3
ACTION=remove
DEVPATH=/devices/vbd-2-51712
SUBSYSTEM=xen-backend
XENBUS_TYPE=vbd
XENBUS_PATH=backend/vbd/2/51712
XENBUS_BASE_PATH=backend
SEQNUM=2539

KERNEL[1278510217.596581] remove   /devices/vbd-2-51744 (xen-backend)
UDEV_LOG=3
ACTION=remove
DEVPATH=/devices/vbd-2-51744
SUBSYSTEM=xen-backend
XENBUS_TYPE=vbd
XENBUS_PATH=backend/vbd/2/51744
XENBUS_BASE_PATH=backend
SEQNUM=2540


KERNEL[1278510217.602293] remove   /devices/vif-2-0 (xen-backend)
UDEV_LOG=3
ACTION=remove
DEVPATH=/devices/vif-2-0
SUBSYSTEM=xen-backend
XENBUS_TYPE=vif
XENBUS_PATH=backend/vif/2/0
XENBUS_BASE_PATH=backend
SEQNUM=2541

> 
> TIA,
> - Philip
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Re: no udev events in netback domU driver domain 2.6.32.14

[philip tricca]

Konrad Rzeszutek Wilk wrote:
> On Tue, Jul 06, 2010 at 10:04:03PM -0400, philip tricca wrote:
>> I've spent a bit trying to configure and unprivileged network driver
>> domain using the current 2.6.32.14 pvops kernel (haven't up'd to .15
>> due to incompatibility with 4.0.0 release).  I've been partially
>> successful but am failing in what I'd think would be the last step:
>> getting udev rules to fire when attaching network devices using
>> xen-netback & xen-netfront drivers.  From my reading of the pvops
>> wiki page there's a possibility that the wiring between the drivers
>> and the udev events may not have been forward ported completely.  In
>> fact, using 'udevadm monitor' I don't see any events at all when the
>> vif is created in the driver domain and when eth0 is created in the
>> client (both are created when xm network-attach is run).
> 
> You won't see the 'eth0' being created from Dom0 side. But you should
> see the rest:

Correct.  I don't see any udev events from dom0: the nic is passed to a 
domU (call it a driver domain) through pciback so all of the vif events 
happen there.  Another domU is getting its eth0 through the netback 
offered up by the driver domain

>> Is anyone familiar enough with these portions of the driver to
>> comment on this?  If this "should" be working I can post the details
>> of my setup and debug information if necessary but I don't want to
>> flood the list with a huge email if it isn't necessary.
> 
> This is what I get:
> 
> [root@phenom ~]#  udevadm monitor --kernel --env --property
> monitor will print the received events for:
> KERNEL - the kernel uevent

<snip\>

I am now getting the right udev events (the same ones Konrad shows) in 
both my driver domain and in the client.  The problem I am still having 
is related to the xenstore.

The network scripts run by udev access data in the xen store and use it 
to return status information to dom0:
/local/domain/X/backend/vif/Y/Z/hotplug-status
The xenstore is completely inaccessible from my driver domain however. 
I've installed the xenstored daemon in the driver domain which requires 
running it with the --no-domain-init option to keep it from trying to 
execute privileged operations (it's not dom0).

Even with the xenstored daemon running though I (and the networking 
scripts) still can't access then xenstore.

I'm making progress but I could use a hint if someone's got one.

Cheers,
- Philip

Re: no udev events in netback domU driver domain 2.6.32.14

[Konrad Rzeszutek Wilk]

> it to return status information to dom0:
> /local/domain/X/backend/vif/Y/Z/hotplug-status
> The xenstore is completely inaccessible from my driver domain
> however. I've installed the xenstored daemon in the driver domain
> which requires running it with the --no-domain-init option to keep
> it from trying to execute privileged operations (it's not dom0).

You should be able to read it? At least for your domain entry?
> 
> Even with the xenstored daemon running though I (and the networking
> scripts) still can't access then xenstore.

Well, you can't access all of it b/c there is an ownership of the
/local/domain/0 being under Dom0. Not under DomU. AFAICT the backends
use those entries (for example, netback detects via watches that
somebody wrote in /local/domain/0/backend/vif and calls its .probe
function), they will interogate the XenStore and then create udev events
which are fired off to userspace.

Re: no udev events in netback domU driver domain 2.6.32.14

[philip tricca]

Konrad Rzeszutek Wilk wrote:
>> it to return status information to dom0:
>> /local/domain/X/backend/vif/Y/Z/hotplug-status
>> The xenstore is completely inaccessible from my driver domain
>> however. I've installed the xenstored daemon in the driver domain
>> which requires running it with the --no-domain-init option to keep
>> it from trying to execute privileged operations (it's not dom0).
> 
> You should be able to read it? At least for your domain entry?
>> Even with the xenstored daemon running though I (and the networking
>> scripts) still can't access then xenstore.
> 
> Well, you can't access all of it b/c there is an ownership of the
> /local/domain/0 being under Dom0. Not under DomU. AFAICT the backends
> use those entries (for example, netback detects via watches that
> somebody wrote in /local/domain/0/backend/vif and calls its .probe
> function), they will interogate the XenStore and then create udev events
> which are fired off to userspace.

The problem I was having (past tense since I've figured it out) was a 
number of small things going wrong at the same time.  The udev events 
weren't the problem it was figuring out what tools needed to be 
installed in the driver domain, what modules needed to be loaded, as 
well as some debugging in the vif scripts.

Thanks for your input Konrad.  I'll be adding a few notes to the xen 
wiki page for driver domains.

Cheers,
- Philip

-- 
Ruin is the destination toward which all men rush,
each pursuing his own best interest.

Re: no udev events in netback domU driver domain, 2.6.32.14

[Steven Harp]

> I am now getting the right udev events (the same ones Konrad shows) in 
> both my driver domain and in the client.  The problem I am still having 
> is related to the xenstore.
> 
> The network scripts run by udev access data in the xen store and use it 
> to return status information to dom0:
> /local/domain/X/backend/vif/Y/Z/hotplug-status
> The xenstore is completely inaccessible from my driver domain however. 
> I've installed the xenstored daemon in the driver domain which requires 
> running it with the --no-domain-init option to keep it from trying to 
> execute privileged operations (it's not dom0).
> 
> Even with the xenstored daemon running though I (and the networking 
> scripts) still can't access then xenstore.
> 
> I'm making progress but I could use a hint if someone's got one.

Possibly not related, but I have seen it hang in a similar place.
It turned out that my driver domain had failed to prepare a /var/log/xen
(where the scripts want to log errors).

Adding this did the trick here.

// Steve Harp